8000 test-pgp and test-pgpmime fails non-deterministically · Issue #177 · jstedfast/gmime · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

test-pgp and test-pgpmime fails non-deterministically #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Apteryks opened this issue Apr 3, 2025 · 8 comments
Open

test-pgp and test-pgpmime fails non-deterministically #177

Apteryks opened this issue Apr 3, 2025 · 8 comments

Comments

@Apteryks
Copy link
Apteryks commented Apr 3, 2025

Hi,

This affects at least 3.2.14 and 3.2.15. Sometimes the test suite passes, sometimes it fails, like so:

Testing GnuPG crypto context: failed (4 errors, 0 warnings)
Testing PGP/MIME implementation: passed
Results: 12 tests passed; 1 tests failed.
Failed tests: 
 test-pgp
make[3]: *** [Makefile:1137: check-local] Error 255
make[3]: Leaving directory '/tmp/guix-build-gmime-3.2.15.drv-0/source/tests'
make[2]: *** [Makefile:974: check-am] Error 2
make[2]: Leaving directory '/tmp/guix-build-gmime-3.2.15.drv-0/source/tests'
make[1]: *** [Makefile:824: check-recursive] Error 1
make[1]: Leaving directory '/tmp/guix-build-gmime-3.2.15.drv-0/source/tests'
make: *** [Makefile:536: check-recursive] Error 1

Test suite failed, dumping logs.
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("check" "-j" "32") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `check' failed after 67.0 seconds
command "make" "check" "-j" "32" failed with status 2
build process 18 exited with status 256

I suppose there is a race somewhere in the build system.

I think I've seen test-pgpmime fail as well.
@Apteryks
Copy link
Author
Apteryks commented Apr 3, 2025

Building with many cores is probably key to reproduce (I'm using 32 here). And I confirm the test-pgpmime fails spuriously as well:

Results: 12 tests passed; 1 tests failed.
Failed tests: 
 test-pgpmime
make[3]: *** [Makefile:1137: check-local] Error 255
make[3]: Leaving directory '/tmp/guix-build-gmime-3.2.15.drv-0/source/tests'
make[2]: *** [Makefile:974: check-am] Error 2
make[2]: Leaving directory '/tmp/guix-build-gmime-3.2.15.drv-0/source/tests'
make[1]: *** [Makefile:824: check-recursive] Error 1
make[1]: Leaving directory '/tmp/guix-build-gmime-3.2.15.drv-0/source/tests'
make: *** [Makefile:536: check-recursive] Error 1

Test suite failed, dumping logs.
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("check" "-j" "32") exit-status: 2 term-signal: #f stop-signal: 
phase `check' failed after 10.5 seconds
command "make" "check" "-j" "32" failed with status 2
build process 18 exited with status 256

The dependencies used are:

autoconf@2.71 automake@1.16.5 glib@2.82.1 gnupg@2.4.7 gobject-introspection@1.82.0
+ gpgme@1.24.1 gtk-doc@1.34.0 libtool@2.4.7 pkg-config@0.29.2 vala@0.56.17 which@2.21 zlib@1.3

It should be relatively easy to reproduce using Guix (e.g. with guix build --no-grafts gmime && guix build --no-grafts gmime --check --rounds=10).

@Apteryks Apteryks changed the title test-pgp fails non-deterministically test-pgp and test-pgpmime fails non-deterministically Apr 3, 2025
@Apteryks
Copy link
Author
Apteryks commented Apr 3, 2025

With this patch removing the two tests, I could build/run the test suites 10 times without a failure.

@jstedfast
Copy link
Owner

Are the tests being run in parallel? This might cause issues since test-pgp and test-pgpmime both setup/use/clobber (the same) temporary GNUPG_HOME environment.

If you just run one of those tests by itself, using a command like ./test-pgp -vvvv, does it ever fail? And if so, what are the diagnostics?

@Apteryks
Copy link
Author
Apteryks commented Apr 4, 2025

I can't investigate much at the moment, but yes, the tests were run in parallel, via make check -j32.

@jstedfast
Copy link
Owner

Then that is probably why.

@Apteryks
Copy link
Author
Apteryks 8000 commented Apr 4, 2025

I was able to reproduce the problem running only test-pgp like so:

$ timeout 5m sh -c 'while ./test-pgp -vvvv; do :; done'
[...]
Checking GMimeGpgContext::encrypt... PASSED
Checking GMimeGpgContext::decrypt... PASSED
ciphertext:
-----BEGIN PGP MESSAGE-----

hQIOAwTJ0obO8knpEAf/TK1IKYjU7ab+p+ghgOraYjxGc34j93fYG1AgUtdZKFWq
jnSBKr+e6HVHgWAttfhWWNz36OliGZNgm6ZJJNJti8aca4gqZ/ucK4QZY84kWRr/
ohNBtPEXDARPCNuruhQwWbpyeoynvPEqfhTVTKSlYvGewZsW4SGCDR7Zt7tuHox5
Md92jCFYRhDKfwouct0iq3mNCMBFkgP+y4IcrfRDQo3SMw6xwKzccXJCqFVhkne/
LnWo9GWMgw7vbVeZeKh4qWV/h3th/fgBOwoTumbMWJgw3fsQB2pHss2hl04ADl8d
5dpgzXJW5ZsnfWxAAbC0xo1AHum/gcWvLdM/+DWicgf/VZ6wPdfE9AxYSFl3WG7C
Mu4r1s9amLzJE6/k/1DkHaDxmu+MVc7MmPi+UpUjeE87Ircnicat7iJT60Q6qJcP
Gn5a8UXBNLl8MotPlzlAsobLYWvYi+MntiBIsXShD/L3nl0ielpv5FCJx26/uNpJ
4JQyYziuGOz/DaX3mfFoWIvGOK9fW4Vk2TZrVuF/qrnxPgy0q63qrE8Ese7JtwHI
xVX4/1TtusB2fl68R2Wcz+/60iVxe9M86/dYAzI/2WvGKDWBV3szRXAl5qkpQZeL
W0xq4/tTIOYR003075u56PYtfpBaoYFPZhoz4YQixNs7TaCiflSvendfUXHlWbXr
SdK1AdtI30lCNHyah56ocxRhsIMdeoX1zYBRQCbWhi3FdtRi7+dcQq5j8u+yxwpa
aDWBhxsDVBAecU8Eti4UfzgrsgINX2CFcAsc0w5GndlLMmpr98Bt+LjI2Rb4aCvg
wjz+sI/ladOiI1g+5YTMw1rtWOlrdJLgxjMtmFRKSzAtlhMqzLb/zjEjuvhnCo/1
55+6+JatJ+73PfSixqaahd54Fc5FXTcGHc/hBtKb2LYrej2BhP82Ow==
=65jU
-----END PGP MESSAGE-----

Checking GMimeGpgContext::encrypt+sign... PASSED
Checking GMimeGpgContext::decrypt+verify... PASSED
Checking GMimeFilterOpenPGP::public key block... PASSED
Checking GMimeFilterOpenPGP::private key block... PASSED
Checking GMimeFilterOpenPGP::signed message block... PASSED
Checking GMimeFilterOpenPGP::encrypted message block... PASSED
Testing GnuPG crypto context: failed (1 errors, 0 warnings)

It probably happened in the first minute of looping the test.

@jstedfast
Copy link
Owner

which test failed?

@Apteryks
Copy link
Author
Apteryks commented Apr 5, 2025

Hm, sorry for the too short output, here's more context including the failure:

Checking GMimeGpgContext::encrypt... PASSED
Checking GMimeGpgContext::decrypt... PASSED
ciphertext:
-----BEGIN PGP MESSAGE-----

hQIOAwTJ0obO8knpEAgAx/bKA4UaimXp1g+4ULMLy9APk3TwFTnZ236TstOyNgx4
G0dpaeEewvUSHugLc0Xrx1+P9kwdPcOr52jVODv1pfa8ifhgcTL1tawSCNLv6Jnp
fjZcUtF3YTDVxEqbYp6Yrckeq3KDxXSvhKZSuNUMU7LX5lpBrHDmE+hk1wgnDfeB
aJrCy6yvUT8Si0pKT4uJgKG+EmTJYjA8sR1SV6seCi3h6UblTf0/SxkmH0UpGafB
VNGJnegcrXbH0KYffXCbix/cX9RczwCoM9saiJNhIQ9x3elNYqPA4sxsE1zJCVY+
vBQviUBSwwo9vUg46nRO8YZ8U2JrdWihhhj5rM3CNggAi8CM/q8Gl0nAaM+brXSZ
JAnGoueJQGtjqnmdiE5tbWzb99F62fR0/aB+rk2IASJz9V8LrR58CrGnB/0W/o2f
dN6DqIB9Du6vww/GNlSJyVRzStsLeOYDOM+0gD1Rg3TDoqv09hgcBo2qziOpPfoT
Yelrh9xP+DUct3t7J00JhJwILWQT8GpVXc/ya0QLWgTQATlyrThy+Wdu8BoA3Lq5
pfuFg1rfaQ8cccpY62Wm6GCn4vBoWMaQJ58zzx1rWE6qhHQH2YSNePcf3eO6Yh6r
Ja/W7ub8Fie/HLhlk9nrnUjdUFuvFNBmD1PuQEowhhTFpfpn4LMnB9OyT6zPn4cY
rdK1AXlu1yUIh+M1d9KoYk5oOeNIorOtaSUEAQ8cqEdH6gJDmm4poN3LctfO8mea
hv6S0cNNis4bEtcaqejiA31BU3YaC17pIPoBhV5MkdVxw1FhmOk0fUhjogI4NBVT
r3vPHKZctO5HyVr4DMDC2f+3WCkgqD8gWgUYyH0sTkj1QVZreHg9JJiG7CP6rCj4
wKR5E2R6iOiLnCZnkZt/7Sebk3aWHEHrtNprIBsugKMIeFcAsiuYFA==
=mXkS
-----END PGP MESSAGE-----

Checking GMimeGpgContext::encrypt+sign... PASSED
Checking GMimeGpgContext::decrypt+verify... PASSED
Checking GMimeFilterOpenPGP::public key block... PASSED
Checking GMimeFilterOpenPGP::private key block... PASSED
Checking GMimeFilterOpenPGP::signed message block... PASSED
Checking GMimeFilterOpenPGP::encrypted message block... PASSED
Testing GnuPG crypto context: passed
Checking GMimeGpgContext::import... PASSED
GMimeGpgContext::export failed: Could not export key data: Input/output error
Checking GMimeGpgContext::export... FAILED
signature (pgp-sha1):
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

this is some cleartext
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQTaJtq3VyocXyqxp0u0d2DpUYLCAAUCZ/ABZQAKCRC0d2DpUYLC
AMoyAKCwhVZcwPseBLQ66v7uRFHtwTsxgwCgk5i8VWFnlfqdRXmLS8Opp52qKd4=
=nbgW
-----END PGP SIGNATURE-----

Checking GMimeGpgContext::sign... PASSED
Checking GMimeGpgContext::verify... PASSED
signature (pgp-sha1):
-----BEGIN PGP SIGNATURE-----

iF0EABECAB0WIQTaJtq3VyocXyqxp0u0d2DpUYLCAAUCZ/ABZgAKCRC0d2DpUYLC
ABhzAJ9/uNUHEmBnkrsROIcJ5WLOFnUZ1ACcD4WcmVygpMHNLtBnb8weSDBCSQk=
=YwIP
-----END PGP SIGNATURE-----

Checking GMimeGpgContext::sign (detached)... PASSED
Checking GMimeGpgContext::verify (detached)... PASSED
ciphertext:
-----BEGIN PGP MESSAGE-----

hQIOAwTJ0obO8knpEAf+P+Skp8aGp8157AOpOLPbVmBYJGsI2J3yBgDoZnASES4v
f3PqLjn9efusR56M3gb/SLAw3OqxMZ7RSXhnNglM0MuoSH5P1xzCBfRuY0gYS2T3
d98ZHKjE+r9CMtGEjuIkGc+1+NbsehpDTXF9y05LWmv/qZ3J1cc3beRuAbjUnUoJ
F0Jla2Pmrbl83mUehelxkTF5viK9GR/Uk5TO+hi4a1ol3IMq1BcB4Z1kOOvoBHUt
D8Sa8SB2C6v6lagpGJc4jf37wLOKa2FBZleuXSEyv0xOZ7SOPMvStKXAm11q20Ej
eoE00CgBerh0H5AMKi9KDefTb0ZCNFb3Km4dMz6zhwgAn7AYHTRPCHXzmlVTimNh
jDJQreshtCN4JOL9fsOYxCEYoWBpt0SOJWYasiaSFbuFw8D341Gndg+7yioTojCs
VeRMB8OCKedeKgD5e2SMYSTYde+/s4GPT498ALg57JeWY+0wKUFJ8gaHmLWuHAfV
aocJVEPaRPn4+DsQ72pNRJt5RUR3YweRWuC1MycG34WG8YF/dv6wWb0n8kkcvFW6
/BnQn5MVBGaBVpbbskGTfI3sLISg/oAVGTSVRLOQMe1+KquZKk8cMKX2IAT0Wpai
g645I17nK+3k1+op6/JmW14qPopZXqvZjkkYvMIEtedgoSn8C7xPKd5Bs/ofymqx
xtJRAcvx77BdqQLurxZnSrB9ngUQXzOI/eTkxEuJlxZtDpLajxwhyShJMrYhYaGo
l59M1XVtZfz2ggopjQC/Y4WU+u6GG1cuY50s2qhTHNUh7LKF
=FBG0
-----END PGP MESSAGE-----

Checking GMimeGpgContext::encrypt... PASSED
Checking GMimeGpgContext::decrypt... PASSED
ciphertext:
-----BEGIN PGP MESSAGE-----

hQIOAwTJ0obO8knpEAf/TK1IKYjU7ab+p+ghgOraYjxGc34j93fYG1AgUtdZKFWq
jnSBKr+e6HVHgWAttfhWWNz36OliGZNgm6ZJJNJti8aca4gqZ/ucK4QZY84kWRr/
ohNBtPEXDARPCNuruhQwWbpyeoynvPEqfhTVTKSlYvGewZsW4SGCDR7Zt7tuHox5
Md92jCFYRhDKfwouct0iq3mNCMBFkgP+y4IcrfRDQo3SMw6xwKzccXJCqFVhkne/
LnWo9GWMgw7vbVeZeKh4qWV/h3th/fgBOwoTumbMWJgw3fsQB2pHss2hl04ADl8d
5dpgzXJW5ZsnfWxAAbC0xo1AHum/gcWvLdM/+DWicgf/VZ6wPdfE9AxYSFl3WG7C
Mu4r1s9amLzJE6/k/1DkHaDxmu+MVc7MmPi+UpUjeE87Ircnicat7iJT60Q6qJcP
Gn5a8UXBNLl8MotPlzlAsobLYWvYi+MntiBIsXShD/L3nl0ielpv5FCJx26/uNpJ
4JQyYziuGOz/DaX3mfFoWIvGOK9fW4Vk2TZrVuF/qrnxPgy0q63qrE8Ese7JtwHI
xVX4/1TtusB2fl68R2Wcz+/60iVxe9M86/dYAzI/2WvGKDWBV3szRXAl5qkpQZeL
W0xq4/tTIOYR003075u56PYtfpBaoYFPZhoz4YQixNs7TaCiflSvendfUXHlWbXr
SdK1AdtI30lCNHyah56ocxRhsIMdeoX1zYBRQCbWhi3FdtRi7+dcQq5j8u+yxwpa
aDWBhxsDVBAecU8Eti4UfzgrsgINX2CFcAsc0w5GndlLMmpr98Bt+LjI2Rb4aCvg
wjz+sI/ladOiI1g+5YTMw1rtWOlrdJLgxjMtmFRKSzAtlhMqzLb/zjEjuvhnCo/1
55+6+JatJ+73PfSixqaahd54Fc5FXTcGHc/hBtKb2LYrej2BhP82Ow==
=65jU
-----END PGP MESSAGE-----

Checking GMimeGpgContext::encrypt+sign... PASSED
Checking GMimeGpgContext::decrypt+verify... PASSED
Checking GMimeFilterOpenPGP::public key block... PASSED
Checking GMimeFilterOpenPGP::private key block... PASSED
Checking GMimeFilterOpenPGP::signed message block... PASSED
Checking GMimeFilterOpenPGP::encrypted message block... PASSED
Testing GnuPG crypto context: failed (1 errors, 0 warnings)

It failed on GMimeGpgContext::export.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jstedfast @Apteryks
0