8000 Timeout due to excessive shrinking in STM Dynarray test parallel · Issue #554 · ocaml-multicore/multicoretests · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Timeout due to excessive shrinking in STM Dynarray test parallel #554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jmid opened this issue May 5, 2025 · 1 comment
Open

Timeout due to excessive shrinking in STM Dynarray test parallel #554

jmid opened this issue May 5, 2025 · 1 comment
Labels
test suite reliability Issue concerns tests that should behave more predictably

Comments

@jmid
Copy link
Collaborator
jmid commented May 5, 2025

On the merge to main of #553 we just saw a timeout after excessive time spent shrinking in STM Dynarray test parallel
on Cygwin 5.3:
https://github.com/ocaml-multicore/multicoretests/actions/runs/14813376084/job/41590879310

random seed: 523011271
generated error fail pass / total     time test name

[ ]    0    0    0    0 / 1000     0.0s STM Dynarray test sequential agreement (int)
[ ]    0    0    0    0 / 1000     0.0s STM Dynarray test sequential agreement (int) (generating)
[✓] 1000    0    0 1000 / 1000     0.0s STM Dynarray test sequential agreement (int)

[ ]    0    0    0    0 / 1000     0.0s STM Dynarray test parallel (int)
[ ]    0    0    0    0 / 1000    60.1s STM Dynarray test parallel (int) (shrinking:   31.0010)
[ ]    0    0    0    0 / 1000   123.7s STM Dynarray test parallel (int) (shrinking:   31.0045)
[ ]    0    0    0    0 / 1000   200.4s STM Dynarray test parallel (int) (shrinking:   31.0051)
[ ]    0    0    0    0 / 1000   269.9s STM Dynarray test parallel (int) (shrinking:   31.0059)
[ ]    0    0    0    0 / 1000   337.0s STM Dynarray test parallel (int) (shrinking:   31.0068)

[...]

[ ]    0    0    0    0 / 1000  5053.1s STM Dynarray test parallel (int) (shrinking:   44.0021)
[ ]    0    0    0    0 / 1000  5141.3s STM Dynarray test parallel (int) (shrinking:   44.0024)
[ ]    0    0    0    0 / 1000  5207.5s STM Dynarray test parallel (int) (shrinking:   44.0027)
[ ]    0    0    0    0 / 1000  5274.2s STM Dynarray test parallel (int) (shrinking:   44.0030)
[ ]    0    0    0    0 / 1000  5341.3s STM Dynarray test parallel (int) (shrinking:   44.0035)
[✓]    1    0    1    0 / 1000  5375.8s STM Dynarray test parallel (int)

[ ]    0    0    0    0 / 1000     0.0s STM Dynarray stress test (int)
Error: The operation was canceled.

This issue is analogous to #514 for the corresponding Lin test fixed in #517.
@jmid jmid added the test suite reliability Issue concerns tests that should behave more predictably label May 5, 2025
@jmid jmid mentioned this issue May 5, 2025
Merged
@jmid jmid mentioned this issue May 21, 2025
Merged
@jmid
Copy link
Collaborator Author
jmid commented May 21, 2025

This happened - again on Cygwin 5.3:
https://github.com/ocaml-multicore/multicoretests/actions/runs/15154617306/job/42606915788

random seed: 499915151
generated error fail pass / total
65B2
     time test name

[ ]    0    0    0    0 / 1000     0.0s STM Dynarray test sequential agreement (int)
[ ]    0    0    0    0 / 1000     0.0s STM Dynarray test sequential agreement (int) (generating)
[✓] 1000    0    0 1000 / 1000     0.0s STM Dynarray test sequential agreement (int)

[ ]    0    0    0    0 / 1000     0.0s STM Dynarray test parallel (int)
[ ]    2    0    0    2 / 1000    60.2s STM Dynarray test parallel (int) (shrinking:   18.0088)
[ ]    2    0    0    2 / 1000   139.2s STM Dynarray test parallel (int) (shrinking:   19.0033)
[ ]    2    0    0    2 / 1000   217.2s STM Dynarray test parallel (int) (shrinking:   19.0070)
[ ]    2    0    0    2 / 1000   287.1s STM Dynarray test parallel (int) (shrinking:   19.0072)
[ ]    2    0    0    2 / 1000   383.6s STM Dynarray test parallel (int) (shrinking:   19.0073)
[ ]    2    0    0    2 / 1000   448.4s STM Dynarray test parallel (int) (shrinking:   19.0074)

[...]

[ ]    2    0    0    2 / 1000  9384.7s STM Dynarray test parallel (int) (shrinking:   19.0222)
[ ]    2    0    0    2 / 1000  9526.1s STM Dynarray test parallel (int) (shrinking:   19.0225)
[ ]    2    0    0    2 / 1000  9604.9s STM Dynarray test parallel (int) (shrinking:   19.0227)
[ ]    2    0    0    2 / 1000  9737.4s STM Dynarray test parallel (int) (shrinking:   19.0229)
[ ]    2    0    0    2 / 1000  9799.3s STM Dynarray test parallel (int) (shrinking:   19.0230)
Error: The operation was canceled.

It seems crazy that ~200 unsuccessful shrink attempts (from 19.0033 to 19.0230) take ~ 2h 41m ~ 9660s
(9799.3s - 139.2s)! 😮

I initially suspected that this could be caused by integer tearing #528, causing the shrinker to go around in circles, but I'm starting to doubt it... 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
test suite reliability Issue concerns tests that should behave more predictably
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jmid
0