8000 fix(test): fix TestBlockPoolMaliciousNode DATA RACE by melekes · Pull Request #4636 · cometbft/cometbft · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix(test): fix TestBlockPoolMaliciousNode DATA RACE #4636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Dec 10, 2024
Merged

fix(test): fix TestBlockPoolMaliciousNode DATA RACE #4636

merged 3 commits into from
Dec 10, 2024

Conversation

melekes
Copy link
Contributor
@melekes melekes commented Dec 9, 2024 •
edited
Loading

follow up to #4633

See https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504

DATA RACE

==================
WARNING: DATA RACE
Write at 0x00c00028f110 by goroutine 507:
runtime.mapassign_faststr()
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/map_faststr.go:223 +0x0
github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).banPeer()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:433 +0x16f
github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).RemovePeerAndRedoAllPeerRequests()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:266 +0x192
github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode.func4()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:353 +0x1ee

Previous read at 0x00c00028f110 by goroutine 501:
runtime.mapaccess1_faststr()
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/map_faststr.go:13 +0x0
github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).isPeerBanned()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:428 +0x128c
github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:381 +0x1205
testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
testing.(*T).Run.gowrap1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x44

Goroutine 507 (running) created at:
github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:341 +0xdc4
testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
testing.(*T).Run.gowrap1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x44

Goroutine 501 (running) created at:
testing.(*T).Run()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x825
testing.runTests.func1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:2168 +0x85
testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
testing.runTests()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:2166 +0x8be
testing.(*M).Run()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:2034 +0xf17
main.main()
_testmain.go:83 +

@melekes melekes added backport-to-v0.38.x Tell Mergify to backport the PR to v0.38.x backport-to-v1.x Tell Mergify to backport the PR to v1.x testing related to unit testing in general labels Dec 9, 2024
@melekes melekes self-assigned this Dec 9, 2024
@melekes melekes requested review from a team as code owners December 9, 2024 11:49
@melekes melekes enabled auto-merge December 10, 2024 12:01
@melekes melekes added this pull request to the merge queue Dec 10, 2024
Merged via the queue into main with commit 2b1db1c Dec 10, 2024
34 of 35 checks passed
@melekes melekes deleted the 4633-follow-up branch December 10, 2024 12:13
mergify bot pushed a commit that referenced this pull request Dec 10, 2024
@melekes @mergify
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (#4636)
c8d27f0 
follow up to #4633

See
https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504

<details>
<summary>DATA RACE</summary>

==================
WARNING: DATA RACE
Write at 0x00c00028f110 by goroutine 507:
  runtime.mapassign_faststr()
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/map_faststr.go:223 +0x0
  github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).banPeer()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:433
+0x16f

github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).RemovePeerAndRedoAllPeerRequests()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:266
+0x192

github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode.func4()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:353
+0x1ee

Previous read at 0x00c00028f110 by goroutine 501:
  runtime.mapaccess1_faststr()
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/map_faststr.go:13 +0x0

github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).isPeerBanned()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:428
+0x128c

github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:381
+0x1[20](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:21)5
  testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
  testing.(*T).Run.gowrap1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x44

Goroutine 507 (running) created at:

github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:341
+0xdc4
  testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
  testing.(*T).Run.gowrap1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x44

Goroutine 501 (running) created at:
  testing.(*T).Run()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x825
  testing.runTests.func1()

/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:[21](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:22)68
+0x85
  testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690
+0x[22](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:23)6
  testing.runTests()

/opt/hostedtoolcache/go/1.[23](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:24).1/x64/src/testing/testing.go:2166
+0x8be
  testing.(*M).Run()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:2034 +0xf17
  main.main()
      _testmain.go:83 +

</details>

(cherry picked from commit 2b1db1c)
@mergify mergify bot mentioned this pull request Dec 10, 2024
Merged
mergify bot pushed a commit that referenced this pull request Dec 10, 2024
@melekes @mergify
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (#4636)
8f739d4 
follow up to #4633

See
https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504

<details>
<summary>DATA RACE</summary>

==================
WARNING: DATA RACE
Write at 0x00c00028f110 by goroutine 507:
  runtime.mapassign_faststr()
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/map_faststr.go:223 +0x0
  github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).banPeer()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:433
+0x16f

github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).RemovePeerAndRedoAllPeerRequests()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:266
+0x192

github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode.func4()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:353
+0x1ee

Previous read at 0x00c00028f110 by goroutine 501:
  runtime.mapaccess1_faststr()
/opt/hostedtoolcache/go/1.23.1/x64/src/runtime/map_faststr.go:13 +0x0

github.com/cometbft/cometbft/internal/blocksync.(*BlockPool).isPeerBanned()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool.go:428
+0x128c

github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:381
+0x1[20](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:21)5
  testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
  testing.(*T).Run.gowrap1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x44

Goroutine 507 (running) created at:

github.com/cometbft/cometbft/internal/blocksync.TestBlockPoolMaliciousNode()
/home/runner/work/cometbft/cometbft/internal/blocksync/pool_test.go:341
+0xdc4
  testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690 +0x226
  testing.(*T).Run.gowrap1()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x44

Goroutine 501 (running) created at:
  testing.(*T).Run()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1743 +0x825
  testing.runTests.func1()

/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:[21](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:22)68
+0x85
  testing.tRunner()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:1690
+0x[22](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:23)6
  testing.runTests()

/opt/hostedtoolcache/go/1.[23](https://github.com/cometbft/cometbft/actions/runs/12247740137/job/34188217504#step:6:24).1/x64/src/testing/testing.go:2166
+0x8be
  testing.(*M).Run()
/opt/hostedtoolcache/go/1.23.1/x64/src/testing/testing.go:2034 +0xf17
  main.main()
      _testmain.go:83 +

</details>

(cherry picked from commit 2b1db1c)
@mergify mergify bot mentioned this pull request Dec 10, 2024
Merged
melekes added a commit that referenced this pull request Dec 11, 2024
@mergify @melekes
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport #4636) (#…
9ae3b9c 
…4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
melekes added a commit that referenced this pull request Dec 11, 2024
@mergify @melekes
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport #4636) (#…
25128b2 
…4640)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
yihuang pushed a commit to yihuang/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @yihuang
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
05f50b3 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
yihuang added a commit to crypto-org-chain/cometbft that referenced this pull request Feb 3, 2025
@yihuang @melekes @mergify
Merge commit from fork (#9)
1964da4 
* Merge commit from fork

lower than what was previously reported
GHSA-22qq-3xwm-r5x4

* Merge commit from fork

* fix test

* fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbft#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

---------

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
roy-dydx pushed a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @roy-dydx
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
0001f4c 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
roy-dydx pushed a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @roy-dydx
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
6999afb 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
roy-dydx pushed a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @roy-dydx
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
f99aa6f 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
roy-dydx pushed a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @roy-dydx
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
0d25aba 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
roy-dydx pushed a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @roy-dydx
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
8000 6d9c50f 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
roy-dydx pushed a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@mergify @melekes @roy-dydx
fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbf…
ca793ba 
…t#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
roy-dydx added a commit to dydxprotocol/cometbft that referenced this pull request Feb 3, 2025
@roy-dydx @melekes
@mergify @greg-szabo @sergio-mena
Cherry-pick fix for ASA-2025-001 (#41)
2067b31 
* feat(blocksync): set the max number of (concurrently) downloaded blocks (backport cometbft#2467) (cometbft#2515)

Manual backport of cometbft#2467

* feat(blocksync): sort peers by download rate & multiple requests for closer blocks (backport cometbft#2475) (cometbft#2576)

This is an automatic backport of pull request cometbft#2475 done by
[Mergify](https://mergify.com).
Cherry-pick of f8366fc has failed:
```
On branch mergify/bp/v0.38.x/pr-2475
Your branch is up to date with 'origin/v0.38.x'.

You are currently cherry-picking commit f8366fc.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   .changelog/unreleased/improvements/2475-blocksync-2nd-request.md
	new file:   .changelog/unreleased/improvements/2475-blocksync-no-block-response.md
	new file:   .changelog/unreleased/improvements/2475-blocksync-sort-peers.md
	modified:   blocksync/reactor.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   blocksync/pool.go

```


To fix up this pull request, you can check it out locally. See
documentation:
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

---


<details>
<summary>Mergify commands and options</summary>

<br />

More conditions and actions can be found in the
[documentation](https://docs.mergify.com/).

You can also trigger Mergify actions by commenting on this pull request:

- `@Mergifyio refresh` will re-evaluate the rules
- `@Mergifyio rebase` will rebase this PR on its base branch
- `@Mergifyio update` will merge the base branch into this PR
- `@Mergifyio backport <destination>` will backport this PR on
`<destination>` branch

Additionally, on Mergify [dashboard](https://dashboard.mergify.com) you
can:

- look at your merge queues
- generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com
</details>

---------

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix(blocksync): use timer instead of time.After (backport cometbft#2584) (cometbft#2587)

This is an automatic backport of pull request cometbft#2584 done by
[Mergify](https://mergify.com).


---


<details>
<summary>Mergify commands and options</summary>

<br />

More conditions and actions can be found in the
[documentation](https://docs.mergify.com/).

You can also trigger Mergify actions by commenting on this pull request:

- `@Mergifyio refresh` will re-evaluate the rules
- `@Mergifyio rebase` will rebase this PR on its base branch
- `@Mergifyio update` will merge the base branch into this PR
- `@Mergifyio backport <destination>` will backport this PR on
`<destination>` branch

Additionally, on Mergify [dashboard](https://dashboard.mergify.com) you
can:

- look at your merge queues
- generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com
</details>

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* ABC-0013 fix and broken test

* blocksync pool ban test

* Simplified blocksync malicious node test, fix nil pointer error

* Test assertions have more detail

* Update blocksync/pool_test.go

Co-authored-by: Sergio Mena <sergio@informal.systems>

* Update blocksync/pool_test.go

Co-authored-by: Sergio Mena <sergio@informal.systems>

* Update blocksync/pool_test.go

* Remove one thread to make test more compact

* Removed defers from test

* Readded good peer to test

* Update blocksync/pool_test.go

* Release/v0.38.8 (cometbft#3350)

<!--

Please add a reference to the issue that this PR addresses and indicate
which
files are most critical to review. If it fully addresses a particular
issue,
please include "Closes #XXX" (where "XXX" is the issue number).

If this PR is non-trivial/large/complex, please ensure that you have
either
created an issue that the team's had a chance to respond to, or had some
discussion with the team prior to submitting substantial pull requests.
The team
can be reached via GitHub Discussions or the Cosmos Network Discord
server in
the #cometbft channel. GitHub Discussions is preferred over Discord as
it
allows us to keep track of conversations topically.
https://github.com/cometbft/cometbft/discussions

If the work in this PR is not aligned with the team's current
priorities, please
be advised that it may take some time before it is merged - especially
if it has
not yet been discussed with the team.

See the project board for the team's current priorities:
https://github.com/orgs/cometbft/projects/1

-->
Release v0.38.8

[CHANGELOG](https://github.com/cometbft/cometbft/blob/6814b5fff269f5ec6988a4832f24f4804d705ca9/CHANGELOG.md)
---

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [ ] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

* fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbft#4636) (cometbft#4641)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix(test): `TestBlockPoolMaliciousNode` shutdown threads at exit (backport cometbft#4633) (cometbft#4635)

This is a drive-by fix of a test that doesn't shut its threads down
until the whole `go test` execution finishes. I think we have a bunch of
these, but I came across this one during an unrelated troubleshooting.

Is it worth fixing this? It's not really causing any issues, it's just
sloppy coding.

The only way to see any difference is to run the `go test` until it
reaches its time limit and panics. In that case, the trace will contain
references to the threads.

For example:
```
go test github.com/cometbft/cometbft/blocksync -v -run TestBlockPoolMaliciousNode -count 100 -failfast -race -timeout 30s
```

After 30 seconds the test didn't run 100 times yet, hence `go test`
panics. Because the test has been run multiple times already, multiple
sets of threads will be reported in the panic. With the fix, only one
set is reported.

Author: @greg-szabo 
<hr>This is an automatic backport of pull request cometbft#4633 done by
[Mergify](https://mergify.com).

---------

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Merge commit from fork

lower than what was previously reported
GHSA-22qq-3xwm-r5x4

---------

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Greg Szabo <greg@philosobear.com>
Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com>
Co-authored-by: Sergio Mena <sergio@informal.systems>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alesforz alesforz alesforz approved these changes

Assignees

@melekes melekes

Labels
backport-to-v0.38.x Tell Mergify to backport the PR to v0.38.x backport-to-v1.x Tell Mergify to backport the PR to v1.x testing related to unit testing in general
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@melekes @alesforz
0