Use require package instead of assert while performing fs operations in unit tests #90
Conversation
# Conflicts: # chain/test_chain.go # chain/test_chain_test.go # fuzzing/fuzzer.go
… tx gas limit * Change contract deployments to not need a deployment order if only one compiled contract is found
…orpus entries can be deep cloned * Added tx-level corpus mutations powered by the ValueGenerator
… to CallMessageDataAbiValues.Resolve(), renamed a variable in abi_values.go
…-assert # Conflicts: # fuzzing/calls/call_sequence.go # fuzzing/fuzzer.go # fuzzing/fuzzer_worker_sequence_generator.go # fuzzing/test_case_assertion.go # fuzzing/valuegeneration/abi_values.go # fuzzing/valuegeneration/abi_values_test.go
|
I updated the branch here to target Upon testing, we can't use // Start the fuzzer
err := f.fuzzer.Start()
assert.NoError(t, err)
// Check for any failed tests and verify coverage was captured
assertFailedTestsExpected(f, true)
assertCorpusCallSequencesCollected(f, true)The first I'll circle back to solving that, and then we can merge it. If you see an obvious solution (that doesn't require an |
When the Rough, but let me know what you think. func Test3(t *testing.T) {
runFuzzerTest(t, &fuzzerSolcFileTest{
filePath: "testdata/contracts/deployments/inner_inner_deployment.sol",
configUpdates: func(config *config.ProjectConfig) {
config.Fuzzing.DeploymentOrder = []string{"InnerDeploymentFactory"}
config.Fuzzing.Workers = 50
config.Fuzzing.TestLimit = 10000
},
method: func(f *fuzzerTestContext) {
// Start the fuzzer
err := f.fuzzer.Start()
err = fmt.Errorf("not very important bug happened")
expect := expectEventEmittedReturn(f, &f.fuzzer.Events.FuzzerStopping)
if !expect {
// Fuzzer did not stop properly, something bug happened
assert.NoError(t, err)
}
// Check for any failed tests and verify coverage was captured
if expect {
// Fuzzer stopped properly, and only these assertions matter
assertFailedTestsExpected(f, true)
assertCorpusCallSequencesCollected(f, true)
}
},
})
}
func expectEventEmittedReturn[T any](f *fuzzerTestContext, eventEmitter *events.EventEmitter[T]) bool {
eventType := eventEmitter.EventType().String()
eventEmitter.Subscribe(func(event T) error {
f.eventCounter[eventType] += 1
return nil
})
if f.eventCounter[eventType] == 0 {
return false
}
return true
} |
|
I think this is a weird model for unit testing, to have to switch on each failure and not execute further code, so I'd prefer to find another solution here if we can. It was kind of the scenario I was trying to avoid with my comment regarding |
Instead of using the
assertpackage, use therequirepackage to stop testing if a test fails while performing filesystem operations (the assert package will only log the failure and continue execution).Closes #86
I created a semgrep rule for this if we want to incorporate this tool into our CI.