8000 feat: Add ToUnstructured method to Event by danielpacak · Pull Request #830 · aquasecurity/tracee · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: Add ToUnstructured method to Event #830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 26, 2021
Merged

feat: Add ToUnstructured method to Event #830

merged 1 commit into from
Jul 26, 2021
+202 −3

Conversation

danielpacak
Copy link
Contributor
@danielpacak danielpacak commented Jul 20, 2021 •
edited
Loading

ToUnstructured returns a JSON compatible map which can
be used as a parsed input with OPA Go SDK to avoid
relatively expensive JSON encoding round trip.

Looking at benchmarks it's not a game changer, but we can observer a slight improvement:

➜  benchmark git:(main) go test -bench=EngineWithNSignatures/rego -benchtime=100x -benchmem
goos: darwin
goarch: amd64
pkg: github.com/aquasecurity/tracee/tracee-rules/benchmark
cpu: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
BenchmarkEngineWithNSignatures/rego/2Signatures-16         	     100	  14135044 ns/op	11575729 B/op	  245983 allocs/op
BenchmarkEngineWithNSignatures/rego/4Signatures-16         	     100	  22670464 ns/op	23365947 B/op	  494621 allocs/op
BenchmarkEngineWithNSignatures/rego/8Signatures-16         	     100	  44750550 ns/op	46439232 B/op	  983656 allocs/op
BenchmarkEngineWithNSignatures/rego/16Signatures-16        	     100	  59082641 ns/op	92543772 B/op	 1971349 allocs/op
BenchmarkEngineWithNSignatures/rego/32Signatures-16        	     100	  87702240 ns/op	185807271 B/op	 3960792 allocs/op
BenchmarkEngineWithNSignatures/rego/64Signatures-16        	     100	 147384928 ns/op	365204354 B/op	 7788096 allocs/op
BenchmarkEngineWithNSignatures/rego/128Signatures-16       	     100	 280088130 ns/op	737913634 B/op	15737458 allocs/op
PASS
ok  	github.com/aquasecurity/tracee/tracee-rules/benchmark	68.119s

➜  benchmark git:(event_to_unstructured) ✗ go test -bench=EngineWithNSignatures/rego -benchtime=100x -benchmem
goos: darwin
goarch: amd64
pkg: github.com/aquasecurity/tracee/tracee-rules/benchmark
cpu: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
BenchmarkEngineWithNSignatures/rego/2Signatures-16         	     100	  12211402 ns/op	11796871 B/op	  222644 allocs/op
BenchmarkEngineWithNSignatures/rego/4Signatures-16         	     100	  20457959 ns/op	23717600 B/op	  447456 allocs/op
BenchmarkEngineWithNSignatures/rego/8Signatures-16         	     100	  35967881 ns/op	47136604 B/op	  889711 allocs/op
BenchmarkEngineWithNSignatures/rego/16Signatures-16        	     100	  53071508 ns/op	94459594 B/op	 1784532 allocs/op
BenchmarkEngineWithNSignatures/rego/32Signatures-16        	     100	  76496260 ns/op	189707373 B/op	 3585234 allocs/op
BenchmarkEngineWithNSignatures/rego/64Signatures-16        	     100	 131655093 ns/op	372987305 B/op	 7050512 allocs/op
BenchmarkEngineWithNSignatures/rego/128Signatures-16       	     100	 246052599 ns/op	753751712 B/op	14247815 allocs/op
PASS

NB To benchmark I had to modify Rego signature implementation, which is not part of this PR:

diff --git a/tracee-rules/signatures/rego/regosig/traceerego.go b/tracee-rules/signatures/rego/regosig/traceerego.go
index 9793a9d..ab979cf 100644
--- a/tracee-rules/signatures/rego/regosig/traceerego.go
+++ b/tracee-rules/signatures/rego/regosig/traceerego.go
@@ -144,7 +144,16 @@ func (sig *RegoSignature) OnEvent(e types.Event) error {
                return fmt.Errorf("invalid event")
        }

-       results, err := sig.matchPQ.Eval(context.TODO(), rego.EvalInput(ee))
+       u ,err := ee.ToUnstructured()
+       if err != nil {
+               return err
+       }
+       value ,err:= ast.InterfaceToValue(u)
+       if err != nil {
+               return err
+       }
+
+       results, err := sig.matchPQ.Eval(context.TODO(), rego.EvalParsedInput(value))
        if err != nil {
                return err
        }

Signed-off-by: Daniel Pacak pacak.daniel@gmail.com

@danielpacak
Copy link
Contributor Author
danielpacak commented Jul 22, 2021 •
edited
Loading

I run the benchmark with ast.InterfaceToValue(u) inside the OnEvent callback. Probably we'll benefit more from ToUnstructure if this is done before event is dispatched to signatures. This way we'll do it once instead of N times, where N is the number of signatures.

@danielpacak danielpacak marked this pull request as draft July 22, 2021 11:38
@simar7 simar7 self-requested a review July 23, 2021 20:51
@simar7
Copy link
Member
simar7 commented Jul 23, 2021 •
edited
Loading

I played around with this quite a bit today. It looks like we're on to something.

Running all open source rules (1 Golang + 6 Rego) and kernel compile
image

Although with running a lot more Rego rules (~60) the numbers look rather similar across the two. Left current main, right this branch.

image

@itaysk itaysk mentioned this pull request Jul 25, 2021
Merged
@danielpacak danielpacak marked this pull request as ready for review July 25, 2021 06:46
@danielpacak
feat: Add ToUnstructured method to Event
ed05395 
ToUnstructured returns a JSON compatible map which can
be used as a parsed input with OPA Go SDK to avoid
relatively expensive JSON encoding round trip.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
@danielpacak danielpacak requested a review from simar7 July 26, 2021 13:10
@simar7 simar7 merged commit 443955e into aquasecurity:main Jul 26, 2021
yanivagman pushed a commit to yanivagman/tracee that referenced this pull request Jul 26, 2021
@danielpacak @yanivagman
feat: Add ToUnstructured method to Event (aquasecurity#830)
3b73f35 
ToUnstructured returns a JSON compatible map which can
be used as a parsed input with OPA Go SDK to avoid
relatively expensive JSON encoding round trip.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
simar7 added a commit to simar7/tracee that referenced this pull request Aug 27, 2021
@simar7
Revert "feat: Add ToUnstructured method to Event (aquasecurity#830)"
35f9cfb 
This reverts commit 443955e

Signed-off-by: Simar <simar@linux.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simar7 simar7 simar7 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@danielpacak @simar7
0