10000 Exception when using ArrayBoundsChecker · Issue #1354 · soot-oss/soot · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Exception when using ArrayBoundsChecker #1354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
tiziw opened this issue May 13, 2020 · 2 comments
Open

Exception when using ArrayBoundsChecker #1354

tiziw opened this issue May 13, 2020 · 2 comments

Comments

@tiziw
Copy link
tiziw commented May 13, 2020 •
edited
Loading

Hi,

When enabling the ArrayBounds checker optimization it sometimes throws a NullPointerException. It happens randomly, sometimes the program runs fine.

[Thread-6] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: null java.lang.NullPointerException at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.doAnalysis(ArrayBoundsCheckerAnalysis.java:372) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.<init>(ArrayBoundsCheckerAnalysis.java:170) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsChecker.internalTransform(ArrayBoundsChecker.java:104) at soot.BodyTransformer.transform(BodyTransformer.java:51) at soot.Transform.apply(Transform.java:120) at soot.BodyPack.internalApply(BodyPack.java:50) at soot.Pack.apply(Pack.java:125) at soot.PackManager.runBodyPacks(PackManager.java:1010) at soot.PackManager.lambda$runBodyPacks$0(PackManager.java:660) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Exception in thread "main" java.lang.NullPointerException at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.doAnalysis(ArrayBoundsCheckerAnalysis.java:372) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.<init>(ArrayBoundsCheckerAnalysis.java:170) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsChecker.internalTransform(ArrayBoundsChecker.java:104) at soot.BodyTransformer.transform(BodyTransformer.java:51) at soot.Transform.apply(Transform.java:120) at soot.BodyPack.internalApply(BodyPack.java:50) at soot.Pack.apply(Pack.java:125) at soot.PackManager.runBodyPacks(PackManager.java:1010) at soot.PackManager.lambda$runBodyPacks$0(PackManager.java:660) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Exception in thread "Thread-6" java.lang.NullPointerException at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.doAnalysis(ArrayBoundsCheckerAnalysis.java:372) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.<init>(ArrayBoundsCheckerAnalysis.java:170) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsChecker.internalTransform(ArrayBoundsChecker.java:104) at soot.BodyTransformer.transform(BodyTransformer.java:51) at soot.Transform.apply(Transform.java:120) at soot.BodyPack.internalApply(BodyPack.java:50) at soot.Pack.apply(Pack.java:125) at soot.PackManager.runBodyPacks(PackManager.java:1010) at soot.PackManager.lambda$runBodyPacks$0(PackManager.java:660) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Flags used:
PhaseOptions.v().setPhaseOption("jap.npc", "enabled:true") PhaseOptions.v().setPhaseOption("jap.abc", "enabled:true") PhaseOptions.v().setPhaseOption("jap.abc", "with-cse:true") PhaseOptions.v().setPhaseOption("jap.abc", "with-classfield:true")
When disabling with-classfield and with-cse it still happens.

The bug started to occur after I added this custom transformation:

        PackManager.v().getPack("wjtp").add(Transform("wjtp.setup", object: SceneTransformer() {
            override fun internalTransform(p0: String, p1: MutableMap<String, String>) {
                Scene.v().reachableMethods.listener().asSequence().map {it.method().declaringClass}.distinct().forEach {
                    it.setApplicationClass()
                }
            }

        }))

The reason I added that transformation is, I'm using Soot with a custom (java) runtime to generate C code. Without setting all classes (which I generate into source code, including the runtime) to Application Classes, optimizations (BodyTransformer's) only get applied to the Main class and not the runtime classes, which I also need to be optimized. I can only know using the callgraph which classes end up being part of the application ((in)directly used by the Main class). I added this transformation to do that in the phase when the callgraph is available. Did I do something wrong? Is there a better way to do it? Before I would just my (custom) optimizations after all the packs were finished and modify the bodies, but I suspect the jop/wjop packs don't optimize the runtime classes since they're not application classes (by default).

Thanks!

I'm also using Soot version 4.1.0.
@tiziw
Copy link
Author
tiziw commented May 16, 2020 •
edited
Loading

Just had the same error, but with some more stack traces:

First:
Exception in thread "Thread-6" java.util.ConcurrentModificationException at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966) at java.util.LinkedList$ListItr.next(LinkedList.java:888) at java.util.AbstractCollection.addAll(AbstractCollection.java:343) at java.util.HashSet.<init>(HashSet.java:120) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.doAnalysis(ArrayBoundsCheckerAnalysis.java:289) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.<init>(ArrayBoundsCheckerAnalysis.java:170) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsChecker.internalTransform(ArrayBoundsChecker.java:104) at soot.BodyTransformer.transform(BodyTransformer.java:51) at soot.Transform.apply(Transform.java:120) at soot.BodyPack.internalApply(BodyPack.java:50) at soot.Pack.apply(Pack.java:125) at soot.PackManager.runBodyPacks(PackManager.java:1010) at soot.PackManager.lambda$runBodyPacks$0(PackManager.java:660) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Second:
heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: null java.lang.NullPointerException at soot.toolkits.graph.SlowPseudoTopologicalOrderer.visitNode(SlowPseudoTopologicalOrderer.java:186) at soot.toolkits.graph.SlowPseudoTopologicalOrderer.computeOrder(SlowPseudoTopologicalOrderer.java:112) at soot.toolkits.graph.SlowPseudoTopologicalOrderer.newList(SlowPseudoTopologicalOrderer.java:76) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.doAnalysis(ArrayBoundsCheckerAnalysis.java:273) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsCheckerAnalysis.<init>(ArrayBoundsCheckerAnalysis.java:170) at soot.jimple.toolkits.annotation.arraycheck.ArrayBoundsChecker.internalTransform(ArrayBoundsChecker.java:104) at soot.BodyTransformer.transform(BodyTransformer.java:51) at soot.Transform.apply(Transform.java:120) at soot.BodyPack.internalApply(BodyPack.java:50) at soot.Pack.apply(Pack.java:125) at soot.PackManager.runBodyPacks(PackManager.java:1010) at soot.PackManager.lambda$runBodyPacks$0(PackManager.java:660) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

If there's a workaround I can use, I'm also fine with that. I can imagine that the ArrayBounds analysis is rarely used and thus not actively maintained.

@tiziw
Copy link
Author
tiziw commented May 29, 2020 •
edited
Loading

EDIT: I've found the real issue after some painful debugging. I suspect the cause is the fact that this optimization is from before soot parallelized the transformation of classes. The problem is that it's a BodyTransformer, while one of the analysis that it uses (depending on the arraybounds config), uses other classes.

It was a while ago that I figured it out, I don't remember what analysis/class that caused the problem, but I think it was SimpleLocalDefs. The thing that went wrong was another class (running parallel) was also having the ArrayBoundsAnalysis executed and it also ran SimpleLocalDefs on the same method, then SimpleLocalDefs renumbers locals (local.setNumber) and the number is then used to access an array. Sometimes two SimpleLocalDefs would be ran on the same metodh and the number would get overwritten resulting into an ArrayOutOfBounds exception. The fix is easy, namely run the optimization in whole-program mode as a SceneTransformer. I might submit a PR in the future

@tiziw tiziw mentioned this issue Sep 2, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@tiziw
0