sparse visualize command
#22
Comments
|
Actually already implemented this using digraphs. I just didn't merge into streamparse and add a CLI for it yet, but the viz code already works. |
|
Merge that! |
|
See above -- I have it now merged into the JVM layer. Example usage: Resulting in this image: Now just needs to be added to |
|
Problems. Turns out storm-spirit depends on dorothy, which in turn depends on Clojure 1.5. Since Storm still isn't upgraded to Clojure 1.5 until Storm 0.9.2 is released, we can't really use this yet. I'm going to drop it out of our project.clj to make sure it doesn't pull in Clojure 1.5 accidentally. Pity. |
|
Storm JIRA related to 1.5 upgrade, STORM-265. |
|
STORM FAIL :( |
|
For now, I kept the visualize command in, but removed the dependency to storm-spirit, left a note for ourselves, and issue a warning when you try to use the subcommand. Once Storm 0.9.2 we can probably enable the command safely, although we'll have to use some lein magic to make the command unavailable when using 0.8 or 0.9.1 streamparse projects. See commit a7d1c2d. |
sparse visualize command
|
Now that Storm 0.9.2 is out, are you going to re-enable this? It seems pretty handy. |
|
@dan-blanchard thanks for reminding me of this. The above commit brings the lein command back, which can now be issued from inside And it will open a visualization on your operating system. I also updated the bootstrap template for project.clj in new streamparse projects to include the 0.9.2 dependency by default. The next step that needs to happen on this issue is to write the |
- storm-spirit depends on old version of storm - dorothy depends on old version of clojure This causes classpath woes. Though I added the stub for `sparse visualize` here, we'll need more work to get it fully working. Related to issue #22.
|
@dan-blanchard Gah, classpath woes. I tried to get this working today but hit a roadblock which you can read about in commit 52efb03; I'll need to vendorize storm-spirit and dorothy to get this to work. |
|
@amontalenti Have you made any progress on vendorizing storm-spirit and dorothy? I was just asked for a graph of our rather complex Storm topology, and I was disappointed to find that there was no simple way to generate one. I'd offer to help out on this, but Clojure packaging is not really my forte. |
|
Didn't get to it yet. Thanks for the reminder though. Working my way through the queue. :) |
|
I'm actually thinking I may be able to implement |
|
Hmm, I think I like the idea of being able to visualize your topology before submitting it to Storm. I don't think it'd be too bad given that we already have a simple DAG model and plan to render to Clojure, so adding a renderer for Graphviz's digraph format would probably a piece of cake. |
That's true; that would be nice.
👍 I like that idea. |
|
The current |
|
@rduplain I wanted to mention that now that Topology DSL is coming, we can write this in pure Python. And further, that we have some prior art for this that may be easy to adapt, which is the dask visualizer. It's described briefly here: |
|
+1 |
Use an existing topology visualization tool like:
or some mix of these, add a visualization command for topologies:
The text was updated successfully, but these errors were encountered: