sparse stats sub-command for fetching cluster/topo statistics
#17
Labels
Milestone
Comments
|
That core.clj file actually defines a restful api (via compojure) that runs on the storm head. The 'topology summary' page is populated by https://path.to.head:8080/api/v1/topology/index-id. We have the path to the head from config.json (assuming nimbus and head are always on the same machine on different ports?). We can get the index-id from https://path.to.head:8080/api/v1/topology/summary. Somehow the local streamparse client will need to authenticate on the storm head to access the api. I'm going to try to sort that out. |
|
Determined I can ssh into Storm head and access the API without authenticating. PR coming. |
|
Someone on our team noted that ssh tunnelling won't always be necessary (which I assumed it is). I'll fix. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This Clojure file implements the whole Storm UI, basically:
https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/ui/core.clj
We could use a similar recipe in our Clojure interop project to fetch cluster/topology statistics from the Nimbus over the Thrift interface. We could then expose this in the command-line tool via
sparse stats. I'm imaging an ideal implementation might be anhtop-like curses interface that live refreshes stats from the cluster, but even some basic pretty-printed tables could be a good start.The text was updated successfully, but these errors were encountered: