Dockerized Zeek and Threatbus #195

Th3H0ff · 2023-02-09T11:20:40Z

Th3H0ff
Feb 9, 2023

Hi all
I'm playing around with Threatbus as a Docker instance and I cannot get the integration working with my Zeek also running on Docker. Both Threatbus and Zeek are initialized from the same docker-compose.yaml and Zeek is running fine without the "threatbus" flag calling threatbus.zeek. Both Zeek and Threatbus are running in docker network host mode.
The logs from Threatbus look ok to me and it connects to my MISP instance without error on MISP.

2023-02-09 11:08:41,606 INFO     [threatbus] Ignoring installed, but unconfigured app 'zmq'
2023-02-09 11:08:41,606 INFO     [threatbus] Ignoring installed, but unconfigured app 'cif3'
2023-02-09 11:08:41,606 INFO     [threatbus] Ignoring installed, but unconfigured backbones 'rabbitmq'
2023-02-09 11:08:41,638 INFO     [threatbus] Starting plugins...
2023-02-09 11:08:41,639 INFO     [threatbus_inmem.plugin] In-memory backbone started.
2023-02-09 11:08:41,639 INFO     [threatbus_inmem.plugin] Adding subscription to: threatbus/snapshotrequest
2023-02-09 11:08:41,639 INFO     [threatbus_inmem.plugin] Adding subscription to: threatbus/snapshotenvelope
2023-02-09 11:08:41,850 INFO     [threatbus_inmem.plugin] Adding subscription to: stix2/sighting
2023-02-09 11:08:41,853 INFO     [threatbus_misp.plugin] MISP plugin started
2023-02-09 11:08:41,859 INFO     [threatbus_zeek.plugin] Zeek plugin started

If can do a curl from inside the Zeek docker and get an active response from the Threatbus instance. 172.16.80.2 is the host IP of the server running the Docker instances.

bash-5.0# curl 172.16.80.2:47761
curl: (56) Recv failure: Connection reset by peer
bash-5.0#

If I run Zeek from inside the Zeek Docker container I get:

bash-5.0# zeek -i enx58ef68143cb4 -C threatbus.zeek
listening on enx58ef68143cb4

/usr/local/zeek/share/zeek/site/threatbus.zeek, lines 154-155: subscribing to management topic threatbus/manage with snapshot request for 0 secs
/usr/local/zeek/share/zeek/site/threatbus.zeek, lines 156-157: reporting noisy intel at 100 matches/sec
/usr/local/zeek/share/zeek/site/threatbus.zeek, lines 170-171: peering with Threat Bus at 172.16.80.2:47761/tcp
1675941657.550430 error in /usr/local/zeek/share/zeek/base/frameworks/broker/./log.zeek, line 83: Broker error (Broker::PEER_UNAVAILABLE): (invalid-node, *172.16.80.2:47761, "unable to connect to remote peer")
1675941662.573141 error in /usr/local/zeek/share/zeek/base/frameworks/broker/./log.zeek, line 83: Broker error (Broker::PEER_UNAVAILABLE): (invalid-node, *172.16.80.2:47761, "unable to connect to remote peer")

I've tried a bunch of different settings for the host setting in the Threatbus config. No improvements.

logging:
  console: true
  file: true
  console_verbosity: DEBUG
  filename: /opt/tenzir/threatbus/threatbus.log
  file_verbosity: DEBUG

plugins:
  backbones:
    inmem:
  apps:
    zeek:
      host: "0.0.0.0"
      port: 47761
      module_namespace: Tenzir
    misp:
      api:                       # Optional. If present, all of 'host', 'ssl', 'key' are required.
        host: https://xxxxxxx
        ssl: false
        key: xxxxxxxx
      zmq:
        host: localhost          # Required.
        port: 50000              # Required.

Same thing with the threatbus.zeek, I've tried a bunch of different host IPs

  ## Broker bind address.
  option broker_host = "172.16.80.2" &redef;

  ## Broker port.
  option broker_port = 47761/tcp &redef;

I'd really start using Threatbus instead of Dovehawk for my MISP-Zeek integration but Docker support is a dealbreaker for me.

Regards
Fredrik

Th3H0ff · 2023-02-13T12:32:40Z

Th3H0ff
Feb 13, 2023
Author

Hi again
I realize after digging around alot and even having a talk with ChatGPT that Threatbus might not be the most active project but I guess focus has moved to the Vast product instead. Anyway. I'll answer my own question.
I ran a dockerized Zeek 4.2 during all my troubleshooting and after I moved to Zeek 3.2.3 the subscribtion works and all seems well.

1 reply

mavam Feb 13, 2023
Maintainer

Hi there, apologies for the late response on this.

ChatGPT got it right, Threat Bus is in maintenance mode. Unfortunately we've only written that in the current docs and not the main project README.

Let me try to focus on the outcome to uplevel a bit: you would like synchronize MISP and Zeek, such that MISP observables end up in Zeek's intel framework. Is that correct?

This is a use case we are working towards with VAST in the context of security pipelines. Think of VAST like a "pipeline manager", as in this example:

We are still working out the pipeline syntax for the various carriers, but the concept is already clear: you will be able to use a pipeline to connect security tools, and our integrations will expose security tools (like MISP) as a dataflow that can be reshaped and put into other tools (Zeek, for example). Not shown here is the backchannel into Zeek. Within a dataflow of a pipeline, VAST speaks Apache Arrow. This means that we only need to make sure we're putting a dataframe of Zeek-Intel-Framework-compatible output and then finish the pipeline with some-reshaping-operators | to zeek <args>.

Note that that the storage side of things are entirely optional. In this scenario the pipeline executor can be attached both to realtime streaming data, as well as after a historical query.

Th3H0ff · 2023-02-14T10:35:27Z

Th3H0ff
Feb 14, 2023
Author

Thank you for reaching out and your answer. I appreciate it.
I've had to move my time to analyzing the data I'm getting from the ZEEK-THREATBUS-MISP setup as it is now with Zeek 3.2.3, instead of looking further into why I cannot connect the Zeek 4.2 docker to Threatbus. It's really weird though and I doubt I'll be able to leave the issue but for now I have to.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Dockerized Zeek and Threatbus #195

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Dockerized Zeek and Threatbus #195

Uh oh!

Uh oh!

Th3H0ff Feb 9, 2023

Replies: 2 comments · 1 reply

Uh oh!

Th3H0ff Feb 13, 2023 Author

Uh oh!

mavam Feb 13, 2023 Maintainer

Uh oh!

Th3H0ff Feb 14, 2023 Author

Th3H0ff
Feb 9, 2023

Replies: 2 comments 1 reply

Th3H0ff
Feb 13, 2023
Author

mavam Feb 13, 2023
Maintainer

Th3H0ff
Feb 14, 2023
Author