Bithon is a word combining binocular together with python.
It targets application metrics, logging, distributed tracing, alert and application risk governance under microservice environment.
The above pic illustrates the main components of this project, including:
- Agent, which collects metrics/tracing logs automatically from client application without any code modification at the application side
- Collector, which provides various interfaces (including OpenTelemetry GRPC interface) to receive metrics/tracing logs from clients
- Pipeline, which provides a flexible and robust way to hande small data scale to an extra huge data scale for incoming metrics or tracing logs
- Storage, which provides an abstraction to underlying storages like H2, MySQL or Clickhouse
- Alerting, which allows us to set up alerts by using MetricSQL style expression on existing metrics or tracing logs and data in external storages
- Web, which provides NextJS-based web page for metrics/tracing/alerting visualization
- Around 200 built-in metrics for various JDK or various Java middlewares like Apache Http Components
- Compatible with multiple tracing context propagation formats, including OpenTelemetry, Zipkin, Jaeger and Pinpoint
- Support to receive tracing span logs via multiple channels
- Open Telemetry Tracing format via GRPC or HTTP
- Zipkin Tracing format via HTTP
- Jaeger Tracing Thrift format via HTTP or UDP
- Built-in debugging diagnosis commands, including JMX bean realtime monitoring for target application
- Flexible deployment to adapt small data scale and huge data scale use cases
- Fast queries and very low storage cost benefit from ClickHouse
- PromQL style alerting expression support
You can use the docker-compose.yml to start the whole system for preview.
docker-compose -f docker/docker-compose.yml upOnce all services in the docker-compose are up, you can visit http://localhost:9900 to access UI. And by default, the application itself is configured to be self-monitored, you will see the metrics/tracing of the application itself.
A demo is provided by this demo repo with a docker-compose file. You can follow the README on that demo repo to start the demo within just 3 steps.
After cloning this project along with all submodules by following commands
git clone https://github.com/FrankChen021/bithon.git
cd bithon && git submodule update --initJDK 17 and above are required to build this project.
If you have multiple JDKs on your machine, use export JAVA_HOME={YOUR_JDK_HOME} command to set correct JDK.
For example
export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-17.jdk/Contents/HomeFor the first time to build this project, use the following command to build dependencies first:
mvn clean install --activate-profiles shaded,jooq -T 1Cand then execute the following command to build the project.
mvn clean install -DskipTests -T 1CAfter the first build, we don't need to build the dependencies anymore unless there are changes in these dependencies.
Once the project has been built, you could run the project in a standalone mode to evaluate this project.
To launch server in evaluation mode, execute the following command:
java -Dspring.profiles.active=all-in-one -jar server/server-starter/target/server-starter.jarBy default, the application opens and listens on following ports at local
| Function | Port |
|---|---|
| tracing | 9895 |
| event | 9896 |
| metric | 9898 |
| ctrl | 9899 |
| web | 9897 |
Note:
-Dspring.profiles.includeparameter here is just for demo.You can make changes to
server/server-starter/src/main/resources/application.ymlto reflect your own settings.You can also use enable Alibaba Nacos as your configuration storage center.
Attach the agent to your java application so that your application can be managed the agent. Add the following VM arguments to your target Java application.
-javaagent:<YOUR_PROJECT_DIRECTORY>/agent/agent-distribution/target/agent-distribution/agent-main.jar -Dbithon.application.name=<YOUR_APPLICATION_NAME> -Dbithon.application.env=<YOUR_APPLICATION_ENV>| Variable | Description |
|---|---|
| YOUR_PROJECT_DIRECTORY | the directory where this project saves |
| YOUR_APPLICATION_NAME | the name of your application. It could be any string |
| YOUR_APPLICATION_ENV | the name of your environment to label your application. It could be any string. Usually it could be dev, test, prd |
By default, the agent connects collector running at local(127.0.0.1).
Collector address could be changed in file agent/agent-main/src/main/resources/agent.yml.
Make sure to re-build the project after changing the configuration file above.
Even the project is built by JDK 17 and above, the agent is compatible with JDK 1.8+. The following matrix lists the JDKs that are compatible with the agent on macOS. And in theory, this matrix works both for Windows and Linux.
| JDK | Supported |
|---|---|
| JDK 1.8.0_291 | ✓ |
| JDK 9.0.4 | ✓ |
| JDK 10.0.2 | ✓ |
| JDK 11.0.12 | ✓ |
| JDK 12.0.2 | ✓ |
| JDK 13.0.2 | ✓ |
| JDK 14.0.2 | ✓ |
| JDK 15.0.2 | ✓ |
| JDK 16.02 | ✓ |
| JDK 17 | ✓ |
| JDK 21 | ✓ |
| JDK 22.0.2 | ✓ |
| JDK 23.0.2 | ✓ |
| JDK 24 | ✓ |
NOTE: For applications running on JDK 24, the agent may not work properly to handle tracing across threads.
If the target application runs under JDK 11 and above, the following arguments should be added to JVM command to allow the agent to use Java Reflection on corresponding packages.
--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.net.www=ALL-UNNAMED
| Component | Min Version | Max Version | Metrics | Tracing |
|---|---|---|---|---|
| JVM | 1.8 | ✓ | ||
| JDK - Thread Pool | 1.8 | ✓ | ✓ | |
| JDK - HTTP Client | 1.8 | ✓ | ✓ | |
| Apache Druid(1) | 0.16 | 31.0 | ✓ | |
| Apache Kafka(2) | 0.10.0.0 | 3.9.0 | ✓ | ✓ |
| Apache OZone | 1.3.0 | ✓ | ||
| Apache ZooKeeper Client | 3.5 | 3.9 | ✓ | |
| Eclipse Glassfish | 2.34 | ✓ | ||
| GRPC | 1.57.0 | ✓ | ✓ | |
| Google Guice | 4.1.0 | ✓ | ||
| HTTP Client - Apache | 4.5.2 | 5.x | ✓ | ✓ |
| HTTP Client - Jetty | 9.4.6 | ✓ | ✓ | |
| HTTP Client - Netty | 3.10.6 | < 4.0 | ✓ | ✓ |
| HTTP Client - okhttp3 | 3.2 | 4.9 | ✓ | ✓ |
| HTTP Client - reactor-netty | 1.0.11 | ✓ | ✓ | |
| Jersey | 1.19.4 | ✓ | ||
| JDBC - Alibaba Druid | 1.0.28 | ✓ | ✓ | |
| JDBC - Apache Derby | 10.14.2 | ✓ | ✓ | |
| JDBC - H2 | 2.2.224 | ✓ | ✓ | |
| JDBC - MySQL | 5.x | 8.x | ✓ | ✓ |
| JDBC - PostgreSQL | 42.4.3 | ✓ | ✓ | |
| MongoDB | 3.4.2 | ✓ | ||
| Open Feign | 10.8 | ✓ | ||
| Quartz | 2.x | ✓ | ✓ | |
| Redis - Jedis | 2.9 | 5.x | ✓ | ✓ |
| Redis - Lettuce(3) | 5.1.2 | 6.x | ✓ | ✓ |
| Redis - Redisson | 3.19.0 | ✓ | ✓ | |
| Spring Boot | 1.5 | 3.0+ | ✓ | |
| Spring Bean | 4.3.12 | ✓ | ||
| Spring Open Feign | 10.8 | ✓ | ||
| Spring Rest Template | 4.3.12 | ✓ | ||
| Spring Scheduling | 4.3.12 | ✓ | ||
| Spring Gateway | 3.0.0 | ✓ | ✓ | |
| HTTP Server - Jetty | 9.4.41 | 12.0.x | ✓ | ✓ |
| HTTP Server - Netty | 2.0.0 | ✓ | ||
| HTTP Server - Tomcat | 8.5.20 | ✓ | ✓ | |
| HTTP Server - Undertow | 1.4.12 | ✓ | ✓ | |
| xxl-job | 2.3.0 | ✓ |
- For Apache Druid, the Jersey plugin is required to be enabled to collect query information.
- From Apache Kafka clients 3.7, the consumer metrics only works when the
group.protocolis configured asclassicwhich is the default configuration of the consumer client. - For Lettuce, the tracing support is only available when it's used with Spring Data Redis API.
To develop for this project, intellij is recommended.
A code style template file(dev/bithon_intellij_code_style) must be imported into intellij for coding.
For more information, check the development doc.