A toolkit for visualizing and analyzing MQTT connection, messaging performance, and latency data.
This project provides scripts for generating various visualizations of MQTT performance data:
- Connection latencies (TCP, TLS handshake, MQTT Connect)
- Publishing latencies
- Success rates for various operations
- Latency distributions and trends over time
The scripts require Python 3.6+ and the following packages:
pip install pandas numpy matplotlibOr install using the requirements file:
pip install -r requirements.txtbin/batch.sh: Batch processing script that generates all visualization types for a given log directory
bin/plot_qoe.py: Generate box plots for QoE metrics (latency distributions)bin/plot_time2.py: Create time-series plots showing latency trendsbin/plot_bar_qoe.py: Create bar charts showing latency distributionsbin/plot_rate.py: Generate plots showing success rates and cumulative operationsbin/plot_time.py: Alternative time-series plot script
bin/batch.sh /path/to/log/directorybin/batch.sh
# This will process all directories under ~/Home/tmp/toy_bench_databin/plot_qoe.py 'TCP,Handshake,Connect' /path/to/publog-*.csvbin/plot_qoe.py Publish /path/to/sublog-*.csvbin/plot_time2.py Publish /path/to/sublog-*.csvbin/plot_time2.py Connect /path/to/publog-*.csvbin/plot_bar_qoe.py Publish /path/to/sublog-*.csvbin/plot_bar_qoe.py Connect /path/to/*.csvbin/plot_rate.py Handshake /path/to/*.csvThe CSV files used by these scripts are generated by emqtt_bench, a benchmarking tool for MQTT brokers.
The scripts support analyzing CSV files with the following format:
ClientId,TS,TCP,Handshake,Connect[,Subscribe]
client1,1616168562000,35,120,50,-1
client2,1616168562500,42,115,48,-1
ClientId,TS,TCP,Handshake,Connect,Subscribe,Publish
sub1,1616168562000,30,118,45,25,73
sub2,1616168562500,38,125,52,28,65
ClientId: Unique identifier for the MQTT clientTS: Timestamp in milliseconds when the operation was initiated (epoch time)TCP: Time in milliseconds taken to establish the TCP connection- Measured from socket creation to successful connection
Handshake: Time in milliseconds for TLS handshake completion- Measured from from socket creation to successful completion
- Includes certificate validation and cipher negotiation
Connect: Time in milliseconds for MQTT CONNECT operation- Measured from socket creation to receiving CONNACK
Subscribe: Time in milliseconds for MQTT SUBSCRIBE operation- Measured from from socket creation to receiving SUBACK
-1indicates operation didn't occur or failed
Publish: End-to-end latency in milliseconds for message delivery- Measured from message publication to message receipt by subscriber
-1indicates operation didn't occur or failed
Note: The -1 values in the data are for compatibility with legacy data formats. Newer formats use NaN instead of -1 to represent missing or failed operations. The scripts handle both representations by converting them to NaN values during processing.
Scripts generate PNG files with naming patterns like:
{testname}_{operation}_bar_qoe.png: Latency distribution bar charts{testname}_{operation}_latency_trends.png: Time-series latency trends{testname}_{operation}_rate.png: Operation rate charts{testname}_{fields}_qoe.png: QoE distribution box plots