This repo contains code to test compliance with various Prometheus standards. Anyone taking part in CNCF's Prometheus Conformance Program will need to run the tests in here against their own implementations.
If you are reading this as someone testing their own implementation or considering to do so: There is a LOT of work that's planned but not executed yet. If you have time or headcount to invest in uplifting everyone's compliance, please talk to us.
There are several software categories something can be tested in. If something does not seem to fit existing categories, please also talk to us.
The alert_generator directory contains a shim at the moment. It will test correct generation and emitting of alerts towards Alertmanager.
The openmetrics directory contains a reference to the OpenMetrics test suite.
The promql directory contains code to test compliance with the native Prometheus PromQL implementation.
The promql_quick directory contains Go Module for performing compliance tests PromQL correctness. It's designed for smaller in-place quick unit tests, e.g. on per-PR basis, using docker based test framework. Useful as an acceptance tests for vendors or those who wish to maintain high Prometheus compatibility over time.
The remotewrite/sender directory contains code to test compliance with the Prometheus Remote Write specification as a sender.