Chief-of-State (CoS) is an Open Source clustered persistence tool for building event sourced applications. CoS supports CQRS and event-sourcing through simple, language-agnostic interfaces via gRPC, and it allows developers to describe their schema with Protobuf. Under the hood, CoS leverages Apache Pekko to scale out and guarantee performant, reliable persistence.
Chief-of-State was built with the following principles:
- Wire format should be the same as persistence
- Scaling should not require re-architecture
- Developers shouldn't face race conditions or database locks
- Rules should be enforced with interfaces
- Event sourcing is valuable, but challenging to implement
- An ideal event-sourcing datastore would offer random access by key, streaming, and atomic writes
Chief-of-State was created in year 2020 at Namely based upon the principles aforementioned.
If a company is still using the Namely version and wants to migrate to this, it will have to be at least on the namely v0.9.2-rc.1 version. One can refer to the documentation for set up.
- Journal and Snapshot serialization using google protocol buffer message format
- Preconfigured clustering and domain entity sharding with the split-brain-resolver algorithm
- Automatic caching and entity passivation
- Automatic configuration of postgres storage on boot
- Opentelemetry integration for tracing and metrics
- Journal Compaction
- Direct integration to Kubernetes to form a cluster using the kubernetes API
- Read Side Management using the CLI tool
- Skip offset per shard and across the whole CoS cluster
- Pause read sides per shard and across the whole CoS cluster
- Resume read sides per shard and across the whole CoS cluster
- Restart read sides per shard and across the whole CoS cluster
- List read sides' offsets per shard and across the whole CoS cluster
Chief-of-State has been used in production by notable companies since its birth in 2020.
Developers implement two gRPC interfaces: a write handler(gRPC service) to handle commands/events and, optionally, many read handlers(gRPC services) for
reacting to state changes by consuming the events processed by the write handler.
The main entry point of a chief-of-state based application is the Service. Developers will interact with chief of state via:
ProcessCommandis used by the application to send commands to process via Write Handler.GetStateis used by the application to retrieve the current state of a persistent entity
Developers describe state mutations by implementing two RPC’s in the WriteSideHandlerService:
HandleCommandaccepts a command and the prior state of an entity and returns an Event. For example, given a command to UpdateUserEmail and a User, this RPC might return UserEmailUpdated.HandleEventaccepts an event and the prior state of an entity and returns a new state. For example, given a UserEmailUpdated event and a User, this RPC would return a new User instance with the email updated.
In response to state mutations, COS is able to send changes to many ReadSideHandlerService implementations, which may take any action. CoS guarantees at-least-once delivery of events and resulting state to each read side in the order they were persisted.
Some potential read side handlers might:
- Write state changes to a special data store like elastic
- Publish changes to kafka topics
- Send notifications to users in response to specific events
The following docs are available:
You can join these groups and chat to discuss and ask Chief Of State related questions on:
Contributions are welcome!
The project adheres to Semantic Versioning and Conventional Commits. If you see an issue that you'd like to see fixed, the best way to make it happen is to help out by submitting a pull request implementing it. To test your implementation locally follow the steps below:
# install earthly cli
brew install earthly/earthly/earthly (for mac users)
# locally build the image
earthly +build-image
# run tests
earthly -P --no-output +test-local
Chief-of-State is free, and it will remain so without any paid license requirement.