End-to-end open-source voice agents platform: Quickly build voice firsts conversational assistants through a json.
Note
We are actively looking for maintainers.
Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.
demo-create-agent-and-make-calls.mp4
This repository contains the entire orchestration platform to build voice AI applications. It technically orchestrates voice conversations using combination of different ASR+LLM+TTS providers and models over websockets.
Bolna helps you create AI Voice Agents which can be instructed to do tasks beginning with:
- Orchestration platform (this open source repository)
- Hosted APIs (https://docs.bolna.dev/api-reference/introduction) built on top of this orchestration platform [currently closed source]
- No-code UI playground at https://playground.bolna.dev/ using the hosted APIs + tailwind CSS [currently closed source]
- Any integration, enhancement or feature initially lands on this open source package since it forms the backbone of our Hosted APIs and dashboard
- Post that we expose APIs or make changes to existing APIs as required for the same
- Thirdly, we push it to the UI dashboard
graph LR;
A[Bolna open source] -->B[Hosted APIs];
B[Hosted APIs] --> C[Hosted Playground]
- Initiating a phone call using telephony providers like
Twilio,Plivo,Exotel(coming soon),Vonage(coming soon) etc. - Transcribing the conversations using
Deepgram, etc. - Using LLMs like
OpenAI,DeepSeek,Llama,Cohere,Mistral, etc to handle conversations - Synthesizing LLM responses back to telephony using
AWS Polly,ElevenLabs,Deepgram,OpenAI,Azure,Cartesia,Smallestetc.
Refer to the docs for a deepdive into all supported providers.
A basic local setup includes usage of Twilio or Plivo for telephony. We have dockerized the setup in local_setup/. One will need to populate an environment .env file from .env.sample.
The setup consists of four containers:
- Telephony web server:
- Choosing Twilio: for initiating the calls one will need to set up a Twilio account
- Choosing Plivo: for initiating the calls one will need to set up a Plivo account
- Bolna server: for creating and handling agents
ngrok: for tunneling. One will need to add theauthtokentongrok-config.ymlredis: for persisting agents & prompt data
Use docker to build the images using .env file as the environment file and run them locally
docker-compose build --no-cache bolna-app <twilio-app | plivo-app>: rebuild imagesdocker-compose up bolna-app <twilio-app | plivo-app>: run the build images
Once the docker containers are up, you can now start to create your agents and instruct them to initiate calls.
You may try out different agents from example.bolna.dev.
You can populate the .env file to use your own keys for providers.
ASR Providers
These are the current supported ASRs Providers:
| Provider | Environment variable to be added in .env file |
|---|---|
| Deepgram | DEEPGRAM_AUTH_TOKEN |
LLM Providers
Bolna uses LiteLLM package to support multiple LLM integrations.
These are the current supported LLM Provider Family: https://github.com/bolna-ai/bolna/blob/10fa26e5985d342eedb5a8985642f12f1cf92a4b/bolna/providers.py#L30-L47
For LiteLLM based LLMs, add either of the following to the .env file depending on your use-case:
LITELLM_MODEL_API_KEY: API Key of the LLM
LITELLM_MODEL_API_BASE: URL of the hosted LLM
LITELLM_MODEL_API_VERSION: API VERSION for LLMs like Azure
For LLMs hosted via VLLM, add the following to the .env file:
VLLM_SERVER_BASE_URL: URL of the hosted LLM using VLLM
TTS Providers
These are the current supported TTS Providers: https://github.com/bolna-ai/bolna/blob/c8a0d1428793d4df29133119e354bc2f85a7ca76/bolna/providers.py#L7-L14
| Provider | Environment variable to be added in .env file |
|---|---|
| AWS Polly | Accessed from system wide credentials via ~/.aws |
| Elevenlabs | ELEVENLABS_API_KEY |
| OpenAI | OPENAI_API_KEY |
| Deepgram | DEEPGRAM_AUTH_TOKEN |
| Cartesia | CARTESIA_API_KEY |
| Smallest | SMALLEST_API_KEY |
Telephony Providers
These are the current supported Telephony Providers:
| Provider | Environment variable to be added in .env file |
|---|---|
| Twilio | TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_PHONE_NUMBER |
| Plivo | PLIVO_AUTH_ID, PLIVO_AUTH_TOKEN, PLIVO_PHONE_NUMBER |
We have in the past tried to maintain both the open source and the hosted solution (via APIs and a UI dashboard).
We have fluctuated b/w maintaining this repository purely from a point of time crunch and not interest.
Currently, we are continuing to maintain it for the community and improving the adoption of Voice AI.
Though the repository is completely open source, you can connect with us if interested in managed hosted offerings or more customized solutions.
In case you wish to extend and add some other Telephony like Vonage, Telnyx, etc. following the guidelines below:
- Make sure bi-directional streaming is supported by the Telephony provider
- Add the telephony-specific input handler file in input_handlers/telephony_providers writing custom functions extending from the telephony.py class
- This file will mainly contain how different types of event packets are being ingested from the telephony provider
- Add telephony-specific output handler file in output_handlers/telephony_providers writing custom functions extending from the telephony.py class
- This mainly concerns converting audio from the synthesizer class to a supported audio format and streaming it over the websocket provided by the telephony provider
- Lastly, you'll have to write a dedicated server like the example twilio_api_server.py provided in local_setup to initiate calls over websockets.
We love all types of contributions: whether big or small helping in improving this community resource.
- There are a number of open issues present which can be good ones to start with
- If you have suggestions for enhancements, wish to contribute a simple fix such as correcting a typo, or want to address an apparent bug, please feel free to initiate a new issue or submit a pull request
- If you're contemplating a larger change or addition to this repository, be it in terms of its structure or the features, kindly begin by creating a new issue open a new issue
and outline your proposed changes. This will allow us to engage in a discussion before you dedicate a significant amount of time or effort. Y 573E our cooperation and understanding are appreciated