Golang Web Service Example using Databento and DuckDB
https://github.com/NimbleMarkets/dbn-duckduck-goose
This dbn-duckduck-goose repository contains an example Golang webservice that uses Databento's data-as-a-service backend and embedded DuckDB for storage and query.
Open source technologies used include:
- Golang for the programming language
- Databento for data-as-a-service
- DuckDB for embedded SQL database
- OpenAPI for API interop
- Swaggo's
swagfor code-driven OpenAPI generation - Gin for the web service framework
dbn-gofor DataBento Golang support- Apache ECharts for JavaScript charting
go-echartsfor Golang bindings to ECharts- Taskfile task runner
CAUTION: This program incurs DataBento billing!
Build and run the dbn-duckduck-goose server:
$ git clone https://github.com/NimbleMarkets/dbn-duckduck-go.git
$ cd dbn-duckduck-go
$ task build
# run server
$ export DATABENTO_API_KEY="<your_api_key>"
$ ./bin/dbn-duckduck-goose --dataset DBEQ.BASIC --out qqq.dbn.zst QQQThen test in another terminal or a web browser:
# read the specification
$ curl -v http://localhost:8888/docs/doc.json | less
# or open the Web GUI in a web browser
$ open http://localhost:8888/docs/index.html
# query for latest trades as JSON
$ curl http://localhost:8888/api/v1/last-trades/json/DBEQ.BASIC/QQQ
# query for latest trades as CSV
$ curl http://localhost:8888/api/v1/last-trades/csv/DBEQ.BASIC/QQQ
# query for latest trades as Excel
$ curl http://localhost:8888/api/v1/last-trades/excel/DBEQ.BASIC/QQQ
# query for candlesticks as JSON
$ curl http://localhost:8888/api/v1/candles/DBEQ.BASIC/QQQ
# interact with a chart in a web browser
$ open http://localhost:8888/api/v1/charts/candles/DBEQ.BASIC/SPYSince those charts are self-contained, we've bundled an SPY chart example here for you to try out.
The following environment variables control some behavior:
| Variable | Default | Description |
|---|---|---|
DATABENTO_API_KEY |
"" | DataBento API key to use for authorization |
GIN_MODE |
"debug" | Affects logging and Gin. May be debug, release, or test |
usage: ./bin/dbn-duckduck-goose -d <dataset> [opts] symbol1 symbol2 ...
-d, --dataset string Dataset to subscribe to
--db string DuckDB datate file to use (default: ':memory:')
-h, --help Show help
-p, --hostport string 'host:port' to service HTTP (default "localhost:8888")
-k, --key string Databento API key (or set 'DATABENTO_API_KEY' envvar)
-o, --out string Output filename for DBN stream ('-' for stdout)
-n, --snapshot Enable snapshot on subscription request
-t, --start string Start time to request as ISO 8601 format (default: now)
-v, --verbose Verbose logging
There is also a Dockerfile which is built by GitHub Actions. It can be run with:
$ export DATABENTO_API_KEY="<your_api_key>"
$ mkdir logs
$ docker run -it \
--env DATABENTO_API_KEY \
--volume logs:/logs \
--publish 8888:8888 \
ghcr.io/nimblemarkets/dbn-duckduck-goose \
--dataset DBEQ.BASIC \
--out /logs/test.dbn.zst
--db /logs/test.duckdb
--hostport 0.0.0.0:8888
Here we are pulling the image from GitHub, storing the log files in the bind-mounted logs directory, and exposing the web service on port 8888. The --hostport 0.0.0.0:8888 ensures that the web service binds a public interface of the container, otherwise it will bind the container's internal, inaccessible localhost.
Building depends on Taskfile Taskfile) and Swag for OpenAPI generation. Install the latter with go install github.com/swaggo/swag/cmd/swag@latest or task dev-deps. Run task build to build the binary.
# Quickstart for OSX users
$ brew install go-task/tap/go-task
$ task dev-deps
$ task buildNOTE: This library is not affiliated with Databento. It is for educational use. Please be careful as this service incurs billing. We are not responsible for any charges you incur.
Adapted from sample code in NimbleMarkets dbn-go library and other Neomantra/NimbleMarkets code.
Released under the MIT License, see LICENSE.txt.
Copyright (c) 2025 Neomantra Corp.
Made with ❤️ and 🔥 by the team behind Nimble.Markets.