This repo contains Content owned TS microservices structured in a monorepo.
Some of the patterns such as Docker & Github Actions have been adopted from Pocket's Monorepo setup by Daniel & Kat.
- Turborepo for build manager
- pnpm for package manager
- Docker for running dependent services
- Node TS services run with
pnpm/turbo - ESLint for code linting
- Prettier for code formatting
The following is an outline of how the monorepo is structured:
monorepo /
.docker /
shared resources here
.github /
CI workflows for all services and packages
infrastructure /
infra code per service
apps /
TBD (front-end apps go here?)
lambdas
service AWS lambdas go here
packages
internal packages shared between services
servers /
back-end services go here
Purpose of some important files in the root of the monorepo:
.npmrc - pnpm configuration
.syncpackrc - configuration for keeping dependencies in sync across all package.json files
pnpm-workspace.yml - Defines the root of the pnpm workspace and enables to include / exclude directories from the workspace
pnpm-lock.yml - Lock file generated by the pnpm package manager contains a complete, resolved list of all dependencies and their versions.
turbo.json - Configure behavior for turbo
We use Github Actions to:
- Build Docker images
- Push Docker images to AWS ECR. For
main/devpushes, also push the image to AWS ECS. - Execute unit and integration tests for all pull requests
- The integration tests in Github use the same
docker-compose.ymlfile that is used locally.
- The integration tests in Github use the same
- Build, plan (for pull requests), and apply (for
main/devpushes) terraform - Perform Apollo schema checks and updates
- Notify Sentry of new releases for
main/devpushes. - Start AWS CodeDeploy for pushes to
mainanddev.
Note - We currently are not queueing deployments. While a deployment is in progress, any new deployments will fail. This is a point of reslience we should build.
Build and deployment progress can be monitored in the "Actions" tab in the Github Repository.
CodeDeploy progress can be monitored directly in AWS.
First, create a local .env file and copy the contents of example.env.
Don't forget to use the node version configured for this monorepo (v20).
To build all services:
cd content-monorepo
nvm use
pnpm install
pnpm build
Sample build output:
Tasks: 8 successful, 8 total
Cached: 8 cached, 8 total
Time: 416ms >>> FULL TURBO
If there are no changes to the services, turbo uses the cached results of the previous build.
To spin up all services from the root of the monorepo:
cd content-monorepo
docker compose up --wait
pnpm dev
This will spin up dependent docker services (detached mode), and run the node ts services.
To spin up a service individually, pass the --filter={service_name} to the pnpm command:
cd content-monorepo
docker compose up --wait
pnpm build
pnpm dev --filter=prospect-api
Note that the GraphQL endpoints can be reached on the ip:port shown at startup. For example, curated-corpus-api can be reached at:
http://127.0.0.1:4025
http://127.0.0.1:4025/admin
To run all unit tests for all services & packages:
cd content-monorepo
pnpm build
pnpm test
To run unit tests individually per service:
cd content-monorepo
pnpm build
pnpm test --filter=prospect-api
While it is possible to run all integration tests at the same time, this isn't advised as you'll likely get errors related to Snowplow. This is because the integration tests will run concurrently, and all hit the same Snowplow endpoint. This means there will likely be collisions between tests, resulting in unexpected data/failures.
If you are undeterred and want to run all integration tests at the same time:
cd content-monorepo
docker compose up --wait
pnpm build
pnpm test-integrations
To run integration tests for a specific service:
cd content-monorepo
docker compose up --wait
pnpm build
pnpm test-integrations --filter=curated-corpus-api
To enable step-through debugging in VSCode:
- Install the Jest Runner extension. (Be sure it's this one! There are a couple other extensions with almost the same name.)
- Open up your settings and search for
jest-runner. - In the "Jest-Runner Config" section, make sure the "Jestrunner: Config Path" setting is empty. (It's probably got a default value.)
- Open up a test file and click the
RunandDebugcommands that float above the test definition. It should work!
If you have trouble with any of the steps above, try re-starting VSCode.
Prospect-api uses dynamodb as the db system. When running docker compose up, the localstack container executes a dynamodb.sh script where the prospect-api table
is created.
To seed the table with data, run the seeding script:
cd content-monorepo
pnpm build
pnpm db:dynamo-seed
If you want to delete dynamodb database, restart the docker and re-run the seed
docker restart content-monorepo-localstack-1
Collection API & Curated Corpus API use prisma as their ORM, and to set up & seed the tables, some tasks need to be run separately from docker compose.
cd content-monorepo
pnpm db:generate-prisma-client
Make sure the .env under each service using prisma contains the appropriate DATABASE_URL.
For curated-corpus-api: DATABASE_URL=mysql://root:@localhost:3306/curation_corpus?connect_timeout=300
For collection-api: DATABASE_URL=mysql://root:@localhost:3306/collections?connect_timeout=300
Run:
cd content-monorepo
pnpm prisma db seed
Please refer to the specific README of the service for applying a prisma migration.
Navigate to the individual project and use npm install to update the item
Then rebuild from the root of the monorepo with pnpm:
pnpm update
pnpm build
We use Syncpack to keep package versions consistent across servers, lambdas, and shared packages. Outside of the consistent functional expecations of using the same package version in all places, it's important to keep some package groups in sync to mitigate cross-package bugs, e.g AWS and Prisma packages.
The syncpack config can be found in the ./syncpackrc file.
There are two command line operations associated with Syncpack:
-
pnpm list-mismatcheswill tell you if any packages are out of sync/in violation of the rules set in.syncpackrc. This should only happen if your current branch is changing packages/package versions. Our CI will error if the rules in.syncpackrcare in violation. -
pnpm fix-mismatcheswill automatically fixpackage.jsonfiles that are in violation of the rules set in.syncpackrcby changing package versions. This is a quick and easy way to perform an upgrade, but as with any operation that can change things at scale, be sure you check the result is what you expect.
We leverage Pocket's tracing package to perform traces in our server applications:
-
Collection API
- Unleash feature flag (Dev)
- Unleash feature flag (Prod)
-
Curated Corpus API (coming soon)
- Unleash feature flag (Dev)
- Unleash feature flag (Prod)
-
Prospect API
- Unleash feature flag (Dev)
- Unleash feature flag (Prod)
Traces and logs for the above services can be found in GCP (filter by service name):
Tracing is performed using Open Telemetry NPM packages that send trace data to a standalone collector ECS service in AWS, which in turn exports trace data to GCP. The Pocket tracing package also implements an
Open Telemetry package that hooks into the Winston logger (which we implement via the Pocket ts-logger package) to auto-forward log data to GCP.
Tracing can be enabled and disabled using an Unleash feature flag, which exists per service implementing tracing.
To enable and configure the feature flag:
- Visit the feature flag URL for the environment in question (linked above)
- Toggle the
defaultenvironment to "On" in the left hand "Enabled in environments" box - Expand the
defaultenvironment in the main, right-hand panel - Click the ✎ pencil icon to edit the "Gradual rollout" strategy
- Move the "Rollout" slider to 100%
- Click the "Variants" tab and adjust the "Payload" number to the sample rate you'd like for your traces
- In production, this should usually be 1% (0.01) to begin with, and can be increased slowly if needed
- Click "Save strategy"
- After some requests have been made to the service, go look at traces in GCP (using links above)
To disable tracing on a service, simply toggle the default environment to "Off".
Local tracing is enabled by default and sends trace data to a Grafana Docker image. To view traces locally:
- Make sure the local service you want to trace has activity, e.g. by running a query in the Apollo Server Playground
- Navigate to the Grafana docker image endpoint at
http://localhost:3000/explore - Click "Explore" in the left hand menu
- In the dropdown at the top left of the middle pane, select "Tempo"
- In the main panel, select the "Service Graph" for "Query type"
- Click the service you want to view traces for and select "View traces"
- Trace away!