😋 This repository hosts my Backend Project for A Simple Bank, a Golang-based web service to create and manage money transfers between available accounts. 😋
Through this project, I've developed a robust backend system for a banking application.
Key features include account management, transaction records, and inter-account money transfers.
The project spans several critical areas of backend development, including database design with DBML, transaction handling, API development using the Gin framework, user authentication with JWT and PASETO, and implementing robust testing strategies. Additionally, the repository demonstrates proficiency in deploying applications using Docker and Kubernetes on AWS, complete with domain registration and traffic routing.
In this project, I've delved deeply into backend web service development, showcasing my practical abilities and grasp of key tools and technologies. I want to use this project as a platform to demonstrate my dedication and eager to learn backend development and my ability to contribute meaningfully in a professional context.
This project incorporates a diverse set of technologies and concepts essential for backend development:
- Programming Language: Golang
- Web Framework: Gin
- API Documentation: Swagger
- Database Design and Interaction: DBML, SQL, Database Isolation Levels
- Security and Authentication: JWT, PASETO, HTTPS, TLS, Let's Encrypt
- Containerization and Orchestration: Docker, Kubernetes
- Cloud Computing: AWS, EKS
- Continuous Integration/Deployment: GitHub Actions
I designed DB schema and generated SQL code with dbdiagram.io
Docker + Postgres + TablePlus
I used Docker to run containers and chose Postgres 12 as my relational database. For easier look up to the actual data in the local database, I used the TablePlus GUI. When using Docker, I use terminal command extentively, such as docker ps -a, docker start, and docker exec.
Database Migration
When working with a database, schema migration is often necessary to adapt to new business requirements. I ran and managed these database schema migrations using the Golang Migrate library. This library offers various customized commands for migrating schemas up and down. The SQL code for schema migration is stored in the 'db/migration' folder.
Makefile and .Phony list
Remembering and entering various lengthy terminal commands can be exhausting, so I defined those commonly used commands in my Makefile and listed them under .PHONY targets for easy execution.
Generate CRUD Golang code from SQL
After comparing various libraries for converting SQL queries into type-safe Go code, such as database/sql, Gorm, sqlx, and sqlc, I decided to use sqlc for interacting (Create, Read, Update, and Delete operations) with my database. After executing sqlc, it automatically generates struct definitions for models, function definitions with parameters, and the dbtx interface(this allows me to use either a database or a transaction to execute a query).
Unit Tests for database CRUD & Github Actions CI/CD
Using Go's testing package and Testify library(require package), I wrote various unit tests for the database CRUD operations. The files are named ending with "_test.go".
I used Github Action for running automated tests, I defined test and deployment workflow in .yaml files.
DB Transaction lock, and handle Deadlock
In order to show deadlock, I used TDD (test driven development) to create multiple go routine to execute transfer transactions concurrrently, then iterate through the list of results to check the created transfer and entry objects, and finally check the balances of those accounts accordingly.
A deadlock occurs when multiple processes try to access or modify the same data, with each of them waiting for the others to release the data while continuing to hold onto it themselves. This leads to a situation where none of the processes can actually control the data. A simple solution to prevent deadlocks is to include the 'FOR NO KEY UPDATE' clause in the SQL code. This informs PostgreSQL that the current operation will not modify the foreign key (account id), even though the main purpose of the SQL statement is to change the balance. As a result, the transaction lock does not hold onto the accounts table (where the account id is the primary key), thus reducing the risk of deadlocks. Deadlocks can also occur due to the order in which transactions update shared resources. For example, if Transaction T1 locks Account A first and then tries to lock Account B, while simultaneously Transaction T2 locks Account B and then tries to lock Account A, a deadlock ensues. To resolve this, I implemented a solution that enforces a consistent ordering: always locking the account with the smaller ID before locking the account with a larger ID.
ACID Property
The default isolation level for my PostgreSQL database is 'Repeatable Read'.
### 2. Building RESTFul HTTP JSON API [Gin + JWT + PASETO]
RESTFul HTTP API
I implemented HTTP API in Go using Gin web framework, there are many popular frameworks out there such as Beego, Echo, Revel, Martini, FastHttp, Gorilla Mux, but I chose Gin to do it. Furthermore, I used Postman to test the requests.
Load Config from files & environment variables
I used Viper to load Config from files & environment variables.
Security
I used Bcrypt package to hash user passwords, and upgraded JWT into PASETO token for authentication for login user API. Also, I added bearer-token based authentication layer to the API using Gin so that only the account owner can see the returned list of accounts owned by the corresponding user.
I learned
- How to build a small Golang docker image with a multistage dockerfile.
- How to use docker network to connect 2 stand-alone containers.
- How to write docker-compose file and control service start-up orders.
- Auto build & push docker image to AWS ECR(Amazon Elastic Container Registry) with Github Actions.
- How to create a production database on AWS RDS(Amazon Relational Database Service).
- Store & retrieve production secrets with AWS secrets manager.
- Kubernetes architexture & How to create an EKS cluster on AWS.
- How to use kubectl to connect to a kubernetes cluster on AWS EKS(Elastic Kubernetes Service).
- How to deploy a web app to Kubernetes cluster on AWS EKS.
- Register a domain & set up A-record using AWS Route53(DNS Service).
- How to use Ingress to route traffics to different services in Kubernetes.
- Auto issue & renew TSL certificates with cert-manager and Let's Encrypt.
- Automatic deploy to Kubernetes with Github Action.
✨ See my iOS Work Portfolio
✨ See my Task Management mini iOS App with CoreData
✨ See my Add, Delete, & Draggable Capsule Tags iOS Project
✨ See my iOS Boomerang Cards 3D Animation Project
✨ See my iOS Realtime Messaging App using Stream SKD
✨ See my iOS Book/Reading App Page Flipping 3D Animation Project
There is more to explore in my public repositories, and there are many interesting projects that I didn't set to public such as OpenAI API Chatbots and etc.
Cecilia Chen
I learned this Golang Back-end Course from TechSchool's Quang Pham and organized this repository myself. I love his teaching style and highly recommend his courses to anyone who needs it. I took notes in Notion and learned a great deal from his backend series course, really appreciate it!
- LinkedIn: https://www.linkedin.com/in/ceciliaguochen/
- Github: @ceciliachenguo