simplebank-golang-backend

README

Simple Bank

This repository contains the codes of the Backend Master Class course by TECH SCHOOL.

You can also find it on Udemy at this link.

And don't hesitate to join Tech School's Discord group to chat directly with me and other students.

In this course, you will learn step-by-step how to design, develop and deploy a backend web service from scratch. I believe the best way to learn programming is to build a real application. Therefore, throughout the course, you will learn how to build a backend web service for a simple bank. It will provide APIs for the frontend to do the following things:

• Create and manage bank accounts.
• Record all balance changes to each of the accounts.
• Perform a money transfer between 2 accounts.

The programming language we will use to develop the service is Golang, but the course is not just about coding in Go. You will learn a lot of different topics regarding backend web development. They are presented in 6 sections:

1. In the 1st section, you will learn deeply about how to design the database, generate codes to talk to the DB in a consistent and reliable way using transactions, understand the DB isolation levels, and how to use it correctly in production. Besides the database, you will also learn how to use docker for local development, how to use Git to manage your codes, and how to use GitHub Action to run unit tests automatically.

2. In the 2nd section, you will learn how to build a set of RESTful HTTP APIs using Gin - one of the most popular Golang frameworks for building web services. This includes everything from loading app configs, mocking DB for more robust unit tests, handling errors, authenticating users, and securing the APIs with JWT and PASETO access tokens.

3. In the 3rd section, you will learn how to build your app with Docker and deploy it to a production Kubernetes cluster on AWS. The lectures are very detailed with a step-by-step guide, from how to build a minimal docker image, set up a free-tier AWS account, create a production database, store and retrieve production secrets, create a Kubernetes cluster with EKS, use GitHub Action to automatically build and deploy the image to the EKS cluster, buy a domain name and route the traffics to the service, secure the connection with HTTPS and auto-renew TLS certificate from Let's Encrypt.

4. In the 4th section, we will discuss several advanced backend topics such as managing user sessions, building gRPC APIs, using gRPC gateway to serve both gRPC and HTTP requests at the same time, embedding Swagger documentation as part of the backend service, partially updating a record using optional parameters, and writing structured logger HTTP middlewares and gRPC interceptors.

5. Then the 5th section will introduce you to asynchronous processing in Golang using background workers and Redis as its message queue. We'll also learn how to create and send emails to users via Gmail SMTP server. Along the way, we'll learn more about writing unit tests for our gRPC services that might involve mocking multiple dependencies at once.

6. The final section 6th concludes the course with lectures about how to improve the stability and security of the server. We'll keep updating dependency packages to the latest version, use Cookies to make the refresh token more secure, and learn how to gracefully shut down the server to protect the processing resources. As this part is still a work in progress, we will keep making and uploading new videos about new topics in the future. So please come back here to check them out from time to time.

This course is designed with a lot of details, so that everyone, even with very little programming experience can understand and do it by themselves. I strongly believe that after the course, you would be able to work much more confidently and effectively on your projects.

Course videos

• Lecture #0: Setup development environment on Windows: WSL2 + Go + VSCode + Docker + Make + Sqlc

Section 1: Working with database [Postgres]

• Lecture #1: Design DB schema and generate SQL code with dbdiagram.io
• Lecture #2: Install & use Docker + Postgres + TablePlus to create DB schema
• Lecture #3: How to write & run database migration in Golang
• Lecture #4: Generate CRUD Golang code from SQL | Compare db/sql, gorm, sqlx & sqlc
• Lecture #5: Write unit tests for database CRUD with random data in Golang
• Lecture #6: A clean way to implement database transaction in Golang
• Lecture #7: DB transaction lock & How to handle deadlock in Golang
• Lecture #8: How to avoid deadlock in DB transaction? Queries order matters!
• Lecture #9: Deeply understand transaction isolation levels & read phenomena in MySQL & PostgreSQL
• Lecture #10: Setup Github Actions for Golang + Postgres to run automated tests

Section 2: Building RESTful HTTP JSON API [Gin]

• Lecture #11: Implement RESTful HTTP API in Go using Gin
• Lecture #12: Load config from file & environment variables in Go with Viper
• Lecture #13: Mock DB for testing HTTP API in Go and achieve 100% coverage
• Lecture #14: Implement transfer money API with a custom params validator
• Lecture #15: Add users table with unique & foreign key constraints in PostgreSQL
• Lecture #16: How to handle DB errors in Golang correctly
• Lecture #17: How to securely store passwords? Hash password in Go with Bcrypt!
• Lecture #18: How to write stronger unit tests with a custom gomock matcher
• Lecture #19: Why PASETO is better than JWT for token-based authentication?
• Lecture #20: How to create and verify JWT & PASETO token in Golang
• Lecture #21: Implement login user API that returns PASETO or JWT access token in Go
• Lecture #22: Implement authentication middleware and authorization rules in Golang using Gin

Section 3: Deploying the application to production [Kubernetes + AWS]

• Lecture #23: Build a minimal Golang Docker image with a multistage Dockerfile
• Lecture #24: How to use docker network to connect 2 stand-alone containers
• Lecture #25: How to write docker-compose file and control service start-up orders with wait-for.sh
• Lecture #26: How to create a free tier AWS account
• Lecture #27: Auto build & push docker image to AWS ECR with Github Actions
• Lecture #28: How to create a production DB on AWS RDS
• Lecture #29: Store & retrieve production secrets with AWS secrets manager
• Lecture #30: Kubernetes architecture & How to create an EKS cluster on AWS
• Lecture #31: How to use kubectl & k9s to connect to a kubernetes cluster on AWS EKS
• Lecture #32: How to deploy a web app to Kubernetes cluster on AWS EKS
• Lecture #33: Register a domain name & set up A-record using Route53
• Lecture #34: How to use Ingress to route traffics to different services in Kubernetes
• Lecture #35: Automatic issue TLS certificates in Kubernetes with Let's Encrypt
• Lecture #36: Automatic deploy to Kubernetes with Github Action

Section 4: Advanced Backend Topics [Sessions + gRPC]

• Lecture #37: How to manage user session with refresh token - Golang
• Lecture #38: Generate DB documentation page and schema SQL dump from DBML
• Lecture #39: Introduction to gRPC
• Lecture #40: Define gRPC API and generate Go code with protobuf
• Lecture #41: How to run a golang gRPC server and call its API
• Lecture #42: Implement gRPC API to create and login users in Go
• Lecture #43: gRPC gateway: write code once, serve both gRPC & HTTP requests
• Lecture #44: How to extract info from gRPC metadata
• Lecture #45: Automatic generate & serve Swagger docs from Go server
• Lecture #46: Embed static frontend files inside Golang backend server's binary
• Lecture #47: Validate gRPC parameters and send human/machine friendly response
• Lecture #48: Run DB migrations directly inside Golang code
• Lecture #49: Partial update DB record with SQLC nullable parameters
• Lecture #50: Build gRPC update API with optional parameters
• Lecture #51: Add authorization to protect gRPC API
• Lecture #52: Write structured logs for gRPC APIs
• Lecture #53: How to write HTTP logger middleware in Go

Section 5: Asynchronous processing with background workers [Asynq + Redis]

• Lecture #54: Implement background worker in Go with Redis and Asynq
• Lecture #55: Integrate async worker to Go web server
• Lecture #56: Send async tasks to Redis within a DB transaction
• Lecture #57: How to handle errors and print logs for Go Asynq workers
• Lecture #58: A bit of delay might be good for your async tasks
• Lecture #59: How to send emails in Go via Gmail
• Lecture #60: How to skip test in Go and config test flag in vscode
• Lecture #61: Email verification in Go: design DB and send email
• Lecture #62: Implement email verification API in Go
• Lecture #63: Unit test gRPC API with mock DB & Redis
• Lecture #64: How to test a gRPC API that requires authentication

Section 6: Improve the stability and security of the server

• Lecture #65: Config sqlc version 2 for Go and Postgres
• Lecture #66: Switch DB driver from lib/pq to pgx
• Lecture #67: How to handle DB errors with PGX driver
• Lecture #68: Docker compose: port + volume mapping
• Lecture #69: How to install & use binary packages in Go
• Lecture #70: Implement role-based access control (RBAC) in Go
• Lecture #71: Grant AWS EKS cluster access to Postgres and Redis using security group
• Lecture #72: Deploy gRPC + HTTP server to AWS EKS cluster
• Lecture #73: Don't lose money on AWS
• Lecture #74: Go 1.22 fixed the most common for-loop trap

Simple bank service

The service that we’re going to build is a simple bank. It will provide APIs for the frontend to do following things:

1. Create and manage bank accounts, which are composed of owner’s name, balance, and currency.
2. Record all balance changes to each of the account. So every time some money is added to or subtracted from the account, an account entry record will be created.
3. Perform a money transfer between 2 accounts. This should happen within a transaction, so that either both accounts’ balance are updated successfully or none of them are.

Setup local development

Install tools

• Docker desktop

• TablePlus

• Golang

• Homebrew

• Migrate

  brew install golang-migrate
  

• DB Docs

  npm install -g dbdocs
  dbdocs login
  

• DBML CLI

  npm install -g @dbml/cli
  dbml2sql --version
  

• Sqlc

  brew install sqlc
  

• Gomock

  go install github.com/golang/mock/mockgen@v1.6.0
  

Setup infrastructure

• Create the bank-network

  make network
  

• Start postgres container:

  make postgres
  

• Create simple_bank database:

  make createdb
  

• Run db migration up all versions:

  make migrateup
  

• Run db migration up 1 version:

  make migrateup1
  

• Run db migration down all versions:

  make migratedown
  

• Run db migration down 1 version:

  make migratedown1
  

Documentation

• Generate DB documentation:

  make db_docs
  

• Access the DB documentation at this address. Password: secret

How to generate code

• Generate schema SQL file with DBML:

  make db_schema
  

• Generate SQL CRUD with sqlc:

  make sqlc
  

• Generate DB mock with gomock:

  make mock
  

• Create a new db migration:

  migrate create -ext sql -dir db/migration -seq <migration_name>
  

How to run

• Run server:

  make server
  

• Run test:

  make test
  

Deploy to kubernetes cluster

• Install nginx ingress controller:

  kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.48.1/deploy/static/provider/aws/deploy.yaml
  

• Install cert-manager:

  kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml