go-ms-boilerplate

README

Bulut-Bilisimciler-Go-Gin-Kafka-Nats-Microservice-Template

This is the simple boilerplate repository inspired by THIS_REPO

To Use This Template

• You need to clone repository to your local machine by using git clone command.
• You need to replaceAll go.mod module name and all imported class in project yours (You can use vscode "SHIFT + CTRL + F" and replace all.). Old name is "github.com/Bulut-Bilisimciler/go-ms-boilerplate" and new name is your new project name like "github.com/MYNAME/MYSUPERAPP".
• You need to update all service-specific global variables like "APP_NAME, API_PREFIX, RN_PREFIX, FN_PREFIX" in config.go file.
• Last, you need to replace HTTP Swagger documentation variables with your new name in main.go Swagger comments.

Core Utilities

• Uber/Fx - for module based dependency injection. This project includes HTTP Server, Kafka Consumer and NATS Consumer services with Fx Injection.
• Gin HTTP Server - used for handling all HTTP requests.
• Kafka Consumer - implemented as a service for consuming messages from Kafka. You can use it for common Kafka operations like producing, consuming topics for your microservice.
• Nats Req-Reply - used for intra microservices instant endpointless request-reply communication.
• Cron Scheduler - used for application scheduled jobs. You can use it for any scheduled jobs like cron jobs. E.g. usage: push application metrics to push-gateway every 5 minutes, or nitfy admins for error queues or retry dead letter queue items.
• Prometheus Metric Exporter - used for export in-app-metrics from application like example files 1, 2, 3 .
• In-App-Event-Emitter you can use go built-in "sync" library as global-level and inject it to every service individually. In final, this emitter acts like a in-app event emitter. You can emit any message from any in-app-service and listen from its listener.
• Mage - used for Makefile alternative in go. You can run your in-app-level operations or any level operations in go and you can run it with mage command like make. You can look our examples in THIS_FILE. You can check available commands with mage -l command on directory (e.g. cd ./tools/mage && mage -l).
• Taskfile - used to run out-of-code commands like "task gen-swag-docs" for documentation and any out-of-code level operations. For all taskfile commands, run task -a. (task installation required)
• DevContainer - used for auto ready VSCode development environment. You can run your development environment in a container with VSCode. You can check our devcontainer.json file for configuration.
• PlantUML - used for every service handlers documentation. You can create same name with ".go" files like ".plantuml" and explain your endpoints as sequence diagrams. You can copy-paste it to PlantUML Server and see your diagram. You can also use VSCode PlantUML Extension for previewing your diagrams. (e.g. create_comment.go -> create_comment.plantuml docs file.)

Directories

Core Directories

• iac Infrastucture as Code (Terraform, Cloudformation etc.) or applciation definition yamls for kubernetes etc. or any other implementation specific yamls. (e.g. prometheus-metric)
• internal/config for app-level .env.yaml file and configurations
• pkg/infrastructure any third-party infrastructure implementations (e.g. kafka, nats, redis, db etc.)
• pkg/domain for domain application classes specific implementations (e.g. user, comment, post class and interface definitions etc.)
• pkg/api for handling all endpoint requests client facing implementation. This handlers is bridge for application user and pkg/application app service core handlers. It is better to write middlewares and call service handlers here. (e.g. http routes, kafka consumer topic listeners, nats subject listener definitions etc.)
• pkg/application for handling all business logic and application core handlers. Core handlers of services should be here. For instance:
  • Example HTTP Route -> Example Route Handler - Dont forget to add Gin Swagger Documentation for further documentation.
  • Example Kafka Route -> Example Route Handler
  • Example Nats Route -> Example Route Handler
  • Example Job Schedule Route -> Example Scheduled Job Handler

Utility Directories

• .devcontainer - VSCode development container definition
• .github - GitHub Actions workflows
• .vscode - VSCode settings
• .husky - Husky Git Hooks
• docs - documentation (for swagger outputs. individual service docs are in their own directories -plantuml-)
• tools - tools for development (mage etc.)

Libraries

• Testing - testify - for testing
• Concurrency - conc - for manage concurrency multi-threading
• Config - viper - for configuration
• Json - jsoniter - for json encode/decode
• Log - zerolog - for logging
• Gin-Log - gin-zerolog - for gin http logging
• Makefile - Mage - go-level alternative for make and makefile (our use-case is in-app-level operations)
• Taskfile - Task - for out-of-code makefile alternative (our use-case is out-of-code-level operations)
• Git Hooks - Husky - for manage git hooks
• Cache - go-redis/v9 - for redis connection
• Kafka - segmentio - for manage kafka connection
• ReqReply - NATS - for intra-communication nats connection
• Cron - gocron - for in-app scheduled jobs

Running locally

git clone https://github.com/Bulut-Bilisimciler/go-ms-boilerplate.git .
docker-compose up -d # waits all required services to be ready
cp ./internal/config/sample.env.yaml ./internal/config/.env.yaml # copy env file and update it with your own values
go run .

For generating swagger documentation, run:

task gen-swag-docs

for sync local compose postgres db with remote db, there are several scripts in mage, run:

# mage database:syncDB LOCAL_DB_DSN REMOTE_DB_DSN TO_LOCAL_SCHEMA_NAME TABLES_WITH_COMMA_STRING
# example usage below
mage database:syncDB postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable&search_path=public postgres://remoteuser:remotepass@remoteurl:5432/remotedb?sslmode=disable&search_path=public remotesyncrev1 users,profiles,comments