README
¶
Sputnik Popularity
Sputnik populairty keeps track of how busy is my local climbing gym and allows to see how popular it has been over the past few days.
It scrapes my gym's capacity utilization endpoint and store the data in an InfluxDB instance. A web front end shows the utilization during the last couple of weeks.
My Production Environment
The main restriction when building my production environemnt for this project is that everything should be free (as in beer).
Software:
- The Git repository is available on Github.
Hardware:
- Database: an InfluxDB instance running on Amazon Web Services (AWS) in Germany, EU.
- Scraper and Web front-end: running as a single docker container on Google Cloud Platform (GCP) in South Carolina, USA.
Develop/Deploy process:
- The master branch contains the version of the project currently running in production.
- I develop all new functionalities and bug fixes as separate git branches; they get merged into master via Github pull requests.
- I verify all tests are passing on every pull request using CircleCI. You can do the same locally using
make test. - I use Github Actions to lint every pull request using golangci-lint. You do the same locally using
make lint. - When a pull request is approved and merged into master, a Github Action creates the new docker image for the project and push it to my private Google Cloud Registry.
How to run the tests
There are 3 types of tests: unit, integration and e2e.
You can run all of them with make test.
To run them individually use make unit, make integration or make e2e.
How to run the project
Some environment variables are required to run the project:
| Environment variable | Description |
|---|---|
| SPUTNIK_INFLUXDB_URL | your InfluxDB URL |
| SPUTNIK_INFLUXDB_TOKEN_WRITE | your InfluxDB write token |
| SPUTNIK_INFLUXDB_TOKEN_READ | your InfluxDB read token |
| SPUTNIK_SCRAPE_URL | the URL to fetch the gym's current capacity utilization |
| SPUTNIK_SCRAPE_GYM_NAME | the gym name to pass to the scrape URL |
| SPUTNIK_SCRAPE_GYM_ID | the gym ID to pass to the scrape URL |
Once this environment variables have been set you can run the project locally with:
; go run ./app/cmd/sputnik-popularity
Run as a docker container in Google Compute Engine
First build a docker image of the project:
; make docker-image
[...]
Successfully tagged sputnik:latest
This will generate an small image (~11MB) tagged sputnik:latest with the binary of the project.
Now, push the image to a docker container registry of your choosing. Here I'm using the Google Cloud Container Registry and I assume PROJECT_ID holds the ID of one of my projects in Google Cloud:
; docker tag sputnik:latest gcr.io/$PROJECT_ID/sputnik:latest
; docker push gcr.io/$PROJECT_ID/sputnik:latest
Connect to a Google Compute Engine instance and run the docker image. Here I assume INSTANCE_NAME holds the name of one of my virtual machines in Google Compute Engine:
; gcloud compute ssh $INSTANCE_NAME
name@instance ~ $ docker run \
--name sputnik \
--detach \
--env SPUTNIK_INFLUXDB_URL="..." \
--env SPUTNIK_INFLUXDB_TOKEN_WRITE="..." \
--env SPUTNIK_INFLUXDB_TOKEN_READ="..." \
--env SPUTNIK_SCRAPE_URL="..." \
--env SPUTNIK_SCRAPE_GYM_NAME="..." \
--env SPUTNIK_SCRAPE_GYM_ID="..." \
--publish 8080:8080 \
--log-driver=gcplogs \
gcr.io/$PROJECT_ID/sputnik_popularity
This will create a container named sputnik and send the app logs to the Google Cloud Logs Viewer.
Note that if your image is stored as private in your registry you will need to configure docker in your instance to authenticate with it before running or pulling the image.
Directories