stockermart
Cloud Functions
Stock market price downloader to BigQuery
using Cloud Scheduler
and Cloud PubSub
.
Setup
Create PubSub topic
gcloud beta pubsub topics create stocker-schedule
Create BigQuery price table
bq query --use_legacy_sql=false "
CREATE OR REPLACE TABLE stocker.price (
symbol STRING NOT NULL,
price FLOAT64 NOT NULL,
closingPrice FLOAT64 NOT NULL,
closingDate STRING NOT NULL,
quotedAt TIMESTAMP NOT NULL
)"
Deploy
Create a Cloud Function to download and save the stock prices
gcloud functions deploy stocker-price \
--entry-point GetStockPrices \
--set-env-vars "PID=${GCP_PROJECT},STOCK_PRICER_API_KEY=${STOCK_PRICER_API_KEY}" \
--memory 512MB \
--region us-central1 \
--runtime go112 \
--trigger-topic stocker-schedule \
--timeout=540s
Then create a Cloud Scheduler job to execute the above function every 30 min by publishing to the stocker-schedule
topic.
gcloud beta scheduler jobs create pubsub stocker-price-scheduler \
--schedule "*/30 * * * *" \
--topic stocker-schedule \
--message-body "refresh"
Cleanup
Deleting price function
gcloud functions delete stocker-price --region us-central1
Delete the source topics
gcloud pubsub topics delete stocker-schedule
Delete BigQuery content table
bq rm stocker.price