README
¶
The Go Cloud Development Kit (Go CDK)
Write once, run on any cloud ☁️
The Go Cloud Development Kit (Go CDK) allows Go application developers to
seamlessly deploy cloud applications on any combination of cloud providers. It
does this by providing stable, idiomatic interfaces for common uses like storage
and databases. Think database/sql for cloud products.
Imagine writing this to read from blob storage (like Google Cloud Storage or S3):
ctx := context.Background()
bucket, err := blob.OpenBucket(ctx, "s3://my-bucket")
if err != nil {
return err
}
defer bucket.Close()
blobReader, err := bucket.NewReader(ctx, "my-blob", nil)
if err != nil {
return err
}
and being able to run that code on any cloud you want, avoiding all the ceremony of cloud-specific authorization, tracing, SDKs and all the other code required to make an application portable across cloud platforms.
The project works well with a code generator called
Wire. It creates
human-readable code that only imports the cloud SDKs for services you use. This
allows the Go CDK to grow to support any number of cloud services, without
increasing compile times or binary sizes, and avoiding any side effects from
init() functions.
You can learn more about the project from our announcement blog post, or our talk at Next 2018:
Installation
# First "cd" into your project directory if you have one to ensure "go get" uses
# Go modules (or not) appropriately. See "go help modules" for more info.
go get gocloud.dev
The Go CDK builds at the latest stable release of Go. Previous Go versions may compile but are not supported.
Documentation
Documentation for the project lives primarily on https://gocloud.dev/, including tutorials.
You can also browse Go package reference on pkg.go.dev.
Project status
The APIs are still in alpha, but we think they are production-ready and are actively looking for feedback from early adopters. If you have comments or questions please open an issue.
At this time we prefer to focus on maintaining the existing APIs and drivers,
and are unlikely to accept new ones into the go-cloud repository. The modular
nature of the Go CDK makes it simple to host new APIs and drivers for existing
APIs externally, in separate repositories.
If you have a new API or driver that you believe are important and mature enough to be included, feel free to open an issue to discuss this; our default will likely be to suggest starting in a separate repository. We'll also be happy to maintain a list of such external APIs and drivers in this README.
Current features
The Go CDK provides generic APIs for:
- Unstructured binary (blob) storage
- Publish/Subscribe (pubsub)
- Variables that change at runtime (runtimevar)
- Connecting to MySQL and PostgreSQL databases (mysql, postgres)
- Server startup and diagnostics: request logging, tracing, and health checking (server)
Contributing
Thank you for your interest in contributing to the Go Cloud Development Kit! ❤
Everyone is welcome to contribute, whether it's in the form of code, documentation, bug reports, feature requests, or anything else. We encourage you to experiment with the Go CDK and make contributions to help evolve it to meet your needs!
The GitHub repository at google/go-cloud contains some driver implementations for each portable API. We intend to include Google Cloud Platform, Amazon Web Services, and Azure implementations, as well as prominent open source services and at least one implementation suitable for use in local testing. Unfortunately, we cannot support every service directly from the project; however, we encourage contributions in separate repositories.
If you create a repository that implements the Go CDK interfaces for other services, let us know! We would be happy to link to it here and give you a heads-up before making any breaking changes.
See the contributing guide for more details.
Community
This project is covered by the Go Code of Conduct.
Legal disclaimer
The Go CDK is open-source and released under an Apache 2.0 License. Copyright © 2018–2019 The Go Cloud Development Kit Authors.
If you are looking for the website of GoCloud Systems, which is unrelated to the Go CDK, visit https://gocloud.systems.
Documentation
¶
Overview ¶
Package cloud contains a library and tools for open cloud development in Go.
The Go Cloud Development Kit (Go CDK) allows application developers to seamlessly deploy cloud applications on any combination of cloud providers. It does this by providing stable, idiomatic interfaces for common uses like storage and databases. Think `database/sql` for cloud products.
At the core of the Go CDK are common "portable types", implemented on top of service-specific drivers for supported cloud services. For example, objects of the blob.Bucket portable type can be created using gcsblob.OpenBucket, s3blob.OpenBucket, or any other Go CDK driver. Then, the blob.Bucket can be used throughout your application without worrying about the underlying implementation.
The Go CDK works well with a code generator called Wire (https://github.com/google/wire/blob/master/README.md). It creates human-readable code that only imports the cloud SDKs for drivers you use. This allows the Go CDK to grow to support any number of cloud services, without increasing compile times or binary sizes, and avoiding any side effects from `init()` functions.
For non-reference documentation, see https://gocloud.dev/
URLs ¶
See https://gocloud.dev/concepts/urls/ for a discussion of URLs in the Go CDK.
As ¶
See https://gocloud.dev/concepts/as/ for a discussion of how to write service-specific code with the Go CDK.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package aws provides fundamental Wire providers for Amazon Web Services (AWS).
|
Package aws provides fundamental Wire providers for Amazon Web Services (AWS). |
|
awscloud
Package awscloud contains Wire providers for AWS services.
|
Package awscloud contains Wire providers for AWS services. |
|
rds
Package rds contains Wire providers that are common across RDS.
|
Package rds contains Wire providers that are common across RDS. |
|
azure
|
|
|
azurecloud
Package azurecloud contains Wire providers for Azure services.
|
Package azurecloud contains Wire providers for Azure services. |
|
azuredb
Package azuredb contains Wire providers that are common across Azure Database.
|
Package azuredb contains Wire providers that are common across Azure Database. |
|
Package blob provides an easy and portable way to interact with blobs within a storage location.
|
Package blob provides an easy and portable way to interact with blobs within a storage location. |
|
azureblob
Package azureblob provides a blob implementation that uses Azure Storage’s BlockBlob.
|
Package azureblob provides a blob implementation that uses Azure Storage’s BlockBlob. |
|
driver
Package driver defines interfaces to be implemented by blob drivers, which will be used by the blob package to interact with the underlying services.
|
Package driver defines interfaces to be implemented by blob drivers, which will be used by the blob package to interact with the underlying services. |
|
drivertest
Package drivertest provides a conformance test for implementations of driver.
|
Package drivertest provides a conformance test for implementations of driver. |
|
fileblob
Package fileblob provides a blob implementation that uses the filesystem.
|
Package fileblob provides a blob implementation that uses the filesystem. |
|
gcsblob
Package gcsblob provides a blob implementation that uses GCS.
|
Package gcsblob provides a blob implementation that uses GCS. |
|
memblob
Package memblob provides an in-memory blob implementation.
|
Package memblob provides an in-memory blob implementation. |
|
s3blob
Package s3blob provides a blob implementation that uses S3.
|
Package s3blob provides a blob implementation that uses S3. |
|
Package docstore provides a portable way of interacting with a document store.
|
Package docstore provides a portable way of interacting with a document store. |
|
awsdynamodb
Package awsdynamodb provides a docstore implementation backed by Amazon DynamoDB.
|
Package awsdynamodb provides a docstore implementation backed by Amazon DynamoDB. |
|
driver
Package driver defines interfaces to be implemented by docstore drivers, which will be used by the docstore package to interact with the underlying services.
|
Package driver defines interfaces to be implemented by docstore drivers, which will be used by the docstore package to interact with the underlying services. |
|
drivertest
Package drivertest provides a conformance test for implementations of driver.
|
Package drivertest provides a conformance test for implementations of driver. |
|
gcpfirestore
Package gcpfirestore provides a docstore implementation backed by Google Cloud Firestore.
|
Package gcpfirestore provides a docstore implementation backed by Google Cloud Firestore. |
|
internal/fields
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
|
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity. |
|
memdocstore
Package memdocstore provides an in-process in-memory implementation of the docstore API.
|
Package memdocstore provides an in-process in-memory implementation of the docstore API. |
|
mongodocstore
Module
|
|
|
Package gcerrors provides support for getting error codes from errors returned by Go CDK APIs.
|
Package gcerrors provides support for getting error codes from errors returned by Go CDK APIs. |
|
Package gcp provides fundamental Wire providers and types for Google Cloud Platform (GCP).
|
Package gcp provides fundamental Wire providers and types for Google Cloud Platform (GCP). |
|
cloudsql
Package cloudsql contains Wire providers that are common across Google Cloud SQL.
|
Package cloudsql contains Wire providers that are common across Google Cloud SQL. |
|
gcpcloud
Package gcpcloud contains Wire providers for GCP services.
|
Package gcpcloud contains Wire providers for GCP services. |
|
internal
|
|
|
escape
Package escape includes helpers for escaping and unescaping strings.
|
Package escape includes helpers for escaping and unescaping strings. |
|
gcerr
Package gcerr provides an error type for Go CDK APIs.
|
Package gcerr provides an error type for Go CDK APIs. |
|
oc
Package oc supports OpenCensus tracing and metrics for the Go Cloud Development Kit.
|
Package oc supports OpenCensus tracing and metrics for the Go Cloud Development Kit. |
|
openurl
Package openurl provides helpers for URLMux and URLOpeners in portable APIs.
|
Package openurl provides helpers for URLMux and URLOpeners in portable APIs. |
|
releasehelper
Helper tool for creating new releases of the Go CDK.
|
Helper tool for creating new releases of the Go CDK. |
|
retry
Package retry provides retry logic.
|
Package retry provides retry logic. |
|
testing/octest
Package octest supports testing of OpenCensus integrations.
|
Package octest supports testing of OpenCensus integrations. |
|
testing/terraform
Package terraform provides a function to read Terraform output.
|
Package terraform provides a function to read Terraform output. |
|
testing/test-summary
Summarizes the output of go test.
|
Summarizes the output of go test. |
|
useragent
Package useragent includes constants and utilitiesfor setting the User-Agent for Go CDK connections to GCP.
|
Package useragent includes constants and utilitiesfor setting the User-Agent for Go CDK connections to GCP. |
|
cmd/gocdk
Module
|
|
|
contributebot
Module
|
|
|
Package mysql provides functions to open MySQL databases with OpenCensus instrumentation.
|
Package mysql provides functions to open MySQL databases with OpenCensus instrumentation. |
|
awsmysql
Package awsmysql provides connections to AWS RDS MySQL instances.
|
Package awsmysql provides connections to AWS RDS MySQL instances. |
|
azuremysql
Package azuremysql provides connections to Azure Database for MySQL.
|
Package azuremysql provides connections to Azure Database for MySQL. |
|
gcpmysql
Package gcpmysql provides connections to managed MySQL Cloud SQL instances.
|
Package gcpmysql provides connections to managed MySQL Cloud SQL instances. |
|
Package postgres provides functions to open PostgreSQL databases with OpenCensus instrumentation.
|
Package postgres provides functions to open PostgreSQL databases with OpenCensus instrumentation. |
|
awspostgres
Package awspostgres provides connections to AWS RDS PostgreSQL instances.
|
Package awspostgres provides connections to AWS RDS PostgreSQL instances. |
|
gcppostgres
Package gcppostgres provides connections to managed PostgreSQL Cloud SQL instances.
|
Package gcppostgres provides connections to managed PostgreSQL Cloud SQL instances. |
|
Package pubsub provides an easy and portable way to interact with publish/subscribe systems.
|
Package pubsub provides an easy and portable way to interact with publish/subscribe systems. |
|
awssnssqs
Package awssnssqs provides two implementations of pubsub.Topic, one that sends messages to AWS SNS (Simple Notification Service), and one that sends messages to SQS (Simple Queuing Service).
|
Package awssnssqs provides two implementations of pubsub.Topic, one that sends messages to AWS SNS (Simple Notification Service), and one that sends messages to SQS (Simple Queuing Service). |
|
azuresb
Package azuresb provides an implementation of pubsub using Azure Service Bus Topic and Subscription.
|
Package azuresb provides an implementation of pubsub using Azure Service Bus Topic and Subscription. |
|
batcher
Package batcher supports batching of items.
|
Package batcher supports batching of items. |
|
driver
Package driver defines interfaces to be implemented by pubsub drivers, which will be used by the pubsub package to interact with the underlying services.
|
Package driver defines interfaces to be implemented by pubsub drivers, which will be used by the pubsub package to interact with the underlying services. |
|
drivertest
Package drivertest provides a conformance test for implementations of driver.
|
Package drivertest provides a conformance test for implementations of driver. |
|
gcppubsub
Package gcppubsub provides a pubsub implementation that uses GCP PubSub.
|
Package gcppubsub provides a pubsub implementation that uses GCP PubSub. |
|
mempubsub
Package mempubsub provides an in-memory pubsub implementation.
|
Package mempubsub provides an in-memory pubsub implementation. |
|
kafkapubsub
Module
|
|
|
natspubsub
Module
|
|
|
rabbitpubsub
Module
|
|
|
Package runtimevar provides an easy and portable way to watch runtime configuration variables.
|
Package runtimevar provides an easy and portable way to watch runtime configuration variables. |
|
awsparamstore
Package awsparamstore provides a runtimevar implementation with variables read from AWS Systems Manager Parameter Store (https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html) Use OpenVariable to construct a *runtimevar.Variable.
|
Package awsparamstore provides a runtimevar implementation with variables read from AWS Systems Manager Parameter Store (https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html) Use OpenVariable to construct a *runtimevar.Variable. |
|
awssecretsmanager
Package awssecretsmanager provides a runtimevar implementation with variables read from AWS Secrets Manager (https://aws.amazon.com/secrets-manager) Use OpenVariable to construct a *runtimevar.Variable.
|
Package awssecretsmanager provides a runtimevar implementation with variables read from AWS Secrets Manager (https://aws.amazon.com/secrets-manager) Use OpenVariable to construct a *runtimevar.Variable. |
|
blobvar
Package blobvar provides a runtimevar implementation with variables read from a blob.Bucket.
|
Package blobvar provides a runtimevar implementation with variables read from a blob.Bucket. |
|
constantvar
Package constantvar provides a runtimevar implementation with Variables that never change.
|
Package constantvar provides a runtimevar implementation with Variables that never change. |
|
driver
Package driver defines interfaces to be implemented by runtimevar drivers, which will be used by the runtimevar package to interact with the underlying services.
|
Package driver defines interfaces to be implemented by runtimevar drivers, which will be used by the runtimevar package to interact with the underlying services. |
|
drivertest
Package drivertest provides a conformance test for implementations of runtimevar.
|
Package drivertest provides a conformance test for implementations of runtimevar. |
|
filevar
Package filevar provides a runtimevar implementation with variables backed by the filesystem.
|
Package filevar provides a runtimevar implementation with variables backed by the filesystem. |
|
gcpruntimeconfig
Package gcpruntimeconfig provides a runtimevar implementation with variables read from GCP Cloud Runtime Configurator (https://cloud.google.com/deployment-manager/runtime-configurator).
|
Package gcpruntimeconfig provides a runtimevar implementation with variables read from GCP Cloud Runtime Configurator (https://cloud.google.com/deployment-manager/runtime-configurator). |
|
gcpsecretmanager
Package gcpsecretmanager provides a runtimevar implementation with secrets read from GCP Secret Manager (https://cloud.google.com/secret-manager).
|
Package gcpsecretmanager provides a runtimevar implementation with secrets read from GCP Secret Manager (https://cloud.google.com/secret-manager). |
|
httpvar
Package httpvar provides a runtimevar implementation with variables backed by http endpoint.
|
Package httpvar provides a runtimevar implementation with variables backed by http endpoint. |
|
etcdvar
Module
|
|
|
Package secrets provides an easy and portable way to encrypt and decrypt messages.
|
Package secrets provides an easy and portable way to encrypt and decrypt messages. |
|
awskms
Package awskms provides a secrets implementation backed by AWS KMS.
|
Package awskms provides a secrets implementation backed by AWS KMS. |
|
azurekeyvault
Package azurekeyvault provides a secrets implementation backed by Azure KeyVault.
|
Package azurekeyvault provides a secrets implementation backed by Azure KeyVault. |
|
driver
Package driver defines interfaces to be implemented by secrets drivers, which will be used by the secrets package to interact with the underlying services.
|
Package driver defines interfaces to be implemented by secrets drivers, which will be used by the secrets package to interact with the underlying services. |
|
drivertest
Package drivertest provides a conformance test for implementations of the secrets driver.
|
Package drivertest provides a conformance test for implementations of the secrets driver. |
|
gcpkms
Package gcpkms provides a secrets implementation backed by Google Cloud KMS.
|
Package gcpkms provides a secrets implementation backed by Google Cloud KMS. |
|
localsecrets
Package localsecrets provides a secrets implementation using a locally provided symmetric key.
|
Package localsecrets provides a secrets implementation using a locally provided symmetric key. |
|
hashivault
Module
|
|
|
vault
Module
|
|
|
Package server provides a preconfigured HTTP server with diagnostic hooks.
|
Package server provides a preconfigured HTTP server with diagnostic hooks. |
|
driver
Package driver defines an interface for custom HTTP listeners.
|
Package driver defines an interface for custom HTTP listeners. |
|
health
Package health provides health check handlers.
|
Package health provides health check handlers. |
|
health/sqlhealth
Package sqlhealth provides a health check for a SQL database connection.
|
Package sqlhealth provides a health check for a SQL database connection. |
|
requestlog
Package requestlog provides an http.Handler that logs information about requests.
|
Package requestlog provides an http.Handler that logs information about requests. |
|
sdserver
Package sdserver provides the diagnostic hooks for a server using Stackdriver.
|
Package sdserver provides the diagnostic hooks for a server using Stackdriver. |
|
xrayserver
Package xrayserver provides the diagnostic hooks for a server using AWS X-Ray.
|
Package xrayserver provides the diagnostic hooks for a server using AWS X-Ray. |