gofaas

package module
v0.0.0-...-a712179 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2018 License: Apache-2.0 Imports: 27 Imported by: 0

README

Go Functions-as-a-Service

Running a Go application on AWS Lambda is easier than ever, once you figure out how to configure Lambda, API Gateway and 10 or other "serverless" services to support the Go functions.

This is a boilerplate app with all the AWS pieces configured correctly and explained in depth. See the docs folder for detailed guides about functions, tracing, security, automation and more with AWS and Go.

With this foundation you can skip over all the undifferentiated setup, and focus entirely on your Go code.

Motivation

Functions-as-a-Service (FaaS) like AWS Lambda are one of the latest advances in cloud Infrastructure-as-a-Service (IaaS). Go is particularly well-suited to run in Lambda due to its speed, size and cross-compiler. Check out the Intro to Go Functions-as-a-Service and Lambda doc for more explanation.

For a long time, Go in Lambda was only possible through hacks -- execution shims, 3rd party frameworks and middleware, and little dev/prod parity. But in January 2018, AWS launched official Go support for Lambda and Go released v1.10 paving the clearest path yet for us Gophers.

This project demonstrates a simple and clean foundation for Go in Lambda. You can clone and deploy it with a few commands to get a feel for the stack. Or you can fork and rework it to turn it into your own web app.

It demonstrates:

Component Via Config, Code
HTTP functions Lambda, API Gateway 💾
Worker functions (one-off and periodic) Lambda, Invoke API, CloudWatch Events 💾
Development, packaging and deployment make, go, aws-sam-cli, CloudFormation ⚙️
Per-function environment and policies Lambda, IAM ⚙️
Custom domains CloudFront, ACM ⚙️
Static web content S3, CloudFront, ACM ⚙️
Static web security with Google OAuth 2.0 CloudFront, Lambda@Edge, SSM Parameters 💾
Function security with CORS and JWT API Gateway, jwt-go 💾
Function traces and logs CloudWatch Logs, X-Ray, AWS SDKs for Go 💾
Notifications SNS 💾
Databases and encryption at rest DynamoDB, KMS 💾
Testing with mock AWS clients Go interfaces, aws-sdk-go 💾

What's remarkable is how little work is required to get all functionality for our app. We don't need a framework, platform-as-a-service, or even any 3rd party software-as-a-service. And no, we don't need servers. By standing on the shoulders of Go and AWS, all the undifferentiated heavy lifting is managed for us.

We just need an expert CloudFormation config file and a simple Makefile, then we can focus entirely on writing Go functions.

Quick Start

This project uses :

Install the CLI tools and Docker CE

$ brew install awscli go node python@2 watchexec
$ pip2 install aws-sam-cli
$ open https://store.docker.com/search?type=edition&offering=community
We may want to upgrade existing tools...   
$ brew upgrade awscli go node python@2 watchexec
$ pip2 install --upgrade aws-sam-cli
We may want to double check the installed versions...   
$ aws --version
aws-cli/1.16.20 Python/3.7.0 Darwin/17.7.0 botocore/1.12.10

$ sam --version
SAM CLI, version 0.6.0

$ docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:31 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:29:02 2018
  OS/Arch:          linux/amd64
  Experimental:     true


$ go version
go version go1.11.1 darwin/amd64

$ watchexec --version
watchexec 1.9.2
We may also want to configure the AWS CLI with IAM keys to develop and deploy our application...  

Follow the Creating an IAM User in Your AWS Account doc to create a IAM user with programmatic access. Call the user gofaas-admin and attach the "Administrator Access" policy for now.

Then configure the CLI. Here we are creating a new profile that we can switch to with export AWS_PROFILE=gofaas. This will help us isolate our experiments from other AWS work.

Configure an AWS profile with keys and switch to the profile:

$ aws configure --profile gofaas
AWS Access Key ID [None]: AKIA................
AWS Secret Access Key [None]: PQN4CWZXXbJEgnrom2fP0Z+z................
Default region name [None]: us-east-1
Default output format [None]: json

$ export AWS_PROFILE=gofaas
$ aws iam get-user
{
    "User": {
        "Path": "/",
        "UserName": "gofaas-admin",
        "UserId": "AIDAJA44LJEOECDPZ3S5U",
        "Arn": "arn:aws:iam::572007530218:user/gofaas-admin",
        "CreateDate": "2018-02-16T16:17:24Z"
    }
}
Get the App

We start by getting and testing the github.com/nzoschke/gofaas.

$ git clone https://github.com/nzoschke/gofaas.git ~/dev/gofaas
$ cd ~/dev/gofaas

$ make test
go test -v ./...
go: finding github.com/aws/aws-xray-sdk-go v1.0.0-rc.8
go: finding github.com/aws/aws-lambda-go v1.6.0
go: finding github.com/aws/aws-sdk-go v1.15.49
...
=== RUN   TestUserCreate
--- PASS: TestUserCreate (0.00s)
...
ok     github.com/nzoschke/gofaas      0.014s
PASS

This gives us confidence in our Go environment.

Develop the App

We can then build the app and start a development server:

$ make dev
cd ./handlers/dashboard && GOOS=linux go build...
2018/02/25 08:03:12 Connected to Docker 1.35
2018/02/16 07:40:32 Fetching lambci/lambda:go1.x image for go1.x runtime...

Mounting handler (go1.x) at http://127.0.0.1:3000/users/{id} [DELETE]
Mounting handler (go1.x) at http://127.0.0.1:3000/users/{id} [PUT]
Mounting handler (go1.x) at http://127.0.0.1:3000/users/{id} [GET]
Mounting handler (go1.x) at http://127.0.0.1:3000/ [GET]
Mounting handler (go1.x) at http://127.0.0.1:3000/users [POST]

Now we can access our HTTP functions on port 3000:

$ curl http://localhost:3000
<html><body><h1>gofaas dashboard</h1></body></html>

We can also invoke a function directly:

$ echo '{}' | sam local invoke WorkerFunction
...
START RequestId: 36d6d40e-0d4b-168c-63d5-76b25f543d21 Version: $LATEST
2018/02/25 16:05:21 Worker Event: {SourceIP: TimeEnd:0001-01-01 00:00:00 +0000 UTC TimeStart:0001-01-01 00:00:00 +0000 UTC}
END RequestId: 36d6d40e-0d4b-168c-63d5-76b25f543d21
REPORT RequestId: 36d6d40e-0d4b-168c-63d5-76b25f543d21  Duration: 681.67 ms  Billed Duration: 700 ms  Memory Size: 128 MB  Max Memory Used: 14 MB

Note: if you see No AWS credentials found. Missing credentials may lead to slow startup..., review aws configure list and your AWS_PROFILE env var.

This gives us confidence in our development environment.

Deploy the App

Now we can package and deploy the app:

$ make deploy
make_bucket: pkgs-572007530218-us-east-1
Uploading to 59d2ea5b6bdf38fcbcf62236f4c26f21  3018471 / 3018471.0  (100.00%)
Waiting for changeset to be created
Waiting for stack create/update to complete
Successfully created/updated stack - gofaas

ApiUrl	https://x19vpdk568.execute-api.us-east-1.amazonaws.com/Prod

Now we can access our HTTP functions on AWS:

$ curl https://x19vpdk568.execute-api.us-east-1.amazonaws.com/Prod
<html><body><h1>gofaas dashboard</h1></body></html>

We can also invoke a function directly:

$ aws lambda invoke --function-name gofaas-WorkerFunction --log-type Tail --output text --query 'LogResult' out.log | base64 -D
START RequestId: 0bb47628-1718-11e8-ad73-c58e72b8826c Version: $LATEST
2018/02/21 15:01:07 Worker Event: {SourceIP: TimeEnd:0001-01-01 00:00:00 +0000 UTC TimeStart:0001-01-01 00:00:00 +0000 UTC}
END RequestId: 0bb47628-1718-11e8-ad73-c58e72b8826c
REPORT RequestId: 0bb47628-1718-11e8-ad73-c58e72b8826c  Duration: 11.11 ms  Billed Duration: 100 ms  Memory Size: 128 MB  Max Memory Used: 41 MB

Look at that speedy 11 ms duration! Go is faster than the minimum billing duration of 100 ms.

This gives us confidence in our production environment.

Development Environment

If we want to work on the worker or database functions locally, we need to give the functions environment variables with pointers to DynamoDB, KMS and S3. Open up env.json and set BUCKET, etc. with the ids of the resources we just created on deploy:

$ aws cloudformation describe-stack-resources --output text --stack-name gofaas \
  --query 'StackResources[*].{Name:LogicalResourceId,Id:PhysicalResourceId,Type:ResourceType}' | \
  grep 'Bucket\|Key\|UsersTable'

gofaas-bucket-aykdokk6aek8            Bucket      AWS::S3::Bucket
8eb8e209-51fb-41fa-adfe-1ec401667df4  Key         AWS::KMS::Key
gofaas-UsersTable-1CYAQH3HHHRGW       UsersTable  AWS::DynamoDB::Table
Integration Testing

We can verify the app functionality by creating an isolated testing stack, testing all the endpoints, then deleting the stack. The ci.sh script automates this:

$ ./ci.sh
aws cloudformation package ...
aws cloudformation deploy ...
...
<title>My first gofaas/Vue app</title>
"username": "test"
{"ExecutedVersion":null,"FunctionError":null,"LogResult":null,"Payload":"","StatusCode":202}
...
✅ SUCCESS!

Docs

Check out the docs folder where each component is explained in more detail.

Contributing

Find a bug or see a way to improve the project? Open an issue.

License

Apache 2.0 © 2018 Noah Zoschke

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	APIGateway = NewAPIGateway()
	DynamoDB   = NewDynamoDB()
	KMS        = NewKMS()
	Lambda     = NewLambda()
	S3         = NewS3()
	SNS        = NewSNS()
)

AWS Clients that can be mocked for testing

View Source 
var UUIDGen = func() uuid.UUID {
	return uuid.NewV4()
}

UUIDGen is a UUID generator that can be mocked for testing

Functions

func CFResource 

func CFResource(ctx context.Context, e CFEvent) (string, error)

CFResource dispatches a CloudFormation Custom Resource event to a create, update or delete function

func CFResourceCreate 

func CFResourceCreate(ctx context.Context, e CFEvent, props map[string]string) (string, error)

CFResourceCreate creates a custom resource and returns a new PhysicalResourceID This delegates to ResourceUpdate since there are no resources to create

func CFResourceDelete 

func CFResourceDelete(ctx context.Context, e CFEvent, props map[string]string) (string, error)

CFResourceDelete deletes a custom resource and returns its old PhysicalResourceID This delegates to ResourceUpdate since there are no resources to delete

func CFResourceUpdate 

func CFResourceUpdate(ctx context.Context, e CFEvent, props map[string]string) (string, error)

CFResourceUpdate updates a custom resource and returns its existing PhysicalResourceID This toggles settings on the Stage specified by Stage and RestApiId parameters

func CFRespond 

func CFRespond(ctx context.Context, e CFEvent) error

CFRespond creates, updates or deletes a CloudFormation custom resource then PUTs the status to the given ResponseURL

func Dashboard 

func Dashboard(ctx context.Context, e events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

Dashboard returns a dashboard HTML page

func JWTClaims 

func JWTClaims(e events.APIGatewayProxyRequest, claims jwt.Claims) (events.APIGatewayProxyResponse, jwt.Claims, error)

JWTClaims validates the token in the Authorization header It returns a response with standard headers and claims if valid And an error response and an error if invalid

func NewAPIGateway 

func NewAPIGateway() *apigateway.APIGateway

NewAPIGateway is an xray instrumented APIGateway client

func NewLambda 

func NewLambda() *lambda.Lambda

NewLambda is an xray instrumented Lambda client

func NewS3 

func NewS3() *s3.S3

NewS3 is an xray instrumented S3 client

func NewSNS 

func NewSNS() *sns.SNS

NewSNS is an xray instrumented SNS client

func UserCreate 

func UserCreate(ctx context.Context, e events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

UserCreate creates a user

func UserDelete 

func UserDelete(ctx context.Context, e events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

UserDelete deletes a user by id

func UserRead 

func UserRead(ctx context.Context, e events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

UserRead returns a user by id

func UserUpdate 

func UserUpdate(ctx context.Context, e events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

UserUpdate updates a user by id

func WorkCreate 

func WorkCreate(ctx context.Context, e events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

WorkCreate invokes the worker func

func Worker 

func Worker(ctx context.Context, e WorkerEvent) error

Worker is invoked directly to perform work then upload a report to S3

func WorkerPeriodic 

func WorkerPeriodic(ctx context.Context, e events.CloudWatchEvent) error

WorkerPeriodic runs on a schedule to clean S3

Types

type CFEvent 

type CFEvent struct {
	LogicalResourceID     string          `json:"LogicalResourceId"`
	OldResourceProperties json.RawMessage `json:"OldResourceProperties"`
	PhysicalResourceID    string          `json:"PhysicalResourceId"`
	RequestID             string          `json:"RequestId"`
	RequestType           string          `json:"RequestType"`
	ResourceProperties    json.RawMessage `json:"ResourceProperties"`
	ResourceType          string          `json:"ResourceType"`
	ResponseURL           string          `json:"ResponseURL"`
	StackID               string          `json:"StackId"`
}

CFEvent is a CloudFormation Custom Resource Event that Lambda is invoked with

type CFResponse 

type CFResponse struct {
	Data struct {
		Value string `json:"Value"`
	}
	LogicalResourceID  string `json:"LogicalResourceId"`
	PhysicalResourceID string `json:"PhysicalResourceId"`
	Reason             string `json:"Reason"`
	RequestID          string `json:"RequestId"`
	StackID            string `json:"StackId"`
	Status             string `json:"Status"`
}

CFResponse is a CloudFormation Custom Resource Response that is PUT to S3

type DynamoDBAPI 

type DynamoDBAPI interface {
	DeleteItemWithContext(ctx aws.Context, input *dynamodb.DeleteItemInput, opts ...request.Option) (*dynamodb.DeleteItemOutput, error)
	GetItemWithContext(ctx aws.Context, input *dynamodb.GetItemInput, opts ...request.Option) (*dynamodb.GetItemOutput, error)
	PutItemWithContext(ctx aws.Context, input *dynamodb.PutItemInput, opts ...request.Option) (*dynamodb.PutItemOutput, error)
}

DynamoDBAPI is a subset of dynamodbiface.DynamoDBAPI

func NewDynamoDB 

func NewDynamoDB() DynamoDBAPI

NewDynamoDB is an xray instrumented DynamoDB client

type HandlerAPIGateway 

type HandlerAPIGateway func(context.Context, events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error)

HandlerAPIGateway is an API Gateway Proxy Request handler function

func NotifyAPIGateway 

func NotifyAPIGateway(h HandlerAPIGateway) HandlerAPIGateway

NotifyAPIGateway wraps a handler func and sends an SNS notification on error

type HandlerCloudWatch 

type HandlerCloudWatch func(context.Context, events.CloudWatchEvent) error

HandlerCloudWatch is a CloudWatchEvent handler function

func NotifyCloudWatch 

func NotifyCloudWatch(h HandlerCloudWatch) HandlerCloudWatch

NotifyCloudWatch wraps a handler func and sends an SNS notification on error

type HandlerWorker 

type HandlerWorker func(context.Context, WorkerEvent) error

HandlerWorker is a Worker handler function

func NotifyWorker 

func NotifyWorker(h HandlerWorker) HandlerWorker

NotifyWorker wraps a handler func and sends an SNS notification on error

type KMSAPI 

type KMSAPI interface {
	DecryptWithContext(ctx aws.Context, input *kms.DecryptInput, opts ...request.Option) (*kms.DecryptOutput, error)
	EncryptWithContext(ctx aws.Context, input *kms.EncryptInput, opts ...request.Option) (*kms.EncryptOutput, error)
}

KMSAPI is a subset of kmsiface.KMSAPI

func NewKMS 

func NewKMS() KMSAPI

NewKMS is an xray instrumented KMS client

type ResponseError 

type ResponseError struct {
	Body       string
	StatusCode int
}

ResponseError is an error type that indicates a non-200 response

func (ResponseError) Error 

func (e ResponseError) Error() string

func (ResponseError) Response 

func (e ResponseError) Response() (events.APIGatewayProxyResponse, error)

Response returns an API Gateway Response event

type User 

type User struct {
	ID         string `json:"id"`
	Token      []byte `json:"-"`
	TokenPlain string `json:"token,omitempty"`
	Username   string `json:"username"`
}

User represents a user

type WorkerEvent 

type WorkerEvent struct {
	SourceIP  string    `json:"source_ip"`
	TimeEnd   time.Time `json:"time_end"`
	TimeStart time.Time `json:"time_start"`
}

WorkerEvent is foo

Source Files

View all Source files

Directories

Path Synopsis
handlers
custom-resource
dashboard
user-create
user-delete
user-read
user-update
work-create
worker
worker-periodic

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.