weasel

README

yummy-weasel

A simple frontend (App Engine app) that serves content from a Google Cloud Storage (GCS) bucket, while allowing for:

• HTTPS on custom domain
• support for Service Worker (works only over HTTPS)
• HTTP/2 push
• more robust redirect from naked custom domain
• serving directly from naked custom domain
• keep the deployment/publishing flow using just the existing GCS bucket, which has proven to be fast and reliable, with partial content updates.
• dynamic server-side logic

design

The design is simple. Suppose you have a static website www.example.com, served directly from a GCS gs://www.example.com.

A very simplified picture of the serving path can be shown as follows.

+-----+    GET http://www.example.com/dir/index.html
| GCS |  <-----------------------------------------+  visitor
+-----+                                                 :-/

Weasel enhancement consists of modifying the serving path to:

+-----+   (2) GET gs://www.example.com/dir/index.html
| GCS |  <------------------------------+--------+
+-----+                                 | yummy  |
                                        | weasel |
                                        +--------+
                                           ↑  |
                                      (1)  |  | (3)
         GET https://www.example.com/dir/  |  ↓ Link: <asset>; rel=preload

                                         visitor
                                           :-)

1. A visitor requests the website page. Note that /index.html is now optional. Due to GCS restrictions, such suffixes were previously required for sub-folders, but with this approach they no longer need to be specified. Also, requests can (and will) be made over HTTPS.

2. Weasel fetches the object content from the original GCS bucket and caches it locally. This step is necessary only if the object hasn't been cached already or the cache has expired. Cache expiration and invalidation is based on GCS object cache-control header settings.

3. Weasel responds with the GCS object contents. Note that we can optionally push additional assets related to the requested file by using Link: <asset>; rel=preload header supported by GFE.

license

Apache License 2.0.

This is not an official Google product.

Documentation

Overview

Package weasel provides means for serving content from a Google Cloud Storage (GCS) bucket, suitable for hosting on Google App Engine. See README.md for the design details.

This package is a work in progress and makes no API stability promises.

Index

• Variables
• func ValidMethod(m string) bool
• type CORS
• type FetchError
  • func (e *FetchError) Error() string
• type Object
  • func (o *Object) Redirect() string
  • func (o *Object) RedirectCode() int
• type Storage
  • func (s *Storage) CacheKey(bucket, name string) string
  • func (s *Storage) HandleChangeHook(w http.ResponseWriter, r *http.Request)
  • func (s *Storage) Open(ctx context.Context, bucket, name string) (*Object, error)
  • func (s *Storage) OpenFile(ctx context.Context, bucket, name string) (*Object, error)
  • func (s *Storage) PurgeCache(ctx context.Context, bucket, name string) error
  • func (s *Storage) ServeObject(w http.ResponseWriter, r *http.Request, o *Object) error
  • func (s *Storage) Stat(ctx context.Context, bucket, name string) (*Object, error)

Variables

var DefaultStorage = &Storage{
	Base:  "https://storage.googleapis.com",
	Index: "index.html",
	CORS: CORS{
		Origin: []string{"*"},
		MaxAge: "86400",
	},
}

DefaultStorage is a Storage with sensible default parameters.

Functions

func ValidMethod

func ValidMethod(m string) bool

ValidMethod reports whether m is a supported HTTP method.

Types

type CORS

type CORS struct {
	Origin []string // allowed origins
	MaxAge string   // preflight cache, in seconds
}

CORS is a Storage cross-origin settings.

type FetchError

type FetchError struct {
	Msg  string
	Code int
}

FetchError contains error code and message from a GCS response.

func (*FetchError) Error

func (e *FetchError) Error() string

Error returns formatted FetchError.

type Object

type Object struct {
	Meta map[string]string
	Body io.ReadCloser
}

Object represents a single GCS object.

func (*Object) Redirect

func (o *Object) Redirect() string

Redirect returns o's redirect URL, zero string otherwise.

func (*Object) RedirectCode

func (o *Object) RedirectCode() int

RedirectCode returns o's HTTP response code for redirect. It defaults to http.StatusMovedPermanently.

type Storage

type Storage struct {
	Base  string // GCS service base URL, e.g. "https://storage.googleapis.com".
	Index string // Appended to an object name in certain cases, e.g. "index.html".
	CORS  CORS
}

Storage incapsulates configuration params for retrieveing and serving GCS objects.

func (*Storage) CacheKey

func (s *Storage) CacheKey(bucket, name string) string

CacheKey returns a key to cache an object under, computed from s.Base, bucket and then name.

func (*Storage) HandleChangeHook

func (s *Storage) HandleChangeHook(w http.ResponseWriter, r *http.Request)

HandleChangeHook handles Object Change Notifications as described at https://cloud.google.com/storage/docs/object-change-notification. It removes objects from cache.

func (*Storage) Open

func (s *Storage) Open(ctx context.Context, bucket, name string) (*Object, error)

Open retrieves GCS object name of the bucket from cache or network. Objects fetched from the network are cached before returning from this function.

func (*Storage) OpenFile

func (s *Storage) OpenFile(ctx context.Context, bucket, name string) (*Object, error)

OpenFile abstracts Open and treats object name like a file path.

func (*Storage) PurgeCache

func (s *Storage) PurgeCache(ctx context.Context, bucket, name string) error

PurgeCache removes cached object from memcache. It does not return an error in the case of cache miss.

func (*Storage) ServeObject

func (s *Storage) ServeObject(w http.ResponseWriter, r *http.Request, o *Object) error

ServeObject writes object o to w, with optional body and CORS headers, based on the in-flight request r.

func (*Storage) Stat

func (s *Storage) Stat(ctx context.Context, bucket, name string) (*Object, error)

Stat is similar to Read except the returned Object.Body may be nil. In the case where Body is not nil, calling Body.Close() is not required.