cloud

package
v0.0.0-...-8b7da69 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 27, 2020 License: Apache-2.0 Imports: 47 Imported by: 10

Documentation

Overview

Package cloud provides an implementation of services based on regular Cloud APIs and works from anywhere (not necessarily from AppEngine).

Index

Constants

View Source 
const (
	// DefaultFlexRequestLogName is the default value for Flex's RequestLogName
	// field.
	DefaultFlexRequestLogName = "gae.request"

	// DefaultFlexDebugLogName is the default value for Flex's DebugLogName field.
	DefaultFlexDebugLogName = "gae.request_logs"
)

Variables

View Source 
var (
	// LogIDRegexp is a regular expression that can be used to validate a Log ID.
	//
	// From: https://godoc.org/cloud.google.com/go/logging#Client.Logger
	LogIDRegexp = regexp.MustCompile(`^[A-Za-z0-9/_\-\.]{0,512}$`)

	// TraceIDLogLabel is the label used to express a request's trace ID in
	// Stackdriver LogEntry messages. Attaching a consistent trace ID label to
	// different logs allows them to be associated with the same trace.
	TraceIDLogLabel = "appengine.googleapis.com/trace_id"
)
View Source 
var ErrNotImplemented = errors.New("not implemented")

ErrNotImplemented is an error that can be returned to indicate that the requested functionality is not implemented.

Functions

func HTTPRequest 

func HTTPRequest(c context.Context) *http.Request

HTTPRequest returns the http.Request object associated with the current Flex request Context.

c must be a Context supplied by Use. If the source Request passed to Use did not have an HTTP request associated with it, HTTPRequest will return nil.

func NewConsoleLogger 

func NewConsoleLogger(c context.Context) logging.Logger

NewConsoleLogger instantiates a new Logger instance that logs to STDERR.

STDERR is independently captured in Flex environments.

func ScopedRequestPanic 

func ScopedRequestPanic(err error) *paniccatcher.Panic

ScopedRequestPanic returns the panic that a ScopedRequestHandler error hasl been tagged with.

If the error was not triggered by a captured Panic, ScopedRequestPanic will return nil.

func WithLogger 

func WithLogger(c context.Context, l *cloudLogging.Logger, cfg *LoggerConfig) context.Context

WithLogger installs an instance of a logging.Logger that forwards logs to an underlying Stackdriver Logger into the supplied Context.

An alternative is to construct Logger per request, setting request labels via CommonLabels(...) option. But it is more heavy solution, and at the time of writing it leads to memory leaks: https://github.com/GoogleCloudPlatform/google-cloud-go/issues/720#issuecomment-346199870

Types

type Config 

type Config struct {
	// IsDev is true if this is a development execution.
	IsDev bool

	// ProjectID, if not empty, is the project ID returned by the "info" service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	ProjectID string

	// ServiceName, if not empty, is the service (module) name returned by the
	// "info" service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	ServiceName string

	// VersionName, if not empty, is the version name returned by the "info"
	// service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	VersionName string

	// InstanceID, if not empty, is the instance ID returned by the "info"
	// service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	InstanceID string

	// ServiceAccountName, if not empty, is the service account name returned by
	// the "info" service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	ServiceAccountName string

	// ServiceProvider, if not nil, is the system service provider to use for
	// non-cloud external resources and services.
	//
	// If nil, the service will treat requests for services as not implemented.
	ServiceProvider ServiceProvider

	// LogToSTDERR, if true, indicates that log messages should be tee'd to
	// a simple STDERR logger prior to being written.
	//
	// This is be useful because Flex environment has a separate processing
	// pipeline for logs written to STDOUT/STDERR, so this ensures that even if
	// something goes wrong with the Cloud Logging client, the log may still be
	// recorded.
	LogToSTDERR bool

	// DS is the cloud datastore client. If populated, the datastore service will
	// be installed.
	DS *datastore.Client

	// MC is the memcache service client. If populated, the memcache service will
	// be installed.
	MC *memcache.Client

	// RequestLogger, if not nil, will be used by ScopedRequest to log
	// request-level logs.
	//
	// The request log is a per-request high-level log that shares a Trace ID
	// with individual debug logs, has request-wide metadata, and is given the
	// severity of the highest debug log emitted during the handling of the
	// request.
	RequestLogger *cloudLogging.Logger

	// DebugLogger, if not nil, will cause a Stackdriver Logging client to be
	// installed into the Context by Use for debug logging messages.
	//
	// Debug logging messages are individual application log messages emitted
	// through the "logging.Logger" interface installed by Use. All logs emitted
	// through the Logger are considered debug logs, regardless of theirl
	// individual Level.
	DebugLogger *cloudLogging.Logger
}

Config is a full-stack cloud service configuration. A user can selectively populate its fields, and services for the populated fields will be installed in the Context and available.

Because the "impl/cloud" service collection is a composite set of cloud services, the user can choose services based on their configuration.

The parameters of Config are mostly consumed by the "service/info" service implementation, which describes the environment in which the service is run.

func (*Config) Use 

func (cfg *Config) Use(c context.Context, req *Request) context.Context

Use installs the Config into the supplied Context. Services will be installed based on the fields that are populated in Config.

req is optional. If not nil, its fields will be used to initialize the services installed into the Context.

Any services that are missing will have "impl/dummy" stubs installed. These stubs will panic if called.

type ConfigLite 

type ConfigLite struct {
	// IsDev is true if this is a development execution.
	IsDev bool

	// ProjectID, if not empty, is the project ID returned by the "info" service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	ProjectID string

	// ServiceName, if not empty, is the service (module) name returned by the
	// "info" service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	ServiceName string

	// VersionName, if not empty, is the version name returned by the "info"
	// service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	VersionName string

	// InstanceID, if not empty, is the instance ID returned by the "info"
	// service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	InstanceID string

	// RequestID, if not empty, is the request ID returned by the "info"
	// service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	RequestID string

	// ServiceAccountName, if not empty, is the service account name returned by
	// the "info" service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	ServiceAccountName string

	// DS is the Cloud Datastore client.
	//
	// If populated, the datastore service will be installed.
	DS *datastore.Client

	// MC is the memcache service client.
	//
	// If populated, the memcache service will be installed.
	MC *memcache.Client
}

ConfigLite can be used to configure a context to have supported Cloud Services clients and ONLY them.

Currently supports only datastore and its required dependencies.

Unlike Config, it doesn't try to setup logging, intercept HTTP requests, provide auth, etc.

func (*ConfigLite) Use 

func (cfg *ConfigLite) Use(c context.Context) context.Context

Use configures the context with implementation of Cloud Services.

Any services that are missing will have "impl/dummy" stubs installed. These stubs will panic if called.

type Flex 

type Flex struct {
	// Cache is the process-global LRU cache instance that Flexi services can use
	// to cache data.
	//
	// If Cache is nil, a default cache will be used.
	Cache *lru.Cache

	// RequestLogName is the name of the per-request log entry that is generated
	// on request completion.
	//
	// If empty, RequestLogName will default to DefaultRequestLogName.
	RequestLogName string

	// DebugLogName is the log name that will be used for debug logger entries.
	// See Config.DebugLogger for more information on logging types.
	//
	// If empty, DebugLogName will default to DefaultDebugLogName.
	//
	// The debug log name is largely cosmetic, as request logs will be a more
	// useful entry point in practice.
	DebugLogName string
}

Flex defines a Google AppEngine Flex Environment platform.

func (*Flex) Configure 

func (f *Flex) Configure(c context.Context, opts ...option.ClientOption) (cfg *Config, err error)

Configure constructs a Config based on the current Flex environment.

Configure will instantiate some cloud clients. It is the responsibility of the client to close those instances when finished.

opts is the optional set of client options to pass to cloud platform clients that are instantiated.

func (*Flex) Request 

func (*Flex) Request(c context.Context, req *http.Request) *Request

Request probes Request parameters from a AppEngine Flex Environment HTTP request.

type GoogleServiceProvider 

type GoogleServiceProvider struct {
	// ServiceAccount is the name of the system's service account.
	ServiceAccount string

	// Cache is the LRU cache to use to store values that are fetched from remote
	// services.
	Cache *lru.Cache
}

GoogleServiceProvider is a ServiceProvider implementation that uses Google services.

func (*GoogleServiceProvider) PublicCertificates 

func (gsp *GoogleServiceProvider) PublicCertificates(c context.Context) (certs []infoS.Certificate, err error)

PublicCertificates implements ServiceProvider's PublicCertificates using Google's public certificate endpoint.

func (*GoogleServiceProvider) SignBytes 

func (gsp *GoogleServiceProvider) SignBytes(c context.Context, bytes []byte) (keyName string, signature []byte, err error)

SignBytes implements ServiceProvider's SignBytes using Google Cloud IAM's "SignBlob" endpoint.

The SignBlob RPC request that the GAE/Flex service account account is granted the "iam.serviceAccountActor" role, which is NOT default.

https://cloud.google.com/iam/reference/rest/v1/projects.serviceAccounts/signBlob

func (*GoogleServiceProvider) TokenSource 

func (gsp *GoogleServiceProvider) TokenSource(c context.Context, scopes ...string) (oauth2.TokenSource, error)

TokenSource implements ServiceProvider's TokenSource API using the default Google token source.

The way TokenSource is implemented, the service is vulnerable to a "cache stampede" effect where multiple access tokens invalidate at the same time and need to be refreshed.

TokenSource instances for a set of scopes are cached so that their access tokens will similarly be cached.

type HandlerFunc 

type HandlerFunc func(context.Context, http.ResponseWriter)

HandlerFunc is a callback to handle the HTTP request.

type InsertIDGenerator 

type InsertIDGenerator struct {
	Counter int64
	Base    string
}

InsertIDGenerator generates logging Insert IDs from a base key and an atomically-incrementing number. It is safe for concurrent usage.

Insert IDs must be unique, so it's critical that Base is unique for any given InsertIDGenerator.

func (*InsertIDGenerator) Next 

func (gen *InsertIDGenerator) Next() string

Next returns the next InsertID in the InsertIDGenerator sequence.

As a convenience, a nil InsertIDGenerator will return an empty string.

type LogSeverityTracker 

type LogSeverityTracker struct {
	// contains filtered or unexported fields
}

LogSeverityTracker tracks the highest observed log severity. It is safe for concurrent access.

func (*LogSeverityTracker) HighestSeverity 

func (lst *LogSeverityTracker) HighestSeverity() cloudLogging.Severity

HighestSeverity returns the highest logging severity that is been observed.

If no logging severity has been explicitly observed, Default severity will be returned.

func (*LogSeverityTracker) Observe 

func (lst *LogSeverityTracker) Observe(s cloudLogging.Severity)

Observe updates the LogSeverityTracker's highest observed log severity.

Observe is safe for concurrent usage.

type LoggerConfig 

type LoggerConfig struct {
	// SeverityTracker, if not nil, will be updated with the severity of log
	// messages sent through the generated Logger(s).
	SeverityTracker *LogSeverityTracker
	// InsertIDGenerator, if not nil, will be used to generate InsertIDs for
	// generated logs.
	InsertIDGenerator *InsertIDGenerator

	// Trace, if not empty, is the trace to associated with each log entry.
	Trace string

	// LogToSTDERR, if true, indicates that log messages should be tee'd to
	// a simple STDERR logger prior to being written.
	LogToSTDERR bool

	// The given labels will be applied to each log entry. This allows to reuse
	// Logger instance between requests, even if they have different default
	// labels.
	Labels map[string]string
}

LoggerConfig configures a Logger instance.

type Request 

type Request struct {
	// TraceID, if not empty, is the request's trace ID returned by the "info"
	// service.
	//
	// If empty, the service will treat requests for this field as not
	// implemented.
	TraceID string

	// HTTPRequest, if not nil, is the HTTP request that is being handled.
	HTTPRequest *http.Request

	// StartTime is the time when this request started. If empty, the current
	// clock time at the point when Use is called will be recorded.
	StartTime time.Time

	// LocalAddr is the local address handling this request. It may be empty
	// if the local address is unknown.
	LocalAddr string

	// SeverityTracker tracks the severity of the overall request. Callers may use
	// this manually. If not nil, it will be supplied to the request's Logger.
	//
	// If nil, the highest logging severity will still be tracked.
	SeverityTracker *LogSeverityTracker
}

Request is the set of request-specific parameters.

type ScopedRequestHandler 

type ScopedRequestHandler struct {
	// CapturePanics, if true, instructs the ScopedRequestHandler to capture and
	// handle any panics that occur during its request processing. IF false,
	// panics will not be caught or logged by ScopedRequestHandler.
	//
	// If a panic is caught, it will be logged to the installed logger. Handle
	// will return an error indicating that a panic was observed.
	CapturePanics bool
}

ScopedRequestHandler offers a middleware functionality that adds AppEngine-like handling to individual requests.

ScopedRequestHandler is not required to use the cloud implementation. If used, a ScopedRequestHandler's Handle method should be called during processing of individual HTTP requests to add pre- and post-request processing to these requests.

AppEngine has the behavior of grouping logs emitted during the processing of an individual request underneath of a top-level log for that request, allowing a user to visually isolate messages associated with a request. AppEngine Flex does not offer the same functionality by default, requiring users to manually emit the appropriate log messages to achieve a similar per-request grouping effect.

The construction used by ScopedRequestHandler works as follows: a "debug log" Logger is installed into the handler Context by Use, causing all logs generated by the Handler to be associated with the request's Trace ID. After the request is completed, ScopedRequestHandler emits a single "request log" representing the status of the request's handling and referencing the same Trace ID as the debug logs. A user viewing the request log stream would see one log entry per request; however, when that log entry is expanded, the viewer populates it with the debug logs that share its Trace ID, creating the effect of per-request logs.

func (*ScopedRequestHandler) Handle 

func (sr *ScopedRequestHandler) Handle(c context.Context, rw http.ResponseWriter, fn HandlerFunc) (err error)

Handle is a middleware function that wraps an HTTP request. It sets up a preprocessing environment, invokes the supplied Handler, and operates on the output after the Handler has finished.

If a panic occurs during Handle, and CapturePanics is true, the panic will be logged and consumed, and Handle will return an error. The panic's metadata will be available in the error by calling ScopedRequestPanic.

Handle requires a Flex GAE environment to be installed in the supplied Context (see Use).

type ServiceProvider 

type ServiceProvider interface {
	// PublicCertificates returns the set of public certificates belonging to the
	// current service's service account.
	PublicCertificates(c context.Context) ([]infoS.Certificate, error)

	// TokenSource returns a token source for the specified combination of scopes.
	TokenSource(c context.Context, scopes ...string) (oauth2.TokenSource, error)

	// SignBytes signs the specified bytes with a private key belonging to the
	// current system service.
	SignBytes(c context.Context, bytes []byte) (keyName string, signature []byte, err error)
}

ServiceProvider is a set of functionality which can be used to fetch system data.

Service methods should implement their own caching as appropriate.

Service methods should return ErrNotImplemented if a given function is not implemented.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.