azcore

package module
v0.9.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2020 License: Apache-2.0 Imports: 22 Imported by: 1,546

Documentation

Overview

Package azcore implements an HTTP request/response middleware pipeline whose policy objects mutate an HTTP request's URL, query parameters, and/or headers before the request is sent over the wire.

Not all policy objects mutate an HTTP request; some policy objects simply impact the flow of requests/responses by performing operations such as logging, retry policies, timeouts, failure injection, and deserialization of response payloads.

Implementing the Policy Interface

To implement a policy, define a struct that implements the pipeline.Policy interface's Do method. Your Do method is called when an HTTP request wants to be sent over the network. Your Do method can perform any operation(s) it desires. For example, it can log the outgoing request, mutate the URL, headers, and/or query parameters, inject a failure, etc. Your Do method must then forward the HTTP request to next Policy object in a linked-list ensuring that the remaining Policy objects perform their work. Ultimately, the last Policy object sends the HTTP request over the network (by calling the HTTPSender's Do method).

When an HTTP response comes back, each Policy object in the linked-list gets a chance to process the response (in reverse order). The Policy object can log the response, retry the operation if due to a transient failure or timeout, deserialize the response body, etc. Ultimately, the last Policy object returns the HTTP response to the code that initiated the original HTTP request.

Here is a template for how to define a pipeline.Policy object: 

type myPolicy struct {
   node   PolicyNode
   // TODO: Add configuration/setting fields here (if desired)...
}

func (p *myPolicy) Do(ctx context.Context, request pipeline.Request) (pipeline.Response, error) {
   // TODO: Mutate/process the HTTP request here...
   response, err := p.node.Do(ctx, request)	// Forward HTTP request to next Policy & get HTTP response
   // TODO: Mutate/process the HTTP response here...
   return response, err	// Return response/error to previous Policy
}

Implementing the Factory Interface

Each Policy struct definition requires a factory struct definition that implements the pipeline.Factory interface's New method. The New method is called when application code wants to initiate a new HTTP request. Factory's New method is passed a pipeline.PolicyNode object which contains a reference to the owning pipeline.Pipeline object (discussed later) and a reference to the next Policy object in the linked list. The New method should create its corresponding Policy object passing it the PolicyNode and any other configuration/settings fields appropriate for the specific Policy object.

Here is a template for how to define a pipeline.Policy object: 

// NOTE: Once created & initialized, Factory objects should be goroutine-safe (ex: immutable);
// this allows reuse (efficient use of memory) and makes these objects usable by multiple goroutines concurrently.
type myPolicyFactory struct {
   // TODO: Add any configuration/setting fields if desired...
}

func (f *myPolicyFactory) New(node pipeline.PolicyNode) Policy {
   return &myPolicy{node: node} // TODO: Also initialize any configuration/setting fields here (if desired)...
}

Using your Factory and Policy objects via a Pipeline

To use the Factory and Policy objects, an application constructs a slice of Factory objects and passes this slice to the pipeline.NewPipeline function. 

func NewPipeline(factories []pipeline.Factory, sender pipeline.HTTPSender) Pipeline

This function also requires an object implementing the HTTPSender interface. For simple scenarios, passing nil for HTTPSender causes a standard Go http.Client object to be created and used to actually send the HTTP response over the network. For more advanced scenarios, you can pass your own HTTPSender object in. This allows sharing of http.Client objects or the use of custom-configured http.Client objects or other objects that can simulate the network requests for testing purposes.

Now that you have a pipeline.Pipeline object, you can create a pipeline.Request object (which is a simple wrapper around Go's standard http.Request object) and pass it to Pipeline's Do method along with passing a context.Context for cancelling the HTTP request (if desired). 

type Pipeline interface {
   Do(ctx context.Context, methodFactory pipeline.Factory, request pipeline.Request) (pipeline.Response, error)
}

Do iterates over the slice of Factory objects and tells each one to create its corresponding Policy object. After the linked-list of Policy objects have been created, Do calls the first Policy object passing it the Context & HTTP request parameters. These parameters now flow through all the Policy objects giving each object a chance to look at and/or mutate the HTTP request. The last Policy object sends the message over the network.

When the network operation completes, the HTTP response and error return values pass back through the same Policy objects in reverse order. Most Policy objects ignore the response/error but some log the result, retry the operation (depending on the exact reason the operation failed), or deserialize the response's body. Your own Policy objects can do whatever they like when processing outgoing requests or incoming responses.

Note that after an I/O request runs to completion, the Policy objects for that request are garbage collected. However, Pipeline object (like Factory objects) are goroutine-safe allowing them to be created once and reused over many I/O operations. This allows for efficient use of memory and also makes them safely usable by multiple goroutines concurrently.

Inserting a Method-Specific Factory into the Linked-List of Policy Objects

While Pipeline and Factory objects can be reused over many different operations, it is common to have special behavior for a specific operation/method. For example, a method may need to deserialize the response's body to an instance of a specific data type. To accommodate this, the Pipeline's Do method takes an additional method-specific Factory object. The Do method tells this Factory to create a Policy object and injects this method-specific Policy object into the linked-list of Policy objects.

When creating a Pipeline object, the slice of Factory objects passed must have 1 (and only 1) entry marking where the method-specific Factory should be injected. The Factory marker is obtained by calling the pipeline.MethodFactoryMarker() function: 

func MethodFactoryMarker() pipeline.Factory

Creating an HTTP Request Object

The HTTP request object passed to Pipeline's Do method is not Go's http.Request struct. Instead, it is a pipeline.Request struct which is a simple wrapper around Go's standard http.Request. You create a pipeline.Request object by calling the pipeline.NewRequest function: 

func NewRequest(method string, url url.URL, options pipeline.RequestOptions) (request pipeline.Request, err error)

To this function, you must pass a pipeline.RequestOptions that looks like this: 

type RequestOptions struct {
   // The readable and seekable stream to be sent to the server as the request's body.
   Body io.ReadSeeker

   // The callback method (if not nil) to be invoked to report progress as the stream is uploaded in the HTTP request.
   Progress ProgressReceiver
}

The method and struct ensure that the request's body stream is a read/seekable stream. A seekable stream is required so that upon retry, the final Policy object can seek the stream back to the beginning before retrying the network request and re-uploading the body. In addition, you can associate a ProgressReceiver callback function which will be invoked periodically to report progress while bytes are being read from the body stream and sent over the network.

Processing the HTTP Response

When an HTTP response comes in from the network, a reference to Go's http.Response struct is embedded in a struct that implements the pipeline.Response interface: 

type Response interface {
   Response() *http.Response
}

This interface is returned through all the Policy objects. Each Policy object can call the Response interface's Response method to examine (or mutate) the embedded http.Response object.

A Policy object can internally define another struct (implementing the pipeline.Response interface) that embeds an http.Response and adds additional fields and return this structure to other Policy objects. This allows a Policy object to deserialize the body to some other struct and return the original http.Response and the additional struct back through the Policy chain. Other Policy objects can see the Response but cannot see the additional struct with the deserialized body. After all the Policy objects have returned, the pipeline.Response interface is returned by Pipeline's Do method. The caller of this method can perform a type assertion attempting to get back to the struct type really returned by the Policy object. If the type assertion is successful, the caller now has access to both the http.Response and the deserialized struct object.

Index

Constants

View Source 
const (
	HeaderAuthorization      = "Authorization"
	HeaderCacheControl       = "Cache-Control"
	HeaderContentEncoding    = "Content-Encoding"
	HeaderContentDisposition = "Content-Disposition"
	HeaderContentLanguage    = "Content-Language"
	HeaderContentLength      = "Content-Length"
	HeaderContentMD5         = "Content-MD5"
	HeaderContentType        = "Content-Type"
	HeaderDate               = "Date"
	HeaderIfMatch            = "If-Match"
	HeaderIfModifiedSince    = "If-Modified-Since"
	HeaderIfNoneMatch        = "If-None-Match"
	HeaderIfUnmodifiedSince  = "If-Unmodified-Since"
	HeaderMetadata           = "Metadata"
	HeaderRange              = "Range"
	HeaderRetryAfter         = "Retry-After"
	HeaderURLEncoded         = "application/x-www-form-urlencoded"
	HeaderUserAgent          = "User-Agent"
	HeaderXmsDate            = "x-ms-date"
	HeaderXmsVersion         = "x-ms-version"
)

Constants ensuring that header names are correctly spelled and consistently cased.

View Source 
const (
	// UserAgent is the string to be used in the user agent string when making requests.
	UserAgent = "azcore/" + Version

	// Version is the semantic version (see http://semver.org) of the pipeline package.
	Version = "0.1.0"
)

Variables

View Source 
var (
	// ErrNoMorePolicies is returned from Request.Next() if there are no more policies in the pipeline.
	ErrNoMorePolicies = errors.New("no more policies")
)
View Source 
var (
	// StackFrameCount contains the number of stack frames to include when a trace is being collected.
	StackFrameCount = 32
)
View Source 
var (
	// StatusCodesForRetry is the default set of HTTP status code for which the policy will retry.
	// Changing its value will affect future created clients that use the default values.
	StatusCodesForRetry = []int{
		http.StatusRequestTimeout,
		http.StatusInternalServerError,
		http.StatusBadGateway,
		http.StatusServiceUnavailable,
		http.StatusGatewayTimeout,
	}
)

Functions

func NewResponseBodyProgress 

func NewResponseBodyProgress(responseBody io.ReadCloser, pr ProgressReceiver) io.ReadCloser

NewResponseBodyProgress adds progress reporting to an HTTP response's body stream.

func RedactSigQueryParam 

func RedactSigQueryParam(rawQuery string) (bool, string)

RedactSigQueryParam redacts the 'sig' query parameter in URL's raw query to protect secret.

func RetryAfter added in v0.7.0 

func RetryAfter(resp *http.Response) time.Duration

RetryAfter returns non-zero if the response contains a Retry-After header value.

func WithHTTPHeader added in v0.9.3 

func WithHTTPHeader(parent context.Context, header http.Header) context.Context

WithHTTPHeader adds the specified http.Header to the parent context. Use this to specify custom HTTP headers at the API-call level. Any overlapping headers will have their values replaced with the values specified here.

func WithRetryOptions added in v0.4.0 

func WithRetryOptions(parent context.Context, options RetryOptions) context.Context

WithRetryOptions adds the specified RetryOptions to the parent context. Use this to specify custom RetryOptions at the API-call level.

func WriteRequestWithResponse 

func WriteRequestWithResponse(b *bytes.Buffer, request *Request, response *Response, err error)

WriteRequestWithResponse appends a formatted HTTP request into a Buffer. If request and/or err are not nil, then these are also written into the Buffer.

Types

type AccessToken 

type AccessToken struct {
	Token     string
	ExpiresOn time.Time
}

AccessToken represents an Azure service bearer access token with expiry information.

type AuthenticationPolicyOptions 

type AuthenticationPolicyOptions struct {
	// Options contains the TokenRequestOptions that includes a scopes field which contains
	// the list of OAuth2 authentication scopes used when requesting a token.
	// This field is ignored for other forms of authentication (e.g. shared key).
	Options TokenRequestOptions
}

AuthenticationPolicyOptions contains various options used to create a credential policy.

type Base64Encoding added in v0.8.0 

type Base64Encoding int

Base64Encoding is usesd to specify which base-64 encoder/decoder to use when encoding/decoding a slice of bytes to/from a string. 

const (
	// Base64StdFormat uses base64.StdEncoding for encoding and decoding payloads.
	Base64StdFormat Base64Encoding = 0

	// Base64URLFormat uses base64.RawURLEncoding for encoding and decoding payloads.
	Base64URLFormat Base64Encoding = 1
)

type Credential 

type Credential interface {
	// AuthenticationPolicy returns a policy that requests the credential and applies it to the HTTP request.
	AuthenticationPolicy(options AuthenticationPolicyOptions) Policy
}

Credential represents any credential type.

func AnonymousCredential 

func AnonymousCredential() Credential

AnonymousCredential is for use with HTTP(S) requests that read public resource or for use with Shared Access Signatures (SAS).

type HTTPResponse added in v0.9.0 

type HTTPResponse interface {
	RawResponse() *http.Response
}

HTTPResponse provides access to an HTTP response when available. Errors returned from failed API calls will implement this interface. Use errors.As() to access this interface in the error chain. If there was no HTTP response then this interface will be omitted from any error in the chain.

type Listener 

type Listener func(LogClassification, string)

Listener is the function signature invoked when writing log entries. A Listener is required to perform its own synchronization if it's expected to be called from multiple Go routines.

type LogClassification 

type LogClassification string

LogClassification is used to group entries. Each group can be toggled on or off. 

const (
	// LogError entries contain detailed error information.
	// This includes the error message and stack trace.
	LogError LogClassification = "Error"

	// LogRequest entries contain information about HTTP requests.
	// This includes information like the URL, query parameters, and headers.
	LogRequest LogClassification = "Request"

	// LogResponse entries contain information about HTTP responses.
	// This includes information like the HTTP status code, headers, and request URL.
	LogResponse LogClassification = "Response"

	// LogRetryPolicy entries contain information specific to the retry policy in use.
	LogRetryPolicy LogClassification = "RetryPolicy"

	// LogSlowResponse entries contain information for responses that take longer than the specified threshold.
	LogSlowResponse LogClassification = "SlowResponse"
)

type Logger 

type Logger struct {
	// contains filtered or unexported fields
}

Logger controls which classifications to log and writing to the underlying log.

func Log 

func Log() *Logger

Log returns the process-wide logger.

func (*Logger) SetClassifications 

func (l *Logger) SetClassifications(cls ...LogClassification)

SetClassifications is used to control which classifications are written to the log. By default all log classifications are written.

func (*Logger) SetListener 

func (l *Logger) SetListener(lst Listener)

SetListener will set the Logger to write to the specified Listener.

func (*Logger) Should 

func (l *Logger) Should(cls LogClassification) bool

Should returns true if the specified log classification should be written to the log. By default all log classifications will be logged. Call SetClassification() to limit the log classifications for logging. If no listener has been set this will return false. Calling this method is useful when the message to log is computationally expensive and you want to avoid the overhead if its log classification is not enabled.

func (*Logger) Write 

func (l *Logger) Write(cls LogClassification, message string)

Write invokes the underlying Listener with the specified classification and message. If the classification shouldn't be logged or there is no listener then Write does nothing.

type Pipeline 

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline represents a primitive for sending HTTP requests and receiving responses. Its behavior can be extended by specifying policies during construction.

func NewPipeline 

func NewPipeline(transport Transport, policies ...Policy) Pipeline

NewPipeline creates a new goroutine-safe Pipeline object from the specified Policies. If no transport is provided then the default HTTP transport will be used.

func (Pipeline) Do 

func (p Pipeline) Do(ctx context.Context, req *Request) (*Response, error)

Do is called for each and every HTTP request. It passes the Context and request through all the Policy objects (which can transform the Request's URL/query parameters/headers) and ultimately sends the transformed HTTP request over the network.

type Policy 

type Policy interface {
	// Do applies the policy to the specified Request.  When implementing a Policy, mutate the
	// request before calling req.Do() to move on to the next policy, and respond to the result
	// before returning to the caller.
	Do(ctx context.Context, req *Request) (*Response, error)
}

Policy represents an extensibility point for the Pipeline that can mutate the specified Request and react to the received Response.

func NewRequestLogPolicy 

func NewRequestLogPolicy(o RequestLogOptions) Policy

NewRequestLogPolicy creates a RequestLogPolicy object configured using the specified options.

func NewRetryPolicy 

func NewRetryPolicy(o *RetryOptions) Policy

NewRetryPolicy creates a policy object configured using the specified options. Pass nil to accept the default values; this is the same as passing the result from a call to DefaultRetryOptions().

func NewTelemetryPolicy 

func NewTelemetryPolicy(o TelemetryOptions) Policy

NewTelemetryPolicy creates a telemetry policy object that adds telemetry information to outgoing HTTP requests.

func NewUniqueRequestIDPolicy 

func NewUniqueRequestIDPolicy() Policy

NewUniqueRequestIDPolicy creates a policy object that sets the request's x-ms-client-request-id header if it doesn't already exist.

type PolicyFunc 

type PolicyFunc func(context.Context, *Request) (*Response, error)

PolicyFunc is a type that implements the Policy interface. Use this type when implementing a stateless policy as a first-class function.

func (PolicyFunc) Do 

func (pf PolicyFunc) Do(ctx context.Context, req *Request) (*Response, error)

Do implements the Policy interface on PolicyFunc.

type ProgressReceiver 

type ProgressReceiver func(bytesTransferred int64)

ProgressReceiver defines the signature of a callback function invoked as progress is reported.

type ReadSeekCloser 

type ReadSeekCloser interface {
	io.ReadCloser
	io.Seeker
}

ReadSeekCloser is the interface that groups the io.ReadCloser and io.Seeker interfaces.

func NewRequestBodyProgress 

func NewRequestBodyProgress(requestBody ReadSeekCloser, pr ProgressReceiver) ReadSeekCloser

NewRequestBodyProgress adds progress reporting to an HTTP request's body stream.

func NopCloser 

func NopCloser(rs io.ReadSeeker) ReadSeekCloser

NopCloser returns a ReadSeekCloser with a no-op close method wrapping the provided io.ReadSeeker.

type Request 

type Request struct {
	*http.Request
	// contains filtered or unexported fields
}

Request is an abstraction over the creation of an HTTP request as it passes through the pipeline.

func NewRequest 

func NewRequest(httpMethod string, endpoint url.URL) *Request

NewRequest creates a new Request with the specified input.

func (*Request) Close 

func (req *Request) Close() error

Close closes the request body.

func (*Request) MarshalAsByteArray added in v0.7.1 

func (req *Request) MarshalAsByteArray(v []byte, format Base64Encoding) error

MarshalAsByteArray will base-64 encode the byte slice v, then calls SetBody. The encoded value is treated as a JSON string.

func (*Request) MarshalAsJSON added in v0.2.0 

func (req *Request) MarshalAsJSON(v interface{}) error

MarshalAsJSON calls json.Marshal() to get the JSON encoding of v then calls SetBody. If json.Marshal fails a MarshalError is returned. Any error from SetBody is returned.

func (*Request) MarshalAsXML 

func (req *Request) MarshalAsXML(v interface{}) error

MarshalAsXML calls xml.Marshal() to get the XML encoding of v then calls SetBody. If xml.Marshal fails a MarshalError is returned. Any error from SetBody is returned.

func (*Request) Next 

func (req *Request) Next(ctx context.Context) (*Response, error)

Next calls the next policy in the pipeline. If there are no more policies, nil and ErrNoMorePolicies are returned. This method is intended to be called from pipeline policies. To send a request through a pipeline call Pipeline.Do().

func (*Request) OperationValue 

func (req *Request) OperationValue(value interface{}) bool

OperationValue looks for a value set by SetOperationValue().

func (*Request) RewindBody 

func (req *Request) RewindBody() error

RewindBody seeks the request's Body stream back to the beginning so it can be resent when retrying an operation.

func (*Request) SetBody 

func (req *Request) SetBody(body ReadSeekCloser) error

SetBody sets the specified ReadSeekCloser as the HTTP request body.

func (*Request) SetOperationValue 

func (req *Request) SetOperationValue(value interface{})

SetOperationValue adds/changes a mutable key/value associated with a single operation.

func (*Request) SkipBodyDownload 

func (req *Request) SkipBodyDownload()

SkipBodyDownload will disable automatic downloading of the response body.

type RequestLogOptions 

type RequestLogOptions struct {
	// LogWarningIfTryOverThreshold logs a warning if a tried operation takes longer than the specified
	// duration (-1=no logging; 0=default threshold).
	LogWarningIfTryOverThreshold time.Duration
}

RequestLogOptions configures the retry policy's behavior.

type Response 

type Response struct {
	*http.Response
}

Response represents the response from an HTTP request.

func (*Response) Drain 

func (r *Response) Drain()

Drain reads the response body to completion then closes it. The bytes read are discarded.

func (*Response) HasStatusCode 

func (r *Response) HasStatusCode(statusCodes ...int) bool

HasStatusCode returns true if the Response's status code is one of the specified values.

func (*Response) UnmarshalAsByteArray added in v0.7.1 

func (r *Response) UnmarshalAsByteArray(v **[]byte, format Base64Encoding) error

UnmarshalAsByteArray will base-64 decode the received payload and place the result into the value pointed to by v.

func (*Response) UnmarshalAsJSON added in v0.2.0 

func (r *Response) UnmarshalAsJSON(v interface{}) error

UnmarshalAsJSON calls json.Unmarshal() to unmarshal the received payload into the value pointed to by v. If no payload was received a RequestError is returned. If json.Unmarshal fails a UnmarshalError is returned.

func (*Response) UnmarshalAsXML 

func (r *Response) UnmarshalAsXML(v interface{}) error

UnmarshalAsXML calls xml.Unmarshal() to unmarshal the received payload into the value pointed to by v. If no payload was received a RequestError is returned. If xml.Unmarshal fails a UnmarshalError is returned.

type Retrier 

type Retrier interface {
	// IsNotRetriable returns true for error types that are not retriable.
	IsNotRetriable() bool
}

Retrier provides methods describing if an error should be considered as transient.

type RetryOptions 

type RetryOptions struct {
	// MaxRetries specifies the maximum number of attempts a failed operation will be retried
	// before producing an error.  A value of zero means one try and no retries.
	MaxRetries int32

	// TryTimeout indicates the maximum time allowed for any single try of an HTTP request.
	TryTimeout time.Duration

	// RetryDelay specifies the amount of delay to use before retrying an operation.
	// The delay increases exponentially with each retry up to a maximum specified by MaxRetryDelay.
	// If you specify 0, then you must also specify 0 for MaxRetryDelay.
	// If you specify RetryDelay, then you must also specify MaxRetryDelay, and MaxRetryDelay should be
	// equal to or greater than RetryDelay.
	RetryDelay time.Duration

	// MaxRetryDelay specifies the maximum delay allowed before retrying an operation.
	// If you specify 0, then you must also specify 0 for RetryDelay.
	MaxRetryDelay time.Duration

	// StatusCodes specifies the HTTP status codes that indicate the operation should be retried.
	// If unspecified it will default to the status codes in StatusCodesForRetry.
	StatusCodes []int
}

RetryOptions configures the retry policy's behavior.

func DefaultRetryOptions added in v0.3.0 

func DefaultRetryOptions() RetryOptions

DefaultRetryOptions returns an instance of RetryOptions initialized with default values.

type TelemetryOptions 

type TelemetryOptions struct {
	// Value is a string prepended to each request's User-Agent and sent to the service.
	// The service records the user-agent in logs for diagnostics and tracking of client requests.
	Value string
}

TelemetryOptions configures the telemetry policy's behavior.

type TokenCredential 

type TokenCredential interface {
	Credential
	// GetToken requests an access token for the specified set of scopes.
	GetToken(ctx context.Context, options TokenRequestOptions) (*AccessToken, error)
}

TokenCredential represents a credential capable of providing an OAuth token.

type TokenRequestOptions 

type TokenRequestOptions struct {
	// Scopes contains the list of permission scopes required for the token.
	Scopes []string
}

TokenRequestOptions contain specific parameter that may be used by credentials types when attempting to get a token.

type Transport 

type Transport interface {
	// Do sends the HTTP request and returns the HTTP response or error.
	Do(ctx context.Context, req *http.Request) (*http.Response, error)
}

Transport represents an HTTP pipeline transport used to send HTTP requests and receive responses.

func DefaultHTTPClientTransport 

func DefaultHTTPClientTransport() Transport

DefaultHTTPClientTransport ...

type TransportFunc added in v0.5.0 

type TransportFunc func(context.Context, *http.Request) (*http.Response, error)

TransportFunc is a type that implements the Transport interface. Use this type when implementing a stateless transport as a first-class function.

func (TransportFunc) Do added in v0.5.0 

func (tf TransportFunc) Do(ctx context.Context, req *http.Request) (*http.Response, error)

Do implements the Transport interface on TransportFunc.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.