httpclient

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 6, 2020 License: Apache-2.0 Imports: 24 Imported by: 20

Documentation

Overview

Package httpclient provides round trippers/transport wrappers for http clients.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ContextWithRPCMethodName 

func ContextWithRPCMethodName(ctx context.Context, name string) context.Context

ContextWithRPCMethodName returns a copy of ctx with the rpcMethodName key set. This enables instrumentation (metrics, tracing, etc) to better identify metrics.

func NewHTTPClient 

func NewHTTPClient(params ...HTTPClientParam) (*http.Client, error)

NewHTTPClient returns a configured http client ready for use. We apply "sane defaults" before applying the provided params.

func StatusCodeFromError 

func StatusCodeFromError(err error) (statusCode int, ok bool)

StatusCodeFromError wraps the internal StatusCodeFromError func. For behavior details, see its docs.

Types

type Client 

type Client interface {
	// Do executes a full request. Any input or output should be specified via params.
	// By the time it is returned, the response's body will be fully read and closed.
	// Use the WithResponse* params to unmarshal the body before Do() returns.
	//
	// In the case of a response with StatusCode >= 400, Do() will return a nil response and a non-nil error.
	// Use StatusCodeFromError(err) to retrieve the code from the error.
	// Use WithDisableRestErrors() to disable this middleware on your client.
	// Use WithErrorDecoder(errorDecoder) to replace this default behavior with custom error decoding behavior.
	Do(ctx context.Context, params ...RequestParam) (*http.Response, error)

	Get(ctx context.Context, params ...RequestParam) (*http.Response, error)
	Head(ctx context.Context, params ...RequestParam) (*http.Response, error)
	Post(ctx context.Context, params ...RequestParam) (*http.Response, error)
	Put(ctx context.Context, params ...RequestParam) (*http.Response, error)
	Delete(ctx context.Context, params ...RequestParam) (*http.Response, error)
}

A Client executes requests to a configured service.

The Get/Head/Post/Put/Delete methods are for conveniently setting the method type and calling Do()

func NewClient 

func NewClient(params ...ClientParam) (Client, error)

NewClient returns a configured client ready for use. We apply "sane defaults" before applying the provided params.

type ClientConfig 

type ClientConfig struct {
	ServiceName string `json:"-" yaml:"-"`
	// URIs is a list of fully specified base URIs for the service. These can optionally include a path
	// which will be prepended to the request path specified when invoking the client.
	URIs []string `json:"uris,omitempty" yaml:"uris,omitempty"`
	// APIToken is a string which, if provided, will be used as a Bearer token in the Authorization header.
	// This takes precedence over APITokenFile.
	APIToken *string `json:"api-token,omitempty" yaml:"api-token,omitempty"`
	// APITokenFile is an on-disk location containing a Bearer token. If APITokenFile is provided and APIToken
	// is not, the content of the file will be used as the APIToken.
	APITokenFile *string `json:"api-token-file,omitempty" yaml:"api-token-file,omitempty"`
	// DisableHTTP2, if true, will prevent the client from modifying the *tls.Config object to support H2 connections.
	DisableHTTP2 *bool `json:"disable-http2,omitempty" yaml:"disable-http2,omitempty"`
	// ProxyFromEnvironment enables reading HTTP proxy information from environment variables.
	// See 'http.ProxyFromEnvironment' documentation for specific behavior.
	ProxyFromEnvironment *bool `json:"proxy-from-environment,omitempty" yaml:"proxy-from-environment,omitempty"`
	// ProxyURL uses the provided URL for proxying the request. Schemes http, https, and socks5 are supported.
	ProxyURL *string `json:"proxy-url,omitempty" yaml:"proxy-url,omitempty"`

	// MaxNumRetries controls the number of times the client will retry retryable failures.
	// If unset, this defaults to twice the number of URIs provided.
	MaxNumRetries *int `json:"max-num-retries,omitempty" yaml:"max-num-retries,omitempty"`
	// InitialBackoff controls the duration of the first backoff interval. This delay will double for each subsequent backoff, capped at the MaxBackoff value.
	InitialBackoff *time.Duration `json:"initial-backoff,omitempty" yaml:"initial-backoff,omitempty"`
	// MaxBackoff controls the maximum duration the client will sleep before retrying a request.
	MaxBackoff *time.Duration `json:"max-backoff,omitempty" yaml:"max-backoff,omitempty"`

	// ConnectTimeout is the maximum time for the net.Dialer to connect to the remote host.
	ConnectTimeout *time.Duration `json:"connect-timeout,omitempty" yaml:"connect-timeout,omitempty"`
	// ReadTimeout is the maximum timeout for non-mutating requests.
	// NOTE: The current implementation uses the max(ReadTimeout, WriteTimeout) to set the http.Client timeout value.
	ReadTimeout *time.Duration `json:"read-timeout,omitempty" yaml:"read-timeout,omitempty"`
	// WriteTimeout is the maximum timeout for mutating requests.
	// NOTE: The current implementation uses the max(ReadTimeout, WriteTimeout) to set the http.Client timeout value.
	WriteTimeout *time.Duration `json:"write-timeout,omitempty" yaml:"write-timeout,omitempty"`
	// IdleConnTimeout sets the timeout for idle connections.
	IdleConnTimeout *time.Duration `json:"idle-conn-timeout,omitempty" yaml:"idle-conn-timeout,omitempty"`
	// TLSHandshakeTimeout sets the timeout for TLS handshakes
	TLSHandshakeTimeout *time.Duration `json:"tls-handshake-timeout,omitempty" yaml:"tls-handshake-timeout,omitempty"`
	// IdleConnTimeout sets the timeout to receive the server's first response headers after
	// fully writing the request headers if the request has an "Expect: 100-continue" header.
	ExpectContinueTimeout *time.Duration `json:"expect-continue-timeout,omitempty" yaml:"expect-continue-timeout,omitempty"`

	// Metrics allows disabling metric emission or adding additional static tags to the client metrics.
	Metrics MetricsConfig `json:"metrics,omitempty" yaml:"metrics,omitempty"`
	// Security configures the TLS configuration for the client. It accepts file paths which should be
	// absolute paths or relative to the process's current working directory.
	Security SecurityConfig `json:"security,omitempty" yaml:"security,omitempty"`
}

ClientConfig represents the configuration for a single REST client.

type ClientOrHTTPClientParam 

type ClientOrHTTPClientParam interface {
	ClientParam
	HTTPClientParam
}

ClientOrHTTPClientParam is a param that can be used to build a Client or an http.Client

func WithAddHeader 

func WithAddHeader(key, value string) ClientOrHTTPClientParam

func WithAuthToken 

func WithAuthToken(bearerToken string) ClientOrHTTPClientParam

WithAuthToken sets the Authorization header to a static bearerToken.

func WithAuthTokenProvider 

func WithAuthTokenProvider(provideToken TokenProvider) ClientOrHTTPClientParam

WithAuthTokenProvider calls provideToken() and sets the Authorization header.

func WithBytesBufferPool 

func WithBytesBufferPool(pool bytesbuffers.Pool) ClientOrHTTPClientParam

WithBytesBufferPool stores a bytes buffer pool on the client for use in encoding request bodies. This prevents allocating a new byte buffer for every request.

func WithDialTimeout 

func WithDialTimeout(timeout time.Duration) ClientOrHTTPClientParam

WithDialTimeout sets the timeout on the Dialer. If unset, the client defaults to 30 seconds.

func WithDisableHTTP2 

func WithDisableHTTP2() ClientOrHTTPClientParam

WithDisableHTTP2 skips the default behavior of configuring the transport with http2.ConfigureTransport.

func WithDisablePanicRecovery 

func WithDisablePanicRecovery() ClientOrHTTPClientParam

WithDisablePanicRecovery disables the enabled-by-default panic recovery middleware. If the request was otherwise succeeding (err == nil), we return a new werror with the recovered object as an unsafe param. If there's an error, we werror.Wrap it. If errMiddleware is not nil, it is invoked on the recovered object.

func WithDisableTracing 

func WithDisableTracing() ClientOrHTTPClientParam

WithDisableTracing disables the enabled-by-default tracing middleware which instructs the client to propagate trace information using the go-zipkin libraries method of attaching traces to requests. The server at the other end of such a request, should be instrumented to read zipkin-style headers

If a trace is already attached to a request context, then the trace is continued. Otherwise, no trace information is propagate. This will not create a span if one does not exist.

func WithExpectContinueTimeout 

func WithExpectContinueTimeout(timeout time.Duration) ClientOrHTTPClientParam

WithExpectContinueTimeout sets the timeout to receive the server's first response headers after fully writing the request headers if the request has an "Expect: 100-continue" header. If unset, the client defaults to 1 second.

func WithHTTPTimeout 

func WithHTTPTimeout(timeout time.Duration) ClientOrHTTPClientParam

WithHTTPTimeout sets the timeout on the http client. If unset, the client defaults to 1 minute.

func WithIdleConnTimeout 

func WithIdleConnTimeout(timeout time.Duration) ClientOrHTTPClientParam

WithIdleConnTimeout sets the timeout for idle connections. If unset, the client defaults to 90 seconds.

func WithKeepAlive 

func WithKeepAlive(keepAlive time.Duration) ClientOrHTTPClientParam

WithKeepAlive sets the keep alive frequency on the Dialer. If unset, the client defaults to 30 seconds.

func WithMaxIdleConns 

func WithMaxIdleConns(conns int) ClientOrHTTPClientParam

WithMaxIdleConns sets the number of reusable TCP connections the client will maintain. If unset, the client defaults to 32.

func WithMaxIdleConnsPerHost 

func WithMaxIdleConnsPerHost(conns int) ClientOrHTTPClientParam

WithMaxIdleConnsPerHost sets the number of reusable TCP connections the client will maintain per destination. If unset, the client defaults to 32.

func WithMetrics 

func WithMetrics(tagProviders ...TagsProvider) ClientOrHTTPClientParam

WithMetrics enables the "client.response" metric. See MetricsMiddleware for details. The serviceName will appear as the "service-name" tag.

func WithMiddleware 

func WithMiddleware(h Middleware) ClientOrHTTPClientParam

WithMiddleware will be invoked for custom HTTP behavior after the underlying transport is initialized. Each handler added "wraps" the previous round trip, so it will see the request first and the response last.

func WithNoProxy 

func WithNoProxy() ClientOrHTTPClientParam

WithNoProxy nils out the Proxy field of the http.Transport, ignoring any proxy set in the process's environment. If unset, the default is http.ProxyFromEnvironment.

func WithProxyFromEnvironment 

func WithProxyFromEnvironment() ClientOrHTTPClientParam

WithProxyFromEnvironment can be used to set the HTTP(s) proxy to use the Go standard library's http.ProxyFromEnvironment.

func WithProxyURL 

func WithProxyURL(proxyURLString string) ClientOrHTTPClientParam

WithProxyURL can be used to set a socks5 or HTTP(s) proxy.

func WithResponseHeaderTimeout 

func WithResponseHeaderTimeout(timeout time.Duration) ClientOrHTTPClientParam

WithResponseHeaderTimeout specifies the amount of time to wait for a server's response headers after fully writing the request (including its body, if any). This time does not include the time to read the response body. If unset, the client defaults to having no response header timeout.

func WithServiceName 

func WithServiceName(serviceName string) ClientOrHTTPClientParam

func WithSetHeader 

func WithSetHeader(key, value string) ClientOrHTTPClientParam

func WithTLSConfig 

func WithTLSConfig(conf *tls.Config) ClientOrHTTPClientParam

WithTLSConfig sets the SSL/TLS configuration for the HTTP client's Transport using a copy of the provided config. The palantir/pkg/tlsconfig package is recommended to build a tls.Config from sane defaults.

func WithTLSHandshakeTimeout 

func WithTLSHandshakeTimeout(timeout time.Duration) ClientOrHTTPClientParam

WithTLSHandshakeTimeout sets the timeout for TLS handshakes. If unset, the client defaults to 10 seconds.

func WithUserAgent 

func WithUserAgent(userAgent string) ClientOrHTTPClientParam

WithUserAgent sets the User-Agent header.

type ClientParam 

type ClientParam interface {
	// contains filtered or unexported methods
}

ClientParam is a param that can be used to build

func WithBaseURLs 

func WithBaseURLs(urls []string) ClientParam

WithBaseURLs sets the base URLs for every request. This is meant to be used in conjunction with WithPath.

func WithBasicAuth 

func WithBasicAuth(username, password string) ClientParam

WithBasicAuth sets the request's Authorization header to use HTTP Basic Authentication with the provided username and password.

func WithConfig 

func WithConfig(c ClientConfig) ClientParam

func WithDisableRestErrors 

func WithDisableRestErrors() ClientParam

WithDisableRestErrors disables the middleware which sets Do()'s returned error to a non-nil value in the case of >= 400 HTTP response.

func WithDisableTraceHeaderPropagation 

func WithDisableTraceHeaderPropagation() ClientParam

WithDisableTraceHeaderPropagation disables the enabled-by-default traceId header propagation By default, if witchcraft-logging has attached a traceId to the context of the request (for service and request logging), then the client will attach this traceId as a header for future services to do the same if desired

func WithErrorDecoder 

func WithErrorDecoder(errorDecoder ErrorDecoder) ClientParam

func WithInitialBackoff 

func WithInitialBackoff(initialBackoff time.Duration) ClientParam

WithInitialBackoff sets the initial backoff between retried calls to the same URI. Defaults to 250ms.

func WithMaxBackoff 

func WithMaxBackoff(maxBackoff time.Duration) ClientParam

WithMaxBackoff sets the maximum backoff between retried calls to the same URI. Defaults to no limit.

func WithMaxRetries 

func WithMaxRetries(maxTransportRetries int) ClientParam

WithMaxRetries sets the maximum number of retries on transport errors for every request. Backoffs are also capped at this. If unset, the client defaults to 2 * size of URIs

type ErrorDecoder 

type ErrorDecoder interface {
	// Handles returns whether or not the decoder considers the response an error.
	Handles(resp *http.Response) bool
	// DecodeError returns a decoded error, or an error encountered while trying to decode.
	// DecodeError should never return nil.
	DecodeError(resp *http.Response) error
}

ErrorDecoder implementations declare whether or not they should be used to handle certain http responses, and return decoded errors when invoked. Custom implementations can be used when consumers expect structured errors in response bodies.

type HTTPClientParam 

type HTTPClientParam interface {
	// contains filtered or unexported methods
}

func WithConfigForHTTPClient 

func WithConfigForHTTPClient(c ClientConfig) HTTPClientParam

type MetricsConfig 

type MetricsConfig struct {
	// Enabled can be used to disable metrics with an explicit 'false'. Metrics are enabled if this is unset.
	Enabled *bool `json:"enabled,omitempty" yaml:"enabled,omitempty"`
	// Tags allows setting arbitrary additional tags on the metrics emitted by the client.
	Tags map[string]string `json:"tags,omitempty" yaml:"tags,omitempty"`
}

type Middleware 

type Middleware interface {
	// RoundTrip mimics the API of http.RoundTripper, but adds a 'next' argument.
	// RoundTrip is responsible for invoking next.RoundTrip(req) and returning the response.
	RoundTrip(req *http.Request, next http.RoundTripper) (*http.Response, error)
}

A Middleware wraps an http client's request and is able to read or modify the request and response.

func MetricsMiddleware 

func MetricsMiddleware(serviceName string, tagProviders ...TagsProvider) (Middleware, error)

MetricsMiddleware updates the "client.response" timer metric on every request. By default, metrics are tagged with 'service-name', 'method', and 'family' (of the status code). This metric name and tag set matches http-remoting's DefaultHostMetrics: https://github.com/palantir/http-remoting/blob/develop/okhttp-clients/src/main/java/com/palantir/remoting3/okhttp/DefaultHostMetrics.java

type MiddlewareFunc 

type MiddlewareFunc func(req *http.Request, next http.RoundTripper) (*http.Response, error)

MiddlewareFunc is a convenience type alias that implements Middleware.

func (MiddlewareFunc) RoundTrip 

func (f MiddlewareFunc) RoundTrip(req *http.Request, next http.RoundTripper) (*http.Response, error)

type RequestParam 

type RequestParam interface {
	// contains filtered or unexported methods
}

func WithCompressedRequest 

func WithCompressedRequest(input interface{}, codec codecs.Codec) RequestParam

WithCompressedRequest wraps the 'codec'-encoded request body in zlib compression.

func WithHeader 

func WithHeader(key, value string) RequestParam

WithHeader sets a header on a request.

func WithJSONRequest 

func WithJSONRequest(input interface{}) RequestParam

WithJSONRequest sets the request body to the input marshaled using the JSON codec.

func WithJSONResponse 

func WithJSONResponse(output interface{}) RequestParam

WithJSONResponse unmarshals the response body using the JSON codec. The request will return an error if decoding fails.

func WithPath 

func WithPath(path string) RequestParam

WithPath sets the path for the request. This will be joined with one of the BaseURLs set on the client

func WithPathf 

func WithPathf(format string, args ...interface{}) RequestParam

WithPathf sets the path for the request. This will be joined with one of the BaseURLs set on the client

func WithQueryValues 

func WithQueryValues(query url.Values) RequestParam

WithQueryValues sets a header on a request.

func WithRPCMethodName 

func WithRPCMethodName(name string) RequestParam

WithRPCMethodName configures the requests's context with the RPC method name, like "GetServiceRevision". This is read by the tracing and metrics middlewares.

func WithRawRequestBody deprecated 

func WithRawRequestBody(input io.ReadCloser) RequestParam

WithRawRequestBody uses the provided io.ReadCloser as the request body. Example: 

input, _ := os.Open("file.txt")
resp, err := client.Do(..., WithRawRequestBody(input), ...)

Deprecated: Retries don't include the body for WithRawRequestBody. Use WithRawRequestBodyProvider for full retry support.

func WithRawRequestBodyProvider 

func WithRawRequestBodyProvider(getBody func() io.ReadCloser) RequestParam

WithRawRequestBodyProvider uses the io.ReadCloser provided by getBody as the request body. The getBody parameter must not be nil. Example: 

provider := func() io.ReadCloser {
    input, _ := os.Open("file.txt")
    return input
}
resp, err := client.Do(..., WithRawRequestBodyProvider(provider), ...)

func WithRawResponseBody 

func WithRawResponseBody() RequestParam

WithRawResponseBody configures the request such that the response body will not be read or drained after the request is executed. In this case, it is the responsibility of the caller to read and close the returned reader. Example: 

resp, err := client.Do(..., WithRawResponseBody(), ...)
defer resp.Body.Close()
bytes, err := ioutil.ReadAll(resp.Body)

In the case of an empty response, output will be unmodified (left nil).

func WithRequestBasicAuth 

func WithRequestBasicAuth(username, password string) RequestParam

WithRequestBasicAuth sets the request's Authorization header to use HTTP Basic Authentication with the provided username and password for this request only and takes precedence over any client-scoped authorization.

func WithRequestBody 

func WithRequestBody(input interface{}, encoder codecs.Encoder) RequestParam

WithRequestBody provides a struct to marshal and use as the request body. Encoding is handled by the impl passed to WithRequestBody. Example: 

input := api.RequestInput{Foo: "bar"}
resp, err := client.Do(..., WithRequestBody(input, codecs.JSON), ...)

func WithRequestErrorDecoder 

func WithRequestErrorDecoder(errorDecoder ErrorDecoder) RequestParam

WithRequestErrorDecoder sets an ErrorDecoder to use for this request only. It will take precedence over any ErrorDecoder set on the client. If this request-scoped ErrorDecoder does not handle the response, the client-scoped ErrorDecoder will be consulted in the usual way.

func WithRequestMethod 

func WithRequestMethod(method string) RequestParam

WithRequestMethod sets the HTTP method of the request, e.g. GET or POST.

func WithResponseBody 

func WithResponseBody(output interface{}, decoder codecs.Decoder) RequestParam

WithResponseBody provides a struct into which the body middleware will decode as the response body. Decoding is handled by the impl passed to WithResponseBody. Example: 

var output api.RequestOutput
resp, err := client.Do(..., WithResponseBody(&output, codecs.JSON), ...)
return output, nil

In the case of an empty response, output will be unmodified (left nil).

type SecurityConfig 

type SecurityConfig struct {
	CAFiles  []string `json:"ca-files,omitempty" yaml:"ca-files,omitempty"`
	CertFile string   `json:"cert-file,omitempty" yaml:"cert-file,omitempty"`
	KeyFile  string   `json:"key-file,omitempty" yaml:"key-file,omitempty"`
}

type ServicesConfig 

type ServicesConfig struct {
	// Default values will be used for any field which is not set for a specific client.
	Default ClientConfig `json:",inline" yaml:",inline"`
	// Services is a map of serviceName (e.g. "my-api") to service-specific configuration.
	Services map[string]ClientConfig `json:"services,omitempty" yaml:"services,omitempty"`
}

ServicesConfig is the top-level configuration struct for all HTTP clients. It supports setting default values and overriding those values per-service. Use ClientConfig(serviceName) to retrieve a specific service's configuration, and the httpclient.WithConfig() param to construct a Client using that configuration. The fields of this struct should generally not be read directly by application code.

func (ServicesConfig) ClientConfig 

func (c ServicesConfig) ClientConfig(serviceName string) ClientConfig

ClientConfig returns the default configuration merged with service-specific configuration. If the serviceName is not in the service map, an empty configuration (plus defaults) is used.

func (ServicesConfig) MustClientConfig 

func (c ServicesConfig) MustClientConfig(serviceName string) (ClientConfig, error)

MustClientConfig returns an error if the service name is not configured.

type TagsProvider 

type TagsProvider interface {
	Tags(*http.Request, *http.Response) metrics.Tags
}

A TagsProvider returns metrics tags based on an http round trip.

type TagsProviderFunc 

type TagsProviderFunc func(*http.Request, *http.Response) metrics.Tags

TagsProviderFunc is a convenience type that implements TagsProvider.

func (TagsProviderFunc) Tags 

func (f TagsProviderFunc) Tags(req *http.Request, resp *http.Response) metrics.Tags

type TokenProvider 

type TokenProvider func(context.Context) (string, error)

TokenProvider accepts a context and returns either:

(1) a nonempty token and a nil error, or

(2) an empty string and a non-nil error.

A good implementation will request and cache an ephemeral client token.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.