http2

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 8, 2021 License: BSD-3-Clause Imports: 20 Imported by: 0

Documentation

Overview

http2 is a collection of functions meant to supplement the capabilities provided by the standard "net/http" package.

Index

Constants

View Source 
const (
	GzipEncoding          string = "gzip"
	AcceptEncodingHeader         = "Accept-Encoding"
	ContentTypeHeader            = "Content-Type"
	ContentEncodingHeader        = "Content-Encoding"
	ConnectionHeader             = "Connection"
	CloseConnection              = "close"
	KeepAliveConnection          = "keep-alive"
)
View Source 
const (
	// Reasonable default for HTTP connect timeouts
	DefaultConnectTimeout = time.Second

	// Reasonable default for HTTP timeouts
	DefaultTimeout = 5 * time.Second

	// Reasonable default for maximum idle connections
	DefaultMaxIdle = 10
)

Variables

This section is empty.

Functions

func AcceptsGzipResponse 

func AcceptsGzipResponse(httpRequest *http.Request) bool

func NewGzipResponseWriter 

func NewGzipResponseWriter(writer http.ResponseWriter, compressionLevel int) gzipResponseWriter

compressionLevel - one of the compression levels in the gzip package.

Types

type ConnectionParams 

type ConnectionParams struct {
	// Name of the connection pool. It's useful to identify connection-pool stats for different
	// services
	Name string

	// The maximum number of concurrent connections that can be open by the
	// pool.  Non-positive means unlimited.
	MaxConns int

	// Number of idle HTTP clients we allow to remain in the pool
	MaxIdle int

	// Use SSL transport?
	UseSSL bool

	// The tls config to use for the transport.
	TLSClientConfig *tls.Config

	// Timeout for acquiring connection in case we hit MaxConns. Only applicable if MaxConns > 0
	ConnectionAcquireTimeout time.Duration

	// Timeout for connection (includes DNS resolution)
	ConnectTimeout time.Duration

	// Timeout for waiting for an HTTP response header
	ResponseTimeout time.Duration

	// Host header to use instead of address.
	HostHeader *string

	// When true, and http.Request.Host is not empty, set http.Request.URL.Host
	// to http.Request.Host.  Otherwise, set http.Request.URL.Host to
	// the pool's address (or HostHeader).
	UseRequestHost bool

	// When true, do not follow redirects.  When false, use the http.Client's
	// default policy, which follows at most 10 consecutive requests.
	DisableFollowRedirect bool

	// Dial function to use instead of the default
	Dial func(network, addr string) (net.Conn, error)

	// Function to determine proxy
	Proxy func(*http.Request) (*url.URL, error)

	// For logging stats
	StatsFactory stats.StatsFactory
}

func DefaultPoolParams 

func DefaultPoolParams() ConnectionParams

func (ConnectionParams) String 

func (p ConnectionParams) String() string

type ConnectionTracker 

type ConnectionTracker struct {
	// contains filtered or unexported fields
}

func NewConnectionTracker 

func NewConnectionTracker(
	maxConnections int,
	dial DialFunc,
	statsFactory stats.StatsFactory) *ConnectionTracker

func (*ConnectionTracker) Dial 

func (c *ConnectionTracker) Dial(
	network string,
	addr string) (
	net.Conn,
	error)

func (*ConnectionTracker) DisallowNewConn 

func (c *ConnectionTracker) DisallowNewConn()

func (*ConnectionTracker) ForceCloseAll 

func (c *ConnectionTracker) ForceCloseAll()

func (*ConnectionTracker) NumAlive 

func (c *ConnectionTracker) NumAlive() int

type ConsistentHashFunc 

type ConsistentHashFunc func(key []byte, seed uint32) uint32

type DialError 

type DialError struct {
	errors.DropboxError
}

type DialFunc 

type DialFunc func(network string, add string) (net.Conn, error)

type DoParams 

type DoParams struct {
	// Timeout is optional and specifies timeout for Do operation
	Timeout time.Duration
	// Key is optional for most pools except those that shard requests
	// (i.e., consistent-hashing)
	Key []byte
	// MaxInstances specifies how many instances to try at most. It's only
	// relevant if a key is specified. This is useful for cases where we
	// need to balance the load between multiple instances for the same key
	MaxInstances int
}

type LBPoolInstanceInfo 

type LBPoolInstanceInfo struct {
	// Optional InstanceId that can later on be used to look up specific instances
	// from the LoadBalancedPool.
	InstanceId int
	Addr       string
}

type LBStrategy 

type LBStrategy int
const (
	// In 'RoundRobin' load balancing strategy requests are sent to
	// different hosts in round robin fashion.
	//
	// Note: Order of hosts to try is changed after each update.
	LBRoundRobin LBStrategy = 0
	// In 'SortedFixed' load balancing strategy requests are routed to same host,
	// others are used only in case of failover. Order of hosts to try is determined
	// by instance id.
	//
	// Note: Order of hosts to try is the same for all instances of LoadBalancedPool.
	LBSortedFixed LBStrategy = 1
	// In 'ShuffledFixed' load balancing strategy requests are routed to same host,
	// others are used only in case of failover. Order of hosts to try is determined
	// by instance id and shuffle seed which is picked at pool's initialization.
	//
	// Note: Order of hosts to try is specific to instance of LoadBalancedPool.
	LBShuffledFixed LBStrategy = 2
	// In 'LBConsistentHashing' strategy, requests will be routed to host(s) that are
	// the closest to the hash for the request key in the consistent hash ring. The number
	// of closest hosts to send the request to can be more than 1 in order to load-balance
	// between the different hosts, which is important in the case of hot-keys. Load balancing
	// is done by picking a random host from the closest set of host(s). The default behavior
	// is to use only a single host.
	// LBConsistentHashing will prefer availability over consistency. In the case some server
	// is down, it will try new servers in the hash-ring
	LBConsistentHashing LBStrategy = 3
)

type LoadBalancedPool 

type LoadBalancedPool struct {
	// contains filtered or unexported fields
}

func NewLoadBalancedPool 

func NewLoadBalancedPool(params LoadBalancedPoolParams) *LoadBalancedPool

func (*LoadBalancedPool) Close 

func (pool *LoadBalancedPool) Close()

func (*LoadBalancedPool) CloseIdleConnections 

func (pool *LoadBalancedPool) CloseIdleConnections()

func (*LoadBalancedPool) Do 

func (pool *LoadBalancedPool) Do(req *http.Request) (resp *http.Response, err error)

func (*LoadBalancedPool) DoWithParams 

func (pool *LoadBalancedPool) DoWithParams(
	req *http.Request,
	params DoParams) (*http.Response, error)

func (*LoadBalancedPool) DoWithTimeout 

func (pool *LoadBalancedPool) DoWithTimeout(req *http.Request,
	timeout time.Duration) (*http.Response, error)

Issues an HTTP request, distributing more load to relatively unloaded instances.

func (*LoadBalancedPool) Get 

func (pool *LoadBalancedPool) Get() (*http.Client, error)

Checks out an HTTP connection from an instance pool, favoring less loaded instances.

func (*LoadBalancedPool) GetInstancePool 

func (pool *LoadBalancedPool) GetInstancePool(instanceId int) (*SimplePool, error)

Returns a SimplePool for given instanceId, or an error if it does not exist. TODO(zviad): right now this scans all instances, thus if there are a lot of instances per partition it can become very slow. If it becomes a problem, fix it!

func (*LoadBalancedPool) GetSingleInstance 

func (pool *LoadBalancedPool) GetSingleInstance() (Pool, error)

Returns a Pool for an instance selected based on load balancing strategy.

func (*LoadBalancedPool) GetWithKey 

func (pool *LoadBalancedPool) GetWithKey(key []byte, limit int) (*http.Client, error)

func (*LoadBalancedPool) HasInstances 

func (pool *LoadBalancedPool) HasInstances() bool

func (*LoadBalancedPool) SetActiveSetSize 

func (pool *LoadBalancedPool) SetActiveSetSize(size uint64)

For the round robin strategy, sets the number of servers to round-robin between. The default is 6.

func (*LoadBalancedPool) SetMarkDownDuration 

func (pool *LoadBalancedPool) SetMarkDownDuration(duration time.Duration)

When a server returns a 500, we mark it unusable. Configure how long we will avoid sending requests to it. Must be called before the pool is used.

func (*LoadBalancedPool) SetStrategy 

func (pool *LoadBalancedPool) SetStrategy(strategy LBStrategy)

Sets Load Balancing strategy. Must be called before pool is actually put to use.

func (*LoadBalancedPool) Update 

func (pool *LoadBalancedPool) Update(instanceInfos []LBPoolInstanceInfo)

type LoadBalancedPoolParams 

type LoadBalancedPoolParams struct {
	// Parameters for creating SimplePool-s.
	ConnParams ConnectionParams

	// How long to mark a server unusable for.
	MarkDownDuration time.Duration
	// Load balancing strategy.
	Strategy LBStrategy
	// Number of instances to round-robin between
	ActiveSetSize uint64
	// Specifies consistent hash function to use.
	HashFunction ConsistentHashFunc
	// Specifies the seed for hashing.
	HashSeed uint32
}

func DefaultConsistentHashPoolParams 

func DefaultConsistentHashPoolParams(
	hashFunc ConsistentHashFunc, hashSeed uint32) LoadBalancedPoolParams

func DefaultLoadBalancedPoolParams 

func DefaultLoadBalancedPoolParams() LoadBalancedPoolParams

type Pool 

type Pool interface {
	// Similar interface as net/http.Client.Do()
	// Most important note is that: Callers should close resp.Body
	// when done reading from it. If resp.Body is not closed, the
	// Client's underlying RoundTripper (typically Transport) may not
	// be able to re-use a persistent TCP connection to the server
	// for a subsequent "keep-alive" request.
	Do(*http.Request) (*http.Response, error)

	// Perform request and properly tear down connection if it times out.
	DoWithTimeout(*http.Request, time.Duration) (*http.Response, error)

	// Provides a more generic Do with extra options
	DoWithParams(*http.Request, DoParams) (*http.Response, error)

	// Returns http.Client to perform http requests with, preferable
	// to just use Do() function instead of this.
	Get() (*http.Client, error)

	// Returns http.Client to perform http requests with, preferable
	// to just use Do() function instead of this.
	GetWithKey(key []byte, limit int) (*http.Client, error)

	// Closes idle connections.  Active connections are uneffected.
	// The user may continue to use the pool for further processing.
	CloseIdleConnections()

	// Closes underlying connections.  The user must abandon the
	// pool after closing.
	Close()
}

A generic interface for HTTP connection pools

type SimplePool 

type SimplePool struct {
	// contains filtered or unexported fields
}

Pool of persistent HTTP connections. The only limit is on the max # of idle connections we cache. Like Python's dropbox.curllib.CurlConnectionPool.

func NewSimplePool 

func NewSimplePool(addr string, params ConnectionParams) *SimplePool

Creates a new HTTP connection pool using the given address and pool parameters.

'addr' is a net.Dial()-style 'host:port' destination for making the TCP connection for HTTP/HTTPS traffic. It will be used as the hostname by default for virtual hosting and SSL certificate validation; if you'd like to use a different hostname, set params.HostHeader.

func (*SimplePool) Addr 

func (pool *SimplePool) Addr() string

func (*SimplePool) Close 

func (pool *SimplePool) Close()

func (*SimplePool) CloseIdleConnections 

func (pool *SimplePool) CloseIdleConnections()

Closes all idle connections in this pool

func (*SimplePool) Do 

func (pool *SimplePool) Do(req *http.Request) (resp *http.Response, err error)

Performs the HTTP request using our HTTP client

func (*SimplePool) DoWithParams 

func (pool *SimplePool) DoWithParams(
	req *http.Request,
	params DoParams) (resp *http.Response, err error)

func (*SimplePool) DoWithTimeout 

func (pool *SimplePool) DoWithTimeout(req *http.Request,
	timeout time.Duration) (resp *http.Response, err error)

Set a local timeout the actually cancels the request if we've given up.

func (*SimplePool) Get 

func (pool *SimplePool) Get() (*http.Client, error)

Returns the HTTP client, which is thread-safe.

Note that we use http.Client, rather than httputil.ClientConn, despite http.Client being higher- level. This is normally a liability for backend code, but it has more robust error handling and provides functionality that's more comparable to pycurl/curllib.

func (*SimplePool) GetWithKey 

func (pool *SimplePool) GetWithKey(key []byte, limit int) (*http.Client, error)

SimplePool doesn't care about the key

func (*SimplePool) Params 

func (pool *SimplePool) Params() ConnectionParams

func (*SimplePool) Transport 

func (pool *SimplePool) Transport() *http.Transport

Source Files

View all Source files

Directories

Path Synopsis
Utility functions for testing net2/http2
 Utility functions for testing net2/http2

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.