client

package
v0.3.0-docs.0...-f96dd80 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2019 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package client implements the client to interact with QED servers.

Index

Constants

View Source 
const (
	// DefaultTimeout is the default number of seconds to wait for a request to QED.
	DefaultTimeout = 10 * time.Second

	// DefaultDialTimeout is the default number of seconds to wait for the connection
	// to be established.
	DefaultDialTimeout = 5 * time.Second

	// DefaultHandshakeTimeout is the default number of seconds to wait for a handshake
	// negotiation.
	DefaultHandshakeTimeout = 5 * time.Second

	// DefaultInsecure sets if the client verifies, by default, the server's
	// certificate chain and host name, allowing MiTM vector attacks.
	DefaultInsecure = false

	// DefaultMaxRetries sets the default maximum number of retries before giving up
	// when performing an HTTP request to QED.
	DefaultMaxRetries = 0

	// DefaultHealthCheckEnabled specifies if healthchecks are enabled by default.
	DefaultHealthCheckEnabled = true

	// DefaultHealthCheckTimeout specifies the time the healtch checker waits for
	// a response from QED.
	DefaultHealthCheckTimeout = 2 * time.Second

	// DefaultHealthCheckInterval is the default interval between two health checks
	// of the nodes in the QED cluster.
	DefaultHealthCheckInterval = 60 * time.Second

	// DefaultTopologyDiscoveryEnabled specifies if the discoverer is enabled by default.
	DefaultTopologyDiscoveryEnabled = true
)

Variables

View Source 
var (
	// ErrNoEndpoint is raised when no QED node is available.
	ErrNoEndpoint = errors.New("no QED node available")

	// ErrNoPrimary is raised when no QED primary node is available.
	ErrNoPrimary = errors.New("no QED primary node available")

	// ErrNoPrimary is raised when the current QED primary node is dead.
	ErrPrimaryDead = errors.New("current QED primary node is dead")

	// ErrRetry is raised when a request cannot be executed after
	// the configured number of retries.
	ErrRetry = errors.New("cannot connect after serveral retries")

	// ErrTimeout is raised when a request timed out.
	ErrTimeout = errors.New("timeout")
)

Functions

func NewTestHttpClient 

func NewTestHttpClient(fn RoundTripFunc) *http.Client

NewTestHttpClient returns *http.Client with Transport replaced to avoid making real calls

Types

type Backoff 

type Backoff interface {
	// Next implements a BackoffF.
	Next(attempt int) (time.Duration, bool)
}

Backoff allows callers to implement their own Backoff strategy.

type BackoffF 

type BackoffF func(attempt int) (time.Duration, bool)

BackoffF specifies the signature of a function that returns the time to wait before the next call to a resource. To stop retrying return false in the 2nd return value.

type BackoffRequestRetrier 

type BackoffRequestRetrier struct {
	*http.Client
	// contains filtered or unexported fields
}

BackoffRequestRetrier is an implementation that uses the given backoff strategy.

func NewBackoffRequestRetrier 

func NewBackoffRequestRetrier(httpClient *http.Client, maxRetries int, backoff Backoff) *BackoffRequestRetrier

NewBackoffRequestRetrier returns a retrier that uses the given backoff strategy.

func (*BackoffRequestRetrier) DoReq 

func (r *BackoffRequestRetrier) DoReq(req *RetriableRequest) (*http.Response, error)

type Config 

type Config struct {
	// Log level
	Log string `desc:"Set log level to info, error or debug"`

	// Endpoints [host:port,host:port,...] to ask for QED cluster-topology.
	Endpoints []string `desc:"REST QED Log service endpoint list http://ip1:port1,http://ip2:port2... "`

	// Snapshot store [host:port] to ask for QED published signed snapshots.
	SnapshotStoreURL string `desc:"REST Snapshot store service endpoint http://ip:port "`

	// ApiKey to query the server endpoint.
	APIKey string `desc:"Set API Key to talk to QED Log service"`

	// Insecure enables the verification of the server's certificate chain
	// and host name, allowing MiTM vector attacks.
	Insecure bool `desc:"Set it to true to disable the verification of the server's certificate chain"`

	// Timeout is the time to wait for a request to QED.
	Timeout time.Duration `desc:"Time to wait for a request to QED"`

	// DialTimeout is the time to wait for the connection to be established.
	DialTimeout time.Duration `desc:"Time to wait for the connection to be established"`

	// HandshakeTimeout is the time to wait for a handshake negotiation.
	HandshakeTimeout time.Duration `desc:"Time to wait for a handshake negotiation"`

	// Controls how the client will route all queries to members of the cluster.
	ReadPreference ReadPref `flag:"-"`

	// MaxRetries sets the maximum number of retries before giving up
	// when performing an HTTP request to QED.
	MaxRetries int `desc:"Sets the maximum number of retries before giving up"`

	// EnableTopologyDiscovery enables the process of discovering the cluster
	// topology when requests fail.
	EnableTopologyDiscovery bool `desc:"Enables the process of discovering the cluster topology when requests fail"`

	// EnableHealthChecks enables helthchecks of all endpoints in the current cluster topology.
	EnableHealthChecks bool `desc:"Enables helthchecks of all endpoints in the current cluster topology"`

	// HealthCheckTimeout is the time the healthcheck waits for a response
	// from a QED server.
	HealthCheckTimeout time.Duration `desc:"Time the healthcheck waits for a response from QED"`

	// HealthCheckInterval is the interval between two health checks of the nodes in the QED cluster.
	HealthCheckInterval time.Duration `desc:"Interval between two health checks of the nodes in the QED cluster"`

	// AttemptToReviveEndpoints sets if dead endpoints will be marked alive again after a
	// round-robin round. This way, they will be picked up in the next try.
	AttemptToReviveEndpoints bool `desc:"Set if dead endpoints will be marked alive again after a round-robin round"`

	// HasherFunction sets which function will use the client to do its work: verify, ask for proofs, ...
	HasherFunction func() hashing.Hasher `desc:"Hashing function to verify proofs"`
}

Config sets the HTTP client configuration

func DefaultConfig 

func DefaultConfig() *Config

DefaultConfig creates a Config structures with default values.

type ConstantBackoff 

type ConstantBackoff struct {
	// contains filtered or unexported fields
}

ConstantBackoff is a backoff policy that always returns the same delay.

func NewConstantBackoff 

func NewConstantBackoff(interval time.Duration) *ConstantBackoff

NewConstantBackoff returns a new ConstantBackoff.

func (*ConstantBackoff) Next 

func (b *ConstantBackoff) Next(attempt int) (time.Duration, bool)

Next implements BackoffF for ConstantBackoff.

type ExponentialBackoff 

type ExponentialBackoff struct {
	// contains filtered or unexported fields
}

ExponentialBackoff implements the simple exponential backoff described by Douglas Thain at http://dthain.blogspot.de/2009/02/exponential-backoff-in-distributed.html.

func NewExponentialBackoff 

func NewExponentialBackoff(initialTimeout, maxTimeout time.Duration) *ExponentialBackoff

NewExponentialBackoff returns a ExponentialBackoff backoff policy. Use initialTimeout to set the first/minimal interval and maxTimeout to set the maximum wait interval.

func (*ExponentialBackoff) Next 

func (b *ExponentialBackoff) Next(attempt int) (time.Duration, bool)

Next implements BackoffF for ExponentialBackoff.

type HTTPClient 

type HTTPClient struct {
	// contains filtered or unexported fields
}

HTTPClient is an HTTP QED client.

func NewHTTPClient 

func NewHTTPClient(options ...HTTPClientOptionF) (*HTTPClient, error)

NewHTTPClient creates a new HTTP client to work with QED.

The client, by default, is meant to be long-lived and shared across your application. If you need a short-lived client, e.g. for request-scope, consider using NewSimpleHttpClient instead.

func NewHTTPClientFromConfig 

func NewHTTPClientFromConfig(conf *Config) (*HTTPClient, error)

NewHTTPClientFromConfig initializes a client from a configuration.

func NewSimpleHTTPClient 

func NewSimpleHTTPClient(httpClient *http.Client, urls []string, snapshotStoreURL string) (*HTTPClient, error)

NewSimpleHTTPClient creates a new short-lived client thath can be used in use cases where you need one client per request.

All checks are disabled, including timeouts and periodic checks. The number of retries is set to 0.

func (*HTTPClient) Add 

func (c *HTTPClient) Add(event string) (*protocol.Snapshot, error)

Add will do a request to the server with a post data to store a new event.

func (*HTTPClient) AddBulk 

func (c *HTTPClient) AddBulk(events []string) ([]*protocol.Snapshot, error)

AddBulk will do a request to the server with a post data to store a bulk of new events.

func (*HTTPClient) Close 

func (c *HTTPClient) Close()

Close stops the background processes that the client is running, i.e. sniffing the cluster periodically and running health checks on the nodes.

If the background processes are not running, this is a no-op.

func (*HTTPClient) GetSnapshot 

func (c *HTTPClient) GetSnapshot(version uint64) (*protocol.Snapshot, error)

GetSnapshot will ask for a given snapshot version to the snapshot store and returns the required snapshot

func (*HTTPClient) Incremental 

func (c *HTTPClient) Incremental(start, end uint64) (*balloon.IncrementalProof, error)

Incremental will ask for an IncrementalProof to the server.

func (*HTTPClient) IncrementalAutoVerify 

func (c *HTTPClient) IncrementalAutoVerify(
	start, end uint64,
) (bool, error)

IncrementalAutoVerify will ask for an Incremental proof to the server, given both a start and end versions. With these versions, it will ask to the snapshot store to get both snapshots, and finally it will verify the proof. It returns the verification result.

func (*HTTPClient) IncrementalVerify 

func (c *HTTPClient) IncrementalVerify(
	proof *balloon.IncrementalProof,
	startSnapshot, endSnapshot *balloon.Snapshot,
) (bool, error)

IncrementalVerify will verify a proof against two snapshots, given these 3 elements. It returns the verification result.

func (*HTTPClient) Membership 

func (c *HTTPClient) Membership(key []byte, version *uint64) (*balloon.MembershipProof, error)

Membership will ask for a Proof to the server.

func (*HTTPClient) MembershipAutoVerify 

func (c *HTTPClient) MembershipAutoVerify(eventDigest hashing.Digest, version *uint64) (bool, error)

MembershipAutoVerify will compute the Proof given in Membership, get hyper and history digests from the snapshot store, and returns the verification result.

func (*HTTPClient) MembershipDigest 

func (c *HTTPClient) MembershipDigest(keyDigest hashing.Digest, version *uint64) (*balloon.MembershipProof, error)

Membership will ask for a Proof to the server.

func (*HTTPClient) MembershipVerify 

func (c *HTTPClient) MembershipVerify(
	eventDigest hashing.Digest,
	proof *balloon.MembershipProof,
	snapshot *balloon.Snapshot,
) (bool, error)

MembershipVerify will compute the Proof given in Membership and the snapshot from the add and returns the verification result.

func (*HTTPClient) Ping 

func (c *HTTPClient) Ping() error

Ping will do a healthcheck request to the primary node

type HTTPClientOptionF 

type HTTPClientOptionF func(*HTTPClient) error

HTTPClientOptionF is a function that configures an HTTPClient.

func SetAPIKey 

func SetAPIKey(key string) HTTPClientOptionF

func SetAttemptToReviveEndpoints 

func SetAttemptToReviveEndpoints(value bool) HTTPClientOptionF

func SetHasherFunction 

func SetHasherFunction(hasherF func() hashing.Hasher) HTTPClientOptionF

func SetHealthCheckInterval 

func SetHealthCheckInterval(seconds time.Duration) HTTPClientOptionF

func SetHealthCheckTimeout 

func SetHealthCheckTimeout(seconds time.Duration) HTTPClientOptionF

func SetHealthChecks 

func SetHealthChecks(enable bool) HTTPClientOptionF

func SetHttpClient 

func SetHttpClient(client *http.Client) HTTPClientOptionF

func SetMaxRetries 

func SetMaxRetries(retries int) HTTPClientOptionF

func SetReadPreference 

func SetReadPreference(preference ReadPref) HTTPClientOptionF

func SetRequestRetrier 

func SetRequestRetrier(retrier RequestRetrier) HTTPClientOptionF

func SetSnapshotStoreURL 

func SetSnapshotStoreURL(url string) HTTPClientOptionF

func SetTopologyDiscovery 

func SetTopologyDiscovery(enable bool) HTTPClientOptionF

func SetURLs 

func SetURLs(primary string, secondaries ...string) HTTPClientOptionF

type NoRequestRetrier 

type NoRequestRetrier struct {
	*http.Client
}

NoRequestRetrier is an implementation that does no retries.

func NewNoRequestRetrier 

func NewNoRequestRetrier(httpClient *http.Client) *NoRequestRetrier

NewNoRequestRetrier returns a retrier that does no retries.

func (*NoRequestRetrier) DoReq 

func (r *NoRequestRetrier) DoReq(req *RetriableRequest) (*http.Response, error)

type ReadPref 

type ReadPref int

ReadPref specifies the preferred type of node in the cluster to send request to. 

const (
	// Primary forces to read only from the primary node (or leader).
	Primary ReadPref = iota

	// PrimaryPreferred aims to read from the primary node (or leader).
	//
	// Use PrimaryPreferred if you want an application to read from the primary
	// under normal circumstances, but to allow stale reads from secondaries when
	// the primary is unavailable. This provides a "read-only mode" for your
	// application during a failover.
	PrimaryPreferred

	// Secondary force to read only from secondary nodes (or replicas).
	Secondary

	// SecondaryPreferred aims to read from secondary nodes (or replicas).
	//
	// In general, do not use SecondaryPreferred to provide extra capacity for reads,
	// because all members of a cluster have roughly equivalent write traffic; as
	// a result, secondaries will service reads at roughly the same rate as the
	// primary. In addition, although replication is synchronous, there is some amount
	// of dely between event replication to secondaries and change application
	// to the corresponding balloon. Reading from a secondary can return stale data.
	SecondaryPreferred

	// Any forces to read from any node in the cluster including the leader.
	Any
)

type ReaderFunc 

type ReaderFunc func() (io.Reader, error)

ReaderFunc is the type of function that can be given natively to NewRetriableRequest.

type RequestRetrier 

type RequestRetrier interface {
	// DoReq executes the given request and if fails, it decides whether to retry
	// the call, how long to wait for the next call, or whether to return an
	// error (which will be returned to the service that started the HTTP
	// request in the first place).
	DoReq(req *RetriableRequest) (*http.Response, error)
}

RequestRetrier decides whether to retry a failed HTTP request.

type RetriableRequest 

type RetriableRequest struct {

	// Embed an HTTP request directly. This makes a *Request act exactly
	// like an *http.Request so that all meta methods are supported.
	*http.Request
	// contains filtered or unexported fields
}

RetriableRequest wraps the metadata needed to create HTTP requests and allow to reused it between retries.

func NewRetriableRequest 

func NewRetriableRequest(method, url string, rawBody []byte) (*RetriableRequest, error)

NewRetriableRequest creates a new retriable request.

type RoundTripFunc 

type RoundTripFunc func(req *http.Request) (*http.Response, error)

func NewFailingTransport 

func NewFailingTransport(path string, fail RoundTripFunc, next http.RoundTripper) RoundTripFunc

NewFailingTransport will run a fail callback if it sees a given URL path prefix.

func (RoundTripFunc) RoundTrip 

func (f RoundTripFunc) RoundTrip(req *http.Request) (*http.Response, error)

type SimpleBackoff 

type SimpleBackoff struct {
	sync.Mutex
	// contains filtered or unexported fields
}

SimpleBackoff takes a list of fixed values for backoff intervals. Each call to Next returns the next value from that fixed list. After each value is returned, subsequent calls to Next will only return the last element.

func NewSimpleBackoff 

func NewSimpleBackoff(ticks ...int) *SimpleBackoff

NewSimpleBackoff creates a SimpleBackoff algorithm with the specified list of fixed intervals in milliseconds.

func (*SimpleBackoff) Next 

func (b *SimpleBackoff) Next(attempt int) (time.Duration, bool)

Next implements BackoffF for SimpleBackoff.

type StopBackoff 

type StopBackoff struct{}

StopBackoff is a fixed backoff policy that always returns false for Next(), meaning that the operation should never be retried.

func NewStopBackoff 

func NewStopBackoff() *StopBackoff

NewStopBackoff returns a new StopBackoff.

func (StopBackoff) Next 

func (b StopBackoff) Next(attempt int) (time.Duration, bool)

Next implements BackoffF for StopBackoff.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.