httpclient

Documentation

Index

• func AuthFieldSpecs() []*service.ConfigField
• func AuthFieldSpecsExpanded() []*service.ConfigField
• func BasicAuthField() *service.ConfigField
• func ConfigField(defaultVerb string, forOutput bool, extraChildren ...*service.ConfigField) *service.ConfigField
• func OldFieldSpec(defaultVerb string, forOutput bool, extraChildren ...docs.FieldSpec) docs.FieldSpec
• type AuthConfig
  • func NewAuthConfig() AuthConfig
  • func (c AuthConfig) Sign(f ifs.FS, req *http.Request) error
• type BasicAuthConfig
  • func NewBasicAuthConfig() BasicAuthConfig
  • func (basic BasicAuthConfig) Sign(req *http.Request) error
• type Client
  • func NewClientFromOldConfig(conf OldConfig, mgr bundle.NewManagement, opts ...RequestOpt) (*Client, error)
  • func (h *Client) Close(ctx context.Context) error
  • func (h *Client) ResponseToBatch(res *http.Response) (message.Batch, error)
  • func (h *Client) Send(ctx context.Context, sendMsg message.Batch) (message.Batch, error)
  • func (h *Client) SendToResponse(ctx context.Context, sendMsg message.Batch) (res *http.Response, err error)
• type JWTConfig
  • func NewJWTConfig() JWTConfig
  • func (j JWTConfig) Sign(f ifs.FS, req *http.Request) error
• type MultipartExpressions
• type OAuth2Config
  • func NewOAuth2Config() OAuth2Config
  • func (oauth OAuth2Config) Client(ctx context.Context, base *http.Client) *http.Client
• type OAuthConfig
  • func NewOAuthConfig() OAuthConfig
  • func (oauth OAuthConfig) Sign(req *http.Request) error
• type OldConfig
  • func ConfigFromAny(v any) (conf OldConfig, err error)
  • func NewOldConfig() OldConfig
• type RequestCreator
  • func RequestCreatorFromOldConfig(conf OldConfig, mgr bundle.NewManagement, opts ...RequestOpt) (*RequestCreator, error)
  • func (r *RequestCreator) Create(refBatch message.Batch) (req *http.Request, err error)
• type RequestOpt
  • func WithExplicitBody(e *field.Expression) RequestOpt
  • func WithExplicitMultipart(m []MultipartExpressions) RequestOpt
• type RequestSigner
  • func AuthSignerFromParsed(conf *service.ParsedConfig) (RequestSigner, error)

Functions

func AuthFieldSpecs

func AuthFieldSpecs() []*service.ConfigField

AuthFieldSpecs returns a map of field specs for an auth type.

func AuthFieldSpecsExpanded

func AuthFieldSpecsExpanded() []*service.ConfigField

AuthFieldSpecsExpanded includes OAuth2 and JWT fields that might not be appropriate for all components.

func BasicAuthField

func BasicAuthField() *service.ConfigField

BasicAuthField returns a config field spec for basic authentication.

func ConfigField

func ConfigField(defaultVerb string, forOutput bool, extraChildren ...*service.ConfigField) *service.ConfigField

ConfigField returns a public API config field spec for an HTTP component, with optional extra fields added to the end.

func OldFieldSpec

func OldFieldSpec(defaultVerb string, forOutput bool, extraChildren ...docs.FieldSpec) docs.FieldSpec

OldFieldSpec returns a field spec for an http client component.

Types

type AuthConfig

type AuthConfig struct {
	OAuth     OAuthConfig     `json:"oauth" yaml:"oauth"`
	BasicAuth BasicAuthConfig `json:"basic_auth" yaml:"basic_auth"`
	JWT       JWTConfig       `json:"jwt" yaml:"jwt"`
}

AuthConfig contains configuration params for various HTTP auth strategies.

func NewAuthConfig

func NewAuthConfig() AuthConfig

NewAuthConfig creates a new Config with default values.

func (AuthConfig) Sign

func (c AuthConfig) Sign(f ifs.FS, req *http.Request) error

Sign method to sign an HTTP request for configured auth strategies.

type BasicAuthConfig

type BasicAuthConfig struct {
	Enabled  bool   `json:"enabled" yaml:"enabled"`
	Username string `json:"username" yaml:"username"`
	Password string `json:"password" yaml:"password"`
}

BasicAuthConfig contains fields for setting basic authentication in HTTP requests.

func NewBasicAuthConfig

func NewBasicAuthConfig() BasicAuthConfig

NewBasicAuthConfig returns a default configuration for basic authentication in HTTP client requests.

func (BasicAuthConfig) Sign

func (basic BasicAuthConfig) Sign(req *http.Request) error

Sign method to sign an HTTP request for an OAuth exchange.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a component able to send and receive Benthos messages over HTTP.

func NewClientFromOldConfig

func NewClientFromOldConfig(conf OldConfig, mgr bundle.NewManagement, opts ...RequestOpt) (*Client, error)

NewClientFromOldConfig creates a new request creator from an old struct style config. Eventually I'd like to phase these out for the more dynamic service style parses, but it'll take a while so we have this for now.

func (*Client) Close

func (h *Client) Close(ctx context.Context) error

Close the client.

func (*Client) ResponseToBatch

func (h *Client) ResponseToBatch(res *http.Response) (message.Batch, error)

ResponseToBatch attempts to parse an HTTP response into a 2D slice of bytes.

func (*Client) Send

func (h *Client) Send(ctx context.Context, sendMsg message.Batch) (message.Batch, error)

Send creates an HTTP request from the client config, a provided message to be sent as the body of the request, and a reference message used to establish interpolated fields for the request (which can be the same as the message used for the body).

If the request is successful then the response is parsed into a message, including headers added as metadata (when configured to do so).

func (*Client) SendToResponse

func (h *Client) SendToResponse(ctx context.Context, sendMsg message.Batch) (res *http.Response, err error)

SendToResponse attempts to create an HTTP request from a provided message, performs it, and then returns the *http.Response, allowing the raw response to be consumed.

type JWTConfig

type JWTConfig struct {
	Enabled        bool           `json:"enabled" yaml:"enabled"`
	Claims         jwt.MapClaims  `json:"claims" yaml:"claims"`
	Headers        map[string]any `json:"headers" yaml:"headers"`
	SigningMethod  string         `json:"signing_method" yaml:"signing_method"`
	PrivateKeyFile string         `json:"private_key_file" yaml:"private_key_file"`
	// contains filtered or unexported fields
}

JWTConfig holds the configuration parameters for an JWT exchange.

func NewJWTConfig

func NewJWTConfig() JWTConfig

NewJWTConfig returns a new JWTConfig with default values.

func (JWTConfig) Sign

func (j JWTConfig) Sign(f ifs.FS, req *http.Request) error

Sign method to sign an HTTP request for an JWT exchange.

type MultipartExpressions

type MultipartExpressions struct {
	ContentDisposition *field.Expression
	ContentType        *field.Expression
	Body               *field.Expression
}

MultipartExpressions represents three dynamic expressions that define a multipart message part in an HTTP request. Specifying one or more of these can be used as a way of creating HTTP requests that overrides the default behaviour.

type OAuth2Config

type OAuth2Config struct {
	Enabled      bool     `json:"enabled" yaml:"enabled"`
	ClientKey    string   `json:"client_key" yaml:"client_key"`
	ClientSecret string   `json:"client_secret" yaml:"client_secret"`
	TokenURL     string   `json:"token_url" yaml:"token_url"`
	Scopes       []string `json:"scopes" yaml:"scopes"`
}

OAuth2Config holds the configuration parameters for an OAuth2 exchange.

func NewOAuth2Config

func NewOAuth2Config() OAuth2Config

NewOAuth2Config returns a new OAuth2Config with default values.

func (OAuth2Config) Client

func (oauth OAuth2Config) Client(ctx context.Context, base *http.Client) *http.Client

Client returns an http.Client with OAuth2 configured.

type OAuthConfig

type OAuthConfig struct {
	Enabled           bool   `json:"enabled" yaml:"enabled"`
	ConsumerKey       string `json:"consumer_key" yaml:"consumer_key"`
	ConsumerSecret    string `json:"consumer_secret" yaml:"consumer_secret"`
	AccessToken       string `json:"access_token" yaml:"access_token"`
	AccessTokenSecret string `json:"access_token_secret" yaml:"access_token_secret"`
}

OAuthConfig holds the configuration parameters for an OAuth exchange.

func NewOAuthConfig

func NewOAuthConfig() OAuthConfig

NewOAuthConfig returns a new OAuthConfig with default values.

func (OAuthConfig) Sign

func (oauth OAuthConfig) Sign(req *http.Request) error

Sign method to sign an HTTP request for an OAuth exchange.

type OldConfig

type OldConfig struct {
	URL                 string                       `json:"url" yaml:"url"`
	Verb                string                       `json:"verb" yaml:"verb"`
	Headers             map[string]string            `json:"headers" yaml:"headers"`
	Metadata            metadata.IncludeFilterConfig `json:"metadata" yaml:"metadata"`
	ExtractMetadata     metadata.IncludeFilterConfig `json:"extract_headers" yaml:"extract_headers"`
	RateLimit           string                       `json:"rate_limit" yaml:"rate_limit"`
	Timeout             string                       `json:"timeout" yaml:"timeout"`
	Retry               string                       `json:"retry_period" yaml:"retry_period"`
	MaxBackoff          string                       `json:"max_retry_backoff" yaml:"max_retry_backoff"`
	NumRetries          int                          `json:"retries" yaml:"retries"`
	BackoffOn           []int                        `json:"backoff_on" yaml:"backoff_on"`
	DropOn              []int                        `json:"drop_on" yaml:"drop_on"`
	SuccessfulOn        []int                        `json:"successful_on" yaml:"successful_on"`
	DumpRequestLogLevel string                       `json:"dump_request_log_level" yaml:"dump_request_log_level"`
	TLS                 tls.Config                   `json:"tls" yaml:"tls"`
	ProxyURL            string                       `json:"proxy_url" yaml:"proxy_url"`
	AuthConfig          `json:",inline" yaml:",inline"`
	OAuth2              OAuth2Config `json:"oauth2" yaml:"oauth2"`
}

OldConfig is a configuration struct for an HTTP client.

func ConfigFromAny

func ConfigFromAny(v any) (conf OldConfig, err error)

ConfigFromAny is a temporary work around for components that get a new style parsed config and need to get an old style struct config.

func NewOldConfig

func NewOldConfig() OldConfig

NewOldConfig creates a new Config with default values.

type RequestCreator

type RequestCreator struct {
	// contains filtered or unexported fields
}

RequestCreator creates *http.Request types from messages based on various configurable parameters.

func RequestCreatorFromOldConfig

func RequestCreatorFromOldConfig(conf OldConfig, mgr bundle.NewManagement, opts ...RequestOpt) (*RequestCreator, error)

RequestCreatorFromOldConfig creates a new request creator from an old struct style config. Eventually I'd like to phase these out for the more dynamic service style parses, but it'll take a while so we have this for now.

func (*RequestCreator) Create

func (r *RequestCreator) Create(refBatch message.Batch) (req *http.Request, err error)

Create an *http.Request using a reference message batch to extract the body and headers of the request. It's possible that the creator has been given explicit overrides for the body, in which case the reference batch is only used for general request headers/metadata enrichment.

type RequestOpt

type RequestOpt func(r *RequestCreator)

RequestOpt represents a customisation of a request creator.

func WithExplicitBody

func WithExplicitBody(e *field.Expression) RequestOpt

WithExplicitBody modifies the request creator to instead only use input reference messages for headers and metadata, and use the expression for creating a body.

func WithExplicitMultipart

func WithExplicitMultipart(m []MultipartExpressions) RequestOpt

WithExplicitMultipart modifies the request creator to instead only use input reference messages for headers and metadata, and use a list of multipart expressions for creating a body.

type RequestSigner

type RequestSigner func(f ifs.FS, req *http.Request) error

RequestSigner is a closure configured to enrich requests with various functions, usually authentication.

func AuthSignerFromParsed

func AuthSignerFromParsed(conf *service.ParsedConfig) (RequestSigner, error)

AuthSignerFromParsed takes a parsed config which is expected to contain fields from AuthFields, and returns a RequestSigner that implements the configured authentication strategies by enriching a request directly.