httpclient

package
v0.2.16 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 26, 2024 License: MPL-2.0 Imports: 22 Imported by: 4

Documentation

Overview

httpclient/client.go 

The `http_client` package provides a configurable HTTP client tailored for interacting with specific APIs.

It supports different authentication methods, including "bearer" and "oauth". The client is designed with a focus on concurrency management, structured error handling, and flexible configuration options. The package offers a default timeout, custom backoff strategies, dynamic rate limiting, and detailed logging capabilities. The main `Client` structure encapsulates all necessary components, like the baseURL, authentication details, and an embedded standard HTTP client.

httpclient/client_configuration.go Description: This file contains functions to load and validate configuration values from a JSON file or environment variables.

httpclient/integration.go

httpmethod/httpmethod.go

httpclient/request.go

httpclient/utility.go

Index

Constants

View Source 
const (
	DefaultLogLevelString              = "LogLevelInfo"
	DefaultLogOutputFormatString       = "pretty"
	DefaultLogConsoleSeparator         = "	"
	DefaultLogExportPath               = "/defaultlogs"
	DefaultMaxRetryAttempts            = 3
	DefaultMaxConcurrentRequests       = 1
	DefaultExportLogs                  = false
	DefaultHideSensitiveData           = false
	DefaultEnableDynamicRateLimiting   = false
	DefaultCustomTimeout               = 5 * time.Second
	DefaultTokenRefreshBufferPeriod    = 2 * time.Minute
	DefaultTotalRetryDuration          = 5 * time.Minute
	DefaultEnableConcurrencyManagement = false
)
View Source 
const ConfigFileExtension = ".json"

Variables

This section is empty.

Functions

This section is empty.

Types

type APIIntegration added in v0.1.54 

type APIIntegration interface {
	GetFQDN() string
	ConstructURL(endpoint string) string
	GetAuthMethodDescriptor() string
	CheckRefreshToken() error
	PrepRequestParamsAndAuth(req *http.Request) error
	PrepRequestBody(body interface{}, method string, endpoint string) ([]byte, error)
	MarshalMultipartRequest(fields map[string]string, files map[string]string) ([]byte, string, error)
	GetSessionCookies() ([]*http.Cookie, error)
}

APIIntegration is an interface that defines the methods required for an API integration. These are obtained from go-api-http-client-integrations. The methods defined in this interface are used by the HTTP client to authenticate and prepare requests for the API.

type Client 

type Client struct {
	Integration *APIIntegration

	Sugar       *zap.SugaredLogger
	Concurrency *concurrency.ConcurrencyHandler
	// contains filtered or unexported fields
}

Master struct/object

func (*Client) CheckDeprecationHeader added in v0.2.6 

func (c *Client) CheckDeprecationHeader(resp *http.Response)

CheckDeprecationHeader checks the response headers for the Deprecation header and logs a warning if present.

func (*Client) DoImageMultiPartUpload added in v0.2.13 

func (c *Client) DoImageMultiPartUpload(method, endpoint string, fileName string, base64Data string, customBoundary string, out interface{}) (*http.Response, error)

DoImageMultiPartUpload performs a multipart request with a specifically formatted payload. This is designed for APIs that expect a very specific multipart format, where the payload needs to be constructed manually rather than using the standard multipart writer.

func (*Client) DoMultiPartRequest added in v0.1.51 

func (c *Client) DoMultiPartRequest(method, endpoint string, files map[string][]string, formDataFields map[string]string, fileContentTypes map[string]string, formDataPartHeaders map[string]http.Header, out interface{}) (*http.Response, error)

Parameters:

  • method: A string representing the HTTP method to be used for the request. This method should be either POST or PUT as these are the only methods that support multipart/form-data requests.
  • endpoint: The target API endpoint for the request. This should be a relative path that will be appended to the base URL configured for the HTTP client.
  • files: A map where the key is the field name and the value is a slice of file paths to be included in the request.
  • formDataFields: A map of additional form fields to be included in the multipart request, where the key is the field name and the value is the field value.
  • fileContentTypes: A map specifying the content type for each file part. The key is the field name and the value is the content type (e.g., "image/jpeg").
  • formDataPartHeaders: A map specifying custom headers for each part of the multipart form data. The key is the field name and the value is an http.Header containing the headers for that part.
  • out: A pointer to an output variable where the response will be deserialized. This should be a pointer to a struct that matches the expected response schema.

Returns:

  • *http.Response: The HTTP response received from the server. In case of successful execution, this response contains the status code, headers, and body of the response. In case of errors, this response may contain the last received HTTP response that led to the failure.
  • error: An error object indicating failure during request execution. This could be due to network issues, server errors, or a failure in request serialization/deserialization.

Usage: This function is suitable for executing multipart/form-data HTTP requests, particularly for file uploads along with additional form fields. It ensures proper authentication, sets necessary headers, and logs the process for debugging and monitoring purposes. Example: var result MyResponseType resp, err := client.DoMultiPartRequest("POST", "/api/upload", files, formDataFields, fileContentTypes, formDataPartHeaders, &result) 

if err != nil {
    // Handle error
}

// Use `result` or `resp` as needed

func (*Client) DoRequest 

func (c *Client) DoRequest(method, endpoint string, body, out interface{}) (*http.Response, error)

DoRequest constructs and executes an HTTP request based on the provided method, endpoint, request body, and output variable. This function serves as a dispatcher, deciding whether to execute the request with or without retry logic based on the idempotency of the HTTP method. Idempotent methods (GET, PUT, DELETE) are executed with retries to handle transient errors and rate limits, while non-idempotent methods (POST, PATCH) are executed without retries to avoid potential side effects of duplicating non-idempotent operations. The function uses an instance of a logger implementing the logger.Logger interface, used to log informational messages, warnings, and errors encountered during the execution of the request. It also applies redirect handling to the client if configured, allowing the client to follow redirects up to a maximum number of times. Parameters:

  • method: A string representing the HTTP method to be used for the request. This method determines the execution path and whether the request will be retried in case of failures.
  • endpoint: The target API endpoint for the request. This should be a relative path that will be appended to the base URL configured for the HTTP client.
  • body: The payload for the request, which will be serialized into the request body. The serialization format (e.g., JSON, XML) is determined by the content-type header and the specific implementation of the API handler used by the client.
  • out: A pointer to an output variable where the response will be deserialized. The function expects this to be a pointer to a struct that matches the expected response schema.

Returns:

  • *http.Response: The HTTP response received from the server. In case of successful execution, this response contains the status code, headers, and body of the response. In case of errors, particularly after exhausting retries for idempotent methods, this response may contain the last received HTTP response that led to the failure.
  • error: An error object indicating failure during request execution. This could be due to network issues, server errors, or a failure in request serialization/deserialization. For idempotent methods, an error is returned if all retries are exhausted without success.

Usage: This function is the primary entry point for executing HTTP requests using the client. It abstracts away the details of request retries, serialization, and response handling, providing a simplified interface for making HTTP requests. It is suitable for a wide range of HTTP operations, from fetching data with GET requests to submitting data with POST requests. Example: var result MyResponseType resp, err := client.DoRequest("GET", "/api/resource", nil, &result, logger) 

if err != nil {
    // Handle error
}

// Use `result` or `resp` as needed Note:

  • The caller is responsible for closing the response body when not nil to avoid resource leaks.
  • The function ensures concurrency control by managing concurrency tokens internally, providing safe concurrent operations within the client's concurrency model.
  • The decision to retry requests is based on the idempotency of the HTTP method and the client's retry configuration, including maximum retry attempts and total retry duration.

type ClientConfig added in v0.0.12 

type ClientConfig struct {
	// Interface which implements the APIIntegration patterns. Integration handles all server/endpoint specific configuration, auth and vars.
	Integration APIIntegration

	// Sugar is the logger from Zap.
	Sugar *zap.SugaredLogger

	// Wether or not empty values will be set or an error thrown for missing items.
	PopulateDefaultValues bool

	// HideSenitiveData controls if sensitive data will be visible in logs. Debug option which should be True in production use.
	HideSensitiveData bool `json:"hide_sensitive_data"`

	// CustomCookies allows implementation of persistent, session wide cookies.
	CustomCookies []*http.Cookie

	// MaxRetry Attempts limits the amount of retries the client will perform on requests which are deemd retriable.
	MaxRetryAttempts int `json:"max_retry_attempts"`

	// MaxConcurrentRequests limits the amount of Semaphore tokens available to the client and therefor limits concurrent requests.
	MaxConcurrentRequests int `json:"max_concurrent_requests"`

	// EnableDynamicRateLimiting // TODO because I don't know.
	EnableDynamicRateLimiting bool `json:"enable_dynamic_rate_limiting"`

	// CustomTimeout // TODO also because I don't know.
	CustomTimeout time.Duration

	// TokenRefreshBufferPeriod is the duration of time before the token expires in which it's deemed
	// more sensible to replace the token rather then carry on using it.
	TokenRefreshBufferPeriod time.Duration

	// TotalRetryDuration // TODO maybe this should be called context?
	TotalRetryDuration time.Duration

	// EnableCustomRedirectLogic allows the client to follow redirections when they're returned from a request.
	// Toggleable for debug reasons only
	CustomRedirectPolicy *func(req *http.Request, via []*http.Request) error

	// MaxRedirects is the maximum amount of redirects the client will follow before throwing an error.
	MaxRedirects int `json:"max_redirects"`

	// EnableConcurrencyManagement when false bypasses any concurrency management to allow for a simpler request flow.
	EnableConcurrencyManagement bool `json:"enable_concurrency_management"`

	// MandatoryRequestDelay is a short, usually sub 0.5 second, delay after every request as to not overwhelm an endpoint.
	// Can be set to nothing if you want to be lightning fast!
	MandatoryRequestDelay time.Duration

	// RetryEligiableRequests when false bypasses any retry logic for a simpler request flow.
	RetryEligiableRequests bool `json:"retry_eligiable_requests"`

	HTTPExecutor HTTPExecutor
}

Options/Variables for Client

func LoadConfigFromEnv added in v0.0.75 

func LoadConfigFromEnv() (*ClientConfig, error)

LoadConfigFromEnv loads HTTP client configuration settings from environment variables. If any environment variables are not set, the default values defined in the constants are used instead.

func LoadConfigFromFile added in v0.0.77 

func LoadConfigFromFile(filepath string) (*ClientConfig, error)

LoadConfigFromFile loads http client configuration settings from a JSON file.

func (*ClientConfig) Build added in v0.2.6 

func (c *ClientConfig) Build() (*Client, error)

BuildClient creates a new HTTP client with the provided configuration.

func (*ClientConfig) SetDefaultValuesClientConfig added in v0.2.6 

func (c *ClientConfig) SetDefaultValuesClientConfig()

SetDefaultValuesClientConfig sets default values for the client configuration. Ensuring that all fields have a valid or minimum value.

type HTTPExecutor added in v0.2.9 

type HTTPExecutor interface {

	// Inherited
	CloseIdleConnections()
	Do(req *http.Request) (*http.Response, error)
	Get(url string) (resp *http.Response, err error)
	Head(url string) (resp *http.Response, err error)
	Post(url string, contentType string, body io.Reader) (resp *http.Response, err error)
	PostForm(url string, data url.Values) (resp *http.Response, err error)

	// Additional
	SetCookieJar(jar http.CookieJar)
	SetCookies(url *url.URL, cookies []*http.Cookie)
	SetCustomTimeout(time.Duration)
	Cookies(*url.URL) []*http.Cookie
	SetRedirectPolicy(*func(req *http.Request, via []*http.Request) error)
}

HTTPExecutor is an interface which wraps http.Client to allow mocking.

type MockExecutor added in v0.2.9 

type MockExecutor struct {
	LockedResponseCode int
	ResponseBody       string
}

MockExecutor implements the same function pattern above but allows controllable responses for mocking/testing

func (*MockExecutor) CloseIdleConnections added in v0.2.9 

func (m *MockExecutor) CloseIdleConnections()

CloseIdleConnections does nothing.

func (*MockExecutor) Cookies added in v0.2.9 

func (m *MockExecutor) Cookies(*url.URL) []*http.Cookie

Cookies does nothing yet

func (*MockExecutor) Do added in v0.2.9 

func (m *MockExecutor) Do(req *http.Request) (*http.Response, error)

Do returns a http.Response with controllable body and status code.

func (*MockExecutor) Get added in v0.2.9 

func (m *MockExecutor) Get(_ string) (*http.Response, error)

Get returns Do

func (*MockExecutor) Head added in v0.2.9 

func (m *MockExecutor) Head(_ string) (*http.Response, error)

Head returns Do

func (*MockExecutor) Post added in v0.2.9 

func (m *MockExecutor) Post(_ string, _ string, _ io.Reader) (*http.Response, error)

Post returns Do

func (*MockExecutor) PostForm added in v0.2.9 

func (m *MockExecutor) PostForm(_ string, _ url.Values) (*http.Response, error)

PostForm returns Do

func (*MockExecutor) SetCookieJar added in v0.2.9 

func (m *MockExecutor) SetCookieJar(jar http.CookieJar)

SetCookieJar does nothing yet

func (*MockExecutor) SetCookies added in v0.2.9 

func (m *MockExecutor) SetCookies(url *url.URL, cookies []*http.Cookie)

SetCookies does nothing yet

func (*MockExecutor) SetCustomTimeout added in v0.2.9 

func (m *MockExecutor) SetCustomTimeout(time.Duration)

SetCustomTimeout does nothing yet

func (*MockExecutor) SetRedirectPolicy added in v0.2.9 

func (m *MockExecutor) SetRedirectPolicy(*func(req *http.Request, via []*http.Request) error)

SetRedirectPolicy does nothing

type ProdExecutor added in v0.2.9 

type ProdExecutor struct {
	*http.Client
}

ProdExecutor wraps http.Client and implements functions to adjust some of it's attrs

func (*ProdExecutor) Cookies added in v0.2.9 

func (c *ProdExecutor) Cookies(url *url.URL) []*http.Cookie

Cookies wraps http.Client.Jar.Cookies()

func (*ProdExecutor) SetCookieJar added in v0.2.9 

func (c *ProdExecutor) SetCookieJar(jar http.CookieJar)

SetCookieJar func to wrap c.Jar = jar

func (*ProdExecutor) SetCookies added in v0.2.9 

func (c *ProdExecutor) SetCookies(url *url.URL, cookies []*http.Cookie)

SetCookies wraps http.Client.Jar.SetCookies

func (*ProdExecutor) SetCustomTimeout added in v0.2.9 

func (c *ProdExecutor) SetCustomTimeout(timeout time.Duration)

SetCustomTimeout wraps http.Client.Timeout = timeout

func (*ProdExecutor) SetRedirectPolicy added in v0.2.9 

func (c *ProdExecutor) SetRedirectPolicy(policy *func(req *http.Request, via []*http.Request) error)

SetRedirectPolicy wraps http.Client.Jar.CheckRedirect =

type UploadState added in v0.1.53 

type UploadState struct {
	LastUploadedByte int64
	sync.Mutex
}

UploadState represents the state of an upload operation, including the last uploaded byte. This struct is used to track the progress of file uploads for resumable uploads and to resume uploads from the last uploaded byte.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.