httpify

README

httpify

httpify is a Go package designed to simplify and enhance HTTP client functionality by providing customizable retry strategies and transport configurations. It aims to improve the handling of HTTP requests with flexible retry mechanisms and efficient connection management.

Installation

To use httpify in your Go project, you can install it using go get:

go get github.com/cyinnove/httpify

Usage

Here are some examples of how to use the package:

Creating a Retry Strategy

package main

import (
	"fmt"
	"time"
	"github.com/cyinnove/httpify"
)

func main() {
	retryStrategy := httpify.DefaultRetryStrategy()
	
	// Example usage of retryStrategy
	fmt.Println(retryStrategy(1*time.Second, 10*time.Second, 2, nil))
}

Creating an HTTP Client with Custom Transport

package main

import (
	"fmt"
	"net/http"
	"github.com/cyinnove/httpify"
)

func main() {
	client := &http.Client{
		Transport: httpify.NoKeepAliveTransport(),
	}
	
	// Example usage of client
	resp, err := client.Get("https://example.com")
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
	defer resp.Body.Close()
	
	fmt.Println("Response Status:", resp.Status)
}

Inspiration

This package is inspired by the following projects:

• go-retryablehttp: A library for retrying HTTP requests with configurable retry strategies.
• retryablehttp-go: Another library for retrying HTTP requests with customizable retry policies.

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests to help improve httpify.

Documentation

Index

• Variables
• func DefaultClient() *http.Client
• func DefaultHTTPClient(timeout time.Duration) *http.Client
• func DefaultPooledClient() *http.Client
• func Discard(req *Request, resp *http.Response, respReadLimit int64)
• func NoKeepAliveTransport() *http.Transport
• func PassthroughErrorHandler(resp *http.Response, err error, _ int) (*http.Response, error)
• func PooledTransport() *http.Transport
• type CheckRetry
  • func DefaultRetryPolicy() CheckRetry
  • func HostSprayRetryPolicy() CheckRetry
• type Client
  • func NewClient(options Options) *Client
  • func NewWithHTTPClient(client *http.Client, options Options) *Client
  • func (c *Client) Do(req *Request) (*http.Response, error)
  • func (c *Client) Get(url string) (*http.Response, error)
  • func (c *Client) Head(url string) (*http.Response, error)
  • func (c *Client) Post(url, bodyType string, body interface{}) (*http.Response, error)
  • func (c *Client) PostForm(url string, data url.Values) (*http.Response, error)
• type ErrorHandler
• type LenReader
• type Metrics
• type Options
• type ReaderFunc
• type Request
  • func FromRequest(r *http.Request) (*Request, error)
  • func FromRequestWithTrace(r *http.Request) (*Request, error)
  • func NewRequest(method, url string, body interface{}) (*Request, error)
  • func NewRequestWithContext(ctx context.Context, method, url string, body interface{}) (*Request, error)
  • func (r *Request) BodyBytes() ([]byte, error)
  • func (r *Request) WithContext(ctx context.Context) *Request
• type RequestLogHook
• type ResponseLogHook
• type RetryStrategy
  • func DefaultRetryStrategy() RetryStrategy
  • func ExponentialRandomizedRetryStrategy() RetryStrategy
  • func LinearRandomizedRetryStrategy() RetryStrategy
  • func RandomizedFullRetryStrategy() RetryStrategy

Variables

var DefaultOptionsSingle = Options{
	RetryWaitMin:  1 * time.Second,
	RetryWaitMax:  30 * time.Second,
	Timeout:       30 * time.Second,
	RetryMax:      5,
	RespReadLimit: 4096,
	KillIdleConn:  false,
}

Default options for targeting a single host.

var DefaultOptionsSpraying = Options{
	RetryWaitMin:  1 * time.Second,
	RetryWaitMax:  30 * time.Second,
	Timeout:       30 * time.Second,
	RetryMax:      5,
	RespReadLimit: 4096,
	KillIdleConn:  true,
}

Default options for spraying multiple hosts.

Functions

func DefaultClient

func DefaultClient() *http.Client

DefaultClient creates a new http.Client with disabled idle connections and keepalives.

func DefaultHTTPClient

func DefaultHTTPClient(timeout time.Duration) *http.Client

DefaultHTTPClient creates an HTTP client with a default timeout.

func DefaultPooledClient

func DefaultPooledClient() *http.Client

DefaultPooledClient returns an http.Client with a pooled transport for connection reuse.

func Discard

func Discard(req *Request, resp *http.Response, respReadLimit int64)

Discard discards the response body and closes the underlying connection. It reads up to RespReadLimit bytes from the response body to ensure connection reuse. If an error occurs during reading, it increments the DrainErrors metric.

func NoKeepAliveTransport

func NoKeepAliveTransport() *http.Transport

HostSprayingTransport returns a new http.Transport with disabled idle connections and keepalives.

func PassthroughErrorHandler

func PassthroughErrorHandler(resp *http.Response, err error, _ int) (*http.Response, error)

PassthroughErrorHandler directly passes through net/http errors for the final request.

func PooledTransport

func PooledTransport() *http.Transport

PooledTransport returns a new http.Transport for connection reuse.

Types

type CheckRetry

type CheckRetry func(ctx context.Context, resp *http.Response, err error) (bool, error)

CheckRetry defines a policy for retrying requests based on the response and error.

func DefaultRetryPolicy

func DefaultRetryPolicy() CheckRetry

DefaultRetryPolicy retries on connection errors and server errors.

func HostSprayRetryPolicy

func HostSprayRetryPolicy() CheckRetry

HostSprayRetryPolicy retries on connection and server errors for host-spraying use cases.

type Client

type Client struct {
	HTTPClient      *http.Client
	RequestLogHook  RequestLogHook
	ResponseLogHook ResponseLogHook
	ErrorHandler    ErrorHandler
	CheckRetry      CheckRetry
	RetryStrategy   RetryStrategy
	// contains filtered or unexported fields
}

Client wraps an HTTP client with retry and retryStrategy logic.

func NewClient

func NewClient(options Options) *Client

NewClient initializes a Client with specified options.

func NewWithHTTPClient

func NewWithHTTPClient(client *http.Client, options Options) *Client

NewWithHTTPClient initializes a Client with a custom HTTP client.

func (*Client) Do

func (c *Client) Do(req *Request) (*http.Response, error)

Do sends an HTTP request with retries and retryStrategy.

func (*Client) Get

func (c *Client) Get(url string) (*http.Response, error)

Get sends an HTTP GET request.

func (*Client) Head

func (c *Client) Head(url string) (*http.Response, error)

Head sends an HTTP HEAD request.

func (*Client) Post

func (c *Client) Post(url, bodyType string, body interface{}) (*http.Response, error)

Post sends an HTTP POST request.

func (*Client) PostForm

func (c *Client) PostForm(url string, data url.Values) (*http.Response, error)

PostForm sends an HTTP POST request using pre-filled form data.

type ErrorHandler

type ErrorHandler func(resp *http.Response, err error, numTries int) (*http.Response, error)

ErrorHandler handles retries exhaustion and returns custom responses.

type LenReader

type LenReader interface {
	Len() int
}

LenReader interface defines a method to get the length of a reader.

type Metrics

type Metrics struct {
	Failures    int
	Retries     int
	DrainErrors int
}

Metrics stores retry and error metrics for a request.

type Options

type Options struct {
	RetryWaitMin  time.Duration
	RetryWaitMax  time.Duration
	Timeout       time.Duration
	RetryMax      int
	RespReadLimit int64
	KillIdleConn  bool
}

Options defines retryable settings for the HTTP client.

type ReaderFunc

type ReaderFunc func() (io.Reader, error)

ReaderFunc defines a function type for creating readers.

type Request

type Request struct {
	*http.Request
	Metrics Metrics
	// contains filtered or unexported fields
}

Request wraps HTTP request metadata for retries.

func FromRequest

func FromRequest(r *http.Request) (*Request, error)

FromRequest wraps an http.Request into a retryable Request.

func FromRequestWithTrace

func FromRequestWithTrace(r *http.Request) (*Request, error)

FromRequestWithTrace wraps an http.Request into a retryable Request with trace enabled.

func NewRequest

func NewRequest(method, url string, body interface{}) (*Request, error)

NewRequest creates a new wrapped request.

func NewRequestWithContext

func NewRequestWithContext(ctx context.Context, method, url string, body interface{}) (*Request, error)

NewRequestWithContext creates a new wrapped request with a context.

func (*Request) BodyBytes

func (r *Request) BodyBytes() ([]byte, error)

BodyBytes returns a copy of the request body data.

func (*Request) WithContext

func (r *Request) WithContext(ctx context.Context) *Request

WithContext returns a shallow copy of the request with a new context.

type RequestLogHook

type RequestLogHook func(*http.Request, int)

RequestLogHook allows executing custom logic before each retry.

type ResponseLogHook

type ResponseLogHook func(*http.Response)

ResponseLogHook allows executing custom logic after each HTTP request.

type RetryStrategy

type RetryStrategy func(min, max time.Duration, attemptNum int, resp *http.Response) time.Duration

RetryStrategy defines how long to wait between retries.

func DefaultRetryStrategy

func DefaultRetryStrategy() RetryStrategy

DefaultRetryStrategy implements exponential retryStrategy based on attempt count, bounded by min and max durations.

func ExponentialRandomizedRetryStrategy

func ExponentialRandomizedRetryStrategy() RetryStrategy

ExponentialRandomizedRetryStrategy adds randomized to exponential retryStrategy.

func LinearRandomizedRetryStrategy

func LinearRandomizedRetryStrategy() RetryStrategy

LinearRandomizedRetryStrategy adds random randomized to linear retryStrategy.

func RandomizedFullRetryStrategy

func RandomizedFullRetryStrategy() RetryStrategy

RandomizedFullRetryStrategy implements capped exponential retryStrategy with full randomized.