fhttp

package
v1.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 9, 2023 License: Apache-2.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// SocketError is return when a transport error occurred: unexpected EOF, connection error, etc...
	SocketError = -1
	// RetryOnce is used internally as an error code to allow 1 retry for bad socket reuse.
	RetryOnce = -2
)

Extra error codes outside of the HTTP Status code ranges. ie negative.

View Source 
const (

	// HTTPReqTimeOutDefaultValue is the default timeout value. 3s.
	HTTPReqTimeOutDefaultValue = 3 * time.Second
)

Variables

View Source 
var (
	// BufferSizeKb size of the buffer (max data) for optimized client in kilobytes defaults to 128k.
	BufferSizeKb = 128
	// CheckConnectionClosedHeader indicates whether to check for server side connection closed headers.
	CheckConnectionClosedHeader = false
)
View Source 
var (
	// EnvoyRequestID is the header set by envoy and we need to propagate for distributed tracing.
	EnvoyRequestID = textproto.CanonicalMIMEHeaderKey("x-request-id")
	// TraceHeader is the single aggregated open tracing header to propagate when present.
	TraceHeader = textproto.CanonicalMIMEHeaderKey("b3")
	// TraceHeadersPrefix is the prefix for the multi header version of open zipkin.
	TraceHeadersPrefix = textproto.CanonicalMIMEHeaderKey("x-b3-")
)
View Source 
var (

	// EchoRequests is the number of request received. Only updated in Debug mode.
	EchoRequests            int64
	DefaultEchoServerParams = dflag.New("",
		"Default parameters/querystring to use if there isn't one provided explicitly. E.g \"status=404&delay=3s\"")
	Fetch2CopiesAllHeader = dflag.NewBool(true,
		"Determines if only tracing or all headers (and cookies) are copied from request on the fetch2 ui/server endpoint")
	ServerIdleTimeout = dflag.New(30*time.Second, "Default IdleTimeout for servers")
)
View Source 
var MaxDelay = dflag.New(1500*time.Millisecond,
	"Maximum sleep time for delay= echo server parameter. dynamic flag.")

MaxDelay is the maximum delay allowed for the echoserver responses. It is a dynamic flag with default value of 1.5s so we can test the default 1s timeout in envoy.

Functions

func ASCIIToUpper 

func ASCIIToUpper(str string) []byte

ASCIIToUpper returns a byte array equal to the input string but in lowercase. Only works for ASCII, not meant for unicode.

func AddHTTPS 

func AddHTTPS(url string) string

AddHTTPS replaces "http://" in url with "https://" or prepends "https://" if url does not contain prefix "http://".

func CacheOn 

func CacheOn(w http.ResponseWriter)

CacheOn sets the header for indefinite caching.

func CopyHeaders 

func CopyHeaders(req, r *http.Request, all bool)

CopyHeaders copies all or trace headers from `r` into `req`.

func CreateProxyClient 

func CreateProxyClient() *http.Client

CreateProxyClient http client for connection reuse.

func DebugHandler 

func DebugHandler(w http.ResponseWriter, r *http.Request)

DebugHandler returns debug/useful info to http client. Note this can be dangerous and shouldn't be exposed to the internet. A safer version is available as part of fortio's proxy https://github.com/fortio/proxy/blob/main/rp/reverse_proxy.go

func DebugSummary 

func DebugSummary(buf []byte, max int) string

DebugSummary returns a string with the size and escaped first max/2 and last max/2 bytes of a buffer (or the whole escaped buffer if small enough).

func DynamicHTTPServer 

func DynamicHTTPServer(closing bool) (*http.ServeMux, *net.TCPAddr)

DynamicHTTPServer listens on an available port, sets up an http or a closing server simulating an https server (when closing is true) server on it and returns the listening port and mux to which one can attach handlers to. Note: in a future version of istio, the closing will be actually be secure on/off and create an https server instead of a closing server. As this is a dynamic tcp socket server, the address is TCP.

func EchoDebugPath 

func EchoDebugPath(debugPath string) string

EchoDebugPath returns the additional echo handler path behind debugPath (ie /debug -> /debug/echo/).

func EchoHandler 

func EchoHandler(w http.ResponseWriter, r *http.Request)

EchoHandler is an http server handler echoing back the input.

func Fetch 

func Fetch(httpOptions *HTTPOptions) (int, []byte)

Fetch creates a client an performs a fetch according to the http options passed in. To be used only for single fetches or when performance doesn't matter as the client is closed at the end. Deprecated: use StreamFetch instead.

func FetchURL 

func FetchURL(url string) (int, []byte)

FetchURL fetches the data at the given url using the standard client and default options. Returns the http status code (http.StatusOK == 200 for success) and the data. To be used only for single fetches or when performance doesn't matter as the client is closed at the end. Deprecated: use StreamURL instead.

func FetcherHandler 

func FetcherHandler(w http.ResponseWriter, r *http.Request)

FetcherHandler is the handler for the fetcher/proxy.

func FetcherHandler2 

func FetcherHandler2(w http.ResponseWriter, r *http.Request)

FetcherHandler2 is the handler for the fetcher/proxy that supports h2 input and makes a new request with all headers copied (allows to test sticky routing) Note this should only be made available to trusted clients.

func Flush 

func Flush(w http.ResponseWriter)

func FoldFind 

func FoldFind(haystack []byte, needle []byte) (bool, int)

FoldFind searches the bytes assuming ascii, ignoring the lowercase bit for testing. Not intended to work with unicode, meant for http headers and to be fast (see benchmark in test file).

func Gzip 

func Gzip(next http.Handler) http.Handler

Gzip wraps a handler for automatic gzip.

func HTTPSServer 

func HTTPSServer(name string, port string, to *TLSOptions) (*http.ServeMux, net.Addr)

func HTTPServer 

func HTTPServer(name string, port string) (*http.ServeMux, net.Addr)

HTTPServer creates an http server named name on address/port port. Port can include binding address and/or be port 0.

func HTTPServerWithHandler 

func HTTPServerWithHandler(name string, port string, hdlr http.Handler) net.Addr

HTTPServerWithHandler creates and h2c compatible server named name on address/port port. Port can include binding address and/or be port 0. Takes in a handler.

func LogAndCall 

func LogAndCall(msg string, hf http.HandlerFunc) http.HandlerFunc

LogAndCall wraps an HTTP handler to log the request first.

func LogAndCallNoArg 

func LogAndCallNoArg(msg string, f func()) http.HandlerFunc

LogAndCallNoArg is LogAndCall for functions not needing the response/request args.

func MakeSimpleRequest 

func MakeSimpleRequest(url string, r *http.Request, copyAllHeaders bool) *http.Request

MakeSimpleRequest makes a new request for url but copies trace headers from input request r. or all the headers if copyAllHeaders is true.

func MultiServer 

func MultiServer(port string, cfg *MultiServerConfig) (*http.ServeMux, net.Addr)

MultiServer starts fan out http server on the given port. Returns the mux and addr where the listening socket is bound. The port can be retrieved from it when requesting the 0 port as input for dynamic http server.

func NewHTMLEscapeWriter 

func NewHTMLEscapeWriter(w io.Writer) io.Writer

NewHTMLEscapeWriter creates a io.Writer that can safely output to an http.ResponseWrite with HTMLEscape-ing.

func OnBehalfOf 

func OnBehalfOf(o *HTTPOptions, r *http.Request)

OnBehalfOf adds a header with the remote addr to an http options object.

func OnBehalfOfRequest 

func OnBehalfOfRequest(to *http.Request, from *http.Request)

OnBehalfOfRequest same as OnBehalfOf but places the header directly on the dst request object but also adds a X-Proxy-Agent header if the user-agent isn't already the same as this running server's version.

func ParseChunkSize 

func ParseChunkSize(inp []byte) (int64, int64)

ParseChunkSize extracts the chunk size and consumes the line. Returns the offset of the data and the size of the chunk, 0, -1 when not found.

func ParseDecimal 

func ParseDecimal(inp []byte) int64

ParseDecimal extracts the first positive integer number from the input. spaces are ignored. any character that isn't a digit cause the parsing to stop.

func QueryArg 

func QueryArg(r *http.Request, key string) string

QueryArg(r,...) is like r.FormValue(...) but exclusively getting the values from the query string (as we use the body for data). Result of parsing the query string is cached in r.Form so we don't keep parsing it and so r.Form can be used for multivalued entries like r.Form["header"].

func RedirectToHTTPS 

func RedirectToHTTPS(port string) net.Addr

RedirectToHTTPS Sets up a redirector to https on the given port. (Do not create a loop, make sure this is addressed from an ingress).

func RedirectToHTTPSHandler 

func RedirectToHTTPSHandler(w http.ResponseWriter, r *http.Request)

RedirectToHTTPSHandler handler sends a redirect to same URL with https.

func RoundDuration 

func RoundDuration(d time.Duration) time.Duration

RoundDuration rounds to 10th of second.

func Serve 

func Serve(port, debugPath string) (*http.ServeMux, net.Addr)

Serve starts a debug / echo http server on the given port. Returns the mux and addr where the listening socket is bound. The .Port can be retrieved from it when requesting the 0 port as input for dynamic http server.

func ServeTCP 

func ServeTCP(port, debugPath string) (*http.ServeMux, *net.TCPAddr)

ServeTCP is Serve() but restricted to TCP (return address is assumed to be TCP - will panic for unix domain).

func ServeTLS 

func ServeTLS(port, debugPath string, to *TLSOptions) (*http.ServeMux, net.Addr)

ServeTLS starts a debug / echo server on the given port, using TLS if certPath and keyPath aren't not empty.

func SetupPPROF 

func SetupPPROF(mux *http.ServeMux)

SetupPPROF add pprof to the mux (mirror the init() of http pprof).

func StreamFetch 

func StreamFetch(httpOptions *HTTPOptions) int

Fetch creates a client an performs a fetch according to the http options passed in. To be used only for single fetches or when performance doesn't matter as the client is closed at the end.

func StreamURL 

func StreamURL(url string, w io.Writer) int

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client object for making repeated requests of the same URL using the same http client (net/http). TODO: refactor common parts with FastClient.

func NewStdClient 

func NewStdClient(o *HTTPOptions) (*Client, error)

NewStdClient creates a client object that wraps the net/http standard client.

func (*Client) ChangeURL 

func (c *Client) ChangeURL(urlStr string) (err error)

ChangeURL only for standard client, allows fetching a different URL.

func (*Client) Close 

func (c *Client) Close()

Close cleans up any resources used by NewStdClient.

func (*Client) Fetch 

func (c *Client) Fetch(ctx context.Context) (int, []byte, int)

Fetch is the backward compatible version of StreamFetch to avoid rewriting a bunch of tests but should not be used for load tests (where we don't care about the data and only the stats) Deprecated: use StreamFetch instead.

func (*Client) GetIPAddress 

func (c *Client) GetIPAddress() (*stats.Occurrence, *stats.Histogram)

GetIPAddress get the ip address that DNS resolves to when using stdClient and connection stats.

func (*Client) HasBuffer 

func (c *Client) HasBuffer() bool

func (*Client) StreamFetch 

func (c *Client) StreamFetch(ctx context.Context) (int, int64, uint)

StreamFetch fetches the byte and code for pre created std client. header length (3rd returned value) is always 0 for that client and only available with the fastclient.

type CreateClientTrace 

type CreateClientTrace func(ctx context.Context) *httptrace.ClientTrace

type CreateTransport 

type CreateTransport func(base http.RoundTripper) http.RoundTripper

type FastClient 

type FastClient struct {
	// contains filtered or unexported fields
}

FastClient is a fast, lockfree single purpose http 1.0/1.1 client.

func (*FastClient) Close 

func (c *FastClient) Close()

Close cleans up any resources used by FastClient.

func (*FastClient) Fetch 

func (c *FastClient) Fetch(ctx context.Context) (int, []byte, int)

Fetch fetches the url content. Returns http code, data, offset of body.

func (*FastClient) GetIPAddress 

func (c *FastClient) GetIPAddress() (*stats.Occurrence, *stats.Histogram)

GetIPAddress get ip address that DNS resolved to when using fast client and connection stats.

func (*FastClient) HasBuffer 

func (c *FastClient) HasBuffer() bool

func (*FastClient) StreamFetch 

func (c *FastClient) StreamFetch(ctx context.Context) (int, int64, uint)

Fetch fetches the url content. Returns http code, data written to the writer, length of headers.

type Fetcher 

type Fetcher interface {
	// Fetch returns http code, data, offset of body (for client which returns
	// headers)
	// Deprecated: use StreamFetch with a DataWriter (nil if you don't need the data) instead.
	Fetch(context.Context) (int, []byte, int)
	// StreamFetch returns http code and body bytes read
	// (body is streamed to Dest writer or null),and header size for the fast client.
	StreamFetch(context.Context) (int, int64, uint)
	// HasBuffer is true for the fast client and false for golang standard library based client.
	// it's used to know if calling Fetch() is actually better (fast client with headers to stderr)
	HasBuffer() bool
	// Close() cleans up connections and state - must be paired with NewClient calls.
	Close()
	// GetIPAddress() returns the occurrence of ip address used by this client connection,
	// and the connection time histogram (which includes the count).
	GetIPAddress() (*stats.Occurrence, *stats.Histogram)
}

Fetcher is the Url content fetcher that the different client implements.

func NewClient 

func NewClient(o *HTTPOptions) (Fetcher, error)

NewClient creates either a standard or fast client (depending on the DisableFastClient flag).

func NewFastClient 

func NewFastClient(o *HTTPOptions) (Fetcher, error)

NewFastClient makes a basic, efficient http 1.0/1.1 client. This function itself doesn't need to be super efficient as it is created at the beginning and then reused many times.

type FlushWriter 

type FlushWriter struct {
	// contains filtered or unexported fields
}

func (FlushWriter) Write 

func (fw FlushWriter) Write(p []byte) (n int, err error)

type GzipResponseWriter 

type GzipResponseWriter struct {
	io.Writer
	http.ResponseWriter
	// contains filtered or unexported fields
}

GzipResponseWriter wraps the response and gzips the content.

func NewGzipHTTPResponseWriter 

func NewGzipHTTPResponseWriter(w http.ResponseWriter) *GzipResponseWriter

NewGzipHTTPResponseWriter returns a wrapper for gzip'ing the response.

func (*GzipResponseWriter) Close 

func (w *GzipResponseWriter) Close() error

Close must be called in defer inside the handler using this.

func (*GzipResponseWriter) Write 

func (w *GzipResponseWriter) Write(b []byte) (int, error)

Write sends the Write() to the gzip Writer.

func (*GzipResponseWriter) WriteHeader 

func (w *GzipResponseWriter) WriteHeader(status int)

WriteHeader intercepts the actual to remove any Content-Length that may have been added before compression.

type HTMLEscapeWriter 

type HTMLEscapeWriter struct {
	NextWriter io.Writer
	Flusher    http.Flusher
}

HTMLEscapeWriter is an io.Writer escaping the output for safe html inclusion.

func (*HTMLEscapeWriter) Write 

func (w *HTMLEscapeWriter) Write(p []byte) (int, error)

type HTTPOptions 

type HTTPOptions struct {
	TLSOptions
	URL               string
	NumConnections    int  // num connections (for std client)
	Compression       bool // defaults to no compression, only used by std client
	DisableFastClient bool // defaults to fast client
	HTTP10            bool // defaults to http1.1
	H2                bool // defaults to http1.1 (h2 only for stdclient)
	DisableKeepAlive  bool // so default is keep alive
	AllowHalfClose    bool // if not keepalive, whether to half close after request
	FollowRedirects   bool // For the Std Client only: follow redirects.

	Resolve string // resolve Common Name to this ip when use CN as target url

	HTTPReqTimeOut   time.Duration // timeout value for http request
	UserCredentials  string        // user credentials for authorization
	ContentType      string        // indicates request body type, implies POST instead of GET
	Payload          []byte        // body for http request, implies POST if not empty.
	LogErrors        bool          // whether to log non 2xx code as they occur or not
	ID               int           `json:"-"` // thread/connect id to use for logging (thread id when used as a runner)
	SequentialWarmup bool          // whether to do http(s) runs warmup sequentially or in parallel (new default is //)
	ConnReuseRange   [2]int        // range of max number of connection to reuse for each thread.
	// When false, re-resolve the DNS name when the connection breaks.
	NoResolveEachConn bool
	// Optional Offset Duration; to offset the histogram of the Connection duration
	Offset time.Duration
	// Optional resolution divider for the Connection duration histogram. In seconds. Defaults to 0.001 or 1 millisecond.
	Resolution float64
	// Optional ClientTrace factory to use if set. Only effective when using std client.
	ClientTrace CreateClientTrace `json:"-"`
	// Optional Transport chain factory to use if set. Only effective when using std client.
	// pass otelhttp.NewTransport for instance.
	Transport CreateTransport `json:"-"`
	// These following 2 options are only making sense for single operation (curl) mode.
	PayloadReader io.Reader `json:"-"` // if set, Payload is ignored and this is used instead.
	DataWriter    io.Writer `json:"-"` // if set, the response body is written to this writer.
	// contains filtered or unexported fields
}

HTTPOptions holds the common options of both http clients and the headers.

func NewHTTPOptions 

func NewHTTPOptions(url string) *HTTPOptions

NewHTTPOptions creates and initialize a HTTPOptions object. Do not change the URL directly later, make a new HTTPOptions. It replaces plain % to %25 in the url. If you already have properly escaped URLs use o.URL = to set it.

func (*HTTPOptions) AddAndValidateExtraHeader 

func (h *HTTPOptions) AddAndValidateExtraHeader(hdr string) error

AddAndValidateExtraHeader collects extra headers (see commonflags.go for example).

func (*HTTPOptions) AllHeaders 

func (h *HTTPOptions) AllHeaders() http.Header

AllHeaders returns the current set of headers including virtual/special Host header.

func (*HTTPOptions) GenerateHeaders 

func (h *HTTPOptions) GenerateHeaders() http.Header

GenerateHeaders completes the header generation, including Content-Type/Length and user credential coming from the http options in addition to extra headers coming from flags and AddAndValidateExtraHeader(). Warning this gets called more than once, do not generate duplicate headers.

func (*HTTPOptions) Init 

func (h *HTTPOptions) Init(url string) *HTTPOptions

Init initializes the headers in an HTTPOptions (User-Agent).

func (*HTTPOptions) InitHeaders 

func (h *HTTPOptions) InitHeaders()

InitHeaders initialize and/or resets the default headers (ie just User-Agent).

func (*HTTPOptions) Method 

func (h *HTTPOptions) Method() string

Method returns the method of the http req.

func (*HTTPOptions) PayloadUTF8 

func (h *HTTPOptions) PayloadUTF8() string

PayloadUTF8 returns the payload as a string. If payload is null return empty string This is only needed due to grpc ping proto. It takes string instead of byte array.

func (*HTTPOptions) ResetHeaders 

func (h *HTTPOptions) ResetHeaders()

ResetHeaders resets all the headers, including the User-Agent: one (and the Host: logical special header). This is used from the UI as the user agent is settable from the form UI.

func (*HTTPOptions) URLSchemeCheck 

func (h *HTTPOptions) URLSchemeCheck()

URLSchemeCheck makes sure the client will work with the scheme requested. it also adds missing http:// to emulate curl's behavior.

func (*HTTPOptions) ValidateAndAddBasicAuthentication 

func (h *HTTPOptions) ValidateAndAddBasicAuthentication(headers http.Header) error

ValidateAndAddBasicAuthentication validates user credentials and adds basic authentication to http header, if user credentials are valid.

func (*HTTPOptions) ValidateAndSetConnectionReuseRange 

func (h *HTTPOptions) ValidateAndSetConnectionReuseRange(inp string) error

type HTTPRunnerOptions 

type HTTPRunnerOptions struct {
	periodic.RunnerOptions
	HTTPOptions               // Need to call Init() to initialize
	Profiler           string // file to save profiles to. defaults to no profiling
	AllowInitialErrors bool   // whether initial errors don't cause an abort
	// Which status code cause an abort of the run (default 0 = don't abort; reminder -1 is returned for socket errors)
	AbortOn int
}

HTTPRunnerOptions includes the base RunnerOptions plus http specific options.

type HTTPRunnerResults 

type HTTPRunnerResults struct {
	periodic.RunnerResults

	RetCodes   map[int]int64
	IPCountMap map[string]int // TODO: Move it to a shared results struct where all runner should have this field

	// exported result
	HTTPOptions
	Sizes       *stats.HistogramData
	HeaderSizes *stats.HistogramData
	Sockets     []int64
	SocketCount int64
	// Connection Time stats
	ConnectionStats *stats.HistogramData
	// http code to abort the run on (-1 for connection or other socket error)
	AbortOn int
	// contains filtered or unexported fields
}

HTTPRunnerResults is the aggregated result of an HTTPRunner. Also is the internal type used per thread/goroutine.

func RunHTTPTest 

func RunHTTPTest(o *HTTPRunnerOptions) (*HTTPRunnerResults, error)

RunHTTPTest runs an http test and returns the aggregated stats.

func (*HTTPRunnerResults) Run 

func (httpstate *HTTPRunnerResults) Run(ctx context.Context, t periodic.ThreadID) (bool, string)

Run tests http request fetching. Main call being run at the target QPS. To be set as the Function in RunnerOptions.

type MultiServerConfig 

type MultiServerConfig struct {
	Targets []TargetConf
	Serial  bool // Serialize or parallel queries
	//	Javascript bool // return data as UI suitable
	Name string
	// contains filtered or unexported fields
}

MultiServerConfig configures the MultiServer and holds the http client it uses for proxying.

func (*MultiServerConfig) TeeHandler 

func (mcfg *MultiServerConfig) TeeHandler(w http.ResponseWriter, r *http.Request)

TeeHandler common part between TeeSerialHandler and TeeParallelHandler.

func (*MultiServerConfig) TeeParallelHandler 

func (mcfg *MultiServerConfig) TeeParallelHandler(w http.ResponseWriter, r *http.Request, data []byte)

TeeParallelHandler handles teeing off traffic in parallel (one goroutine each) mode.

func (*MultiServerConfig) TeeSerialHandler 

func (mcfg *MultiServerConfig) TeeSerialHandler(w http.ResponseWriter, r *http.Request, data []byte)

TeeSerialHandler handles teeing off traffic in serial (one at a time) mode.

type SyncReader 

type SyncReader struct {
	// contains filtered or unexported fields
}

SyncReader is a thread-safe wrapper for a reader.

func NewSyncReader 

func NewSyncReader(reader io.Reader) *SyncReader

NewSyncReader returns a new thread-safe reader.

func (*SyncReader) Read 

func (r *SyncReader) Read(p []byte) (n int, err error)

type TLSOptions 

type TLSOptions struct {
	Insecure         bool   // Do not verify certs
	MTLS             bool   // Use mutual TLS, require client cert
	CACert           string // `Path` to a custom CA certificate file to be used
	Cert             string // `Path` to the certificate file to be used
	Key              string // `Path` to the key file used
	UnixDomainSocket string // `Path“ of unix domain socket to use instead of host:port
}

TLSOptions are common TLS related options between https and grpc.

func (*TLSOptions) TLSConfig 

func (to *TLSOptions) TLSConfig() (*tls.Config, error)

TLSConfig creates a tls.Config based on input TLSOptions. For https, ServerName is set later (once host is determined after URL parsing and depending on hostOverride). Used for both client and server TLS config.

type TargetConf 

type TargetConf struct {
	Destination  string // Destination URL or base
	MirrorOrigin bool   // wether to use the incoming request as URI and data params to outgoing one (proxy like)

}

TargetConf is the structure to configure one of the multiple targets for MultiServer.

type Transport 

type Transport interface {
	http.RoundTripper
	CloseIdleConnections()
}

Transport common interface between http.Transport and http2.Transport.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.