forwarder

package module
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 11, 2023 License: MPL-2.0 Imports: 37 Imported by: 0

README

🚜 Forwarder Build Status Go Report Card GoDoc GitHub release

Forwarder is HTTP proxy. It can be used to proxy HTTP/HTTPS/HTTP2 requests, Server Sent Events, WebSockets, TCP traffic and more. It supports downstream HTTP(S) and SOCKS5 proxies and basic authentication.

Installation

The release binaries are available on GitHub.

You can also install Forwarder from source:

go get -u github.com/saucelabs/forwarder/cmd/forwarder

Usage

HTTP (forward) proxy server with PAC support and PAC testing tools

Commands:
  run           Start HTTP (forward) proxy server
  pac           Tools for working with PAC files

Other Commands:
  completion    Generate the autocompletion script for the specified shell
  version       Print version information

The following options can be passed to any subcommand:

Options:
    --config-file <path> (env FORWARDER_CONFIG_FILE)
        Configuration file to load options from. The supported formats are: JSON, YAML, TOML, HCL, and Java
        properties. The file format is determined by the file extension, if not specified the default format is YAML.
        The following precedence order of configuration sources is used: command flags, environment variables, config
        file, default values.
Run
Start HTTP (forward) proxy server. You can start HTTP or HTTPS server. If you start an HTTPS server and you don't
provide a certificate, the server will generate a self-signed certificate on startup. The server may be protected by
basic authentication. Whenever applicable, username and password are URL decoded. This allows you to pass in special
characters such as @ by using %%40 or pass in a colon with %%3a.

Examples:
  # HTTP proxy with upstream proxy
  forwarder run --proxy http://localhost:8081

  # Start HTTP proxy with PAC script
  forwarder run --address localhost:3128 --pac https://example.com/pac.js

  # HTTPS proxy server with basic authentication
  forwarder run --protocol https --address localhost:8443 --basic-auth user:password

Options:
    --address <host:port> (default ':3128') (env FORWARDER_ADDRESS)
        The server address to listen on. If the host is empty, the server will listen on all available interfaces.

    --basic-auth <username:password> (env FORWARDER_BASIC_AUTH)
        Basic authentication credentials to protect the server. Username and password are URL decoded. This allows you
        to pass in special characters such as @ by using %40 or pass in a colon with %3a.

    -c, --credentials <username:password@host:port> (env FORWARDER_CREDENTIALS)
        Site or upstream proxy basic authentication credentials. The host and port can be set to "*" to match all
        hosts and ports respectively. The flag can be specified multiple times to add multiple credentials.

    -H, --header <header> (env FORWARDER_HEADER)
        Add or remove HTTP request headers. Use the format "name: value" to add a header, "name;" to set the header to
        empty value, "-name" to remove the header, "-name*" to remove headers by prefix. The header name will be
        normalized to canonical form. The header value should not contain any newlines or carriage returns. The flag
        can be specified multiple times. Example: -H "Host: example.com" -H "-User-Agent" -H "-X-*".

    -p, --pac <path or URL> (env FORWARDER_PAC)
        Proxy Auto-Configuration file to use for upstream proxy selection. It can be a local file or a URL, you can
        also use '-' to read from stdin.

    --protocol <http|https> (default http) (env FORWARDER_PROTOCOL)
        The server protocol. For https and h2 protocols, if TLS certificate is not specified, the server will use a
        self-signed certificate.

    -x, --proxy [protocol://]host[:port] (env FORWARDER_PROXY)
        Upstream proxy to use. The supported protocols are: http, https, socks, socks5. No protocol specified will be
        treated as HTTP proxy. If the port number is not specified, it is assumed to be 1080. The basic authentication
        username and password can be specified in the host string e.g. user:pass@host:port. Alternatively, you can use
        the -c, --credentials flag to specify the credentials.

    --proxy-localhost <allow|deny|direct> (default deny) (env FORWARDER_PROXY_LOCALHOST)
        Setting this to allow enables sending requests to localhost through the upstream proxy. Setting this to direct
        sends requests to localhost directly without using the upstream proxy. By default, requests to localhost are
        denied.

    --read-header-timeout <duration> (default 1m0s) (env FORWARDER_READ_HEADER_TIMEOUT)
        The amount of time allowed to read request headers.

    -R, --response-header <header> (env FORWARDER_RESPONSE_HEADER)
        Add or remove HTTP headers on the received response before sending it to the client. See the documentation for
        the -H, --header flag for more details on the format.

    --tls-cert-file <path> (env FORWARDER_TLS_CERT_FILE)
        TLS certificate to use if the server protocol is https or h2.

    --tls-key-file <path> (env FORWARDER_TLS_KEY_FILE)
        TLS private key to use if the server protocol is https or h2.

API server options:
    --api-address <host:port> (default 'localhost:10000') (env FORWARDER_API_ADDRESS)
        The server address to listen on. If the host is empty, the server will listen on all available interfaces.

    --api-basic-auth <username:password> (env FORWARDER_API_BASIC_AUTH)
        Basic authentication credentials to protect the server. Username and password are URL decoded. This allows you
        to pass in special characters such as @ by using %40 or pass in a colon with %3a.

    --api-log-http <none|url|headers|body|errors> (default errors) (env FORWARDER_API_LOG_HTTP)
        HTTP request and response logging mode. By default, request line and headers are logged if response status
        code is greater than or equal to 500. Setting this to none disables logging.

    --api-read-header-timeout <duration> (default 1m0s) (env FORWARDER_API_READ_HEADER_TIMEOUT)
        The amount of time allowed to read request headers.

DNS options:
    -n, --dns-server <ip>[:<port>] (env FORWARDER_DNS_SERVER)
        DNS server(s) to use instead of system default. If specified multiple times, the first one is used as primary
        server, the rest are used as fallbacks. The port is optional, if not specified the default port is 53.

    --dns-timeout <duration> (default 5s) (env FORWARDER_DNS_TIMEOUT)
        Timeout for dialing DNS servers.

HTTP client options:
    --http-dial-timeout <duration> (default 30s) (env FORWARDER_HTTP_DIAL_TIMEOUT)
        The maximum amount of time a dial will wait for a connect to complete. With or without a timeout, the
        operating system may impose its own earlier timeout. For instance, TCP timeouts are often around 3 minutes. 

    --http-idle-conn-timeout <duration> (default 1m30s) (env FORWARDER_HTTP_IDLE_CONN_TIMEOUT)
        The maximum amount of time an idle (keep-alive) connection will remain idle before closing itself. Zero means
        no limit. 

    --http-response-header-timeout <duration> (default 0s) (env FORWARDER_HTTP_RESPONSE_HEADER_TIMEOUT)
        The amount of time to wait for a server's response headers after fully writing the request (including its
        body, if any).This time does not include the time to read the response body. Zero means no limit. 

    --http-tls-handshake-timeout <duration> (default 10s) (env FORWARDER_HTTP_TLS_HANDSHAKE_TIMEOUT)
        The maximum amount of time waiting to wait for a TLS handshake. Zero means no limit.

    -k, --insecure <value> (default false) (env FORWARDER_INSECURE)
        Don't verify the server's certificate chain and host name. Enable to work with self-signed certificates. 

Logging options:
    --log-file <path> (env FORWARDER_LOG_FILE)
        Path to the log file, if empty, logs to stdout.

    --log-http <none|url|headers|body|errors> (default errors) (env FORWARDER_LOG_HTTP)
        HTTP request and response logging mode. By default, request line and headers are logged if response status
        code is greater than or equal to 500. Setting this to none disables logging.

    --log-level <error|info|debug> (default info) (env FORWARDER_LOG_LEVEL)
        Log level.

Usage:
  forwarder run [--address <host:port>] [--pac <path or url>] [--credentials <username:password@host:port>]... [flags]

Proxy Auto-Configuration (PAC)

Forwarder implements Proxy Auto-Configuration (PAC) file support. PAC file is a JavaScript file that returns a proxy URL for a given URL. It can be used to implement complex proxy rules.

Forwarder also implements Microsoft's PAC extensions for IPv6. So you can use FindProxyForURL and FindProxyForURLEx functions to implement IPv6-aware proxy rules.

Forwarder can be used as a PAC file server (see forwarder pac server command), or as a PAC file test util (see forwarder pac eval command).

Available functions

The following functions are available in PAC files:

  • alert
  • convert_addr
  • dateRange
  • dnsDomainIs
  • dnsDomainLevels
  • dnsResolve
  • dnsResolveEx
  • getClientVersion
  • getDay
  • getMonth
  • isInNet
  • isInNetEx
  • isPlainHostName
  • isResolvable
  • isResolvableEx
  • isValidIpAddress
  • localHostOrDomainIs
  • myIpAddress
  • myIpAddressEx
  • shExpMatch
  • sortIpAddressList
  • timeRange
  • weekdayRange

License

Forwarder is licensed under the Mozilla Public License 2.0.

Documentation

Overview

Package forwarder provides a simple forward proxy server. The proxy can be protected with HTTP basic authentication. It can also forward connections to a parent proxy, and authorize connections against that. Both local, and parent credentials can be set via environment variables.

Index

Constants

View Source 
const ErrorHeader = "X-Forwarder-Error"

ErrorHeader is the header that is set on error responses with the error message.

Variables

View Source 
var ErrProxyLocalhost = denyError{errors.New("localhost proxying is disabled")}

Functions

func LoadCertificateFromTLSConfig added in v1.0.1 

func LoadCertificateFromTLSConfig(dst *tls.Config, src *TLSConfig) error

func NewHTTPTransport 

func NewHTTPTransport(cfg *HTTPTransportConfig, r *net.Resolver) *http.Transport

func NewResolver 

func NewResolver(cfg *DNSConfig, log log.Logger) (*net.Resolver, error)

func OpenFileParser 

func OpenFileParser(flag int, perm, dirPerm os.FileMode) func(val string) (*os.File, error)

OpenFileParser returns a parser that calls os.OpenFile. If dirPerm is set it will create the directory if it does not exist. For empty path the parser returns nil file and nil error.

func ParseDNSAddress 

func ParseDNSAddress(val string) (netip.AddrPort, error)

func ParseProxyURL 

func ParseProxyURL(val string) (*url.URL, error)

func ParseUserInfo 

func ParseUserInfo(val string) (*url.Userinfo, error)

ParseUserInfo parses a user:password string into *url.Userinfo. Username and password cannot be empty.

func ReadURL 

func ReadURL(u *url.URL, rt http.RoundTripper) (string, error)

ReadURL can read a local file, http or https URL or stdin.

func RedactHostPortUser 

func RedactHostPortUser(hpu *HostPortUser) string

Types

type APIHandler 

type APIHandler struct {
	// contains filtered or unexported fields
}

APIHandler serves API endpoints. It provides health and readiness endpoints prometheus metrics, and pprof debug endpoints.

func NewAPIHandler 

func NewAPIHandler(r prometheus.Gatherer, ready func() bool, config, pac string) *APIHandler

func (*APIHandler) ServeHTTP 

func (h *APIHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)

type CredentialsMatcher 

type CredentialsMatcher struct {
	// contains filtered or unexported fields
}

func NewCredentialsMatcher 

func NewCredentialsMatcher(credentials []*HostPortUser, log log.Logger) (*CredentialsMatcher, error)

func (*CredentialsMatcher) Match 

func (m *CredentialsMatcher) Match(hostport string) *url.Userinfo

Match `hostport` to one of the configured input. Priority is exact Match, then host, then port, then global wildcard.

func (*CredentialsMatcher) MatchURL 

func (m *CredentialsMatcher) MatchURL(u *url.URL) *url.Userinfo

MatchURL adds standard http and https ports if they are missing in URL and calls Match function.

type DNSConfig 

type DNSConfig struct {
	// Servers is a list of DNS servers to use.
	// If provided with multiple servers, the first one is used as primary server, the rest are used as a fallback.
	Servers []netip.AddrPort
	// Timeout is the timeout for DNS queries.
	Timeout time.Duration
}

func DefaultDNSConfig 

func DefaultDNSConfig() *DNSConfig

func (*DNSConfig) Validate 

func (c *DNSConfig) Validate() error

type HTTPProxy 

type HTTPProxy struct {
	TLSConfig *tls.Config
	Listener  net.Listener
	// contains filtered or unexported fields
}

func NewHTTPProxy 

func NewHTTPProxy(cfg *HTTPProxyConfig, pr PACResolver, cm *CredentialsMatcher, rt http.RoundTripper, log log.Logger) (*HTTPProxy, error)

func (*HTTPProxy) Addr 

func (hp *HTTPProxy) Addr() string

Addr returns the address the server is listening on or an empty string if the server is not running.

func (*HTTPProxy) Handler 

func (hp *HTTPProxy) Handler() http.Handler

func (*HTTPProxy) Ready 

func (hp *HTTPProxy) Ready() bool

Ready returns true if the server is running and ready to accept requests.

func (*HTTPProxy) Run 

func (hp *HTTPProxy) Run(ctx context.Context) error

type HTTPProxyConfig 

type HTTPProxyConfig struct {
	HTTPServerConfig
	ProxyLocalhost     ProxyLocalhostMode
	UpstreamProxy      *url.URL
	UpstreamProxyFunc  ProxyFunc
	RequestModifiers   []martian.RequestModifier
	ResponseModifiers  []martian.ResponseModifier
	ConnectPassthrough bool
	CloseAfterReply    bool

	// TestingHTTPHandler uses Martian's [http.Handler] implementation
	// over [http.Server] instead of the default TCP server.
	TestingHTTPHandler bool
}

func DefaultHTTPProxyConfig 

func DefaultHTTPProxyConfig() *HTTPProxyConfig

func (*HTTPProxyConfig) Validate 

func (c *HTTPProxyConfig) Validate() error

type HTTPServer 

type HTTPServer struct {
	Listener net.Listener
	// contains filtered or unexported fields
}

func NewHTTPServer 

func NewHTTPServer(cfg *HTTPServerConfig, h http.Handler, log log.Logger) (*HTTPServer, error)

func (*HTTPServer) Addr 

func (hs *HTTPServer) Addr() string

Addr returns the address the server is listening on or an empty string if the server is not running.

func (*HTTPServer) Ready 

func (hs *HTTPServer) Ready() bool

Ready returns true if the server is running and ready to accept requests.

func (*HTTPServer) Run 

func (hs *HTTPServer) Run(ctx context.Context) error

type HTTPServerConfig 

type HTTPServerConfig struct {
	Protocol Scheme
	Addr     string
	TLSConfig
	ReadTimeout       time.Duration
	ReadHeaderTimeout time.Duration
	WriteTimeout      time.Duration
	LogHTTPMode       httplog.Mode

	PromNamespace string
	PromRegistry  prometheus.Registerer
	BasicAuth     *url.Userinfo
}

func DefaultHTTPServerConfig 

func DefaultHTTPServerConfig() *HTTPServerConfig

func (*HTTPServerConfig) Validate 

func (c *HTTPServerConfig) Validate() error

type HTTPTransportConfig 

type HTTPTransportConfig struct {
	// DialTimeout is the maximum amount of time a dial will wait for
	// a connect to complete.
	//
	// With or without a timeout, the operating system may impose
	// its own earlier timeout. For instance, TCP timeouts are
	// often around 3 minutes.
	DialTimeout time.Duration

	// KeepAlive specifies the interval between keep-alive
	// probes for an active network connection.
	// If zero, keep-alive probes are sent with a default value
	// (currently 15 seconds), if supported by the protocol and operating
	// system. Network protocols or operating systems that do
	// not support keep-alives ignore this field.
	// If negative, keep-alive probes are disabled.
	KeepAlive time.Duration

	// TLSHandshakeTimeout specifies the maximum amount of time waiting to
	// wait for a TLS handshake. Zero means no timeout.
	TLSHandshakeTimeout time.Duration

	// MaxIdleConns controls the maximum number of idle (keep-alive)
	// connections across all hosts. Zero means no limit.
	MaxIdleConns int

	// MaxIdleConnsPerHost, if non-zero, controls the maximum idle
	// (keep-alive) connections to keep per-host. If zero,
	// DefaultMaxIdleConnsPerHost is used.
	MaxIdleConnsPerHost int

	// MaxConnsPerHost optionally limits the total number of
	// connections per host, including connections in the dialing,
	// active, and idle states. On limit violation, dials will block.
	//
	// Zero means no limit.
	MaxConnsPerHost int

	// IdleConnTimeout is the maximum amount of time an idle
	// (keep-alive) connection will remain idle before closing
	// itself.
	// Zero means no limit.
	IdleConnTimeout time.Duration

	// ResponseHeaderTimeout, if non-zero, specifies the amount of
	// time to wait for a server's response headers after fully
	// writing the request (including its body, if any). This
	// time does not include the time to read the response body.
	ResponseHeaderTimeout time.Duration

	// ExpectContinueTimeout, if non-zero, specifies the amount of
	// time to wait for a server's first response headers after fully
	// writing the request headers if the request has an
	// "Expect: 100-continue" header. Zero means no timeout and
	// causes the body to be sent immediately, without
	// waiting for the server to approve.
	// This time does not include the time to send the request header.
	ExpectContinueTimeout time.Duration

	TLSConfig
}

func DefaultHTTPTransportConfig 

func DefaultHTTPTransportConfig() *HTTPTransportConfig

type HostPortUser 

type HostPortUser struct {
	Host string
	Port string
	*url.Userinfo
}

func ParseHostPortUser 

func ParseHostPortUser(val string) (*HostPortUser, error)

ParseHostPortUser parses a user:password@host:port string into HostUser. User and password cannot be empty.

func (*HostPortUser) Validate 

func (hpu *HostPortUser) Validate() error

type LoggingPACResolver 

type LoggingPACResolver struct {
	Resolver PACResolver
	Logger   log.Logger
}

func (*LoggingPACResolver) FindProxyForURL 

func (r *LoggingPACResolver) FindProxyForURL(u *url.URL, hostname string) (string, error)

type PACResolver 

type PACResolver interface {
	// FindProxyForURL calls FindProxyForURL or FindProxyForURLEx function in the PAC script.
	// The hostname is optional, if empty it will be extracted from URL.
	FindProxyForURL(url *url.URL, hostname string) (string, error)
}

type ProxyFunc 

type ProxyFunc func(*http.Request) (*url.URL, error)

type ProxyLocalhostMode 

type ProxyLocalhostMode string
const (
	DenyProxyLocalhost   ProxyLocalhostMode = "deny"
	AllowProxyLocalhost  ProxyLocalhostMode = "allow"
	DirectProxyLocalhost ProxyLocalhostMode = "direct"
)

func (ProxyLocalhostMode) String 

func (m ProxyLocalhostMode) String() string

func (*ProxyLocalhostMode) UnmarshalText 

func (m *ProxyLocalhostMode) UnmarshalText(text []byte) error

type Scheme 

type Scheme string
const (
	HTTPScheme   Scheme = "http"
	HTTPSScheme  Scheme = "https"
	HTTP2Scheme  Scheme = "h2"
	TunnelScheme Scheme = "tunnel"
)

func (Scheme) String 

func (s Scheme) String() string

type TLSConfig 

type TLSConfig struct {
	// InsecureSkipVerify controls whether a client verifies the server's
	// certificate chain and host name. If InsecureSkipVerify is true, crypto/tls
	// accepts any certificate presented by the server and any host name in that
	// certificate. In this mode, TLS is susceptible to machine-in-the-middle
	// attacks unless custom verification is used. This should be used only for
	// testing or in combination with VerifyConnection or VerifyPeerCertificate.
	InsecureSkipVerify bool

	// CertFile is the path to the TLS certificate.
	CertFile string

	// KeyFile is the path to the TLS private key of the certificate.
	KeyFile string
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
forwarder
forwarder/httpbin
forwarder/pac
forwarder/pac/eval
forwarder/pac/server
forwarder/run
forwarder/version
e2e
compose
run
internal
version
log
stdlog
Package pac provides a PAC file parser and evaluator.
 Package pac provides a PAC file parser and evaluator.
utils
certutil
cobrautil
cobrautil/templates
cobrautil/term
golden
httpbin
httpexpect

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.