goproxy

package module
v1.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 15, 2019 License: BSD-3-Clause Imports: 48 Imported by: 0

README

Introduction

NOTICE: this is a fork of the original elazarl/goproxy with some radical API changes. - abourget

Package goproxy provides a customizable HTTP proxy library for Go (golang),

It supports regular HTTP proxy, HTTPS through CONNECT, "hijacking" HTTPS connection using "Man in the Middle" style attack, and SNI sniffing.

The intent of the proxy, is to be usable with reasonable amount of traffic yet, customizable and programable.

The proxy itself is simply a net/http handler.

In order to use goproxy, one should set his browser to use goproxy as an HTTP proxy. Here is how you do that in Chrome and in Firefox.

For example, the URL you should use as proxy when running ./bin/basic is localhost:8080, as this is the default binding for the basic proxy.

Mailing List

New features would be discussed on the mailing list before their development.

Latest Stable Release

Get the latest goproxy from gopkg.in/elazarl/goproxy.v1.

Why not Fiddler2?

Fiddler is an excellent software with similar intent. However, Fiddler is not as customable as goproxy intend to be. The main difference is, Fiddler is not intended to be used as a real proxy.

A possible use case that suits goproxy but not Fiddler, is, gathering statisitics on page load times for a certain website over a week. With goproxy you could ask all your users to set their proxy to a dedicated machine running a goproxy server. Fiddler is a GUI app not designed to be ran like a server for multiple users.

A taste of goproxy

To get a taste of goproxy, a basic HTTP/HTTPS transparent proxy

import (
    "github.com/elazarl/goproxy"
    "log"
    "net/http"
)

func main() {
    proxy := goproxy.NewProxyHttpServer()
    proxy.Verbose = true
    log.Fatal(proxy.ListenAndServe(":8080"))
}

This line will add X-GoProxy: yxorPoG-X header to all requests sent through the proxy

proxy.HandleRequestFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
    ctx.Req.Header.Set("X-GoProxy","yxorPoG-X")
    return goproxy.NEXT  // continue on with next handlers
    // or, return goproxy.FORWARD  // to short circuit other handlers, and continue on with forwarding
})

Here is a more complex/complete example:

proxy.HandleConnectFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
    if ctx.SNIHost() == "secure.example.com:443" {
        return goproxy.MITM
    }
    return goproxy.REJECT
})
proxy.HandleRequestFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
    if ctx.IsThroughMITM {
        ctx.Req.Header.Set("X-Snooped-On", "absolutely")
    }
    return goproxy.NEXT  // continue on with next handlers
    // or, return goproxy.FORWARD  // to short circuit other handlers, and continue on with forwarding
})

By setting Request Handlers, you can set a response directly and have it short-circuit the proxying to a remote server:

proxy.HandleRequestFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
    if ctx.URL.Path == "/super/bob" {
        //ctx.NewResponse(...)
        //ctx.NewHTMLResponse("<strong>welcome home</strong>")
        ctx.NewTextResponse("welcome home")
    }
    return goproxy.FORWARD
})

See additional examples in the examples directory.

What's New

  1. Major overhaul of API. Pretty much everything will break if you merely try this version.

  2. Ability to do optional SNI sniffing, and take action based on that information.

  3. Ability to FakeDestinationDNS() and SetDestinationHost().. to redirect requests.

  4. Ability to save network flows in .har format. See http://www.softwareishard.com/har/viewer/ and view with this tool: http://ericduran.github.io/chromeHAR/

  5. Ability to Hijack CONNECT requests. See the eavesdropper example

  6. Transparent proxy support for http/https including MITM certificate generation for TLS. See the transparent example.

License

I put the software temporarily under the Go-compatible BSD license, if this prevents someone from using the software, do let mee know and I'll consider changing it.

At any rate, user feedback is very important for me, so I'll be delighted to know if you're using this package.

Beta Software

I've received a positive feedback from a few people who use goproxy in production settings. I believe it is good enough for usage.

I'll try to keep reasonable backwards compatability. In case of a major API change, I'll change the import path.

Documentation

Overview

Taken from $GOROOT/src/pkg/net/http/chunked needed to write https responses to client.

Package goproxy provides a customizable HTTP proxy, supporting hijacking HTTPS connection.

The intent of the proxy, is to be usable with reasonable amount of traffic yet, customizable and programable.

The proxy itself is simply an `net/http` handler.

Typical usage is 

proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = true
proxy.HandleRequestFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
	if ctx.Host() == "example.com:80" {
		ctx.SetDestinationHost("127.0.0.1:8080")
		return goproxy.FORWARD
	}
	return goproxy.NEXT
})
proxy.ListenAndServe(":8080")

Adding a header to each request: 

proxy.HandleRequestFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
	ctx.Req.Header.Set("X-GoProxy","1")
	return goproxy.NEXT
})

For printing the content type of all incoming responses 

proxy.HandleResponseFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
	fmt.Println(ctx.Req.Host, "->", ctx.Req.Header.Get("Content-Type"))
	return goproxy.NEXT
})

Note the use of `ctx.Req` here. The `ctx` holds `Req` and `Resp`. `Resp` can be nil if not available.

To print the content type of all responses from a certain url, we'll add a "middleware" before: 

golangOrgOnly := goproxy.UrlHasPrefix("golang.org/pkg")
logInYourFace := func(ctx *goproxy.ProxyCtx) goproxy.Next {
	fmt.Println(ctx.Req.Host, "->", ctx.Req.Header.Get("Content-Type"))
	return goproxy.NEXT
}
proxy.HandleResponseFunc(golangOrgOnly(logInYourFace))

To invalide responses based on headers for example, you can: 

proxy.HandleResponseFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
	if ctx.Resp.Header.Get("X-GoProxy") == "" {
		ctx.Resp.Close()
		http.Error(ctx.ResponseWriter, "Denied.. didn't find an X-GoProxy header!", 403)
		return goproxy.DONE
	}
	return goproxy.NEXT
})

We close the body of the original repsonse, and return a new 403 response with a short message.

You can catch traffic going through the proxy selectively, and write it to a HAR formatted file with this code: 

proxy.HandleRequestFunc(func(ctx *goproxy.ProxyCtx) goproxy.Next {
	if ctx.Host() == "example.com:80" {
		ctx.LogToHARFile(true)
	}
	return goproxy.NEXT
})
proxy.NonProxyHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	if r.URL.Path == "/har" {
		proxy.FlushHARToDisk("/tmp/file.har")
		w.WriteHeader(201)
		w.Write([]byte("hello world!\n"))
	}
})

You then "curl http://localhost:8888/har" to provoke a file flush to "/tmp/file.har".

Example use cases:

1. https://github.com/abourget/goproxy/tree/master/examples/goproxy-avgsize

To measure the average size of an Html served in your site. One can ask all the QA team to access the website by a proxy, and the proxy will measure the average size of all text/html responses from your host.

2. [not yet implemented]

All requests to your web servers should be directed through the proxy, when the proxy will detect html pieces sent as a response to AJAX request, it'll send a warning email.

3. https://github.com/abourget/goproxy/blob/master/examples/goproxy-httpdump/

Generate a real traffic to your website by real users using through proxy. Record the traffic, and try it again for more real load testing.

4. https://github.com/abourget/goproxy/tree/master/examples/goproxy-no-reddit-at-worktime

Will allow browsing to reddit.com between 8:00am and 17:00pm

5. https://github.com/abourget/goproxy/tree/master/examples/goproxy-jquery-version

Will warn if multiple versions of jquery are used in the same domain.

6. https://github.com/abourget/goproxy/blob/master/examples/goproxy-upside-down-ternet/

Modifies image files in an HTTP response via goproxy's image extension found in ext/.

RLS 8/31/2018 A responsewriter that will pass headers back to the underlying connection. Based on dumbresponsewriter.

6-29-2017 RLS - Added Tlsfailure method so callers can subscribe to TLS handshake failures 7011-2017 RLS - Added tproxy support to capture the original destination of https requests. This enables support for non-SNI clients. 

Responsible for signing certificates based on Winston root certificate.
* TODO: Check validity of certificates on first load rather than simply passing them on blindly to clients.

Index

Constants

View Source 
const (
	NEXT = Next(iota) // Continue to the next Handler
	MOCK
	DONE      // Implies that no further processing is required. The request has been fulfilled completely.
	FORWARD   // Continue directly with forwarding, going through Response Handlers
	MITM      // Continue with Man-in-the-middle attempt, either through HTTP or HTTPS.
	REJECT    // Reject the CONNECT attempt outright
	SIGNATURE // Return the client TLS signature (RLS 3-5-2018)
)

Variables

View Source 
var AlwaysForward = HandlerFunc(func(ctx *ProxyCtx) Next {
	return FORWARD
})
View Source 
var AlwaysMitm = HandlerFunc(func(ctx *ProxyCtx) Next {
	ctx.SNIHost()
	return MITM
})
View Source 
var AlwaysReject = HandlerFunc(func(ctx *ProxyCtx) Next {
	return REJECT
})
View Source 
var CA_CERT = []byte(`-----BEGIN CERTIFICATE-----
MIIChTCCAe6gAwIBAgIBADANBgkqhkiG9w0BAQsFADA2MQswCQYDVQQGEwJVUzEQ
MA4GA1UECgwHV2luc3RvbjEVMBMGA1UEAwwMd2luc3Rvbi5jb25mMB4XDTE4MDIy
MzE4NTExNVoXDTM4MDIxODE4NTExNVowNjELMAkGA1UEBhMCVVMxEDAOBgNVBAoM
B1dpbnN0b24xFTATBgNVBAMMDHdpbnN0b24uY29uZjCBnzANBgkqhkiG9w0BAQEF
AAOBjQAwgYkCgYEAu+LAQcFv/kIbgTPdmFmIcdFVvIONOiK6yWfjObQBwJOaRuZx
FgfojIFdRmm4SvOEhQp+/FZQQwHunDXy1ICNuEGveJyxxVh+FKfhKvwa7HcejZRe
z9BCun4vtSyOBMFkBn/oXKkK3zG7hZKsAjZiftO2m2CKTpR7mRVZHzoFDXMCAwEA
AaOBojCBnzAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NMIEdlbmVy
YXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUcDIKYDmsRzhFGibcyJ0tCX9OYZYw
HwYDVR0jBBgwFoAUV9oF/beeIssH3fAByRVyt8Mh+kIwFwYDVR0RBBAwDoIMd2lu
c3Rvbi5jb25mMAsGA1UdDwQEAwIFoDANBgkqhkiG9w0BAQsFAAOBgQCEhewUly9a
+LGp3HN0JagOk96cD13fmCTTzCtZH6jPXVoV1li2eQqI1UOxSqAnYD9D/KWqKkPg
zngMDIDjRXBIIMlGldKcPqC6/xKQxavGDZV4dvqRByKfPAJoam66nfN0vNaGS6dF
sCs4ajBfOFjOd62D1lpyKXFyW3dt2rxacg==
-----END CERTIFICATE-----`)
View Source 
var CA_KEY = []byte(`-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQC74sBBwW/+QhuBM92YWYhx0VW8g406IrrJZ+M5tAHAk5pG5nEW
B+iMgV1GabhK84SFCn78VlBDAe6cNfLUgI24Qa94nLHFWH4Up+Eq/Brsdx6NlF7P
0EK6fi+1LI4EwWQGf+hcqQrfMbuFkqwCNmJ+07abYIpOlHuZFVkfOgUNcwIDAQAB
AoGAYoxp+VOD8aItGRTiS1HS7pg1Vz7NKcwjmxahqZeQP7lr93pRoJOfV2tXSGKV
ZsLaJIo/1w1S5gKybD8j0nBnZHGnE+5eViuAwWLmHrcQo2Zx8Wsd1ooQce/UsL27
PjimImkLdLrCDPnavD2jrInZtWKfgP0QpaC1OyW3WHlR5OkCQQD0lnF5fbCPWpLy
qCKEQrPxVugf+sccEJ5u8Zww6MWIU8a5/299WsJL8WolnyM+xsZs7+bnx66PUjMR
FO30zDbFAkEAxKcDY2HNripMpg5uG23EGUcqvtjADnTxATrFl72OIpAHaxGsEowu
iJERWNbJnKr0p6Z0JyA/PX5yixpf5ne21wJBAJXaatHNwVRDYQ8NFoDEQW1XGscl
JcK7J+a/XzvUEdpxwasJpmw+JBbVZXyBYN3Aeaga3/UYMYocCa+ojBZU3CUCQGLd
ZEKuhO0uruI9FvYLNS9QLBlOdx/NguyPU4956N9PcatOcyfP+gUGiaYUNb/h4qX2
dSbGe4S68XMli1kejnkCQFNo6CZx31jiy57J5rjJZE6LM7OtSTBtSYJ3aE1ZUPnk
6fjNzFMU+APTWib1YsZQF/mrVh9q52CkA5Nnwpxfkjc=
-----END RSA PRIVATE KEY-----`)
View Source 
var MaxSerialNumber = big.NewInt(0).SetBytes(bytes.Repeat([]byte{255}, 20))

MaxSerialNumber is the upper boundary that is used to create unique serial numbers for the certificate. This can be any unsigned integer up to 20 bytes (2^(8*20)-1).

View Source 
var NonHTTPRequest = "nonhttprequest"
View Source 
var OrganizationName = "Winston Privacy, Inc."

OrganizationName is the name your CA cert will be signed with. It will show in your different UIs. Change it globally here to show meaningful things to your users.

Functions

func CondRemoteAddrIs 

func CondRemoteAddrIs(ctx *ProxyCtx, ip string) bool

func GenerateSignature 

func GenerateSignature(h *vhost.ClientHelloMsg, debug bool) string

func GetSubdomains 

func GetSubdomains(domain string) []string

Returns a list of more general subdomains down to the TLDPlusOne, including the original. ie: "a.b.example.com" returns "a.b.example.com", "b.example.com" and "example.com" Returns most specific to least specific

func HijackedDNSDialer 

func HijackedDNSDialer() *net.Dialer

Returns a DNS dialer which can bypass local DNS

func HostsToMap 

func HostsToMap(hosts ...string) map[string]bool

func IsExternal 

func IsExternal(hostname string) bool

func LoadDefaultConfig 

func LoadDefaultConfig() error

Used for unit testing

func MatchIsLocalhost 

func MatchIsLocalhost(req *http.Request) bool

func MatchRequestHostMap 

func MatchRequestHostMap(req *http.Request, hosts map[string]bool) bool

func NewResponse 

func NewResponse(r *http.Request, status int, contentType, body string) *http.Response

Will generate a valid http response to the given request the response will have the given contentType, and http status. Typical usage, refuse to process requests to local addresses: 

 proxy.HandleRequest(IsLocalhost(HandlerFunc(func(ctx *ProxyCtx) Next {
	    ctx.NewResponse(http.StatusUnauthorized, "text/html", "<html><body>Can't use proxy for local addresses</body></html>")
	    return FORWARD
  })))

func WhitelistedDNSDialer 

func WhitelistedDNSDialer() *net.Dialer

Returns a DNS dialer for port 54 (unfiltered DNS which allows all requests to succeed) TODO: Should we check for DNS server on port 54 and default to port 53 if not available? For now, caller is responsible for this.

Types

type ChainedHandler 

type ChainedHandler func(Handler) Handler

func RemoteAddrIs 

func RemoteAddrIs(ip string) ChainedHandler

MatchRemoteAddr returns a ReqCondtion testing wether the source IP of the request is the given string, Was renamed from `SrcIpIs`.

func RemoteAddrIsNot 

func RemoteAddrIsNot(ip string) ChainedHandler

func ReqHostMatches 

func ReqHostMatches(regexps ...*regexp.Regexp) ChainedHandler

ReqHostMatches is a middleware that tests whether the host to which the request was directed to matches any of the given regular expressions.

func RequestHostContains 

func RequestHostContains(hosts ...string) ChainedHandler

RequestHostContains is a middleware that tests whether the host to which the request is directed to contains one of the given strings.

func RequestHostIsIn 

func RequestHostIsIn(hosts ...string) ChainedHandler

RequestHostIsIn is a middleware that tests whether the host to which the request is directed to equal to one of the given strings.

This matcher supersedes and combines DstHostIs and ReqHostIs.

func RequestHostIsNotIn 

func RequestHostIsNotIn(hosts ...string) ChainedHandler

func RespContentTypeIs 

func RespContentTypeIs(types ...string) ChainedHandler

RespContentTypeIs is a middleware to filter apply a handler only to those requests matching a given content-type. 

imageHandler := HandlerFunc(func(ctx *ProxyCtx) Next {
    ... convert ctx.Resp.Body, reinject a new Body with a png, and change the Content-Type
    return FORWARD  // side-steps further modifications
    // or return NEXT  // to continue on with other Handlers
}

proxy.HandleRequest(RespContentTypeIs("image/jpeg", "image/gif")(imageHandler))

func UrlHasPrefix 

func UrlHasPrefix(prefix string) ChainedHandler

UrlHasPrefix is a middleware matching when the destination URL the proxy client has requested has the given prefix, with or without the host.

For example UrlHasPrefix("host/x") will match requests of the form 'GET host/x', and will match requests to url 'http://host/x'

func UrlIsIn 

func UrlIsIn(urls ...string) ChainedHandler

UrlIsIn is a middleware that tests if the URL matches `urls`, testing whether or not the request URL is one of the given strings with or without the host prefix.

UrlIsIn("google.com/","foo") will match these three cases: * 'GET /' with 'Host: google.com' * 'GET google.com/' * 'GET /foo' for any host

func UrlMatches 

func UrlMatches(re *regexp.Regexp) ChainedHandler

UrlMatches returns a ReqCondition testing whether the destination URL of the request matches the given regexp, with or without prefix

type CounterEncryptorRand 

type CounterEncryptorRand struct {
	// contains filtered or unexported fields
}

func NewCounterEncryptorRandFromKey 

func NewCounterEncryptorRandFromKey(key interface{}, seed []byte) (r CounterEncryptorRand, err error)

func (*CounterEncryptorRand) Read 

func (c *CounterEncryptorRand) Read(b []byte) (n int, err error)

func (*CounterEncryptorRand) Seed 

func (c *CounterEncryptorRand) Seed(b []byte)

type GoproxyConfigServer 

type GoproxyConfigServer struct {
	Root    *x509.Certificate
	RootCAs *x509.CertPool

	Host map[string]*HostInfo
	//Config		map[string]
	IsExternal func(string) bool
	// contains filtered or unexported fields
}

Maintains a global list of immutable TLS Configs which can be used for TLS handshakes. 

var GoproxyCaConfig *GoproxyConfigServer

func LoadCAConfig 

func LoadCAConfig(filename string, caCert, caKey []byte) (*GoproxyConfigServer, error)

Load a CAConfig bundle from by arrays. You can then load them into the proxy with `proxy.SetMITMCertConfig. If filename is non-nil, will attempt to load from disk.

func NewConfigServer 

func NewConfigServer(filename string, ca *x509.Certificate, privateKey interface{}) (*GoproxyConfigServer, error)

NewConfig creates a MITM config using the CA certificate and private key to generate on-the-fly certificates.

func (*GoproxyConfigServer) Cert 

func (c *GoproxyConfigServer) Cert(hostname string) (*tls.Config, error)

RLS 3/19/2018 - exported for testng

func (*GoproxyConfigServer) FlushCert 

func (c *GoproxyConfigServer) FlushCert(hostname string)

Removes the certificate associated with the given hostname from the cache. This is necessary if we change the whitelist or blacklist settings for a domain.

func (*GoproxyConfigServer) GetTestCertificate 

func (c *GoproxyConfigServer) GetTestCertificate(host string, port string) (*tls.Config, error)

Used for unit testing. Gets a certificate from a server and port and creates a local version suitable for MITM.

type Handler 

type Handler interface {
	Handle(ctx *ProxyCtx) Next
}

func IsLocalhost 

func IsLocalhost(chainedHandler Handler) Handler

IsLocalhost checks whether the destination host is explicitly local host (buggy, there can be IPv6 addresses it doesn't catch)

func IsNotLocalhost 

func IsNotLocalhost(chainedHandler Handler) Handler

type HandlerFunc 

type HandlerFunc func(ctx *ProxyCtx) Next

func (HandlerFunc) Handle 

func (f HandlerFunc) Handle(ctx *ProxyCtx) Next

type HostInfo 

type HostInfo struct {
	LastVerify  time.Time
	NextAttempt time.Time // Set to future time for invalid certs to avoid frequent reloading

	Config *tls.Config
	// contains filtered or unexported fields
}

Stores metadata about a particular host. Used to improve performance.

type IdleTimeoutConn 

type IdleTimeoutConn struct {
	Conn        net.Conn
	IdleTimeout int
	Deadline    time.Time
}

Wraps net.conn to enforce idle connection timeout during io.Copy().

func (*IdleTimeoutConn) Close 

func (idleconn *IdleTimeoutConn) Close() error

func (*IdleTimeoutConn) LocalAddr 

func (idleconn *IdleTimeoutConn) LocalAddr() net.Addr

func (*IdleTimeoutConn) Read 

func (idleconn *IdleTimeoutConn) Read(buf []byte) (int, error)

func (*IdleTimeoutConn) RemoteAddr 

func (idleconn *IdleTimeoutConn) RemoteAddr() net.Addr

func (*IdleTimeoutConn) SetDeadline 

func (idleconn *IdleTimeoutConn) SetDeadline(t time.Time) error

func (*IdleTimeoutConn) SetReadDeadline 

func (idleconn *IdleTimeoutConn) SetReadDeadline(t time.Time) error

func (*IdleTimeoutConn) SetWriteDeadline 

func (idleconn *IdleTimeoutConn) SetWriteDeadline(t time.Time) error

func (*IdleTimeoutConn) Write 

func (idleconn *IdleTimeoutConn) Write(buf []byte) (int, error)

type Next 

type Next int

Next indicates the return values possible for handlers.

In `request` handlers, `NEXT` means continue to the next handler. If none are left, it continues on with `FORWARD`. If you return `FORWARD`, then you skip all the other registered request handlers, and forward the request directly to the remote server, unless you've set a `ctx.Resp` before, in which case the response is sent back without touching any Response Handlers. If you return `REJECT`, the library will return a 502 error to the requester. In the `request` handlers, returning `MITM` will panic.

In Response handlers, `NEXT` means continue with the other registered handlers. If this was the last, the library will finish the forwarding of the current `ctx.Resp`. When `DONE` is returned, the library doesn't do anything else. It assumes you have properly written to `ctx.ResponseWriter` with a proper response (along with `ctx.Req`, this would resemble closely native `net/http` request handling). When `FORWARD` is returned, the lib jumps directly to the forwarding step, instead of going through the other response handlers. When `MITM` is returned, the lib will panic. When `REJECT` is returned, the lib will panic also. You can reject a request, but not a response.

In Connect handlers, `NEXT` means continue on with the next Connect handler. `FORWARD` means continue on with forwarding the raw TCP socket. `MITM` means jump in the middle and try to man-in-the-middle the connection. With the `MITM` option, all future requests to be detected from within the tunnel will trigger the Request handlers, in the order they were registered, just like a normal request arriving directly outside a tunnel. Those requests that are MITM will have the `ctx.IsThroughMITM` property set to `true`. Returning `REJECT` will reject the connection altogether. If you did sniff with `SNIHost()`, then the lib will forcefully close the connection, violating the protocol (you wanted to sniff, so it's your fault, right ? :). If you didn't sniff SNI, then a `502 Rejected` will be sent propertly to the client before closing the connection. `DONE` is not valid in that context, you need take a decision.

type NonHTTPRoundTripper 

type NonHTTPRoundTripper struct {
	// TLSClientConfig specifies the TLS configuration to use with
	// tls.Client.
	// Must be non-nil
	TLSClientConfig *tls.Config

	// TLSHandshakeTimeout specifies the maximum amount of time waiting to
	// wait for a TLS handshake. Zero means no timeout.
	TLSHandshakeTimeout time.Duration

	// DialContext must be non-nil. The caller is responsible for providing this function.
	DialContext func(ctx context.Context, network string, addr string) (net.Conn, error)

	MaxResponseHeaderBytes int64

	ExpectContinueTimeout time.Duration
	ResponseHeaderTimeout time.Duration
	// contains filtered or unexported fields
}

func (*NonHTTPRoundTripper) CancelRequest deprecated 

func (t *NonHTTPRoundTripper) CancelRequest(req *http.Request)

CancelRequest cancels an in-flight request by closing its connection. CancelRequest should only be called after RoundTrip has returned.

Deprecated: Use Request.WithContext to create a request with a cancelable context instead. CancelRequest cannot cancel HTTP/2 requests.

func (*NonHTTPRoundTripper) RoundTrip 

func (tr *NonHTTPRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

type ProxyCtx 

type ProxyCtx struct {
	Method            string
	SourceIP          string
	IsSecure          bool   // Whether we are handling an HTTPS request with the client
	IsThroughMITM     bool   // Whether the current request is currently being MITM'd
	IsThroughTunnel   bool   // Whether the current request is going through a CONNECT tunnel, doing HTTP calls (non-secure)
	IsNonHttpProtocol bool   // Set to true if a MITM request is determined to not be a HTTP 1.0-1.2 protocol.
	NonHTTPRequest    []byte // The original request if a non non-HTTP protocol is detected

	MITMCertConfig *GoproxyConfigServer

	// OriginalRequest holds a copy of the request before doing some HTTP tunnelling
	// through CONNECT, or doing a man-in-the-middle attack.
	OriginalRequest *http.Request

	// Contains the request and response streams from the proxy to the
	// downstream server in the case of a MITM connection
	Req            *http.Request
	ResponseWriter http.ResponseWriter

	// Connections, up (the requester) and downstream (the server we forward to)
	Conn net.Conn

	// Resp contains the remote sever's response (if available). This can be nil if the
	// request wasn't sent yet, or if there was an error trying to fetch the response.
	// In this case, refer to `ResponseError` for the latest error.
	// RLS: In the case of MITM, this is the client's response stream
	Resp *http.Response

	// ResponseError contains the last error, if any, after running `ForwardRequest()`
	// explicitly, or implicitly forwarding a request through other means (like returning
	// `FORWARD` in some handlers).
	ResponseError error

	// RoundTripper is used to send a request to a remote server when
	// forwarding a Request.  If you set your own RoundTripper, then
	// `FakeDestinationDNS` and `LogToHARFile` will have no effect.
	RoundTripper RoundTripper

	// will contain the recent error that occured while trying to send receive or parse traffic
	Error error

	// UserObjects and UserData allow you to keep data between
	// Connect, Request and Response handlers.
	UserObjects map[string]interface{}
	UserData    map[string]string

	// Will connect a request to a response
	Session int64
	Proxy   *ProxyHttpServer

	// Closure to alert listeners that a TLS handshake failed
	// RLS 6-29-2017
	Tlsfailure func(ctx *ProxyCtx, untrustedCertificate bool)

	// References to persistent caches for statistics collection
	// RLS 7-5-2017
	IgnoreCounter bool // if true, this request won't be counted (used for streaming)

	// Client signature
	// https://blog.squarelemon.com/tls-fingerprinting/
	CipherSignature string

	NewBodyLength  int
	VerbosityLevel uint16

	// 11/2/2017 - Used for replacement macros (user agents)
	DeviceType    int
	Whitelisted   bool // If true, response filtering will be completely disabled and local DNS will be bypassed.
	TimeRemaining int  // Time remaining in sec for temporary whitelisting

	// Keeps a list of any messages we want to pass back to the client
	StatusMessage []string

	// Request handler sets this to true if it thinks it is a first party request
	FirstParty bool

	// Set to true to use private network
	PrivateNetwork      bool
	RevealTimeRemaining int // Time remaining in sec for temporary uncloaking

	// If a shadow transport is being used, this points to it.
	ShadowTransport *shadownetwork.ShadowTransport

	// If true, then Winston diagnostic information will be recorded about the current request
	Trace traceRequest

	TraceInfo           *TraceInfo // Information about the original request/response
	SkipRequestHandler  bool       // If set to true, then response handler will be skipped
	SkipResponseHandler bool       // If set to true, then response handler will be skipped
	RequestTime         time.Time  // Time the request was started. Useful for debugging.
	Referrer            string     // Referrer taken from HTTP request. Used for logging.
	CookiesModified     int        // # of cookies blocked or modified for the current request. Used for logging.
	ElementsModified    int        // # of page elements removed or modified for the current request. Used for logging.
	// contains filtered or unexported fields
}

ProxyCtx is the Proxy context, contains useful information about every request. It is passed to every user function. Also used as a logger.

func (*ProxyCtx) BufferResponse 

func (ctx *ProxyCtx) BufferResponse() ([]byte, error)

BufferResponse reads the whole Resp.Body and returns a byte array. It is the caller,s responsibility to set a new Body with `SetResponseBody` afterwards. Otherwise, the Resp.Body will be in a closed state and that is not fun for other parts of your program. This replaces the need for goproxy's previous `HandleBytes` implementation.

func (*ProxyCtx) Charset 

func (ctx *ProxyCtx) Charset() string

Will try to infer the character set of the request from the headers. Returns the empty string if we don't know which character set it used. Currently it will look for charset=<charset> in the Content-Type header of the request.

func (*ProxyCtx) DispatchDoneHandlers 

func (ctx *ProxyCtx) DispatchDoneHandlers() error

func (*ProxyCtx) DispatchResponseHandlers 

func (ctx *ProxyCtx) DispatchResponseHandlers() error

func (*ProxyCtx) FakeDestinationDNS 

func (ctx *ProxyCtx) FakeDestinationDNS(host string)

FakeDestinationDNS will force a connection to the specified host/ip instead of the normal DNS resolution of the `SetDestinationHost()`. This will assume the destination server will answer as if it was ctx.Host().

If you specify a port, it will also serve in the redirection, otherwise the port from `ctx.Host()` will be used.

func (*ProxyCtx) FlushHARToDisk 

func (ctx *ProxyCtx) FlushHARToDisk(filename string)

func (*ProxyCtx) ForwardConnect 

func (ctx *ProxyCtx) ForwardConnect() error

This is used to pipe a request directly through to the target site back to the client via MITM. Note that we have to already have a connection to the remote server before calling.

func (*ProxyCtx) ForwardNonHTTPRequest 

func (ctx *ProxyCtx) ForwardNonHTTPRequest(host string) error

Used to forward protocols that are "http-like" (ie: websockets). This method forwards the original HTTP request before fusing the connection, allowing further communication to take place (if none, it will close). TODO: Should we add websockets protocol support so we can inspect packets? TODO: Add P2P support

func (*ProxyCtx) ForwardRequest 

func (ctx *ProxyCtx) ForwardRequest(host string) error

Forwards a request to a downstream server. This is done after MITM has been established. TODO: Remove host from function parameters

func (*ProxyCtx) ForwardResponse 

func (ctx *ProxyCtx) ForwardResponse(resp *http.Response) error

func (*ProxyCtx) HijackConnect 

func (ctx *ProxyCtx) HijackConnect() net.Conn

func (*ProxyCtx) Host 

func (ctx *ProxyCtx) Host() string

Host() returns the "host:port" to which your request will be forwarded. For a CONNECT request, it is preloaded with the original request's "host:port". For other methods, it is preloaded with the request's host and an added port based on the scheme (unless the port was specified).

If you sniff the SNI host with `ctx.SNIHost()`, it will alter the value returned by `Host()` to reflect what was sniffed. You need that to properly MITM secure CONNECT calls, otherwise the remote end will always fail to recognize the certificates this lib signs on-the-fly.

You can alter this value with `SetDestinationHost()`.

func (*ProxyCtx) LogToHARFile 

func (ctx *ProxyCtx) LogToHARFile(captureContent bool) Next

LogToHARFile collects all the content from the Request/Response roundtrip and stores it in memory until you call `FlushHARToDisk(filename)`.. at which point it will all be flushed to disk in HAR file format.

LogToHARFile alwasy returns `NEXT`.

func (*ProxyCtx) Logf 

func (ctx *ProxyCtx) Logf(level uint16, msg string, argv ...interface{})

Logf prints a message to the proxy's log. Should be used in a ProxyHttpServer's filter This message will be printed only if the Verbose field of the ProxyHttpServer is set to true

func (*ProxyCtx) ManInTheMiddle 

func (ctx *ProxyCtx) ManInTheMiddle() error

ManInTheMiddle triggers either a full-fledged MITM when done through HTTPS, otherwise, simply tunnels future HTTP requests through the CONNECT stream, dispatching calls to the Request Handlers

func (*ProxyCtx) ManInTheMiddleHTTPS 

func (ctx *ProxyCtx) ManInTheMiddleHTTPS() error

ManIntheMiddleHTTPS assumes we're dealing with an TLS-wrapped CONNECT tunnel. It will perform a full-blown man-in-the-middle attack, and forward any future requests received from inside the TSL tunnel to the Request Handlers.

Requests in there will be marked `IsSecure = true` (although, you and me know it's not *totally* secure, huh ?). They will also have the `ctx.IsThroughMITM` flag set to true.

The `ctx.OriginalRequest` will also hold the original CONNECT request from which the tunnel originated.

func (*ProxyCtx) NewEmptyGif 

func (ctx *ProxyCtx) NewEmptyGif()

func (*ProxyCtx) NewEmptyImage 

func (ctx *ProxyCtx) NewEmptyImage(extension string)

func (*ProxyCtx) NewEmptyPng 

func (ctx *ProxyCtx) NewEmptyPng()

func (*ProxyCtx) NewEmptyScript 

func (ctx *ProxyCtx) NewEmptyScript()

Returns an empty script. This is done to be less obvious that we blocked it.

func (*ProxyCtx) NewHTMLResponse 

func (ctx *ProxyCtx) NewHTMLResponse(body string)

func (*ProxyCtx) NewResponse 

func (ctx *ProxyCtx) NewResponse(status int, contentType, body string)

func (*ProxyCtx) NewTextResponse 

func (ctx *ProxyCtx) NewTextResponse(body string)

func (*ProxyCtx) RecordStatus 

func (ctx *ProxyCtx) RecordStatus(msg string)

Append a message to the context. This will be sent back to the client as a "Winston-Response" header.

func (*ProxyCtx) RejectConnect 

func (ctx *ProxyCtx) RejectConnect()

func (*ProxyCtx) RoundTrip 

func (ctx *ProxyCtx) RoundTrip(req *http.Request) (*http.Response, error)

func (*ProxyCtx) SNIHost 

func (ctx *ProxyCtx) SNIHost() string

SNIHost will try preempt the TLS handshake and try to sniff the Server Name Indication. It returns `Host()` for non CONNECT requests, so it is always safe to call. If it sniffed successfully, but didn't find anything, it is possible to return an empty string.

func (*ProxyCtx) SetConnectScheme 

func (ctx *ProxyCtx) SetConnectScheme(scheme string)

SetConnectScheme determines how to interprete the TCP conversation following a CONNECT request. `scheme` can be "http" or "https". By default, it uses a simple heuristic: "http" if CONNECT asked for port 80, otherwise it always assumes "https" when trying to man-in-the-middle. Call this before returning `MITM` from Connect Handlers.

func (*ProxyCtx) SetDestinationHost 

func (ctx *ProxyCtx) SetDestinationHost(host string)

SetDestinationHost sets the "host:port" to which you want to FORWARD or MITM a CONNECT request. Otherwise defaults to what was in the `CONNECT` request. If you call `SNIHost()` to sniff SNI, then this will override the destination host automatically.

If you want to alter the destination host of a *Request* that goes through a tunnel you can eavesdrop, modify `ctx.Req.URL.Host`, the RoundTrip will go to that address, even though the `ctx.Req.Host` is used as the `Host:` header. You can identify those requests with `ctx.IsThroughMITM` or `ctx.IsThroughTunnel`.

func (*ProxyCtx) SetRequestBody 

func (ctx *ProxyCtx) SetRequestBody(req *http.Request, body io.Reader) error

Grafts the provided io.Closer to the provided Request body. Request.Body will be consumed and closed by the native Golang Client methods Do, Post, and PostForm, and Transport.RoundTrip.

If body is of type *bytes.Buffer, *bytes.Reader, or *strings.Reader, the returned request's ContentLength is set to its exact value (instead of -1), GetBody is populated (so 307 and 308 redirects can replay the body), and Body is set to NoBody if the ContentLength is 0.

func (*ProxyCtx) SetResponseBody 

func (ctx *ProxyCtx) SetResponseBody(content []byte)

SetResponseBody overwrites the Resp.Body with the given content. It is the caller's responsibility to ensure the previous Body was read and/or closed properly. Use `BufferResponse()` for that. This call will fail if ctx.Resp is nil.

func (*ProxyCtx) TunnelHTTP 

func (ctx *ProxyCtx) TunnelHTTP() error

TunnelHTTP assumes the current connection is a plain HTTP tunnel, with no security. It then dispatches all future requests in there through the registered Request Handlers.

Requests flowing through this tunnel will be marked `ctx.IsThroughTunnel == true`.

You can also find the original CONNECT request in `ctx.OriginalRequest`.

func (*ProxyCtx) Warnf 

func (ctx *ProxyCtx) Warnf(msg string, argv ...interface{})

Warnf prints a message to the proxy's log. Should be used in a ProxyHttpServer's filter This message will always be printed.

type ProxyHttpServer 

type ProxyHttpServer struct {

	// setting Verbose to true will log information on each request sent to the proxy
	Verbose bool

	// 0 (default) = Startup, service messages  and command output only
	// 1 Serious Errors
	// 2 HTTP/HTTPS blocked
	// 3 HTTP/HTTPS OK
	// 4 White/Blacklisting decisions
	// 5 Image files
	// 6 Warnings
	// 7 Partial content (status code 206) ?
	// 8 ElementHiding matches
	// 9 Allowed/Blocked Statistics logging
	VerbosityLevel uint16 //int

	// SniffSNI enables sniffing Server Name Indicator when doing CONNECT calls.  It will
	// thus answer to CONNECT calls with a "200 OK" even if the remote server might not
	// answer.  The result would be the shutdown of the connection instead of an appropriate
	// HTTP error code if the remote node doesn't answer.
	SniffSNI bool
	Logger   *log.Logger

	// NonProxyHandler will be used to handle direct connections to the proxy. You can
	// assign an `http.ServeMux` or some other routing libs here.  The default will return
	// a 500 error saying this is a proxy and has nothing to serve by itself.
	NonProxyHandler http.Handler

	// Custom transport to be used
	Transport *http.Transport

	// Private transports
	//PrivateNetwork *shadowtransport.PrivateNetwork
	PrivateNetwork *shadownetwork.ShadowNetwork

	// Setting MITMCertConfig allows you to override the default CA cert/key used to sign MITM'd requests.
	MITMCertConfig *GoproxyConfigServer

	// ConnectDial will be used to create TCP connections for CONNECT requests
	// if nil, .Transport.Dial will be used
	ConnectDial func(network string, addr string) (net.Conn, error)

	// RLS 2/15/2018 - New context version of ConnectDial
	ConnectDialContext func(ctx context.Context, network string, addr string) (net.Conn, error)

	// Callback function to determine if request should be traced.
	Trace func(ctx *ProxyCtx) traceRequest

	// Closure to alert listeners that a TLS handshake failed
	// RLS 6-29-2017
	Tlsfailure func(ctx *ProxyCtx, untrustedCertificate bool)

	// Closure to give listeners a chance to service a request directly. Return true if handled.
	HandleHTTP func(ctx *ProxyCtx) bool

	// If set to true, then the next HTTP request will flush all idle connections. Will be reset to false afterwards.
	FlushIdleConnections bool

	// RoundTripper which supports non-http protocols
	NonHTTPRoundTripper *NonHTTPRoundTripper

	UpdateAllowedCounter     func(string, string, string, int, int, int)
	UpdateBlockedCounter     func(string, string, string, int, bool)
	UpdateWhitelistedCounter func(string, string, string, int)
	// contains filtered or unexported fields
}

The basic proxy type. Implements http.Handler.

func NewProxyHttpServer 

func NewProxyHttpServer() *ProxyHttpServer

New proxy server, logs to StdErr by default

func (*ProxyHttpServer) BufferLogEntry 

func (proxy *ProxyHttpServer) BufferLogEntry(signature, entry string)

func (*ProxyHttpServer) DispatchRequestHandlers 

func (proxy *ProxyHttpServer) DispatchRequestHandlers(ctx *ProxyCtx)

RLS 5/22/2018 - exported so that we can use it for unit testing

func (*ProxyHttpServer) FlushHARToDisk 

func (proxy *ProxyHttpServer) FlushHARToDisk(filename string)

func (*ProxyHttpServer) GetLogEntries 

func (proxy *ProxyHttpServer) GetLogEntries(signature string) []string

Retrieves the log entries for the given signature since the prior call. If there was no prior call, it begins buffering log entries for the next 30 seconds. The client must poll this function more frequently to obtain a complete log.

func (*ProxyHttpServer) HandleConnect 

func (proxy *ProxyHttpServer) HandleConnect(f Handler)

func (*ProxyHttpServer) HandleConnectFunc 

func (proxy *ProxyHttpServer) HandleConnectFunc(f func(ctx *ProxyCtx) Next)

HandleConnectFunc and HandleConnect mimic the `net/http` handlers, and register handlers for CONNECT proxy calls.

See `Next` values for the return value meaning

func (*ProxyHttpServer) HandleDone 

func (proxy *ProxyHttpServer) HandleDone(f Handler)

func (*ProxyHttpServer) HandleDoneFunc 

func (proxy *ProxyHttpServer) HandleDoneFunc(f func(ctx *ProxyCtx) Next)

HandleDoneFunc and HandleDone are called at the end of every request. Use them to cleanup.

See `Next` values for the return value meaning

func (*ProxyHttpServer) HandleRequest 

func (proxy *ProxyHttpServer) HandleRequest(f Handler)

func (*ProxyHttpServer) HandleRequestFunc 

func (proxy *ProxyHttpServer) HandleRequestFunc(f func(ctx *ProxyCtx) Next)

HandleRequestFunc and HandleRequest put hooks to handle certain requests. Note that MITM'd and HTTP requests that go through a CONNECT'd connection also go through those Request Handlers.

See `Next` values for the return value meaning

func (*ProxyHttpServer) HandleResponse 

func (proxy *ProxyHttpServer) HandleResponse(f Handler)

func (*ProxyHttpServer) HandleResponseFunc 

func (proxy *ProxyHttpServer) HandleResponseFunc(f func(ctx *ProxyCtx) Next)

HandleResponseFunc and HandleResponse put hooks to handle certain requests. Note that MITM'd and HTTP requests that go through a CONNECT'd connection also go through those Response Handlers.

See `Next` values for the return value meaning

func (*ProxyHttpServer) LastSignature 

func (proxy *ProxyHttpServer) LastSignature() string

func (*ProxyHttpServer) ListenAndServe 

func (proxy *ProxyHttpServer) ListenAndServe(addr string) error

ListenAndServe launches all the servers required and listens. Use this method if you want to start listeners for transparent proxying.

func (*ProxyHttpServer) ListenAndServeTLS 

func (proxy *ProxyHttpServer) ListenAndServeTLS(httpsAddr string) error

func (*ProxyHttpServer) Logf 

func (proxy *ProxyHttpServer) Logf(level uint16, msg string, v ...interface{})

RLS 8/16/2017 Logging now supports multiple levels of verbosity

func (*ProxyHttpServer) NewConnectDialToProxy 

func (proxy *ProxyHttpServer) NewConnectDialToProxy(https_proxy string) func(network, addr string) (net.Conn, error)

func (*ProxyHttpServer) NewConnectDialToProxyContext 

func (proxy *ProxyHttpServer) NewConnectDialToProxyContext(https_proxy string) func(ctx context.Context, network, addr string) (net.Conn, error)

func (*ProxyHttpServer) ServeHTTP 

func (proxy *ProxyHttpServer) ServeHTTP(w http.ResponseWriter, r *http.Request)

Standard net/http function. Shouldn't be used directly, http.Serve will use it.

func (*ProxyHttpServer) SetMITMCertConfig 

func (proxy *ProxyHttpServer) SetMITMCertConfig(config *GoproxyConfigServer)

SetMITMCertConfig sets the CA Config to be used to sign man-in-the-middle'd certificates. You can load some []byte with `LoadCAConfig()`. This bundle gets passed into the `ProxyCtx` and may be overridden in the [TODO: FIXME] `HandleConnect()` callback, before doing SNI sniffing.

func (*ProxyHttpServer) SetShadowNetwork 

func (proxy *ProxyHttpServer) SetShadowNetwork(sn *shadownetwork.ShadowNetwork)

Call after the private network has been initialized to have proxy automatically redirect requests through it. The proxy will simply forward requests through the local network until this is called.

func (*ProxyHttpServer) SetSignature 

func (proxy *ProxyHttpServer) SetSignature(signature string)

func (*ProxyHttpServer) TestConnectDialContext 

func (proxy *ProxyHttpServer) TestConnectDialContext(ctx context.Context, network, addr string) (c net.Conn, err error)

Don't use - this is for unit testing purposes only.

type RequestTracer 

type RequestTracer struct {
	Requests []traceRequest
	// contains filtered or unexported fields
}

func (*RequestTracer) RequestTrace 

func (tr *RequestTracer) RequestTrace(cmd []string, seconds int)
 Requests a trace. By default, will be disabled after two minutes if not triggered.
	[Host] -> Required

Optional parameters: 

modified - display modified trace for next request only
unmodified - display the original trace for next request only
SkipRequest - skip request handling
SkipResponse - skip response handling
SkipInject - skip toolbar and monitor
SkipPrivate - Bypass the private network
SkipMonitor - Bypass the javascript monitor injection
SkipToolbar - Bypass the toolbar injection code

func (*RequestTracer) Trace 

func (tr *RequestTracer) Trace(ctx *ProxyCtx) traceRequest

Returns a trace request if one has been registered for the given ctx

type RoundTripper 

type RoundTripper interface {
	RoundTrip(req *http.Request, ctx *ProxyCtx) (*http.Response, error)
}

type RoundTripperFunc 

type RoundTripperFunc func(req *http.Request, ctx *ProxyCtx) (*http.Response, error)

func (RoundTripperFunc) RoundTrip 

func (f RoundTripperFunc) RoundTrip(req *http.Request, ctx *ProxyCtx) (*http.Response, error)

type SpyConnection 

type SpyConnection struct {
	net.Conn
}

SpyConnection embeds a net.Conn, all reads and writes are output to stderr

func (*SpyConnection) Read 

func (sc *SpyConnection) Read(b []byte) (int, error)

Read writes all data read from the underlying connection to stderr

func (*SpyConnection) Write 

func (sc *SpyConnection) Write(b []byte) (int, error)

Write writes all data written to the underlying connection to stderr

type TraceInfo 

type TraceInfo struct {
	Name            string // Will be printed out at beginning of trace output.
	RequestTime     time.Time
	RequestDuration time.Duration // Time needed to complete the request
	RequestHeaders  []string

	ResponseHeaders []string
	PrivateNetwork  bool     // If true, the request was cloaked
	MITM            bool     // if true, then we were able to intercept the request. Wil be false for clients which don't trust us.
	RoundTripError  string   // Errors recorded by roundtrip
	CookiesSent     []string // Cookies sent with the request
	CookiesReceived []string // Cookies received from the server
	StatusCode      int      // status code of the server response
	ReqBody         *[]byte  // This is a copy of the original request body (used in POSTs) if needed to replay.
	Method          *string  // The original request method.
	// contains filtered or unexported fields
}

Used to store information about a roundtrip.

Source Files

View all Source files

Directories

Path Synopsis
ext
auth
html
extension to goproxy that will allow you to easily filter web browser related content.
 extension to goproxy that will allow you to easily filter web browser related content.
image
Instructions: The following code reproduces a real-world test case encountered in a production * Winston unit.
 Instructions: The following code reproduces a real-world test case encountered in a production * Winston unit.
Reproduces WINSTON-379 - Hashmap collision panics during TLS handshake.
 Reproduces WINSTON-379 - Hashmap collision panics during TLS handshake.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.