README
¶
Proxy
This project provides a proxy server that can proxy multiple websites with a custom http.Transport.
Install
go get github.com/FrauElster/proxy
Why?
I have a automated browser which crawles websites, and I wanted it to use a VPN without the system its running on needing to connect to one. Since I dont have control over the http transport the browser uses, I had to come up with this proxy.
What exactly can it do?
It is basically a man in the middle between your requesting client and the actual website.
Lets say you want to GET www.google.com/search?q=hello+world, you would setup the proxy and send you request to GET localhost:8080/google/search?q=hello+world. The proxy would request the website over a custom http.Transport (e.g. using SOCKS5), swap all URL that resolve to www.google.com with localhost:8080/google and forwards the response to the client.
How to use?
import (
goproxy "golang.org/x/net/proxy"
)
func main() {
// build a custom transport, this can be any http.RoundTripper,
socksAddr := os.Getenv("SOCKS5_PROXY")
user := os.Getenv("SOCKS5_USER")
pass := os.Getenv("SOCKS5_PASS")
transport := proxy.NewStealthTransport(
proxy.WithSocks5(socksAddr, &goproxy.Auth{User: user, Password: pass}),
proxy.WithUserAgents(proxy.CommonUserAgents...)
)
// define website to forward to
targets := []proxy.Target({BaseUrl: "https://www.google.com", Prefix: "/google/"})
// build proxy
addr, _ := url.Parse("http://0.0.0.0:8080")
p, err := proxy.NewProxy(targets,
proxy.WithTransport(transport),
proxy.WithAddr(addr)
)
if err != nil {
panic(err)
}
// start the server
err := proxy.ListenAndServe()
if err != nil && err != http.ErrServerClosed {
panic(err)
}
}
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var CommonUserAgents = []string{
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537",
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko",
"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36 Edge/16.16299",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8",
"Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; AS; rv:11.0) like Gecko",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586",
"Mozilla/5.0 (Windows NT 6.1; Trident/7.0; AS; rv:11.0) like Gecko",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 Edge/16.16299",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0",
"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36",
}
var WithCompression = func(s *stealthTransport) { s.compression = true }
WithCompression enables compression for the stealth transport
Functions ¶
func GenerateSslCerts ¶
func GenerateSslCerts(caOrganisation string) (tls.Certificate, error)
func NewProxy ¶
func NewProxy(targets []Target, opts ...ProxyOption) (*proxy, error)
func NewStealthTransport ¶
func NewStealthTransport(opts ...StealthOption) *stealthTransport
Types ¶
type ProxyOption ¶
type ProxyOption func(*proxy)
func WithAddr ¶
func WithAddr(addr *url.URL) ProxyOption
WithAddr sets the address used by the proxy server
func WithSsl ¶
func WithSsl(cert tls.Certificate) ProxyOption
WithSsl enables SSL for the proxy server ListenAndServe will use http.ListenAndServeTLS instead of http.ListenAndServe
func WithTransport ¶
func WithTransport(transport http.RoundTripper) ProxyOption
WithTransport sets the transport used by the proxy server
type StealthOption ¶
type StealthOption func(*stealthTransport)
func WithDelay ¶
func WithDelay(min, max time.Duration) StealthOption
WithDelay sets the minimum and maximum delay between requests the actual delay will be a random value between min and max
func WithSocks5 ¶
func WithSocks5(proxyAddr string, auth *goProxy.Auth) StealthOption
WithSocks5 sets the SOCKS5 proxy used by the stealth transport
func WithUserAgents ¶
func WithUserAgents(agents ...string) StealthOption
WithUserAgents the stealth transport will randomly choose one of the given user agents the most common user agents can be found in CommonUserAgents
type SupportedCompression ¶
type SupportedCompression string
const ( Gzip SupportedCompression = "gzip" Deflate SupportedCompression = "deflate" Brotli SupportedCompression = "br" )
Source Files