transports

README

transports

A HTTP proxy that aims to support different transports.

Motivation

I started the facebook tunnel project two years ago and I thought that it could be better to follow a modular approach for supporting other services (chat systems, platforms, *gram, *book?).

This repository includes some code to explore the idea.

Why not a TCP/UDP tunnel?

At this time I'm not planning tuntap support (like I did in the previous project). I would like to focus on the transports. Also, I think that a HTTP proxy is easier to port and run, especially when considering that the project is built on Golang, where the output is a static binary. For example, it'll be very easy to build a binary for ARM.

Available transports

I've been working on these transports during the past week:

Facebook Transport (early stage, sorry!)

This transport uses surf, a stateful web browser built in Go.

Load your credentials by using export or the .env file:

FB_LOGIN=youraccount@facebook.com
FB_PASSWORD=supersecretpass
FB_FRIEND=yourtunnelfriend

I'm looking for collaborators from countries where the Internet.org campaigns like "Free Basics" are active, they could benefit from it :)

Whatsapp Transport (status: you can perform some GETs)

This transport uses a HTTP wrapper for yowsup to send/receive Whatsapp messages.

I recorded this small video, showing some interactions with this transport. For the demonstration I point my browser to the proxy and perform a test request to Akamai, the communication happens between two Whatsapp clients running on the same computer:

It would be good to have a "pure Golang" Whatsapp library but I think the current approach is fine for experimentation (anyone considering writing this?).

The following environment variables are used:

WA_CLIENT_LOGIN=123412341
WA_CLIENT_PASSWORD=whatsappgeneratedpassword123
WA_CLIENT_CONTACT=43214321

WA_SERVER_LOGIN=123412341
WA_SERVER_PASSWORD=whatsappgeneratedpassword123

Requires Python 3

Marshalers

I'm working on providing a set of "marshalers" and a simple API to combine them, this could be useful for conducting network/system usage benchmark experiments & performing a good choice.

Protocol buffers sound like a good option, instead of JSON (which is what I'm actually using for the Whatsapp transport). Also brotli looks promising. A combination of these two is a very interesting thing to consider.

Tor support

There's a Tor integration idea for the server side, I've did a few tests with it, under the following scenario:

The Whatsapp transport includes a flag to enable Tor (UseTor), I'm expecting to extend this flag to any transport.

The Tor support is achieved by overriding the default http.Transport dialer with a custom one.

Ideas/to-do

• Handle SSL.
• Extend the test coverage.
• Think about some way of debugging the transports/traffic.
• Run basic tests in additional platforms (e.g. ARM).
• Plan strategies for mobile support, Golang is doing great progress on this.
• Multiplexing

Contributors

• Matias Insaurralde

• Carlos Carvallo

License

MIT

Documentation

Index

• Constants
• Variables
• func MarshalRequest(request *http.Request) []byte
• func TorDialer() proxy.Dialer
• type DefaultSerializer
  • func (serializer *DefaultSerializer) DeserializeRequest(Input []byte) *http.Request
  • func (serializer *DefaultSerializer) DeserializeResponse(Input []byte) Response
  • func (serializer *DefaultSerializer) Serialize(req interface{}, jsonOutput bool) interface{}
• type FacebookTransport
  • func (t *FacebookTransport) DoLogin() bool
  • func (t *FacebookTransport) Handler(w http.ResponseWriter, originalRequest *http.Request)
  • func (t *FacebookTransport) Listen()
  • func (t *FacebookTransport) Prepare()
• type FreeFBTransport
  • func (t *FreeFBTransport) DoLogin() bool
  • func (t *FreeFBTransport) Handler(w http.ResponseWriter, originalRequest *http.Request)
  • func (t *FreeFBTransport) Prepare()
• type Proxy
  • func (proxy *Proxy) Listen()
• type Request
• type Response
• type Transport
  • func (t *Transport) Handler(w http.ResponseWriter, req *http.Request)
  • func (t *Transport) Listen()
  • func (t *Transport) Prepare()
• type WhatsappMessage
• type WhatsappMessageCallback
• type WhatsappTransport
  • func (t *WhatsappTransport) DaemonizeWrapper()
  • func (t *WhatsappTransport) FetchMessages()
  • func (t *WhatsappTransport) GetMessageIDs() []string
  • func (t *WhatsappTransport) HandleClientMessages()
  • func (t *WhatsappTransport) Handler(w http.ResponseWriter, originalRequest *http.Request)
  • func (t *WhatsappTransport) Listen(Callback WhatsappMessageCallback)
  • func (t *WhatsappTransport) Prepare()
  • func (t *WhatsappTransport) PurgeMessage(Id string)
  • func (t *WhatsappTransport) SendMessage(body string)

Constants

const YowsupHTTPWrapperPath = "../yowsup-http-wrapper/run.py"

Variables

var ResponseChannel chan Response

Functions

func MarshalRequest

func MarshalRequest(request *http.Request) []byte

func TorDialer

func TorDialer() proxy.Dialer

Types

type DefaultSerializer

type DefaultSerializer struct {
}

func (*DefaultSerializer) DeserializeRequest

func (serializer *DefaultSerializer) DeserializeRequest(Input []byte) *http.Request

func (*DefaultSerializer) DeserializeResponse

func (serializer *DefaultSerializer) DeserializeResponse(Input []byte) Response

func (*DefaultSerializer) Serialize

func (serializer *DefaultSerializer) Serialize(req interface{}, jsonOutput bool) interface{}

type FacebookTransport

type FacebookTransport struct {
	*Transport
	Login      string
	Password   string
	Friend     string
	Browser    *browser.Browser
	Serializer DefaultSerializer
	ChatURL    string
}

func (*FacebookTransport) DoLogin

func (t *FacebookTransport) DoLogin() bool

func (*FacebookTransport) Handler

func (t *FacebookTransport) Handler(w http.ResponseWriter, originalRequest *http.Request)

func (*FacebookTransport) Listen

func (t *FacebookTransport) Listen()

func (*FacebookTransport) Prepare

func (t *FacebookTransport) Prepare()

type FreeFBTransport

type FreeFBTransport struct {
	*Transport
	Login      string
	Password   string
	Friend     string
	Browser    *browser.Browser
	Serializer DefaultSerializer
	ChatUrl    string
}

func (*FreeFBTransport) DoLogin

func (t *FreeFBTransport) DoLogin() bool

func (*FreeFBTransport) Handler

func (t *FreeFBTransport) Handler(w http.ResponseWriter, originalRequest *http.Request)

func (*FreeFBTransport) Prepare

func (t *FreeFBTransport) Prepare()

type Proxy

type Proxy struct {
	Port      int
	Transport interface{}
}

func (*Proxy) Listen

func (proxy *Proxy) Listen()

type Request

type Request struct {
	Method  string
	URL     string
	Proto   string
	Headers map[string][]string
}

type Response

type Response struct {
	Status     string
	StatusCode int
	Proto      string
	Headers    map[string][]string
	Body       string
}

type Transport

type Transport struct {
	Name string
}

func (*Transport) Handler

func (t *Transport) Handler(w http.ResponseWriter, req *http.Request)

func (*Transport) Listen

func (t *Transport) Listen()

func (*Transport) Prepare

func (t *Transport) Prepare()

type WhatsappMessage

type WhatsappMessage struct {
	ID     string `json:"id,omitempty"`
	Body   string `json:"msg,omitempty"`
	Origin string `json:"origin,omitempty"`
	Dest   string `json:"dest,omitempty"`
}

type WhatsappMessageCallback

type WhatsappMessageCallback func(*WhatsappTransport)

type WhatsappTransport

type WhatsappTransport struct {
	*Transport
	Login             string
	Password          string
	Contact           string
	YowsupWrapperPort string
	YowsupWrapperURL  string
	UseTor            bool
	Serializer        DefaultSerializer
	Messages          []WhatsappMessage
}

func (*WhatsappTransport) DaemonizeWrapper

func (t *WhatsappTransport) DaemonizeWrapper()

func (*WhatsappTransport) FetchMessages

func (t *WhatsappTransport) FetchMessages()

func (*WhatsappTransport) GetMessageIDs

func (t *WhatsappTransport) GetMessageIDs() []string

func (*WhatsappTransport) HandleClientMessages

func (t *WhatsappTransport) HandleClientMessages()

func (*WhatsappTransport) Handler

func (t *WhatsappTransport) Handler(w http.ResponseWriter, originalRequest *http.Request)

func (*WhatsappTransport) Listen

func (t *WhatsappTransport) Listen(Callback WhatsappMessageCallback)

func (*WhatsappTransport) Prepare

func (t *WhatsappTransport) Prepare()

func (*WhatsappTransport) PurgeMessage

func (t *WhatsappTransport) PurgeMessage(Id string)

func (*WhatsappTransport) SendMessage

func (t *WhatsappTransport) SendMessage(body string)