httpteleport

README

httpteleport

Teleports 10Gbps http traffic over 1Gbps networks. Built on top of fastrpc.

Use cases

httpteleport may significantly reduce inter-server network bandwidth overhead and costs for the following cases:

• RTB servers.
• HTTP-based API servers (aka REST, JSON, JSON-RPC or HTTP-RPC services and microservices).
• Reverse proxies.
• Load balancers.

How does it work?

It just sends batched http requests and responses over a single compressed connection. This solves the following issues:

• High network bandwidth usage
• High network packets rate
• A lot of open TCP connections

Unlike http pipelining, httpteleport responses may be sent out-of-order. This resolves head of line blocking issue.

Links

• Docs

• httptp - standalone single-binary reverse proxy and load balancer based on httpteleport. httptp source code may be used as an example of httpteleport usage.

FAQ

• Q: Why httpteleport doesn't use HTTP/2.0?

  A: Because http/2.0 has many features, which aren't used by httpteleport. More features complicate the code, make it more error-prone and may slow it down.

• Q: Why does httpteleport provide fasthttp- based API instead of standard net/http- based API?

  A: Because httpteleport is optimized for speed. So it have to use fasthttp for http-related stuff to be fast.

• Q: Give me performance numbers.

  A: httpteleport achieves 200K qps on a single CPU core in end-to-end test, where a client sends requests to a local server and the server sends responses back to the client:

$ GOMAXPROCS=1 go test -bench=. -benchmem
goos: linux
goarch: amd64
pkg: github.com/valyala/httpteleport
BenchmarkEndToEndGetNoDelay1          	  300000	      4346 ns/op	  60.05 MB/s	       0 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay10         	  300000	      4370 ns/op	  59.71 MB/s	       3 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay100        	  300000	      4406 ns/op	  59.23 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay1000       	  300000	      4457 ns/op	  58.55 MB/s	      24 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay10K        	  300000	      5868 ns/op	  44.48 MB/s	     178 B/op	       1 allocs/op
BenchmarkEndToEndGetDelay1ms          	  300000	      4771 ns/op	  54.70 MB/s	      21 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay2ms          	  200000	      7943 ns/op	  32.86 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay4ms          	  200000	      7741 ns/op	  33.71 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay8ms          	  200000	     10580 ns/op	  24.67 MB/s	      26 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay16ms         	  100000	     16923 ns/op	  15.42 MB/s	      50 B/op	       0 allocs/op
BenchmarkEndToEndGetCompressNone      	  200000	      7899 ns/op	  33.04 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndGetCompressFlate     	  100000	     13257 ns/op	  19.69 MB/s	     129 B/op	       0 allocs/op
BenchmarkEndToEndGetCompressSnappy    	  200000	      8158 ns/op	  31.99 MB/s	      40 B/op	       0 allocs/op
BenchmarkEndToEndGetTLSCompressNone   	  200000	      8692 ns/op	  30.02 MB/s	      39 B/op	       0 allocs/op
BenchmarkEndToEndGetTLSCompressFlate  	  100000	     13710 ns/op	  19.04 MB/s	     131 B/op	       0 allocs/op
BenchmarkEndToEndGetTLSCompressSnappy 	  200000	      8480 ns/op	  30.78 MB/s	      42 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline1         	  300000	      4673 ns/op	  55.85 MB/s	       0 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline10        	  300000	      4610 ns/op	  56.61 MB/s	       3 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline100       	  300000	      4576 ns/op	  57.03 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline1000      	  300000	      4886 ns/op	  53.41 MB/s	      26 B/op	       0 allocs/op

Documentation

Index

• Constants
• Variables
• type Client
  • func (c *Client) DoDeadline(req *fasthttp.Request, resp *fasthttp.Response, deadline time.Time) error
  • func (c *Client) DoTimeout(req *fasthttp.Request, resp *fasthttp.Response, timeout time.Duration) error
  • func (c *Client) PendingRequests() int
• type CompressType
• type Server
  • func (s *Server) ListenAndServe(addr string) error
  • func (s *Server) Serve(ln net.Listener) error

Constants

const (
	// CompressNone disables connection compression.
	//
	// CompressNone may be used in the following cases:
	//
	//   * If network bandwidth between client and server is unlimited.
	//   * If client and server are located on the same physical host.
	//   * If other CompressType values consume a lot of CPU resources.
	//
	CompressNone = CompressType(fastrpc.CompressNone)

	// CompressFlate uses compress/flate with default
	// compression level for connection compression.
	//
	// CompressFlate may be used in the following cases:
	//
	//     * If network bandwidth between client and server is limited.
	//     * If client and server are located on distinct physical hosts.
	//     * If both client and server have enough CPU resources
	//       for compression processing.
	//
	CompressFlate = CompressType(fastrpc.CompressFlate)

	// CompressSnappy uses snappy compression.
	//
	// CompressSnappy vs CompressFlate comparison:
	//
	//     * CompressSnappy consumes less CPU resources.
	//     * CompressSnappy consumes more network bandwidth.
	//
	CompressSnappy = CompressType(fastrpc.CompressSnappy)
)

Variables

var (
	// ErrTimeout is returned from timed out calls.
	ErrTimeout = fastrpc.ErrTimeout

	// ErrPendingRequestsOverflow is returned when Client cannot send
	// more requests to the server due to Client.MaxPendingRequests limit.
	ErrPendingRequestsOverflow = fastrpc.ErrPendingRequestsOverflow
)

Types

type Client

type Client struct {
	// Addr is the httpteleport Server address to connect to.
	Addr string

	// CompressType is the compression type used for requests.
	//
	// CompressFlate is used by default.
	CompressType CompressType

	// Dial is a custom function used for connecting to the Server.
	//
	// fasthttp.Dial is used by default.
	Dial func(addr string) (net.Conn, error)

	// TLSConfig is TLS (aka SSL) config used for establishing encrypted
	// connection to the server.
	//
	// Encrypted connections may be used for transferring sensitive
	// information over untrusted networks.
	//
	// By default connection to the server isn't encrypted.
	TLSConfig *tls.Config

	// MaxPendingRequests is the maximum number of pending requests
	// the client may issue until the server responds to them.
	//
	// DefaultMaxPendingRequests is used by default.
	MaxPendingRequests int

	// MaxBatchDelay is the maximum duration before pending requests
	// are sent to the server.
	//
	// Requests' batching may reduce network bandwidth usage and CPU usage.
	//
	// By default requests are sent immediately to the server.
	MaxBatchDelay time.Duration

	// Maximum duration for full response reading (including body).
	//
	// This also limits idle connection lifetime duration.
	//
	// By default response read timeout is unlimited.
	ReadTimeout time.Duration

	// Maximum duration for full request writing (including body).
	//
	// By default request write timeout is unlimited.
	WriteTimeout time.Duration

	// ReadBufferSize is the size for read buffer.
	//
	// DefaultReadBufferSize is used by default.
	ReadBufferSize int

	// WriteBufferSize is the size for write buffer.
	//
	// DefaultWriteBufferSize is used by default.
	WriteBufferSize int
	// contains filtered or unexported fields
}

Client teleports http requests to the given httpteleport Server over a single connection.

Use multiple clients for establishing multiple connections to the server if a single connection processing consumes 100% of a single CPU core on either multi-core client or server.

func (*Client) DoDeadline

func (c *Client) DoDeadline(req *fasthttp.Request, resp *fasthttp.Response, deadline time.Time) error

DoDeadline teleports the given request to the server set in Client.Addr.

ErrTimeout is returned if the server didn't return response until the given deadline.

func (*Client) DoTimeout

func (c *Client) DoTimeout(req *fasthttp.Request, resp *fasthttp.Response, timeout time.Duration) error

DoTimeout teleports the given request to the server set in Client.Addr.

ErrTimeout is returned if the server didn't return response during the given timeout.

func (*Client) PendingRequests

func (c *Client) PendingRequests() int

PendingRequests returns the number of pending requests at the moment.

This function may be used either for informational purposes or for load balancing purposes.

type CompressType

type CompressType byte

CompressType is a compression type used for connections.

type Server

type Server struct {
	// Handler must process incoming http requests.
	//
	// Handler mustn't use the following features:
	//
	//   - Connection hijacking, i.e. RequestCtx.Hijack
	//   - Streamed response bodies, i.e. RequestCtx.*BodyStream*
	Handler fasthttp.RequestHandler

	// CompressType is the compression type used for responses.
	//
	// CompressFlate is used by default.
	CompressType CompressType

	// Concurrency is the maximum number of concurrent goroutines
	// with Server.Handler the server may run.
	//
	// DefaultConcurrency is used by default.
	Concurrency int

	// TLSConfig is TLS (aka SSL) config used for accepting encrypted
	// client connections.
	//
	// Encrypted connections may be used for transferring sensitive
	// information over untrusted networks.
	//
	// By default server accepts only unencrypted connections.
	TLSConfig *tls.Config

	// MaxBatchDelay is the maximum duration before ready responses
	// are sent to the client.
	//
	// Responses' batching may reduce network bandwidth usage and CPU usage.
	//
	// By default responses are sent immediately to the client.
	MaxBatchDelay time.Duration

	// Maximum duration for reading the full request (including body).
	//
	// This also limits the maximum lifetime for idle connections.
	//
	// By default request read timeout is unlimited.
	ReadTimeout time.Duration

	// Maximum duration for writing the full response (including body).
	//
	// By default response write timeout is unlimited.
	WriteTimeout time.Duration

	// ReduceMemoryUsage leads to reduced memory usage at the cost
	// of higher CPU usage if set to true.
	//
	// Memory usage reduction is disabled by default.
	ReduceMemoryUsage bool

	// ReadBufferSize is the size for read buffer.
	//
	// DefaultReadBufferSize is used by default.
	ReadBufferSize int

	// WriteBufferSize is the size for write buffer.
	//
	// DefaultWriteBufferSize is used by default.
	WriteBufferSize int

	// Logger used for logging.
	//
	// Standard logger from log package is used by default.
	Logger fasthttp.Logger

	// PipelineRequests enables requests' pipelining.
	//
	// Requests from a single client are processed serially
	// if is set to true.
	//
	// Enabling requests' pipelining may be useful in the following cases:
	//
	//   - if requests from a single client must be processed serially;
	//   - if the Server.Handler doesn't block and maximum throughput
	//     must be achieved for requests' processing.
	//
	// By default requests from a single client are processed concurrently.
	PipelineRequests bool
	// contains filtered or unexported fields
}

Server accepts requests from httpteleport Client.

func (*Server) ListenAndServe

func (s *Server) ListenAndServe(addr string) error

ListenAndServe serves httpteleport requests accepted from the given TCP address.

func (*Server) Serve

func (s *Server) Serve(ln net.Listener) error

Serve serves httpteleport requests accepted from the given listener.