spdy

package module
v0.0.0-...-31da8b7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 20, 2015 License: LGPL-3.0 Imports: 20 Imported by: 0

README

Amahi SPDY Build Status GoDoc Coverage Status

Test coverage (v1.1): 72%

Amahi SPDY is a library built from scratch for building SPDY clients and servers in the Go programming language. It was meant to do it in a more "Go way" than other libraries available (which use mutexes liberally). Here is a high-level picture of how it's structured:

SPDY Library Architecture

It supports a subset of SPDY 3.1.

Check the online documentation.

This library is used in a streaming server/proxy implementation for Amahi, the home and media server.

Building a Server

package main

import (
	"fmt"
	"github.com/amahi/spdy"
	"net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
	http.HandleFunc("/", handler)
	
	//use spdy's Listen and serve 
	err := spdy.ListenAndServe("localhost:4040",nil)
	if err != nil {
		//error handling here
	}
}

Building a Client

package main

import (
	"fmt"
	"github.com/amahi/spdy"
	"io"
	"net/http"
)

func main() {
        //make a spdy client with a given address
	client, err := spdy.NewClient("localhost:4040")
	if err != nil {
		//handle error here
	}
	
	//make a request
	req, err := http.NewRequest("GET", "http://localhost:4040/banana", nil)
	if err != nil {
		//handle error here
	}
	
	//now send the request to obtain a http response
	res, err := client.Do(req)
	if err != nil {
		//something went wrong
	}
	
	//now handle the response
	data := make([]byte, int(res.ContentLength))
	_, err = res.Body.(io.Reader).Read(data)
	fmt.Println(string(data))
	res.Body.Close()
}

Examples

We have several examples to help in getting aqcuainted to the Amahi SPDY package.

We also have a reference implementation of clients for the library, which contains an origin server, and a proxy server.

Architecture

The library is divided in Session objects and Stream objects as far as the external interface. Each Session and Stream may have multiple goroutines and channels to manage their structure and communication patterns. Here is an overview diagram of how the pieces fit together:

SPDY Library Architecture

Each Session controls the communication between two net.Conn connected endpoints. Each Session has a server loop and in it there are two goroutines, one for sending frames from the network connection and one for receiving frames from it. These two goroutines are designed to never block. Except of course if there are network issues, which break the Session and all Streams in the Session.

Each Stream has a server and in it there are two goroutines, a Northbound Buffer Sender and a Control Flow Manager. The NorthBound Buffer Sender is in charge of writing data to the http response and causes control flow frames being sent southbound when data is written northbound. The Control Flow Manager is the owner of the control flow window size.

In the end there are two copies of these stacks, one on each side of the connection.

HTTP and SPDY

The goals for the library are reliability, streaming and performance/scalability.

  1. Design for reliability means that network connections are assumed to disconnect at any time, especially when it's most inapropriate for the library to handle. This also includes potential issues with bugs in within the library, so the library tries to handle all crazy errors in the most reasonable way. A client or a server built with this library should be able to run for months and months of reliable operation. It's not there yet, but it will be.

  2. Streaming requests, unlike typical HTTP requests (which are short), require working with an arbitrary large number of open requests (streams) simultaneously, and most of them are flow-constrained at the client endpoint. Streaming clients kind of misbehave too, for example, they open and close many streams rapidly with Range request to check certain parts of the file. This is common with endpoint clients like VLC or Quicktime (Safari on iOS or Mac OS X). We wrote this library with the goal of making it not just suitable for HTTP serving, but also for streaming.

  3. The library was built with performance and scalability in mind, so things have been done using as little blocking and copying of data as possible. It was meant to be implemented in the "go way", using concurrency extensively and channel communication. The library uses mutexes very sparingly so that handling of errors at all manner of inapropriate times becomes easier. It goes to great lengths to not block, establishing timeouts when network and even channel communication may fail. The library should use very very little CPU, even in the presence of many streams and sessions running simultaneously.

This is not to say SPDY compliance/feature-completeness is not a priority. We're definitely interested in that, so that is an good area for contributions.

Testing

The library needs more directed, white-box tests. Most of the testing has been black-box testing. It has been tested as follows:

  1. Building a reference proxy and an origin server, exercising them with multiple streaming clients, stressing them with many streams in parallel.

  2. The reference implementation above also contains some integration tests. These do not cover a lot in terms of stressing the library, but are a great baseline.

As such, the integration tests should be considered more like sanity checks. We're interested in contributions that cover more and more edge cases!

  1. We periorically run apps built with this library, with the Go race detector enabled. We no longer found any more race conditions.

We'd like to beef up the testing to make it scale!

Code Coverage

To get a detailed report of covered code:

 go test -coverprofile=coverage.out && go tool cover -html=coverage.out -o coverage.html

Spec Coverage

A document detailing parts of spdy spec covered by the Amahi SPDY library can be found in docs/specs. The document is labelled with the library version for which it is applicable.

Status

Things implemented:

  • SYN_STREAM, SYN_REPLY and RST_STREAM frames
  • WINDOW_UPDATE and a (fixed) control flow window
  • PING frames
  • Support for other all types of HTTP requests
  • DATA frames, obviously
  • GOAWAY frame
  • NPN negotiation

Things to be implemented:

  • Support for SETTINGS frames
  • Actual implementation of priorities (everything is one priority at the moment)
  • Server push
  • HEADERS frames
  • Variable flow control window size
  • Extensive error handling for all possible rainy-day scenarios specified in the specification
  • Support for pre-3.1 SPDY standards

Contributing

  • Fork it
  • Make changes, test them
  • Submit pull request!

Credits

Credit goes to Jamie Hall for the patience and persistance to debug his excellent SPDY library that lead to the creation of this library.

The library was started from scratch, but some isolated code like the header compression comes from Jamie's library as well as other libraries out there that we used for inspiration. The header dictionary table comes from the SPDY spec definition.

The library has been extended by Nilesh Jagnik to support various new features along with the development of a Friendly API to create SPDY servers and clients.

Documentation

Overview

Amahi SPDY is a library built from scratch in the "Go way" for building SPDY clients and servers in the Go programming language.

It supports a subset of SPDY 3.1 http://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3-1

Check the source code, examples and overview at https://github.com/amahi/spdy

This library is used in a streaming server/proxy implementation for Amahi, the [home and media server](https://www.amahi.org).

Goals

The goals are reliability, streaming and performance/scalability.

1) Design for reliability means that network connections are assumed to disconnect at any time, especially when it's most inapropriate for the library to handle. This also includes potential issues with bugs in within the library, so the library tries to handle all crazy errors in the most reasonable way. A client or a server built with this library should be able to run for months and months of reliable operation. It's not there yet, but it will be.

2) Streaming requests, unlike typical HTTP requests (which are short), require working with an arbitrary large number of open requests (streams) simultaneously, and most of them are flow-constrained at the client endpoint. Streaming clients kind of misbehave too, for example, they open and close many streams rapidly with Range request to check certain parts of the file. This is common with endpoint clients like VLC or Quicktime (Safari on iOS or Mac OS X). We wrote this library with the goal of making it not just suitable for HTTP serving, but also for streaming.

3) The library was built with performance and scalability in mind, so things have been done using as little blocking and copying of data as possible. It was meant to be implemented in the "go way", using concurrency extensively and channel communication. The library uses mutexes very sparingly so that handling of errors at all manner of inapropriate times becomes easier. It goes to great lengths to not block, establishing timeouts when network and even channel communication may fail. The library should use very very little CPU, even in the presence of many streams and sessions running simultaneously.

Index

Constants

View Source 
const (
	FRAME_SYN_STREAM    = 0x0001
	FRAME_SYN_REPLY     = 0x0002
	FRAME_RST_STREAM    = 0x0003
	FRAME_SETTINGS      = 0x0004
	FRAME_PING          = 0x0006
	FRAME_GOAWAY        = 0x0007
	FRAME_HEADERS       = 0x0008
	FRAME_WINDOW_UPDATE = 0x0009
)
View Source 
const (
	FLAG_NONE = frameFlags(0x00)
	FLAG_FIN  = frameFlags(0x01)
)
View Source 
const (
	HEADER_STATUS         string = ":status"
	HEADER_VERSION        string = ":version"
	HEADER_PATH           string = ":path"
	HEADER_METHOD         string = ":method"
	HEADER_HOST           string = ":host"
	HEADER_SCHEME         string = ":scheme"
	HEADER_CONTENT_LENGTH string = "Content-Length"
)
View Source 
const INITIAL_FLOW_CONTOL_WINDOW int32 = 64 * 1024
View Source 
const MAX_DATA_PAYLOAD = 1<<24 - 1

maximum number of bytes in a frame

View Source 
const NORTHBOUND_SLOTS = 5

Variables

This section is empty.

Functions

func EnableDebug 

func EnableDebug()

EnableDebug turns on the output of debugging messages to Stdout

func ListenAndServe 

func ListenAndServe(addr string, handler http.Handler) (err error)

ListenAndServe listens on the TCP network address addr and then calls Serve with handler to handle requests on incoming connections. Handler is typically nil, in which case the DefaultServeMux is used. This creates a spdy only server without TLS

A trivial example server is: 

	package main

	import (
		"io"
		"net/http"
             "github.com/amahi/spdy"
		"log"
	)

	// hello world, the web server
	func HelloServer(w http.ResponseWriter, req *http.Request) {
		io.WriteString(w, "hello, world!\n")
	}

	func main() {
		http.HandleFunc("/hello", HelloServer)
		err := spdy.ListenAndServe(":12345", nil)
		if err != nil {
			log.Fatal("ListenAndServe: ", err)
		}
	}

func ListenAndServeTLS 

func ListenAndServeTLS(addr string, certFile string, keyFile string, handler http.Handler) error

ListenAndServeTLS acts identically to ListenAndServe, except that it expects HTTPS connections. Servers created this way have NPN Negotiation and accept requests from both spdy and http clients. Additionally, files containing a certificate and matching private key for the server must be provided. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server's certificate followed by the CA's certificate.

A trivial example server is: 

	import (
		"log"
		"net/http"
             "github.com/amahi/spdy"
	)

	func handler(w http.ResponseWriter, req *http.Request) {
		w.Header().Set("Content-Type", "text/plain")
		w.Write([]byte("This is an example server.\n"))
	}

	func main() {
		http.HandleFunc("/", handler)
		log.Printf("About to listen on 10443. Go to https://127.0.0.1:10443/")
		err := spdy.ListenAndServeTLS(":10443", "cert.pem", "key.pem", nil)
		if err != nil {
			log.Fatal(err)
		}
	}

One can use makecert.sh in /certs to generate certfile and keyfile

func ListenAndServeTLSSpdyOnly 

func ListenAndServeTLSSpdyOnly(addr string, certFile string, keyFile string, handler http.Handler) error

func PriorityFor 

func PriorityFor(req *url.URL) uint8

PriorityFor returns the recommended priority for the given URL for best opteration with the library.

func SetLog 

func SetLog(w io.Writer)

SetLog sets the output of logging to a given io.Writer

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

spdy client

func NewClient 

func NewClient(addr string) (*Client, error)

returns a client with tcp connection created using net.Dial

func NewClientConn 

func NewClientConn(c net.Conn) (*Client, error)

returns a client that reads and writes on c

func (*Client) Close 

func (c *Client) Close() error

func (*Client) Do 

func (c *Client) Do(req *http.Request) (*http.Response, error)

to get a response from the client

func (*Client) Ping 

func (c *Client) Ping(d time.Duration) (pinged bool, err error)

type ResponseRecorder 

type ResponseRecorder struct {
	Code      int           // the HTTP response code from WriteHeader
	HeaderMap http.Header   // the HTTP response headers
	Body      *bytes.Buffer // if non-nil, the bytes.Buffer to append written data to
	// contains filtered or unexported fields
}

ResponseRecorder is an implementation of http.ResponseWriter that is used to get a response.

func NewRecorder 

func NewRecorder() *ResponseRecorder

NewRecorder returns an initialized ResponseRecorder.

func (*ResponseRecorder) Header 

func (rw *ResponseRecorder) Header() http.Header

Header returns the response headers.

func (*ResponseRecorder) Write 

func (rw *ResponseRecorder) Write(buf []byte) (int, error)

Write always succeeds and writes to rw.Body.

func (*ResponseRecorder) WriteHeader 

func (rw *ResponseRecorder) WriteHeader(code int)

WriteHeader sets rw.Code.

type Server 

type Server struct {
	Handler   http.Handler
	Addr      string
	TLSConfig *tls.Config
	// contains filtered or unexported fields
}

spdy server

func (*Server) Close 

func (s *Server) Close() (err error)

close spdy server and return Any blocked Accept operations will be unblocked and return errors.

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe() (err error)

ListenAndServe listens on the TCP network address s.Addr and then calls Serve to handle requests on incoming connections.

func (*Server) ListenAndServeTLSSpdyOnly 

func (srv *Server) ListenAndServeTLSSpdyOnly(certFile, keyFile string) error

ListenAndServeTLSSpdyOnly listens on the TCP network address srv.Addr and then calls Serve to handle requests on incoming TLS connections. This is a spdy-only server with TLS and no NPN.

Filenames containing a certificate and matching private key for the server must be provided. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server's certificate followed by the CA's certificate.

func (*Server) Serve 

func (s *Server) Serve(ln net.Listener) (err error)

Serve accepts incoming connections on the Listener l, creating a new service goroutine for each. The service goroutines read requests and then call srv.Handler to reply to them.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

func NewClientSession 

func NewClientSession(conn net.Conn) *Session

NewClientSession creates a new Session that should be used as a client. the given http.Server will be used to serve requests arriving. The user should call Serve() once it's ready to start serving. New streams will be created as per the SPDY protocol.

func NewServerSession 

func NewServerSession(conn net.Conn, server *http.Server) *Session

NewServerSession creates a new Session with the given network connection. This Session should be used as a server, and the given http.Server will be used to serve requests arriving. The user should call Serve() once it's ready to start serving. New streams will be created as per the SPDY protocol.

func (*Session) Close 

func (s *Session) Close()

Close closes the Session and the underlaying network connection. It should be called when the Session is idle for best results.

func (*Session) NewClientStream 

func (s *Session) NewClientStream() *Stream

NewClientStream starts a new Stream (in the given Session), to be used as a client

func (*Session) NewStreamProxy 

func (s *Session) NewStreamProxy(r *http.Request, w http.ResponseWriter) (err error)

NewStreamProxy starts a new stream and proxies the given HTTP Request to it, writing the response to the given ResponseWriter. If there is an error, it will be returned, but the ResponseWriter will get a 404 Not Found.

func (*Session) Ping 

func (s *Session) Ping(d time.Duration) (pinged bool)

Ping issues a SPDY PING frame and returns true if it the other side returned the PING frame within the duration, else it returns false. NOTE only one outstanting ping works in the current implementation.

func (*Session) SendGoaway 

func (s *Session) SendGoaway(f frameFlags, dat []byte)

func (*Session) Serve 

func (s *Session) Serve() (err error)

Serve starts serving a Session. This implementation of Serve only returns when there has been an error condition.

type Stream 

type Stream struct {
	// contains filtered or unexported fields
}

func (*Stream) Header 

func (s *Stream) Header() http.Header

Header makes streams compatible with the net/http handlers interface

func (*Stream) Request 

func (s *Stream) Request(request *http.Request, writer http.ResponseWriter) (err error)

Request makes an http request down the client that gets a client Stream started and returning the request in the ResponseWriter

func (*Stream) String 

func (s *Stream) String() string

String returns the Stream ID of the Stream

func (*Stream) Write 

func (s *Stream) Write(p []byte) (n int, err error)

Write makes streams compatible with the net/http handlers interface

func (*Stream) WriteHeader 

func (s *Stream) WriteHeader(code int)

WriteHeader makes streams compatible with the net/http handlers interface

Source Files

View all Source files

Directories

Path Synopsis
examples
client
clientTLS
server
serverTLS

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.