tlj

package module
v0.0.0-...-6470440 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 11, 2017 License: MIT Imports: 7 Imported by: 3

README

TLJ

A simple Type Length Value protocol implemented with JSON to hand structs between Go applications in an event driven and parallel way.

Concepts

TLJ is used to write networked application in Go by expressing the application's behavior in terms of what to do with structs recieved on various sockets.

Here's a rough idea of how TLJ came about:

  • maybe "sockets that have a remote certificate I trust are 'trusted' sockets"
  • or "sockets that send an Authentication{} struct with a valid password are 'trusted' sockets"
  • and "when 'trusted' sockets send a Message{}, save it in the database"
  • and also "when 'trusted' sockets send a Message{}, print it"
  • how could this be expressed easily?

Most generally, when tag receives type, do func. If there are many funcs with the same criteria, run them all in parallel as goroutines. This library is meant to be used on a variety of networks, from traditional TLS sockets on the internet to anonymity networks such as I2P.

Usage

To use TLJ, start by defining some structs you want to pass around. We want to hold on to references to their types for later. These structs are just basic examples, anything that can be marshalled to JSON is ok.

type ExampleEvent struct {
	Parameter1	string
	Parameter2	int
}
example_event_inst := reflect.TypeOf(ExampleEvent{})
example_event_ptr := reflect.TypeOf(&ExampleEvent{})

Type ExampleRequest {
	Parameter1	string
}
example_request_inst := reflect.TypeOf(ExampleRequest{})
example_request_ptr := reflect.TypeOf(&ExampleRequest{})

type ExampleResponse {
	Parameter1	string
	Parameter2	string
	Parameter3	string
}
example_response_inst := reflect.TypeOf(ExampleResponse{})
example_response_ptr := reflect.TypeOf(&ExampleResponse{})

Then, define Builder functions for each struct that will create and validate the struct from a JSON byte array. The TLJContext can be used to access the socket that sent this data. Add these functions to a TypeStore.

func NewExampleEvent(data []byte, context TLJContext) interface{} {
	event := &ExampleEvent{}
	err := json.Unmarshal(data, &event)
	if err != nil { return nil }
	return event
}

func NewExampleRequest(data []byte, context TLJContext) interface{} {
	request := &ExampleRequest{}
	err := json.Unmarshal(data, &request)
	if err != nil { return nil }
	return request
}

func NewExampleResponse(data []byte, context TLJContext) interface{} {
	response := &ExampleResponse{}
	err := json.Unmarshal(data, &response)
	if err != nil { return nil }
	return response
}

type_store := NewTypeStore()
type_store.AddType(example_event_inst, example_event_ptr, NewExampleEvent)
type_store.AddType(example_request_inst, example_event_ptr, NewExampleRequest)
type_store.AddType(example_response_inst, example_event_ptr, NewExampleResponse)

A tagging function is used by the server to tag sockets based on their properties.

func TagSocket(socket *net.Conn, server *Server) {
	server.TagSocket(socket, "all")
	// with TLS sockets, a client certificate could be used to tag sockets
	// in I2P, the remote public key could identify sockets
}

Next create a Server and a Client that contain the same TypeStore.

listener := // Anything that implements net.UnixListener
server := NewServer(listener, TagSocket, type_store)

socket := // Anything that implement net.Conn
client := NewClient(socket, type_store, false)

Hook up some goroutines on the server that run on structs or requests that came from sockets with certain tags. A type assertion is used to avoid needing reflect to access fields.

server.Accept("all", example_event, func(iface interface{}, context TLJContext) {
	if example_event, ok :=  iface.(*ExampleEvent); ok {
		fmt.Println("a socket tagged \"all\" sent an ExampleEvent struct")
		fmt.Println(example_event.Parameter1)
		fmt.Println(example_event.Parameter2)
	}
})

server.AcceptRequest("all", example_request, func(iface interface{}, context TLJContext) {
	if example_request, ok :=  iface.(*ExampleRequest); ok {
		fmt.Println("a socket tagged \"all\" sent an ExampleRequest request")
		resp := ExampleResponse {
			Parameter1:	"hello",
			Parameter2:	"world",
			Parameter3:	"response",
		}
		context.Respond(resp)
		if err != nil {
			fmt.Println("response did not send")
		}
	}
})

It is also possible to insert sockets into an existing server and have them tagged. This lets peer-to-peer applications dial sockets on startup as well as accept connections once started.

socket := // any net.Conn
server.Insert(socket)

Notice how false was passed to NewClient(). This put the Client in Client-Server mode, meaning the Client created a goroutine to read data coming back from the server. This enables stateful requests, but means this socket could not simultaniously be used in a Server. To put a Client in p2p mode, the third argument to NewClient should be true.

// Client-Server mode:
client := NewClient(socket, type_store, false)
// Able to:
client.Message()
req := client.Request()
req.OnResponse()

// P2P mode:
client := NewClient(socket, type_store, true)
// Able to:
server := // a TLJ Server
server.Insert(client.Socket)
client.Message()

This is what it might look like:

event := ExampleEvent {
	Parameter1:	"test",
	Parameter2:	0,
}
err := client.Message(event)
if err != nil {
	fmt.Println("message did not send")
}

request := ExampleRequest {
	Parameter1:	"test",
}
req, err := client.Request(request)
if err != nil {
	fmt.Println("request did not send")
}
req.OnResponse(example_response, func(iface) {
	if example_response, ok :=  iface.(*ExampleResponse); ok {
		fmt.Println("the request got a response of type ExampleResponse")
		fmt.Println(example_response.Parameter1)
		fmt.Println(example_response.Parameter2)
		fmt.Println(example_response.Parameter3)
	}
})

If you only ever want to send one type of struct, create a StreamWriter to avoid calling reflect every time you send a struct. This is like a Client in p2p mode that can only send one type of struct.

writer := NewStreamWriter(client, type_store, example_event_inst)
for {
	writer.Write(<-ExampleEventsChan)
}

Tests

$ go test -race -cover
Running Suite: TLJ Suite
========================
Random Seed: 1465096248
Will run 31 of 31 specs

•••••••••••••••••••••••••••••••
Ran 31 of 31 Specs in 1.012 seconds
SUCCESS! -- 31 Passed | 0 Failed | 0 Pending | 0 Skipped PASS
coverage: 91.9% of statements
ok  	github.com/hkparker/TLJ	2.040s

License

This project is licensed under the MIT license, see LICENSE for more information.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExcludeConn 

func ExcludeConn(list []net.Conn, omit net.Conn) []net.Conn

Given a slice of net.Conn interfaces, return all interfaces that are not equal to omit.

func ExcludeString 

func ExcludeString(list []string, omit string) []string

Given a slice of strings, return all strings that are not equal to omit.

Types

type Builder 

type Builder func([]byte, TLJContext) interface{}

Builders are functions that take the raw payload in the TLV protocol and parse the JSON and run any other validations that may be nessicary based on the context before returning the struct.

type Capsule 

type Capsule struct {
	RequestID uint16
	Type      uint16
	Data      string
}

A capsule is used to maintain a session between a client and a server when using sever.AcceptRequest, client.Request, or request.OnResponse.

type Client 

type Client struct {
	Socket               net.Conn
	TypeStore            TypeStore
	Requests             map[uint16]map[uint16][]func(interface{})
	NextID               uint16
	Writing              *sync.Mutex
	RequestsManipulation *sync.Mutex
	Dead                 chan error
}

A Client is used to wrap a net.Conn interface and send TLJ formatted structs through the interface.

func NewClient 

func NewClient(socket net.Conn, type_store TypeStore, p2p bool) Client

Create a new Client with a net.Conn interface and a TypeStore containing all types that will be seen on the network.

func (*Client) Message 

func (client *Client) Message(instance interface{}) error

Given any struct in the Client's TypeStore, format the struct and write it down the client's net.Conn.

func (*Client) Request 

func (client *Client) Request(instance interface{}) (Request, error)

Given any struct in the Client's TypeStore, format the struct inside a capsule and write it down the client's net.Conn.

type Request 

type Request struct {
	RequestID uint16
	Type      uint16
	Data      string
	Client    *Client
}

Requests are returned by Clients when stateful requests are made, and can be used to handle server responses with request.OnResponse.

func (*Request) OnResponse 

func (request *Request) OnResponse(struct_type reflect.Type, function func(interface{}))

OnResponse is used to define the behaviors used to handle responses to client.Request.

type Responder 

type Responder struct {
	RequestID uint16
	WriteLock sync.Mutex
}

Responders contain information needed to send a stateful response

type Server 

type Server struct {
	Listener        net.Listener
	TypeStore       TypeStore
	Tag             func(net.Conn, *Server)
	Tags            map[net.Conn][]string
	Sockets         map[string][]net.Conn
	Events          map[string]map[uint16][]func(interface{}, TLJContext)
	Requests        map[string]map[uint16][]func(interface{}, TLJContext)
	FailedServer    chan error
	FailedSockets   chan net.Conn
	TagManipulation *sync.Mutex
	InsertRequests  *sync.Mutex
	InsertEvents    *sync.Mutex
}

A Server wraps a net.Listener and accepts incoming connections, tagging them and running any relevant callbacks on valid TLJ structs received on them.

func NewServer 

func NewServer(listener net.Listener, tag func(net.Conn, *Server), type_store TypeStore) Server

Create a new server from a net.Listener, a TypeStore, and a tagging function that will assign tags to all accepted sockets.

func (*Server) Accept 

func (server *Server) Accept(socket_tag string, struct_type reflect.Type, function func(interface{}, TLJContext))

Create a new callback to be ran when a socket with a certain tag receives a specific type of struct. The server will have no ability to respond statefully to this event.

func (*Server) AcceptRequest 

func (server *Server) AcceptRequest(socket_tag string, struct_type reflect.Type, function func(interface{}, TLJContext))

Create a new callback to be ran when a socket with a certain tag receives a capsule containing a specific type of struct. The callback accepts a responder which can be used to respond to the client statefully.

func (*Server) Delete 

func (server *Server) Delete(socket net.Conn)

Remove all tags from a socket, removing it from the server.

func (*Server) Insert 

func (server *Server) Insert(socket net.Conn)

Tag the socket then read an structs from this socket until the socket is closed.

func (*Server) TagSocket 

func (server *Server) TagSocket(socket net.Conn, tag string)

Assign a string tag to a socket in this Server.

func (*Server) UntagSocket 

func (server *Server) UntagSocket(socket net.Conn, tag string)

Disassociate a string tag from a socket on this server.

type StreamWriter 

type StreamWriter struct {
	Socket  net.Conn
	TypeID  uint16
	Writing *sync.Mutex
}

A StreamWriter is a Client that can only be used to send one type. Because of this restriction it takes the reflect.Type directly and avoids a reflect call on every write.

func NewStreamWriter 

func NewStreamWriter(conn net.Conn, type_store TypeStore, struct_type reflect.Type) (StreamWriter, error)

Create a new StreamWriter using a net.Conn, a TypeStore, and the reflect.Type of the type this StreamWriter should be capable of sending.

func (*StreamWriter) Write 

func (writer *StreamWriter) Write(obj interface{}) error

Write a struct using the StreamWriter.

type TLJContext 

type TLJContext struct {
	Server    *Server
	Socket    net.Conn
	Responder Responder
}

Context about TLJ events so Server callbacks can respond statefully and Builders can conditionally validate data and verify signatures.

func (*TLJContext) Respond 

func (context *TLJContext) Respond(object interface{}) error

Respond is used to send a struct down the socket the sent a request with client.Request

type TypeStore 

type TypeStore struct {
	Types      map[uint16]Builder
	TypeCodes  map[reflect.Type]uint16
	NextID     uint16
	InsertType *sync.Mutex
}

A TypeStore stores the information needed to marshal, unmsrahsl, and recognize all types passed between all other instances of TLJ that are communicated with.

func NewTypeStore 

func NewTypeStore() TypeStore

Create a new TypeStore with type 0 being a Capsule.

func (*TypeStore) AddType 

func (store *TypeStore) AddType(inst_type reflect.Type, ptr_type reflect.Type, builder Builder) error

Insert a new type into the TypeStore by providing a reflect.Type of the struct and a pointer to the struct, as well as the Builder that will be used to construct the type. Types must be JSON serializable.

func (*TypeStore) BuildType 

func (store *TypeStore) BuildType(struct_code uint16, data []byte, context TLJContext) interface{}

Call the Builder function for a given type on some data if the type exists in the type store, return nil if the type does not exist.

func (*TypeStore) Format 

func (store *TypeStore) Format(instance interface{}) ([]byte, error)

Take any JSON serializable struct that is in the TypeStore and return the byte sequence to send on the network to deliver the struct to the other instance of TLJ, as well as any errors.

func (*TypeStore) FormatCapsule 

func (store *TypeStore) FormatCapsule(instance interface{}, request_id uint16) ([]byte, error)

Take a struct and format it inside of a Capsule so it can be sent statefully to another TLJ instance.

func (*TypeStore) LookupCode 

func (store *TypeStore) LookupCode(struct_type reflect.Type) (uint16, bool)

Given the reflect.Type of a type in the TypeStore, return the uint16 that is used to identify this type over the network, and a boolean to indicate if the type was present in the TypeStore.

func (*TypeStore) NextStruct 

func (store *TypeStore) NextStruct(socket net.Conn, context TLJContext) (interface{}, error)

Read a struct from a net.Conn interface using the types contained in a TypeStore.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.