drpc

package
v0.0.0-...-235079f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 17, 2024 License: BSD-3-Clause Imports: 24 Imported by: 0

README

Package drpc

DRPC is a lightweight, drop-in, protocol buffer-based gRPC replacement. The DRPC protocol strips out huge swaths of unnecessary complexity; it is implemented in just a few thousand lines of straightforward Go! DRPC is small, extensible, efficient, and can still be autogenerated from your existing Protobuf definition files.

More information: https://github.com/storj/drpc

Server

This package simplifies the process of running a DRPC server in production by providing often required additional functionality, for example:

  • Properly setup TLS to provide secure communication channels
  • Protect against panics produced by external services
  • Generate structured logs for all processed requests
  • Request authentication/authorization (authN, authZ)
  • Rate limiting requests to avoid resource exhaustion attacks

For example, to start a typical production server.

// Token validation function. This will usually handle JWT or PASETO
// tokens for example.
myCustomCredsValidation := func(token string) bool {
  return token == "super-secure-credentials"
}

// Server TLS settings. Cert, key and CA are usually read from a file
// or another secret-management tool.
tlsSettings := ServerTLS{
  Cert:             my-cert-pem,
  PrivateKey:       my-key-pem,
  CustomCAs:        [][]byte{my-ca-cert-pem},
  IncludeSystemCAs: true,
}

// Define the middleware elements to use.
// In this case:
// - Use zerolog to produce "pretty" detailed output
// - Limit the processing to 100 RPC requests per-second
// - Provide a custom token-based authentication mechanism
// - Prevent the server from crashing on `panic` calls
smw := []srvmw.Middleware{
  srvmw.Logging(xlog.WithZero(true, "error").Sub(xlog.Fields{"component": "server"}), nil),
  srvmw.RateLimit(100),
  srvmw.AuthByToken("auth.token", myCustomCredsValidation),
  srvmw.PanicRecovery(),
}

// RPC server settings.
// - The server will listen on TCP port 8080
// - The server will handle DRPC and HTTP requests
// - The server will use TLS for secure connections
// - The server will provide bi-directional streaming over WebSockets (and DRPC)
opts := []Option{
  WithServiceProvider(myServiceImplementation()),
  WithPort(8080),
  WithHTTP(),
  WithMiddleware(smw...),
  WithTLS(tlsSettings),
  WithWebSocketProxy(
    ws.EnableCompression(),
    ws.CheckOrigin(func(r *http.Request) bool { return true }),
    ws.HandshakeTimeout(2*time.Second),
    ws.SubProtocols([]string{"rfb", "sip"}),
  ),
}

// Create and start the new server
srv, err := NewServer(opts...)
if err != nil {
  panic(err)
}
srv.Start()

Client

This package simplifies the process of running a DRPC client in production by providing often required additional functionality, for example:

  • Concurrent RPC requests (using connection pools)
  • Properly setup TLS to provide secure communication channels
  • Protect against panics produced by external services
  • Generate structured logs for all processed requests
  • Provide custom metadata on all requests send to the server (e.g. credentials)

For example, to start a typical production client.

// Custom metadata. Can be used to provide identifiers, credentials or
// additional contextual details.
kv := map[string]string{
  "metadata.user": "rick",
}

// Client TLS settings.
tlsSettings := ClientTLS{
  IncludeSystemCAs: true,
  CustomCAs:        [][]byte{ca-cert-pem},
  ServerName:       "my-server.local.acme.com",
}

// Client middleware.
// - Provide custom details (`kv`) on every request
// - Use zerolog to produce "pretty" detailed output
// - Limit the client to 100 RPC requests per-second
// - Prevent the client from crashing on `panic` calls
cmw := []clmw.Middleware{
  clmw.Metadata(kv),
  clmw.Logging(xlog.WithZero(true, "error").Sub(xlog.Fields{"component": "client"}), nil),
  clmw.RateLimit(100),
  clmw.PanicRecovery(),
}

// Client options.
// - Up to 5 concurrent RPC calls
// - Include a protocol selection header (between DRPC and HTTP)
// - Use TLS to verify server's identity and establish secure connections
opts := []ClientOption{
  WithProtocolHeader(),
  WithPoolCapacity(5),
  WithClientTLS(tlsSettings),
  WithClientMiddleware(cmw...),
}

// Start new client
cl, err := NewClient("tcp", ":8080", opts...)
if err != nil {
  panic(err)
}

// When no longer needed, the client must be closed
defer cl.Close()

// The client can be used directly to create stubs to consume
// specific DRPC services.
// RPC request
mySvc := samplev1.NewDRPCFooAPIClient(cl)
res, _ := mySvc.Ping(context.Background(), &emptypb.Empty{})

Custom Middleware

You can provide your own custom middleware to extend/adjust the processing of RPC requests on both server and client using the decorator pattern. You can then use the WithMiddleware and WithClientMiddleware options when creating a new server or client instance.

Documentation

Overview

Package drpc provides a drop-in, lightweight gRPC replacement.

DRPC is a lightweight, drop-in, protocol buffer-based gRPC replacement. The DRPC protocol strips out huge swaths of unnecessary complexity; it is implemented in just a few thousand lines of straightforward Go! DRPC is small, extensible, efficient, and can still be autogenerated from your existing Protobuf definition files.

More information: https://github.com/storj/drpc

Server

This package simplifies the process of running a DRPC server in production by providing often required additional functionality, for example:

  • Properly setup TLS to provide secure communication channels
  • Protect against `panics` produced by external services
  • Generate structured logs for all processed requests
  • Request authentication/authorization (authN, authZ)
  • Rate limiting requests to avoid resource exhaustion attacks

For example, to start a typical production server. 

// Token validation function. This will usually handle JWT or PASETO
// tokens for example.
myCustomCredsValidation := func(token string) bool {
	return token == "super-secure-credentials"
}

// Server TLS settings. Cert, key and CA are usually read from a file
// or another secret-management tool.
tlsSettings := ServerTLS{
	Cert:             my-cert-pem,
	PrivateKey:       my-key-pem,
	CustomCAs:        [][]byte{my-ca-cert-pem},
	IncludeSystemCAs: true,
}

// Define the middleware elements to use.
// In this case:
// - Use zerolog to produce "pretty" detailed output
// - Limit the processing to 10 RPC requests per-second
// - Prevent the server from crashing on `panic` calls
// - Provide a custom token-based authentication
smw := []srvmw.Middleware{
	srvmw.Logging(xlog.WithZero(true, "error").Sub(xlog.Fields{"component": "server"}), nil),
	srvmw.RateLimit(10),
	srvmw.AuthByToken("auth.token", myCustomCredsValidation),
	srvmw.PanicRecovery(),
}

// RPC server settings.
// - The server will listen on TCP port 8080
// - The server will handle DRPC and HTTP requests
// - The server will use TLS for secure connections
// - The server will provide bi-directional streaming over WebSockets (and DRPC)
opts := []Option{
	WithServiceProvider(myServiceImplementation()),
	WithPort(8080),
	WithHTTP(),
	WithMiddleware(smw...),
	WithTLS(tlsSettings),
	WithWebSocketProxy(
		ws.EnableCompression(),
		ws.CheckOrigin(func(r *http.Request) bool { return true }),
		ws.HandshakeTimeout(2*time.Second),
		ws.SubProtocols([]string{"rfb", "sip"}),
	),
}

// Create and start the new server
srv, err := NewServer(opts...)
if err != nil {
	panic(err)
}
srv.Start()

Client

This package simplifies the process of running a DRPC client in production by providing often required additional functionality, for example:

  • Concurrent RPC requests (using connection pools)
  • Properly setup TLS to provide secure communication channels
  • Protect against `panics` produced by external services
  • Generate structured logs for all processed requests
  • Provide custom metadata on all requests send to the server (e.g. credentials)

For example, to start a typical production client. 

// Custom metadata. Can be used to provide identifiers, credentials or
// additional contextual details.
kv := map[string]string{
	"metadata.user": "rick",
}

// Client TLS settings.
tlsSettings := ClientTLS{
	IncludeSystemCAs: true,
	CustomCAs:        [][]byte{ca-cert-pem},
	ServerName:       "my-server.local.acme.com",
}

// Client middleware.
// - Provide custom details (`kv`) on every request
// - Use zerolog to produce "pretty" detailed output
cmw := []clmw.Middleware{
	clmw.Metadata(kv),
	clmw.Logging(xlog.WithZero(true, "error").Sub(xlog.Fields{"component": "client"}), nil),
	clmw.PanicRecovery(),
	clmw.RateLimit(10),
}

// Client options.
// - Up to 5 concurrent RPC calls
// - Include a protocol selection header (between DRPC and HTTP)
// - Use TLS to verify server's identity and establish secure connections
opts := []ClientOption{
	WithProtocolHeader(),
	WithPoolCapacity(5),
	WithClientTLS(tlsSettings),
	WithClientMiddleware(cmw...),
}

// Start new client
cl, err := NewClient("tcp", ":8080", opts...)
if err != nil {
	panic(err)
}

// When no longer needed, the client must be closed
defer cl.Close()

// The client can be used directly to create stubs to consume
// specific DRPC services.
// RPC request
mySvc := samplev1.NewDRPCFooAPIClient(cl)
res, _ := mySvc.Ping(context.Background(), &emptypb.Empty{})

Custom Middleware

You can provide your own custom middleware to extend/adjust the processing of RPC requests on both server and client using the decorator pattern. You can then use the `WithMiddleware` and `WithClientMiddleware` options when creating a new server or client instance.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func ContextWithMetadata 

func ContextWithMetadata(ctx context.Context, data map[string]string) context.Context

ContextWithMetadata adds custom data to a given context. This is particularly useful when sending outgoing requests on the client side.

func LoadCertificate 

func LoadCertificate(cert []byte, key []byte) (tls.Certificate, error)

LoadCertificate provides a helper method to conveniently parse and existing certificate and corresponding private key.

func MetadataFromContext 

func MetadataFromContext(ctx context.Context) (map[string]string, bool)

MetadataFromContext retrieve any custom metadata available on the provided context. This is particularly useful when processing incoming requests on the server side.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client represents an instance used to consume DRPC services offered by other entities in the network.

func NewClient 

func NewClient(network, address string, options ...ClientOption) (*Client, error)

NewClient returns a ready-to-use DRPC client instance.

Example 
// Client connection
cl, err := NewClient("tcp", ":8080")
if err != nil {
	panic(err)
}

// RPC client
client := sampleV1.NewDRPCEchoAPIClient(cl)

// Consume the RPC service
res, _ := client.Ping(context.Background(), &emptypb.Empty{})
fmt.Printf("ping: %+v", res)

// Close client connection when no longer required
_ = cl.Close()

Output:

func (*Client) Close 

func (cl *Client) Close() error

Close the client and free related resources.

func (*Client) Closed 

func (cl *Client) Closed() <-chan struct{}

Closed returns a channel that is closed when the client is definitely closed.

func (*Client) Invoke 

func (cl *Client) Invoke(ctx context.Context, rpc string, enc drpc.Encoding, in, out drpc.Message) error

Invoke acquires a connection from the pool, dialing if necessary, and issues a unary RPC to the remote on that connection. The connection is put back into the pool after the operation finishes.

func (*Client) IsActive 

func (cl *Client) IsActive() bool

IsActive returns true if the client has any active DRPC connection.

func (*Client) NewStream 

func (cl *Client) NewStream(ctx context.Context, rpc string, enc drpc.Encoding) (drpc.Stream, error)

NewStream acquires a connection from the pool, dialing if necessary, and starts a stream with the remote on that connection. The connection is put back into the pool after the stream is finished.

func (*Client) Transport 

func (cl *Client) Transport() drpc.Transport

Transport returns nil because it does not have a fixed transport.

func (*Client) Use 

func (cl *Client) Use(mw ...clmw.Middleware)

Use will register middleware elements to be applied to the client instance. Middleware is executed before the processing of RPC requests is started. When providing middleware the ordering is very important; middleware will be applied in the same order provided. 

For example:
  Use(foo bar baz)
Will be applied as:
  baz( bar( foo(handler) ) )

type ClientOption 

type ClientOption func(cl *Client) error

ClientOption allows adjusting client settings following a functional pattern.

func WithAuthCertificate 

func WithAuthCertificate(cert, key []byte) ClientOption

WithAuthCertificate enabled certificate-based client authentication with the provided credentials. This requires the client and the server to use a TLS communication channel, otherwise this option will be ignored.

func WithClientMiddleware 

func WithClientMiddleware(mw ...clmw.Middleware) ClientOption

WithClientMiddleware register the provided middleware to customize/extend the processing of RPC requests. When providing middleware the ordering is very important; middleware will be applied in the same order provided. 

For example:
  Use(foo bar baz)
Will be applied as:
  baz( bar( foo(handler) ) )

func WithClientTLS 

func WithClientTLS(opts ClientTLS) ClientOption

WithClientTLS adjust the client to establish a secure communication channel with the server.

func WithPoolCapacity 

func WithPoolCapacity(limit int) ClientOption

WithPoolCapacity adjust the max limit of concurrent DRPC connections a single client instance can support.

func WithProtocolHeader 

func WithProtocolHeader() ClientOption

WithProtocolHeader ensure the client connections include the protocol selection header. This is required when the server supports both DRPC and HTTP requests.

type ClientTLS 

type ClientTLS struct {
	// Whether to include system CAs.
	IncludeSystemCAs bool

	// Custom certificate authorities to include when accepting TLS connections.
	CustomCAs [][]byte

	// Name used to verify the hostname on the returned certificates.
	ServerName string

	// Don't verify the server name on the certificate when establishing a secure
	// TLS channel. THIS IS HIGHLY DANGEROUS, INTENDED FOR TESTING/DEV ONLY.
	SkipVerify bool
}

ClientTLS defines the configuration options available when establishing a secure communication channel with a server.

type Option 

type Option func(srv *Server) error

Option allows adjusting server settings following a functional pattern.

func WithAuthByCertificate 

func WithAuthByCertificate(clientCA []byte) Option

WithAuthByCertificate enables certificate-based authentication on the server. It can be used multiple times to allow for several certificate authorities. This requires the client and the server to use a TLS communication channel, otherwise this option will be ignored.

func WithHTTP 

func WithHTTP() Option

WithHTTP enable access to the services exposed by the server via HTTP / JSON. When using HTTP support on the server, clients MUST properly include the `drpcmigrate.DRPCHeader` header for selecting the protocol to use. To set the header automatically use the `WithProtocolHeader` client option when creating a new connection.

When exposing services via HTTP the default routes are set as: 

POST: {server_url}/{proto_package}.{service}/{method}

func WithMiddleware 

func WithMiddleware(mw ...srvMW.Middleware) Option

WithMiddleware register the provided middleware to customize/extend the processing of RPC requests. When applying middleware the ordering is very important, in this case it will be applied in the same order provided. For example: 

Use(foo bar baz)

Will be applied as: 

baz( bar( foo(handler) ) )

func WithPort 

func WithPort(port uint) Option

WithPort specifies which TCP port the server use.

func WithServiceProvider 

func WithServiceProvider(sp ServiceProvider) Option

WithServiceProvider can be used to expose RPC services described and implemented by "sp" through a server instance.

func WithTLS 

func WithTLS(opts ServerTLS) Option

WithTLS enables the server to use secure communication channels with the provided credentials and settings. If a certificate is provided the server name MUST match the identifier included in the certificate.

func WithUnixSocket 

func WithUnixSocket(socket string) Option

WithUnixSocket specifies the path to a UNIX socket to use as main access point. If the provided socket file doesn't exist it will be created by default.

func WithWebSocketProxy 

func WithWebSocketProxy(opts ...ws.ProxyOption) Option

WithWebSocketProxy enable bidirectional streaming on the DRPC server via websocket connections.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server instances are intended to expose DRPC services for network-based consumption.

func NewServer 

func NewServer(options ...Option) (*Server, error)

NewServer returns a ready-to-use server instance.

Example 
// Get RPC service
myService := sampleServiceProvider()

// Server options
opts := []Option{
	WithPort(8080),
	WithServiceProvider(myService),
}

// Create new server
srv, err := NewServer(opts...)
if err != nil {
	panic(err)
}

// Wait for requests in the background
go func() {
	_ = srv.Start()
}()

// ... do something else ...

Output:

func (*Server) RegisterService 

func (srv *Server) RegisterService(impl interface{}, desc drpc.Description) error

RegisterService associates the RPCs described by `desc` to the provided `impl` element and exposes them through the server instance.

func (*Server) Start 

func (srv *Server) Start() error

Start the server and wait for incoming requests.

func (*Server) Stop 

func (srv *Server) Stop() error

Stop the server's network interfaces. Any blocked operations will be unblocked and return errors.

func (*Server) Use 

func (srv *Server) Use(mw ...srvmw.Middleware)

Use will register middleware elements to be applied to the server instance. Middleware is executed before the processing of RPC requests is started. When providing middleware the ordering is very important; middleware will be applied in the same order provided. 

For example:
  Use(foo bar baz)
Will be applied as:
  baz( bar( foo(handler) ) )

type ServerTLS 

type ServerTLS struct {
	// Server certificate, PEM-encoded.
	Cert []byte

	// Server private key, PEM-encoded.
	PrivateKey []byte

	// List of ciphers to allow.
	SupportedCiphers []uint16

	// Server preferred curves configuration.
	PreferredCurves []tls.CurveID

	// Whether to include system CAs.
	IncludeSystemCAs bool

	// Custom certificate authorities to include when accepting TLS connections.
	CustomCAs [][]byte
}

ServerTLS provides available settings to enable secure TLS communications.

type ServiceProvider 

type ServiceProvider interface {
	// DRPCDescription must return the service description as generated by the
	// "protoc-gen-go-drpc" compiler plugin.
	//   More information: https://storj.github.io/drpc/docs.html
	DRPCDescription() drpc.Description
}

ServiceProvider elements define the services that are to be exposed through a server instance. Some points to note:

  • A single server can expose several services
  • The user is responsible to ensure services are free of collisions
  • The service provider MUST also itself provide the implementation for the service defined by the "DRPCDescription" method. If this is not the case you can use the server's "RegisterService" method to manually specify the implementation element and service description independently

Source Files

View all Source files

Directories

Path Synopsis
middleware
client
Package client provides common extensions (middleware) required for production DRPC clients.
 Package client provides common extensions (middleware) required for production DRPC clients.
server
Package server provides common extensions (middleware) required for production DRPC servers.
 Package server provides common extensions (middleware) required for production DRPC servers.
Package ws provides a WebSocket proxy with support for bidirectional streaming on DRPC servers.
 Package ws provides a WebSocket proxy with support for bidirectional streaming on DRPC servers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.