plugin

package module
v0.0.0-...-8cf118f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 8, 2016 License: MPL-2.0 Imports: 24 Imported by: 0

README

Go Plugin System over RPC

go-plugin is a Go (golang) plugin system over RPC. It is the plugin system that has been in use by HashiCorp tooling for over 3 years. While initially created for Packer, it has since been used by Terraform and Otto, with plans to also use it for Nomad and Vault.

While the plugin system is over RPC, it is currently only designed to work over a local [reliable] network. Plugins over a real network are not supported and will lead to unexpected behavior.

This plugin system has been used on millions of machines across many different projects and has proven to be battle hardened and ready for production use.

Features

The HashiCorp plugin system supports a number of features:

Plugins are Go interface implementations. This makes writing and consuming plugins feel very natural. To a plugin author: you just implement an interface as if it were going to run in the same process. For a plugin user: you just use and call functions on an interface as if it were in the same process. This plugin system handles the communication in between.

Complex arguments and return values are supported. This library provides APIs for handling complex arguments and return values such as interfaces, io.Reader/Writer, etc. We do this by giving you a library (MuxBroker) for creating new connections between the client/server to serve additional interfaces or transfer raw data.

Bidirectional communication. Because the plugin system supports complex arguments, the host process can send it interface implementations and the plugin can call back into the host process.

Built-in Logging. Any plugins that use the log standard library will have log data automatically sent to the host process. The host process will mirror this output prefixed with the path to the plugin binary. This makes debugging with plugins simple.

Protocol Versioning. A very basic "protocol version" is supported that can be incremented to invalidate any previous plugins. This is useful when interface signatures are changing, protocol level changes are necessary, etc. When a protocol version is incompatible, a human friendly error message is shown to the end user.

Stdout/Stderr Syncing. While plugins are subprocesses, they can continue to use stdout/stderr as usual and the output will get mirrored back to the host process. The host process can control what io.Writer these streams go to to prevent this from happening.

TTY Preservation. Plugin subprocesses are connected to the identical stdin file descriptor as the host process, allowing software that requires a TTY to work. For example, a plugin can execute ssh and even though there are multiple subprocesses and RPC happening, it will look and act perfectly to the end user.

Host upgrade while a plugin is running. Plugins can be "reattached" so that the host process can be upgraded while the plugin is still running. This requires the host/plugin to know this is possible and daemonize properly. NewClient takes a ReattachConfig to determine if and how to reattach.

Architecture

The HashiCorp plugin system works by launching subprocesses and communicating over RPC (using standard net/rpc). A single connection is made between any plugin and the host process, and we use a connection multiplexing library to multiplex any other connections on top.

This architecture has a number of benefits:

  • Plugins can't crash your host process: A panic in a plugin doesn't panic the plugin user.

  • Plugins are very easy to write: just write a Go application and go build. Theoretically you could also use another language as long as it can communicate the Go net/rpc protocol but this hasn't yet been tried.

  • Plugins are very easy to install: just put the binary in a location where the host will find it (depends on the host but this library also provides helpers), and the plugin host handles the rest.

  • Plugins can be relatively secure: The plugin only has access to the interfaces and args given to it, not to the entire memory space of the process. More security features are planned (see the coming soon section below).

Usage

To use the plugin system, you must take the following steps. These are high-level steps that must be done. Examples are available in the examples/ directory.

  1. Choose the interface(s) you want to expose for plugins.

  2. For each interface, implement an implementation of that interface that communicates over an *rpc.Client (from the standard net/rpc package) for every function call. Likewise, implement the RPC server struct this communicates to which is then communicating to a real, concrete implementation.

  3. Create a Plugin implementation that knows how to create the RPC client/server for a given plugin type.

  4. Plugin authors call plugin.Serve to serve a plugin from the main function.

  5. Plugin users use plugin.Client to launch a subprocess and request an interface implementation over RPC.

That's it! In practice, step 2 is the most tedious and time consuming step. Even so, it isn't very difficult and you can see examples in the examples/ directory as well as throughout our various open source projects.

For complete API documentation, see GoDoc.

Roadmap

Our plugin system is constantly evolving. As we use the plugin system for new projects or for new features in existing projects, we constantly find improvements we can make.

At this point in time, the roadmap for the plugin system is:

Cryptographically Secure Plugins. We'll implement signing plugins and loading signed plugins in order to allow Vault to make use of multi-process in a secure way.

Semantic Versioning. Plugins will be able to implement a semantic version. This plugin system will give host processes a system for constraining versions. This is in addition to the protocol versioning already present which is more for larger underlying changes.

Plugin fetching. We will integrate with go-getter to support automatic download + install of plugins. Paired with cryptographically secure plugins (above), we can make this a safe operation for an amazing user experience.

What About Shared Libraries?

When we started using plugins (late 2012, early 2013), plugins over RPC were the only option since Go didn't support dynamic library loading. Today, Go still doesn't support dynamic library loading, but they do intend to. Since 2012, our plugin system has stabilized from millions of users using it, and has many benefits we've come to value greatly.

For example, we intend to use this plugin system in Vault, and dynamic library loading will simply never be acceptable in Vault for security reasons. That is an extreme example, but we believe our library system has more upsides than downsides over dynamic library loading and since we've had it built and tested for years, we'll likely continue to use it.

Shared libraries have one major advantage over our system which is much higher performance. In real world scenarios across our various tools, we've never required any more performance out of our plugin system and it has seen very high throughput, so this isn't a concern for us at the moment.

Documentation

Overview

The plugin package exposes functions and helpers for communicating to plugins which are implemented as standalone binary applications.

plugin.Client fully manages the lifecycle of executing the application, connecting to it, and returning the RPC client for dispensing plugins.

plugin.Serve fully manages listeners to expose an RPC server from a binary that plugin.Client can connect to.

Index

Constants

View Source 
const CoreProtocolVersion = 1

CoreProtocolVersion is the ProtocolVersion of the plugin system itself. We will increment this whenever we change any protocol behavior. This will invalidate any prior plugins but will at least allow us to iterate on the core in a safe way. We will do our best to do this very infrequently.

Variables

View Source 
var (
	// ErrProcessNotFound is returned when a client is instantiated to
	// reattach to an existing process and it isn't found.
	ErrProcessNotFound = errors.New("Reattachment process not found")
)

Error types

View Source 
var Killed uint32 = 0

If this is 1, then we've called CleanupClients. This can be used by plugin RPC implementations to change error behavior since you can expected network connection errors at this point. This should be read by using sync/atomic.

Functions

func CleanupClients 

func CleanupClients()

This makes sure all the managed subprocesses are killed and properly logged. This should be called before the parent process running the plugins exits.

This must only be called _once_.

func Discover 

func Discover(glob, dir string) ([]string, error)

Discover discovers plugins that are in a given directory.

The directory doesn't need to be absolute. For example, "." will work fine.

This currently assumes any file matching the glob is a plugin. In the future this may be smarter about checking that a file is executable and so on.

TODO: test

func Serve 

func Serve(opts *ServeConfig)

Serve serves the plugins given by ServeConfig.

Serve doesn't return until the plugin is done being executed. Any errors will be outputted to the log.

This is the method that plugins should call in their main() functions.

func ServeMux 

func ServeMux(m ServeMuxMap)

ServeMux is like Serve, but serves multiple types of plugins determined by the argument given on the command-line.

This command doesn't return until the plugin is done being executed. Any errors are logged or output to stderr.

func TestConn 

func TestConn(t *testing.T) (net.Conn, net.Conn)

TestConn is a helper function for returning a client and server net.Conn connected to each other.

func TestPluginRPCConn 

func TestPluginRPCConn(t *testing.T, ps map[string]Plugin) (*RPCClient, *RPCServer)

TestPluginRPCConn returns a plugin RPC client and server that are connected together and configured.

func TestRPCConn 

func TestRPCConn(t *testing.T) (*rpc.Client, *rpc.Server)

TestRPCConn returns a rpc client and server connected to each other.

Types

type BasicError 

type BasicError struct {
	Message string
}

This is a type that wraps error types so that they can be messaged across RPC channels. Since "error" is an interface, we can't always gob-encode the underlying structure. This is a valid error interface implementer that we will push across.

func NewBasicError 

func NewBasicError(err error) *BasicError

NewBasicError is used to create a BasicError.

err is allowed to be nil.

func (*BasicError) Error 

func (e *BasicError) Error() string

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client handles the lifecycle of a plugin application. It launches plugins, connects to them, dispenses interface implementations, and handles killing the process.

Plugin hosts should use one Client for each plugin executable. To dispense a plugin type, use the `Client.Client` function, and then cal `Dispense`. This awkward API is mostly historical but is used to split the client that deals with subprocess management and the client that does RPC management.

See NewClient and ClientConfig for using a Client.

func NewClient 

func NewClient(config *ClientConfig) (c *Client)

Creates a new plugin client which manages the lifecycle of an external plugin and gets the address for the RPC connection.

The client must be cleaned up at some point by calling Kill(). If the client is a managed client (created with NewManagedClient) you can just call CleanupClients at the end of your program and they will be properly cleaned.

func (*Client) Client 

func (c *Client) Client() (*RPCClient, error)

Client returns an RPC client for the plugin.

Subsequent calls to this will return the same RPC client.

func (*Client) Exited 

func (c *Client) Exited() bool

Tells whether or not the underlying process has exited.

func (*Client) Kill 

func (c *Client) Kill()

End the executing subprocess (if it is running) and perform any cleanup tasks necessary such as capturing any remaining logs and so on.

This method blocks until the process successfully exits.

This method can safely be called multiple times.

func (*Client) ReattachConfig 

func (c *Client) ReattachConfig() *ReattachConfig

ReattachConfig returns the information that must be provided to NewClient to reattach to the plugin process that this client started. This is useful for plugins that detach from their parent process.

If this returns nil then the process hasn't been started yet. Please call Start or Client before calling this.

func (*Client) Start 

func (c *Client) Start() (addr net.Addr, err error)

Starts the underlying subprocess, communicating with it to negotiate a port for RPC connections, and returning the address to connect via RPC.

This method is safe to call multiple times. Subsequent calls have no effect. Once a client has been started once, it cannot be started again, even if it was killed.

type ClientConfig 

type ClientConfig struct {
	// HandshakeConfig is the configuration that must match servers.
	HandshakeConfig

	// Plugins are the plugins that can be consumed.
	Plugins map[string]Plugin

	// One of the following must be set, but not both.
	//
	// Cmd is the unstarted subprocess for starting the plugin. If this is
	// set, then the Client starts the plugin process on its own and connects
	// to it.
	//
	// Reattach is configuration for reattaching to an existing plugin process
	// that is already running. This isn't common.
	Cmd      *exec.Cmd
	Reattach *ReattachConfig

	// Managed represents if the client should be managed by the
	// plugin package or not. If true, then by calling CleanupClients,
	// it will automatically be cleaned up. Otherwise, the client
	// user is fully responsible for making sure to Kill all plugin
	// clients. By default the client is _not_ managed.
	Managed bool

	// The minimum and maximum port to use for communicating with
	// the subprocess. If not set, this defaults to 10,000 and 25,000
	// respectively.
	MinPort, MaxPort uint

	// StartTimeout is the timeout to wait for the plugin to say it
	// has started successfully.
	StartTimeout time.Duration

	// If non-nil, then the stderr of the client will be written to here
	// (as well as the log). This is the original os.Stderr of the subprocess.
	// This isn't the output of synced stderr.
	Stderr io.Writer

	// SyncStdout, SyncStderr can be set to override the
	// respective os.Std* values in the plugin. Care should be taken to
	// avoid races here. If these are nil, then this will automatically be
	// hooked up to os.Stdin, Stdout, and Stderr, respectively.
	//
	// If the default values (nil) are used, then this package will not
	// sync any of these streams.
	SyncStdout io.Writer
	SyncStderr io.Writer
}

ClientConfig is the configuration used to initialize a new plugin client. After being used to initialize a plugin client, that configuration must not be modified again.

type HandshakeConfig 

type HandshakeConfig struct {
	// ProtocolVersion is the version that clients must match on to
	// agree they can communicate. This should match the ProtocolVersion
	// set on ClientConfig when using a plugin.
	ProtocolVersion uint

	// MagicCookieKey and value are used as a very basic verification
	// that a plugin is intended to be launched. This is not a security
	// measure, just a UX feature. If the magic cookie doesn't match,
	// we show human-friendly output.
	MagicCookieKey   string
	MagicCookieValue string
}

HandshakeConfig is the configuration used by client and servers to handshake before starting a plugin connection. This is embedded by both ServeConfig and ClientConfig.

In practice, the plugin host creates a HandshakeConfig that is exported and plugins then can easily consume it.

type MuxBroker 

type MuxBroker struct {
	sync.Mutex
	// contains filtered or unexported fields
}

MuxBroker is responsible for brokering multiplexed connections by unique ID.

It is used by plugins to multiplex multiple RPC connections and data streams on top of a single connection between the plugin process and the host process.

This allows a plugin to request a channel with a specific ID to connect to or accept a connection from, and the broker handles the details of holding these channels open while they're being negotiated.

The Plugin interface has access to these for both Server and Client. The broker can be used by either (optionally) to reserve and connect to new multiplexed streams. This is useful for complex args and return values, or anything else you might need a data stream for.

func (*MuxBroker) Accept 

func (m *MuxBroker) Accept(id uint32) (net.Conn, error)

Accept accepts a connection by ID.

This should not be called multiple times with the same ID at one time.

func (*MuxBroker) AcceptAndServe 

func (m *MuxBroker) AcceptAndServe(id uint32, v interface{})

AcceptAndServe is used to accept a specific stream ID and immediately serve an RPC server on that stream ID. This is used to easily serve complex arguments.

The served interface is always registered to the "Plugin" name.

func (*MuxBroker) Close 

func (m *MuxBroker) Close() error

Close closes the connection and all sub-connections.

func (*MuxBroker) Dial 

func (m *MuxBroker) Dial(id uint32) (net.Conn, error)

Dial opens a connection by ID.

func (*MuxBroker) NextId 

func (m *MuxBroker) NextId() uint32

NextId returns a unique ID to use next.

It is possible for very long-running plugin hosts to wrap this value, though it would require a very large amount of RPC calls. In practice we've never seen it happen.

func (*MuxBroker) Run 

func (m *MuxBroker) Run()

Run starts the brokering and should be executed in a goroutine, since it blocks forever, or until the session closes.

Uses of MuxBroker never need to call this. It is called internally by the plugin host/client.

type Plugin 

type Plugin interface {
	// Server should return the RPC server compatible struct to serve
	// the methods that the Client calls over net/rpc.
	Server(*MuxBroker) (interface{}, error)

	// Client returns an interface implementation for the plugin you're
	// serving that communicates to the server end of the plugin.
	Client(*MuxBroker, *rpc.Client) (interface{}, error)
}

Plugin is the interface that is implemented to serve/connect to an inteface implementation.

type RPCClient 

type RPCClient struct {
	// contains filtered or unexported fields
}

RPCClient connects to an RPCServer over net/rpc to dispense plugin types.

func NewRPCClient 

func NewRPCClient(conn io.ReadWriteCloser, plugins map[string]Plugin) (*RPCClient, error)

NewRPCClient creates a client from an already-open connection-like value. Dial is typically used instead.

func (*RPCClient) Close 

func (c *RPCClient) Close() error

Close closes the connection. The client is no longer usable after this is called.

func (*RPCClient) Dispense 

func (c *RPCClient) Dispense(name string) (interface{}, error)

func (*RPCClient) SyncStreams 

func (c *RPCClient) SyncStreams(stdout io.Writer, stderr io.Writer) error

SyncStreams should be called to enable syncing of stdout, stderr with the plugin.

This will return immediately and the syncing will continue to happen in the background. You do not need to launch this in a goroutine itself.

This should never be called multiple times.

type RPCServer 

type RPCServer struct {
	Plugins map[string]Plugin

	// Stdout, Stderr are what this server will use instead of the
	// normal stdin/out/err. This is because due to the multi-process nature
	// of our plugin system, we can't use the normal process values so we
	// make our own custom one we pipe across.
	Stdout io.Reader
	Stderr io.Reader

	// DoneCh should be set to a non-nil channel that will be closed
	// when the control requests the RPC server to end.
	DoneCh chan<- struct{}
}

RPCServer listens for network connections and then dispenses interface implementations over net/rpc.

func (*RPCServer) Accept 

func (s *RPCServer) Accept(lis net.Listener)

Accept accepts connections on a listener and serves requests for each incoming connection. Accept blocks; the caller typically invokes it in a go statement.

func (*RPCServer) ServeConn 

func (s *RPCServer) ServeConn(conn io.ReadWriteCloser)

ServeConn runs a single connection.

ServeConn blocks, serving the connection until the client hangs up.

type ReattachConfig 

type ReattachConfig struct {
	Addr net.Addr
	Pid  int
}

ReattachConfig is used to configure a client to reattach to an already-running plugin process. You can retrieve this information by calling ReattachConfig on Client.

type ServeConfig 

type ServeConfig struct {
	// HandshakeConfig is the configuration that must match clients.
	HandshakeConfig

	// Plugins are the plugins that are served.
	Plugins map[string]Plugin
}

ServeConfig configures what sorts of plugins are served.

type ServeMuxMap 

type ServeMuxMap map[string]*ServeConfig

ServeMuxMap is the type that is used to configure ServeMux

Source Files

View all Source files

Directories

Path Synopsis
examples
basic

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.