grpcurl

README

gRPCurl

grpcurl is a command-line tool that lets you interact with gRPC servers. It's basically curl for gRPC servers.

The main purpose for this tool is to invoke RPC methods on a gRPC server from the command-line. gRPC servers use a binary encoding on the wire (protocol buffers, or "protobufs" for short). So they are basically impossible to interact with using regular curl (and older versions of curl that do not support HTTP/2 are of course non-starters). This program accepts messages using JSON encoding, which is much more friendly for both humans and scripts.

With this tool you can also browse the schema for gRPC services, either by querying a server that supports service reflection, by reading proto source files, or by loading in compiled "protoset" files (files that contain encoded file descriptor protos). In fact, the way the tool transforms JSON request data into a binary encoded protobuf is using that very same schema. So, if the server you interact with does not support reflection, you will either need the proto source files that define the service or need protoset files that grpcurl can use.

Examples for how to set up server reflection can be found here.

This repo also provides a library package, github.com/fullstorydev/grpcurl, that has functions for simplifying the construction of other command-line tools that dynamically invoke gRPC endpoints. This code is a great example of how to use the various packages of the protoreflect library, and shows off what they can do.

Features

grpcurl supports all kinds of RPC methods, including streaming methods. You can even operate bi-directional streaming methods interactively by running grpcurl from an interactive terminal and using stdin as the request body!

grpcurl supports both plain-text and TLS servers and has numerous options for TLS configuration. It also supports mutual TLS, where the client is required to present a client certificate.

As mentioned above, grpcurl works seamlessly if the server supports the reflection service. If not, you can supply the .proto source files or you can supply protoset files (containing compiled descriptors, produced by protoc) to grpcurl.

Installation

You can use the go tool to install grpcurl:

go get github.com/fullstorydev/grpcurl
go install github.com/fullstorydev/grpcurl/cmd/grpcurl

This installs the command into the bin sub-folder of wherever your $GOPATH environment variable points. If this directory is already in your $PATH, then you should be good to go.

If you have already pulled down this repo to a location that is not in your $GOPATH and want to build from the sources, you can cd into the repo and then run make install.

If you encounter compile errors, you could have out-dated versions of grpcurl's dependencies. You can update the dependencies by running make updatedeps.

Usage

The usage doc for the tool explains the numerous options:

grpcurl -help

In the sections below, you will find numerous examples demonstrating how to use grpcurl.

Invoking RPCs

Invoking an RPC on a trusted server (e.g. TLS without self-signed key or custom CA) that requires no client certs and supports service reflection is the simplest thing to do with grpcurl. This minimal invocation sends an empty request body:

grpcurl grpc.server.com:443 my.custom.server.Service/Method

To send a non-empty request, use the -d argument. Note that all arguments must come before the server address and method name:

grpcurl -d '{"id": 1234, "tags": ["foo","bar"]}' \
    grpc.server.com:443 my.custom.server.Service/Method

As can be seen in the example, the supplied body must be in JSON format. The body will be parsed and then transmitted to the server in the protobuf binary format.

If you want to include grpcurl in a command pipeline, such as when using jq to create a request body, you can use -d @, which tells grpcurl to read the actual request body from stdin:

grpcurl -d @ grpc.server.com:443 my.custom.server.Service/Method <<<EOM
{
  "id": 1234,
  "tags": [
    "foor",
    "bar"
  ]
}
EOM

Listing Services

To list all services exposed by a server, use the "list" verb. When using .proto source or protoset files instead of server reflection, this lists all services defined in the source or protoset files.

# Server supports reflection
grpcurl localhost:8787 list

# Using compiled protoset files
grpcurl -protoset my-protos.bin list

# Using proto sources
grpcurl -import-path ../protos -proto my-stuff.proto list

The "list" verb also lets you see all methods in a particular service:

grpcurl localhost:8787 list my.custom.server.Service

Describing Elements

The "describe" verb will print the type of any symbol that the server knows about or that is found in a given protoset file and also print the full descriptor for the symbol, in JSON.

# Server supports reflection
grpcurl localhost:8787 describe my.custom.server.Service.MethodOne

# Using compiled protoset files
grpcurl -protoset my-protos.bin describe my.custom.server.Service.MethodOne

# Using proto sources
grpcurl -import-path ../protos -proto my-stuff.proto describe my.custom.server.Service.MethodOne

Proto Source Files

To use grpcurl on servers that do not support reflection, you can use .proto source files.

In addition to using -proto flags to point grpcurl at the relevant proto source file(s), you may also need to supply -import-path flags to tell grpcurl the folders from which dependencies can be imported.

Just like when compiling with protoc, you do not need to provide an import path for the location of the standard protos included with protoc (which contain various "well-known types" with a package definition of google.protobuf). These files are "known" by grpcurl as a snapshot of their descriptors is built into the grpcurl binary.

Protoset Files

You can also use compiled protoset files with grpcurl. If you are scripting grpcurl and need to re-use the same proto sources for many invocations, you will see better performance by using protoset files (since it skips the parsing and compilation steps with each invocation).

Protoset files contain binary encoded google.protobuf.FileDescriptorSet protos. To create a protoset file, invoke protoc with the *.proto files that describe the service:

protoc --proto_path=. \
    --descriptor_set_out=myservice.protoset \
    --include_imports \
    my/custom/server/service.proto

The --descriptor_set_out argument is what tells protoc to produce a protoset, and the --include_imports argument is necessary for the protoset to contain everything that grpcurl needs to process and understand the schema.

Documentation

Overview

Package grpcurl provides the core functionality exposed by the grpcurl command, for dynamically connecting to a server, using the reflection service to inspect the server, and invoking RPCs. The grpcurl command-line tool constructs a DescriptorSource, based on the command-line parameters, and supplies an InvocationEventHandler to supply request data (which can come from command-line args or the process's stdin) and to log the events (to the process's stdout).

Index

• Variables
• func BlockingDial(ctx context.Context, network, address string, ...) (*grpc.ClientConn, error)
• func ClientTransportCredentials(insecureSkipVerify bool, cacertFile, clientCertFile, clientKeyFile string) (credentials.TransportCredentials, error)
• func EnsureExtensions(source DescriptorSource, msg proto.Message) proto.Message
• func GetDescriptorText(dsc desc.Descriptor, descSource DescriptorSource) (string, error)
• func InvokeRpc(ctx context.Context, source DescriptorSource, cc *grpc.ClientConn, ...) error
• func ListMethods(source DescriptorSource, serviceName string) ([]string, error)
• func ListServices(source DescriptorSource) ([]string, error)
• func MetadataFromHeaders(headers []string) metadata.MD
• func MetadataToString(md metadata.MD) string
• func ServerTransportCredentials(cacertFile, serverCertFile, serverKeyFile string, requireClientCerts bool) (credentials.TransportCredentials, error)
• type DescriptorSource
  • func DescriptorSourceFromFileDescriptorSet(files *descriptor.FileDescriptorSet) (DescriptorSource, error)
  • func DescriptorSourceFromFileDescriptors(files ...*desc.FileDescriptor) (DescriptorSource, error)
  • func DescriptorSourceFromProtoFiles(importPaths []string, fileNames ...string) (DescriptorSource, error)
  • func DescriptorSourceFromProtoSets(fileNames ...string) (DescriptorSource, error)
  • func DescriptorSourceFromServer(ctx context.Context, refClient *grpcreflect.Client) DescriptorSource
• type InvocationEventHandler
• type RequestMessageSupplier

Variables

var ErrReflectionNotSupported = errors.New("server does not support the reflection API")

ErrReflectionNotSupported is returned by DescriptorSource operations that rely on interacting with the reflection service when the source does not actually expose the reflection service. When this occurs, an alternate source (like file descriptor sets) must be used.

Functions

func BlockingDial

func BlockingDial(ctx context.Context, network, address string, creds credentials.TransportCredentials, opts ...grpc.DialOption) (*grpc.ClientConn, error)

BlockingDial is a helper method to dial the given address, using optional TLS credentials, and blocking until the returned connection is ready. If the given credentials are nil, the connection will be insecure (plain-text).

func ClientTransportCredentials

func ClientTransportCredentials(insecureSkipVerify bool, cacertFile, clientCertFile, clientKeyFile string) (credentials.TransportCredentials, error)

ClientTransportCredentials builds transport credentials for a GRPC client using the given properties. If cacertFile is blank, only standard trusted certs are used to verify the server certs. If clientCertFile is blank, the client will not use a client certificate. If clientCertFile is not blank then clientKeyFile must not be blank.

func EnsureExtensions

func EnsureExtensions(source DescriptorSource, msg proto.Message) proto.Message

EnsureExtensions uses the given descriptor source to download extensions for the given message. It returns a copy of the given message, but as a dynamic message that knows about all extensions known to the given descriptor source.

func GetDescriptorText

func GetDescriptorText(dsc desc.Descriptor, descSource DescriptorSource) (string, error)

GetDescriptorText returns a string representation of the given descriptor.

func InvokeRpc

func InvokeRpc(ctx context.Context, source DescriptorSource, cc *grpc.ClientConn, methodName string,
	headers []string, handler InvocationEventHandler, requestData RequestMessageSupplier) error

InvokeRpc uses te given GRPC connection to invoke the given method. The given descriptor source is used to determine the type of method and the type of request and response message. The given headers are sent as request metadata. Methods on the given event handler are called as the invocation proceeds.

The given requestData function supplies the actual data to send. It should return io.EOF when there is no more request data. If it returns a nil error then the returned JSON message should not be blank. If the method being invoked is a unary or server-streaming RPC (e.g. exactly one request message) and there is no request data (e.g. the first invocation of the function returns io.EOF), then a blank request message is sent, as if the request data were an empty object: "{}".

If the requestData function and the given event handler coordinate or share any state, they should be thread-safe. This is because the requestData function may be called from a different goroutine than the one invoking event callbacks. (This only happens for bi-directional streaming RPCs, where one goroutine sends request messages and another consumes the response messages).

func ListMethods

func ListMethods(source DescriptorSource, serviceName string) ([]string, error)

ListMethods uses the given descriptor source to return a sorted list of method names for the specified fully-qualified service name.

func ListServices

func ListServices(source DescriptorSource) ([]string, error)

ListServices uses the given descriptor source to return a sorted list of fully-qualified service names.

func MetadataFromHeaders

func MetadataFromHeaders(headers []string) metadata.MD

MetadataFromHeaders converts a list of header strings (each string in "Header-Name: Header-Value" form) into metadata. If a string has a header name without a value (e.g. does not contain a colon), the value is assumed to be blank. Binary headers (those whose names end in "-bin") should be base64-encoded. But if they cannot be base64-decoded, they will be assumed to be in raw form and used as is.

func MetadataToString

func MetadataToString(md metadata.MD) string

MetadataToString returns a string representation of the given metadata, for displaying to users.

func ServerTransportCredentials

func ServerTransportCredentials(cacertFile, serverCertFile, serverKeyFile string, requireClientCerts bool) (credentials.TransportCredentials, error)

ServerTransportCredentials builds transport credentials for a GRPC server using the given properties. If cacertFile is blank, the server will not request client certs unless requireClientCerts is true. When requireClientCerts is false and cacertFile is not blank, the server will verify client certs when presented, but will not require client certs. The serverCertFile and serverKeyFile must both not be blank.

Types

type DescriptorSource

type DescriptorSource interface {
	// ListServices returns a list of fully-qualified service names. It will be all services in a set of
	// descriptor files or the set of all services exposed by a GRPC server.
	ListServices() ([]string, error)
	// FindSymbol returns a descriptor for the given fully-qualified symbol name.
	FindSymbol(fullyQualifiedName string) (desc.Descriptor, error)
	// AllExtensionsForType returns all known extension fields that extend the given message type name.
	AllExtensionsForType(typeName string) ([]*desc.FieldDescriptor, error)
}

DescriptorSource is a source of protobuf descriptor information. It can be backed by a FileDescriptorSet proto (like a file generated by protoc) or a remote server that supports the reflection API.

func DescriptorSourceFromFileDescriptorSet

func DescriptorSourceFromFileDescriptorSet(files *descriptor.FileDescriptorSet) (DescriptorSource, error)

DescriptorSourceFromFileDescriptorSet creates a DescriptorSource that is backed by the FileDescriptorSet.

func DescriptorSourceFromFileDescriptors

func DescriptorSourceFromFileDescriptors(files ...*desc.FileDescriptor) (DescriptorSource, error)

DescriptorSourceFromFileDescriptorSet creates a DescriptorSource that is backed by the given file descriptors

func DescriptorSourceFromProtoFiles

func DescriptorSourceFromProtoFiles(importPaths []string, fileNames ...string) (DescriptorSource, error)

DescriptorSourceFromProtoFiles creates a DescriptorSource that is backed by the named files, whose contents are Protocol Buffer source files. The given importPaths are used to locate any imported files.

func DescriptorSourceFromProtoSets

func DescriptorSourceFromProtoSets(fileNames ...string) (DescriptorSource, error)

DescriptorSourceFromProtoSets creates a DescriptorSource that is backed by the named files, whose contents are encoded FileDescriptorSet protos.

func DescriptorSourceFromServer

func DescriptorSourceFromServer(ctx context.Context, refClient *grpcreflect.Client) DescriptorSource

DescriptorSourceFromServer creates a DescriptorSource that uses the given GRPC reflection client to interrogate a server for descriptor information. If the server does not support the reflection API then the various DescriptorSource methods will return ErrReflectionNotSupported

type InvocationEventHandler

type InvocationEventHandler interface {
	// OnResolveMethod is called with a descriptor of the method that is being invoked.
	OnResolveMethod(*desc.MethodDescriptor)
	// OnSendHeaders is called with the request metadata that is being sent.
	OnSendHeaders(metadata.MD)
	// OnReceiveHeaders is called when response headers have been received.
	OnReceiveHeaders(metadata.MD)
	// OnReceiveResponse is called for each response message received.
	OnReceiveResponse(proto.Message)
	// OnReceiveTrailers is called when response trailers and final RPC status have been received.
	OnReceiveTrailers(*status.Status, metadata.MD)
}

InvocationEventHandler is a bag of callbacks for handling events that occur in the course of invoking an RPC. The handler also provides request data that is sent. The callbacks are generally called in the order they are listed below.

type RequestMessageSupplier

type RequestMessageSupplier func() ([]byte, error)

RequestMessageSupplier is a function that is called to retrieve request messages for a GRPC operation. The message contents must be valid JSON. If the supplier has no more messages, it should return nil, io.EOF.