grpcurl

package module
v1.3.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 5, 2020 License: MIT Imports: 34 Imported by: 2

README

gRPCurl

Build Status Go Report Card

grpcurl is a command-line tool that lets you interact with gRPC servers. It's basically curl for gRPC servers.

The main purpose for this tool is to invoke RPC methods on a gRPC server from the command-line. gRPC servers use a binary encoding on the wire (protocol buffers, or "protobufs" for short). So they are basically impossible to interact with using regular curl (and older versions of curl that do not support HTTP/2 are of course non-starters). This program accepts messages using JSON encoding, which is much more friendly for both humans and scripts.

With this tool you can also browse the schema for gRPC services, either by querying a server that supports server reflection, by reading proto source files, or by loading in compiled "protoset" files (files that contain encoded file descriptor protos). In fact, the way the tool transforms JSON request data into a binary encoded protobuf is using that very same schema. So, if the server you interact with does not support reflection, you will either need the proto source files that define the service or need protoset files that grpcurl can use.

This repo also provides a library package, github.com/hoveychen/grpcurl, that has functions for simplifying the construction of other command-line tools that dynamically invoke gRPC endpoints. This code is a great example of how to use the various packages of the protoreflect library, and shows off what they can do.

See also the grpcurl talk at GopherCon 2018.

Features

grpcurl supports all kinds of RPC methods, including streaming methods. You can even operate bi-directional streaming methods interactively by running grpcurl from an interactive terminal and using stdin as the request body!

grpcurl supports both plain-text and TLS servers and has numerous options for TLS configuration. It also supports mutual TLS, where the client is required to present a client certificate.

As mentioned above, grpcurl works seamlessly if the server supports the reflection service. If not, you can supply the .proto source files or you can supply protoset files (containing compiled descriptors, produced by protoc) to grpcurl.

Installation

Binaries

Download the binary from the releases page.

On macOS, grpcurl is available via Homebrew:

brew install grpcurl
From Source

You can use the go tool to install grpcurl:

go get github.com/hoveychen/grpcurl
go install github.com/hoveychen/grpcurl/cmd/grpcurl

This installs the command into the bin sub-folder of wherever your $GOPATH environment variable points. If this directory is already in your $PATH, then you should be good to go.

If you have already pulled down this repo to a location that is not in your $GOPATH and want to build from the sources, you can cd into the repo and then run make install.

If you encounter compile errors, you could have out-dated versions of grpcurl's dependencies. You can update the dependencies by running make updatedeps. You can also use vgo to install, which will use the right versions of dependencies. Or, if you are using Go 1.11, you can add GO111MODULE=on as a prefix to the commands above, which will also build using the right versions of dependencies (vs. whatever you may already in your GOPATH).

Usage

The usage doc for the tool explains the numerous options:

grpcurl -help

In the sections below, you will find numerous examples demonstrating how to use grpcurl.

Invoking RPCs

Invoking an RPC on a trusted server (e.g. TLS without self-signed key or custom CA) that requires no client certs and supports server reflection is the simplest thing to do with grpcurl. This minimal invocation sends an empty request body:

grpcurl grpc.server.com:443 my.custom.server.Service/Method

To send a non-empty request, use the -d argument. Note that all arguments must come before the server address and method name:

grpcurl -d '{"id": 1234, "tags": ["foo","bar"]}' \
    grpc.server.com:443 my.custom.server.Service/Method

As can be seen in the example, the supplied body must be in JSON format. The body will be parsed and then transmitted to the server in the protobuf binary format.

If you want to include grpcurl in a command pipeline, such as when using jq to create a request body, you can use -d @, which tells grpcurl to read the actual request body from stdin:

grpcurl -d @ grpc.server.com:443 my.custom.server.Service/Method <<EOM
{
  "id": 1234,
  "tags": [
    "foor",
    "bar"
  ]
}
EOM
Listing Services

To list all services exposed by a server, use the "list" verb. When using .proto source or protoset files instead of server reflection, this lists all services defined in the source or protoset files.

# Server supports reflection
grpcurl localhost:8787 list

# Using compiled protoset files
grpcurl -protoset my-protos.bin list

# Using proto sources
grpcurl -import-path ../protos -proto my-stuff.proto list

The "list" verb also lets you see all methods in a particular service:

grpcurl localhost:8787 list my.custom.server.Service
Describing Elements

The "describe" verb will print the type of any symbol that the server knows about or that is found in a given protoset file. It also prints a description of that symbol, in the form of snippets of proto source. It won't necessarily be the original source that defined the element, but it will be equivalent.

# Server supports reflection
grpcurl localhost:8787 describe my.custom.server.Service.MethodOne

# Using compiled protoset files
grpcurl -protoset my-protos.bin describe my.custom.server.Service.MethodOne

# Using proto sources
grpcurl -import-path ../protos -proto my-stuff.proto describe my.custom.server.Service.MethodOne

Descriptor Sources

The grpcurl tool can operate on a variety of sources for descriptors. The descriptors are required, in order for grpcurl to understand the RPC schema, translate inputs into the protobuf binary format as well as translate responses from the binary format into text. The sections below document the supported sources and what command-line flags are needed to use them.

Server Reflection

Without any additional command-line flags, grpcurl will try to use server reflection.

Examples for how to set up server reflection can be found here.

When using reflection, the server address (host:port or path to Unix socket) is required even for "list" and "describe" operations, so that grpcurl can connect to the server and ask it for its descriptors.

Proto Source Files

To use grpcurl on servers that do not support reflection, you can use .proto source files.

In addition to using -proto flags to point grpcurl at the relevant proto source file(s), you may also need to supply -import-path flags to tell grpcurl the folders from which dependencies can be imported.

Just like when compiling with protoc, you do not need to provide an import path for the location of the standard protos included with protoc (which contain various "well-known types" with a package definition of google.protobuf). These files are "known" by grpcurl as a snapshot of their descriptors is built into the grpcurl binary.

When using proto sources, you can omit the server address (host:port or path to Unix socket) when using the "list" and "describe" operations since they only need to consult the proto source files.

Protoset Files

You can also use compiled protoset files with grpcurl. If you are scripting grpcurl and need to re-use the same proto sources for many invocations, you will see better performance by using protoset files (since it skips the parsing and compilation steps with each invocation).

Protoset files contain binary encoded google.protobuf.FileDescriptorSet protos. To create a protoset file, invoke protoc with the *.proto files that define the service:

protoc --proto_path=. \
    --descriptor_set_out=myservice.protoset \
    --include_imports \
    my/custom/server/service.proto

The --descriptor_set_out argument is what tells protoc to produce a protoset, and the --include_imports argument is necessary for the protoset to contain everything that grpcurl needs to process and understand the schema.

When using protosets, you can omit the server address (host:port or path to Unix socket) when using the "list" and "describe" operations since they only need to consult the protoset files.

Documentation

Overview

Package grpcurl provides the core functionality exposed by the grpcurl command, for dynamically connecting to a server, using the reflection service to inspect the server, and invoking RPCs. The grpcurl command-line tool constructs a DescriptorSource, based on the command-line parameters, and supplies an InvocationEventHandler to supply request data (which can come from command-line args or the process's stdin) and to log the events (to the process's stdout).

Index

Constants

View Source 
const (
	FormatJSON = Format("json")
	FormatText = Format("text")
)

Variables

View Source 
var ErrReflectionNotSupported = errors.New("server does not support the reflection API")

ErrReflectionNotSupported is returned by DescriptorSource operations that rely on interacting with the reflection service when the source does not actually expose the reflection service. When this occurs, an alternate source (like file descriptor sets) must be used.

Functions

func AnyResolverFromDescriptorSource 

func AnyResolverFromDescriptorSource(source DescriptorSource) jsonpb.AnyResolver

AnyResolverFromDescriptorSource returns an AnyResolver that will search for types using the given descriptor source.

func AnyResolverFromDescriptorSourceWithFallback 

func AnyResolverFromDescriptorSourceWithFallback(source DescriptorSource) jsonpb.AnyResolver

AnyResolverFromDescriptorSourceWithFallback returns an AnyResolver that will search for types using the given descriptor source and then fallback to a special message if the type is not found. The fallback type will render to JSON with a "@type" property, just like an Any message, but also with a custom "@value" property that includes the binary encoded payload.

func BlockingDial 

func BlockingDial(ctx context.Context, network, address string, creds credentials.TransportCredentials, opts ...grpc.DialOption) (*grpc.ClientConn, error)

BlockingDial is a helper method to dial the given address, using optional TLS credentials, and blocking until the returned connection is ready. If the given credentials are nil, the connection will be insecure (plain-text).

func ClientTransportCredentials 

func ClientTransportCredentials(insecureSkipVerify bool, cacertFile, clientCertFile, clientKeyFile string) (credentials.TransportCredentials, error)

ClientTransportCredentials builds transport credentials for a gRPC client using the given properties. If cacertFile is blank, only standard trusted certs are used to verify the server certs. If clientCertFile is blank, the client will not use a client certificate. If clientCertFile is not blank then clientKeyFile must not be blank.

func EnsureExtensions 

func EnsureExtensions(source DescriptorSource, msg proto.Message) proto.Message

EnsureExtensions uses the given descriptor source to download extensions for the given message. It returns a copy of the given message, but as a dynamic message that knows about all extensions known to the given descriptor source.

func GetAllFiles 

func GetAllFiles(source DescriptorSource) ([]*desc.FileDescriptor, error)

GetAllFiles uses the given descriptor source to return a list of file descriptors.

func GetDescriptorText 

func GetDescriptorText(dsc desc.Descriptor, _ DescriptorSource) (string, error)

GetDescriptorText returns a string representation of the given descriptor. This returns a snippet of proto source that describes the given element.

func InvokeRPC 

func InvokeRPC(ctx context.Context, source DescriptorSource, ch grpcdynamic.Channel, methodName string,
	headers []string, handler InvocationEventHandler, requestData RequestSupplier) error

InvokeRPC uses the given gRPC channel to invoke the given method. The given descriptor source is used to determine the type of method and the type of request and response message. The given headers are sent as request metadata. Methods on the given event handler are called as the invocation proceeds.

The given requestData function supplies the actual data to send. It should return io.EOF when there is no more request data. If the method being invoked is a unary or server-streaming RPC (e.g. exactly one request message) and there is no request data (e.g. the first invocation of the function returns io.EOF), then an empty request message is sent.

If the requestData function and the given event handler coordinate or share any state, they should be thread-safe. This is because the requestData function may be called from a different goroutine than the one invoking event callbacks. (This only happens for bi-directional streaming RPCs, where one goroutine sends request messages and another consumes the response messages).

func InvokeRpc deprecated 

func InvokeRpc(ctx context.Context, source DescriptorSource, cc *grpc.ClientConn, methodName string,
	headers []string, handler InvocationEventHandler, requestData RequestMessageSupplier) error

InvokeRpc uses the given gRPC connection to invoke the given method. This function is deprecated and will be removed in a future release. It just delegates to the similarly named InvokeRPC method, whose signature is only slightly different.

Deprecated: use InvokeRPC instead.

func ListMethods 

func ListMethods(source DescriptorSource, serviceName string) ([]string, error)

ListMethods uses the given descriptor source to return a sorted list of method names for the specified fully-qualified service name.

func ListServices 

func ListServices(source DescriptorSource) ([]string, error)

ListServices uses the given descriptor source to return a sorted list of fully-qualified service names.

func MakeTemplate 

func MakeTemplate(md *desc.MessageDescriptor) proto.Message

MakeTemplate returns a message instance for the given descriptor that is a suitable template for creating an instance of that message in JSON. In particular, it ensures that any repeated fields (which include map fields) are not empty, so they will render with a single element (to show the types and optionally nested fields). It also ensures that nested messages are not nil by setting them to a message that is also fleshed out as a template message.

func MetadataFromHeaders 

func MetadataFromHeaders(headers []string) metadata.MD

MetadataFromHeaders converts a list of header strings (each string in "Header-Name: Header-Value" form) into metadata. If a string has a header name without a value (e.g. does not contain a colon), the value is assumed to be blank. Binary headers (those whose names end in "-bin") should be base64-encoded. But if they cannot be base64-decoded, they will be assumed to be in raw form and used as is.

func MetadataToString 

func MetadataToString(md metadata.MD) string

MetadataToString returns a string representation of the given metadata, for displaying to users.

func PrintStatus 

func PrintStatus(w io.Writer, stat *status.Status, formatter Formatter)

PrintStatus prints details about the given status to the given writer. The given formatter is used to print any detail messages that may be included in the status. If the given status has a code of OK, "OK" is printed and that is all. Otherwise, "ERROR:" is printed along with a line showing the code, one showing the message string, and each detail message if any are present. The detail messages will be printed as proto text format or JSON, depending on the given formatter.

func RequestParserAndFormatterFor 

func RequestParserAndFormatterFor(format Format, descSource DescriptorSource, emitJSONDefaultFields, includeTextSeparator bool, in io.Reader) (RequestParser, Formatter, error)

RequestParserAndFormatterFor returns a request parser and formatter for the given format. The given descriptor source may be used for parsing message data (if needed by the format). The flags emitJSONDefaultFields and includeTextSeparator are options for JSON and protobuf text formats, respectively. Requests will be parsed from the given in.

func ServerTransportCredentials 

func ServerTransportCredentials(cacertFile, serverCertFile, serverKeyFile string, requireClientCerts bool) (credentials.TransportCredentials, error)

ServerTransportCredentials builds transport credentials for a gRPC server using the given properties. If cacertFile is blank, the server will not request client certs unless requireClientCerts is true. When requireClientCerts is false and cacertFile is not blank, the server will verify client certs when presented, but will not require client certs. The serverCertFile and serverKeyFile must both not be blank.

Types

type DefaultEventHandler 

type DefaultEventHandler struct {

	// NumResponses is the number of responses that have been received.
	NumResponses int
	// Status is the status that was received at the end of an RPC. It is
	// nil if the RPC is still in progress.
	Status *status.Status
	// contains filtered or unexported fields
}

DefaultEventHandler logs events to a writer. This is not thread-safe, but is safe for use with InvokeRPC as long as NumResponses and Status are not read until the call to InvokeRPC completes.

func NewDefaultEventHandler 

func NewDefaultEventHandler(out io.Writer, descSource DescriptorSource, formatter Formatter, verbose bool) *DefaultEventHandler

NewDefaultEventHandler returns an InvocationEventHandler that logs events to the given output. If verbose is true, all events are logged. Otherwise, only response messages are logged.

func (*DefaultEventHandler) OnReceiveHeaders 

func (h *DefaultEventHandler) OnReceiveHeaders(md metadata.MD)

func (*DefaultEventHandler) OnReceiveResponse 

func (h *DefaultEventHandler) OnReceiveResponse(resp proto.Message)

func (*DefaultEventHandler) OnReceiveTrailers 

func (h *DefaultEventHandler) OnReceiveTrailers(stat *status.Status, md metadata.MD)

func (*DefaultEventHandler) OnResolveMethod 

func (h *DefaultEventHandler) OnResolveMethod(md *desc.MethodDescriptor)

func (*DefaultEventHandler) OnSendHeaders 

func (h *DefaultEventHandler) OnSendHeaders(md metadata.MD)

type DescriptorSource 

type DescriptorSource interface {
	// ListServices returns a list of fully-qualified service names. It will be all services in a set of
	// descriptor files or the set of all services exposed by a gRPC server.
	ListServices() ([]string, error)
	// FindSymbol returns a descriptor for the given fully-qualified symbol name.
	FindSymbol(fullyQualifiedName string) (desc.Descriptor, error)
	// AllExtensionsForType returns all known extension fields that extend the given message type name.
	AllExtensionsForType(typeName string) ([]*desc.FieldDescriptor, error)
}

DescriptorSource is a source of protobuf descriptor information. It can be backed by a FileDescriptorSet proto (like a file generated by protoc) or a remote server that supports the reflection API.

func DescriptorSourceFromFileDescriptorSet 

func DescriptorSourceFromFileDescriptorSet(files *descpb.FileDescriptorSet) (DescriptorSource, error)

DescriptorSourceFromFileDescriptorSet creates a DescriptorSource that is backed by the FileDescriptorSet.

func DescriptorSourceFromFileDescriptors 

func DescriptorSourceFromFileDescriptors(files ...*desc.FileDescriptor) (DescriptorSource, error)

DescriptorSourceFromFileDescriptors creates a DescriptorSource that is backed by the given file descriptors

func DescriptorSourceFromProtoFiles 

func DescriptorSourceFromProtoFiles(importPaths []string, fileNames ...string) (DescriptorSource, error)

DescriptorSourceFromProtoFiles creates a DescriptorSource that is backed by the named files, whose contents are Protocol Buffer source files. The given importPaths are used to locate any imported files.

func DescriptorSourceFromProtoSets 

func DescriptorSourceFromProtoSets(fileNames ...string) (DescriptorSource, error)

DescriptorSourceFromProtoSets creates a DescriptorSource that is backed by the named files, whose contents are encoded FileDescriptorSet protos.

func DescriptorSourceFromServer 

func DescriptorSourceFromServer(_ context.Context, refClient *grpcreflect.Client) DescriptorSource

DescriptorSourceFromServer creates a DescriptorSource that uses the given gRPC reflection client to interrogate a server for descriptor information. If the server does not support the reflection API then the various DescriptorSource methods will return ErrReflectionNotSupported

type Format 

type Format string

type Formatter 

type Formatter func(proto.Message) (string, error)

Formatter translates messages into string representations.

func NewJSONFormatter 

func NewJSONFormatter(emitDefaults bool, resolver jsonpb.AnyResolver) Formatter

NewJSONFormatter returns a formatter that returns JSON strings. The JSON will include empty/default values (instead of just omitted them) if emitDefaults is true. The given resolver is used to assist with encoding of google.protobuf.Any messages.

func NewTextFormatter 

func NewTextFormatter(includeSeparator bool) Formatter

NewTextFormatter returns a formatter that returns strings in the protobuf text format. If includeSeparator is true then, when invoked to format multiple messages, all messages after the first one will be prefixed with the ASCII 'Record Separator' character (0x1E).

type InvocationEventHandler 

type InvocationEventHandler interface {
	// OnResolveMethod is called with a descriptor of the method that is being invoked.
	OnResolveMethod(*desc.MethodDescriptor)
	// OnSendHeaders is called with the request metadata that is being sent.
	OnSendHeaders(metadata.MD)
	// OnReceiveHeaders is called when response headers have been received.
	OnReceiveHeaders(metadata.MD)
	// OnReceiveResponse is called for each response message received.
	OnReceiveResponse(proto.Message)
	// OnReceiveTrailers is called when response trailers and final RPC status have been received.
	OnReceiveTrailers(*status.Status, metadata.MD)
}

InvocationEventHandler is a bag of callbacks for handling events that occur in the course of invoking an RPC. The handler also provides request data that is sent. The callbacks are generally called in the order they are listed below.

type RequestMessageSupplier deprecated 

type RequestMessageSupplier func() ([]byte, error)

RequestMessageSupplier is a function that is called to retrieve request messages for a GRPC operation. This type is deprecated and will be removed in a future release.

Deprecated: This is only used with the deprecated InvokeRpc. Instead, use RequestSupplier with InvokeRPC.

type RequestParser 

type RequestParser interface {
	// Next parses input data into the given request message. If called after
	// input is exhausted, it returns io.EOF. If the caller re-uses the same
	// instance in multiple calls to Next, it should call msg.Reset() in between
	// each call.
	Next(msg proto.Message) error
	// NumRequests returns the number of messages that have been parsed and
	// returned by a call to Next.
	NumRequests() int
}

RequestParser processes input into messages.

func NewJSONRequestParser 

func NewJSONRequestParser(in io.Reader, resolver jsonpb.AnyResolver) RequestParser

NewJSONRequestParser returns a RequestParser that reads data in JSON format from the given reader. The given resolver is used to assist with decoding of google.protobuf.Any messages.

Input data that contains more than one message should just include all messages concatenated (though whitespace is necessary to separate some kinds of values in JSON).

If the given reader has no data, the returned parser will return io.EOF on the very first call.

func NewTextRequestParser 

func NewTextRequestParser(in io.Reader) RequestParser

NewTextRequestParser returns a RequestParser that reads data in the protobuf text format from the given reader.

Input data that contains more than one message should include an ASCII 'Record Separator' character (0x1E) between each message.

Empty text is a valid text format and represents an empty message. So if the given reader has no data, the returned parser will yield an empty message for the first call to Next and then return io.EOF thereafter. This also means that if the input data ends with a record separator, then a final empty message will be parsed *after* the separator.

type RequestSupplier 

type RequestSupplier func(proto.Message) error

RequestSupplier is a function that is called to populate messages for a gRPC operation. The function should populate the given message or return a non-nil error. If the supplier has no more messages, it should return io.EOF. When it returns io.EOF, it should not in any way modify the given message argument.

Source Files

View all Source files

Directories

Path Synopsis
cmd
grpcurl
Command grpcurl makes gRPC requests (a la cURL, but HTTP/2).
 Command grpcurl makes gRPC requests (a la cURL, but HTTP/2).
cmd/bankdemo
cmd/testserver
Command testserver spins up a test GRPC server.
 Command testserver spins up a test GRPC server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.