Documentation ¶
Overview ¶
Package prpc (provisional RPC) implements an RPC client over HTTP 1.x.
Like gRPC:
- services are defined in .proto files
- service implementation does not depend on pRPC.
Unlike gRPC:
- supports HTTP 1.x and AppEngine
- does not support streams.
It is similar in purpose to some other gRPC-compatible HTTP protocols, but predates them or makes different design choices:
- https://github.com/grpc/grpc-web: pRPC avoids a proxy.
- https://connectrpc.com/: pRPC predates it by multiple years.
Compile service definitions ¶
Use cproto tool to compile .proto files to .go files with gRPC and pRPC support. It runs protoc and does some additional postprocessing.
//go:generate cproto
Install cproto:
go install go.chromium.org/luci/grpc/cmd/cproto
Protocol ¶
v1.5 ¶
v1.5 is small, backward-compatible amendment to v1.4 that allows clients to advertise the maximum response size they are willing to deserialize.
A client MAY send "X-Prpc-Max-Response-Size" request header with a positive decimal number of bytes specifying the maximum uncompressed response size the client is willing to deserialize. The server MAY use it to discard responses larger than that size. If the server discards a response, it replies with UNAVAILABLE gRPC status with luci.prpc.ErrorDetails message as google.rpc.Status detail (see v1.2 section below for how it is encoded). This error proto contains ResponseTooBig as the realized oneof "error" alternative.
The response size is calculated as a length of the serialized response message using the negotiated response encoding (i.e. if the client asked for a JSON response, the response size is the length of the JSON blob with the serialized response message). Length of metadata or response compression (if any) have no effect on this check.
v1.4 ¶
v1.4 hides some leaking HTTP v1 transport implementation details from gRPC services and clients by stopping exposing values of the following headers in metadata.MD: "Accept", "Accept-Encoding", "Content-Encoding", "Content-Length", "Content-Type", "X-Content-Type-Options", all "X-Prpc-*" headers.
Note that "X-Prpc-Grpc-Timeout", "X-Prpc-Status-Details-Bin", "Content-Type" and "Accept" headers were already hidden in the previous version of the protocol.
Also note that such commonly present headers as "Host" and "User-Agent" are still exposed as metadata, since they are already used in the wild and the protocol does not depend on them significantly.
v1.3 ¶
v1.3 adds request/response body compression support using GZIP (RFC-1952).
- A request MAY have a header "Content-Encoding: gzip". If it is present, then the server MUST decompress the request body before unmarshaling the request message.
- A request MAY have a header "Accept-Encoding: gzip". If the header is present, then the server response body MAY be compressed; and the server SHOULD decide based on the response message size. If the response body is compressed, then the response MUST include a header "Content-Encoding: gzip".
See also https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding and https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding
v1.2 ¶
v1.2 is small, backward-compatible amendment to v1.1 that adds support for error details.
Response header "X-Prpc-Status-Details-Bin" contains elements of google.rpc.Status.details field, one value per element, in the same order. The header value is a standard base64 string with padding of the encoded google.protobuf.Any, where the message encoding is the same as the response message encoding, i.e. depends on Accept request header.
v1.1 ¶
v1.1 is small, backward-compatible amendment to v1.0 to address a security issue. Since it is backward compatible, it does not introduce a formal protocol version header at this time.
Changes:
- Requests/responses must use "application/json" media type instead of "application/prpc; encoding=json".
- Responses must include "X-Content-Type-Options: nosniff" HTTP header.
This enables CORB protection (which mitigates spectre) and disables content sniffing. For CORB, see https://chromium.googlesource.com/chromium/src/+/master/services/network/cross_origin_read_blocking_explainer.md.
v1.0 ¶
This section describes the base pRPC protocol. It is based on HTTP 1.x and employs gRPC codes.
A pRPC server MUST handle HTTP POST requests at `/prpc/{service}/{method}` where service is a full service name including full package name. The handler must decode an input message from an HTTP request, call the service method implementation and encode the returned output message or error to the HTTP response.
pRPC protocol defines three protocol buffer encodings and media types.
- Binary: "application/prpc; encoding=binary" (default).
- JSON: "application/prpc; encoding=json" or "application/json" A response body MUST have `)]}'\n` prefix to prevent XSSI.
- Text: "application/prpc; encoding=text"
A pRPC server MUST support Binary and SHOULD support JSON and Text.
Request headers:
- "X-Prpc-Grpc-Timeout": specifies request timeout. A client MAY specify it. If a service hits the timeout, a server MUST respond with HTTP 503 and DeadlineExceed gRPC code. Value format: `\d+[HMSmun]` (regular expression), where
- H - hour
- M - minute
- S - second
- m - millisecond
- u - microsecond
- n - nanosecond
- "Content-Type": specifies input message encoding in the body. A client SHOULD specify it. If not present, a server MUST treat the input message as Binary.
- "Accept": specifies the output message encoding for the response. A client MAY specify it, a server MUST support it.
- Any other headers MUST be added to metadata.MD in the context that is passed to the service method implementation (note this behavior was amended in v1.4).
- If a header name has "-Bin" suffix, the server must treat it as standard-base64-encoded with padding and put the decoded binary blob into the metadata under the original key (i.e. the one ending with "-bin").
Response headers:
- "X-Prpc-Grpc-Code": specifies the gRPC code. A server MUST specify it in all cases.
- If it is present, the client MUST ignore the HTTP status code.
- If OK, the client MUST return the output message decoded from the body.
- Otherwise, the client MUST return a gRPC error with the code and message read from the response body.
- If not present, the client MUST return a non-gRPC error with message read from the response body. A client MAY read a portion of the response body.
- "Content-Type": specifies the output message encoding. A server SHOULD specify it. If not specified, a client MUST treat it is as Binary.
- Any metadata returned by a service method implementation MUST go into http headers, unless metadata key starts with "X-Prpc-".
A server MUST always specify "X-Prpc-Grpc-Code". The server SHOULD specify HTTP status corresponding to the gRPC code.
If the "X-Prpc-Grpc-Code" response header value is not 0, the response body MUST contain a description of the error.
If a service/method is not found, the server MUST respond with Unimplemented gRPC code and SHOULD specify HTTP 501 status.
Index ¶
- Constants
- Variables
- func DecodeTimeout(s string) (time.Duration, error)
- func EncodeTimeout(t time.Duration) string
- func ProtocolErrorDetails(err error) *prpcpb.ErrorDetails
- func SetHeader(ctx context.Context, md metadata.MD) error
- type AccessControlDecision
- type CallOption
- type Client
- func (c *Client) Call(ctx context.Context, serviceName, methodName string, in, out proto.Message, ...) error
- func (c *Client) CallWithFormats(ctx context.Context, serviceName, methodName string, in []byte, ...) ([]byte, error)
- func (c *Client) Invoke(ctx context.Context, method string, args any, reply any, ...) error
- func (c *Client) NewStream(ctx context.Context, desc *grpc.StreamDesc, method string, ...) (grpc.ClientStream, error)
- type Format
- type Options
- type Override
- type Registrar
- type ResponseCompression
- type Server
Constants ¶
const ( // HeaderGRPCCode is a name of the HTTP header that specifies the // gRPC code in the response. // A pRPC server must always specify it. HeaderGRPCCode = "X-Prpc-Grpc-Code" // HeaderStatusDetail is a name of the HTTP header that contains // elements of google.rpc.Status.details field, one value per element, // in the same order. // The header value is a standard-base64 string of the encoded google.protobuf.Any, // where the message encoding is the same as the response message encoding, // i.e. depends on Accept request header. HeaderStatusDetail = "X-Prpc-Status-Details-Bin" // HeaderTimeout is HTTP header used to set pRPC request timeout. // The single value should match regexp `\d+[HMSmun]`. HeaderTimeout = "X-Prpc-Grpc-Timeout" // HeaderMaxResponseSize is HTTP request header with the maximum response // size the client will accept. HeaderMaxResponseSize = "X-Prpc-Max-Response-Size" // DefaultMaxResponseSize is the default maximum response size (in bytes) // the client is willing to read from the server. // // It is 32MiB minus 32KiB. Its value is picked to fit into Appengine response // size limits (taking into account potential overhead on headers). DefaultMaxResponseSize = 32*1024*1024 - 32*1024 )
const ( // ContentTypePRPC is the formal MIME content type for pRPC messages. ContentTypePRPC = "application/prpc" // ContentTypeJSON is the JSON content type. ContentTypeJSON = "application/json" // JSONPBPrefix is prepended to a message in JSONPB format to avoid CSRF. JSONPBPrefix = ")]}'\n" )
const DefaultMaxRequestSize = 64 * 1024 * 1024
DefaultMaxRequestSize is the default maximum request size (in bytes) the server is willing to read from the client.
Variables ¶
var ( // DefaultUserAgent is default User-Agent HTTP header for pRPC requests. DefaultUserAgent = "pRPC Client 1.5" // ErrNoStreamingSupport is returned if a pRPC client is used to start a // streaming RPC. They are not supported. ErrNoStreamingSupport = status.Error(codes.Unimplemented, "prpc: no streaming support") )
Functions ¶
func DecodeTimeout ¶
DecodeTimeout decodes a gRPC/pRPC timeout header string into a time.Duration.
func EncodeTimeout ¶
EncodeTimeout encodes a gRPC/pRPC timeout into a timeout header time.Duration.
This rounds the time.Duration to the smallest time unit that can represent it.
func ProtocolErrorDetails ¶
func ProtocolErrorDetails(err error) *prpcpb.ErrorDetails
ProtocolErrorDetails extracts pRPC protocol error details from a gRPC error if they are there.
If err is not a gRPC error, or it is a gRPC but with no ErrorDetails attached, returns nil.
func SetHeader ¶
SetHeader sets the header metadata.
When called multiple times, all the provided metadata will be merged. Headers set this way will be sent whenever the RPC handler finishes (successfully or not).
Some headers are reserved for internal pRPC or HTTP needs and can't be set this way. This includes all "x-prpc-*" headers, various core HTTP headers (like "content-type") and all "access-control-*" headers (if you want to configure CORS policies, use [Server.AccessControl] field instead).
If ctx is not a pRPC server context, then SetHeader calls grpc.SetHeader such that calling prpc.SetHeader works for both pRPC and gRPC.
Types ¶
type AccessControlDecision ¶
type AccessControlDecision struct { // AllowCrossOriginRequests defines if CORS policies should be enabled at all. // // If false, the server will not send any CORS policy headers and the browser // will use the default "same-origin" policy. All fields below will be // ignored. // // See https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy AllowCrossOriginRequests bool // AllowCredentials defines if a cross-origin request is allowed to use // credentials associated with the pRPC server's domain. // // This setting controls if the client can use `credentials: include` option // in Fetch init parameters or `withCredentials = true` in XMLHttpRequest. // // Credentials, as defined by Web APIs, are cookies, HTTP authentication // entries, and TLS client certificates. // // See // https://developer.mozilla.org/en-US/docs/Web/API/fetch#parameters // https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/withCredentials // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials AllowCredentials bool // AllowHeaders is a list of request headers that a cross-origin request is // allowed to have. // // Extends the default list of allowed headers. // // Use this option if the authentication interceptor checks some custom // headers. AllowHeaders []string }
AccessControlDecision describes how to handle a cross-origin request.
It is returned by AccessControl server callback based on the origin of the call and defines what CORS policy headers to add to responses to preflight requests and actual cross-origin requests.
See https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS for the general overview of CORS policies.
func AllowOriginAll ¶
func AllowOriginAll(ctx context.Context, origin string) AccessControlDecision
AllowOriginAll is a CORS policy that allows any origin to send cross-origin requests that can use credentials (e.g. cookies) and "Authorization" header.
Other non-standard headers are forbidden. Use a custom AccessControl callback to allow them.
type CallOption ¶
type CallOption struct { grpc.CallOption // contains filtered or unexported fields }
CallOption mutates Options.
func CallAcceptContentSubtype ¶
func CallAcceptContentSubtype(contentSubtype string) *CallOption
CallAcceptContentSubtype returns a CallOption that sets Content-Type. For example, if content-subtype is "json", the Content-Type over the wire will be "application/json". Unlike that of the grpc.CallContentSubtype, sets Content-Type only for response, not for the request.
func ExpectedCode ¶
func ExpectedCode(codes ...codes.Code) *CallOption
ExpectedCode can be used to indicate that given non-OK codes may appear during normal successful call flow, and thus they must not be logged as erroneous.
Only affects local logging, nothing else.
type Client ¶
type Client struct { C *http.Client // if nil, uses http.DefaultClient Host string // host and optionally a port number of the target server Options *Options // if nil, DefaultOptions() are used // ErrBodySize is the number of bytes to truncate error messages from HTTP // responses to. // // If non-positive, defaults to 256. ErrBodySize int // MaxResponseSize, if > 0, is the maximum response size, in bytes, that a // pRPC client is willing to read from the server. If a larger response is // sent by the server, the client will return UNAVAILABLE error with some // additional details attached (use ProtocolErrorDetails to extract them // from a gRPC error). // // This value is also sent to the server in a request header. The server MAY // use it to skip sending too big response. Instead the server will reply // with the same sort of UNAVAILABLE error (with details attached as well). // // If <= 0, DefaultMaxResponseSize will be used. MaxResponseSize int // MaxConcurrentRequests, if > 0, limits how many requests to the server can // execute at the same time. If 0 (default), there's no limit. // // If there are more concurrent Call(...) calls than the limit, excessive ones // will block until there are execution "slots" available or the context is // canceled. This waiting does not count towards PerRPCTimeout. // // The primary purpose of this mechanism is to reduce strain on the local // network resources such as number of HTTP connections and HTTP2 streams. // Note that it will not help with OOM problems, since blocked calls (and // their bodies) all queue up in memory anyway. MaxConcurrentRequests int // EnableRequestCompression allows the client to compress requests if they // are larger than a certain threshold. // // This is false by default. Use this option only with servers that understand // compressed requests! These are Go servers built after Aug 15 2022. Python // servers and olders Go servers would fail to parse the request with // INVALID_ARGUMENT error. // // The response compression is configured independently on the server. The // client always accepts compressed responses. EnableRequestCompression bool // PathPrefix is the prefix of the URL path, "<PathPrefix>/<service>/<method>" // when making HTTP requests. If not set, defaults to "/prpc". PathPrefix string // contains filtered or unexported fields }
Client can make pRPC calls.
Changing fields after the first Call(...) is undefined behavior.
func (*Client) Call ¶
func (c *Client) Call(ctx context.Context, serviceName, methodName string, in, out proto.Message, opts ...grpc.CallOption) error
Call performs a remote procedure call.
Used by the generated code. Calling from multiple goroutines concurrently is safe.
`opts` must be created by this package. Options from google.golang.org/grpc package are not supported. Panics if they are used.
Propagates outgoing gRPC metadata provided via metadata.NewOutgoingContext. It will be available via metadata.FromIncomingContext on the other side. Similarly, if there is a deadline in the Context, it is be propagated to the server and applied to the context of the request handler there.
Retries on internal transient errors and on gRPC codes considered transient by grpcutil.IsTransientCode. Logs unexpected errors (see ExpectedCode call option).
Returns gRPC errors, perhaps with extra structured details if the server provided them. Context errors are converted into gRPC errors as well. See google.golang.org/grpc/status package.
func (*Client) CallWithFormats ¶
func (c *Client) CallWithFormats(ctx context.Context, serviceName, methodName string, in []byte, inf, outf Format, opts ...grpc.CallOption) ([]byte, error)
CallWithFormats is like Call, but sends and returns raw data without marshaling it.
Trims JSONPBPrefix from the response if necessary.
func (*Client) Invoke ¶
func (c *Client) Invoke(ctx context.Context, method string, args any, reply any, opts ...grpc.CallOption) error
Invoke performs a unary RPC and returns after the response is received into reply.
It is a part of grpc.ClientConnInterface.
func (*Client) NewStream ¶
func (c *Client) NewStream(ctx context.Context, desc *grpc.StreamDesc, method string, opts ...grpc.CallOption) (grpc.ClientStream, error)
NewStream begins a streaming RPC.
It is a part of grpc.ClientConnInterface.
type Format ¶
type Format int
Format is the pRPC protobuf wire format specification.
const ( // FormatBinary indicates that a message is encoded as a raw binary protobuf. FormatBinary Format = iota // FormatJSONPB indicates that a message is encoded as a JSON-serialized // protobuf. FormatJSONPB // FormatText indicates that a message is encoded as a text protobuf. FormatText )
(Ordered by preference).
func FormatFromContentType ¶
FormatFromContentType converts Content-Type header value from a request to a format. Can return only FormatBinary, FormatJSONPB or FormatText. In case of an error, format is undefined.
func FormatFromEncoding ¶
FormatFromEncoding converts a media type encoding parameter into a pRPC Format. Can return only FormatBinary, FormatJSONPB or FormatText. In case of an error, format is undefined.
func FormatFromMediaType ¶
FormatFromMediaType converts a media type ContentType and its parameters into a pRPC Format. Can return only FormatBinary, FormatJSONPB or FormatText. In case of an error, format is undefined.
type Options ¶
type Options struct { // Retry controls a strategy for retrying transient RPC errors and deadlines. // // Each call to this factory should return a new retry.Iterator to use for // controlling retry loop of a single RPC call. Retry retry.Factory // UserAgent is the value of User-Agent HTTP header. // // If empty, DefaultUserAgent is used. UserAgent string // Insecure can be set to true to use HTTP instead of HTTPS. // // Useful for local tests. Insecure bool // PerRPCTimeout, if > 0, is a timeout that is applied to each call attempt. // // If the context passed to the call has a shorter deadline, this timeout will // not be applied. Otherwise, if this timeout is hit, the RPC call attempt // will be considered as failed transiently and it may be retried just like // any other transient error per Retry policy. PerRPCTimeout time.Duration // AcceptContentSubtype defines acceptable Content-Type of responses. // // Valid values are "binary" and "json". Empty value defaults to "binary". // It can be overridden on per-call basis via CallAcceptContentSubtype(). AcceptContentSubtype string // Debug is a flag indicate if we want to print more logs for debug purpose. // // Right now we use this option to print the raw requests when certain // failures occur. Debug bool // contains filtered or unexported fields }
Options controls how RPC requests are sent.
func DefaultOptions ¶
func DefaultOptions() *Options
DefaultOptions are used if no options are specified in Client.
type Override ¶
type Override func(rw http.ResponseWriter, req *http.Request, body func(msg proto.Message) error) (stop bool, err error)
Override is a pRPC method-specific override which may optionally handle the entire pRPC method call. If it returns true, the override is assumed to have fully handled the pRPC method call and processing of the request does not continue. In this case it's the override's responsibility to adhere to all pRPC semantics. However if it returns false, processing continues as normal, allowing the override to act as a preprocessor. In this case it's the override's responsibility to ensure it hasn't done anything that will be incompatible with pRPC semantics (such as writing garbage to the response writer in the router context).
Override receives `body` callback as an argument that can, on demand, read and decode the request body using the decoder constructed based on the request headers. This callback can be used to "peek" inside the deserialized request to decide if the override should be activated. Note that `req.Body` can be read only once, and the callback reads it. To allow reusing `req` after the callback is done, it, as a side effect, replaces `req.Body` with a byte buffer containing the data it just read. The callback can be called at most once.
type Registrar ¶
type Registrar interface { // RegisterService registers a service and its implementation. // Called from the generated code. RegisterService(desc *grpc.ServiceDesc, impl any) }
Registrar can register a service. It is implemented by *grpc.Server and used instead of grpc.Server in the code generated by cproto.
type ResponseCompression ¶
type ResponseCompression string
ResponseCompression controls how the server compresses responses.
See Server doc for details.
const ( CompressDefault ResponseCompression = "" // same as "never" CompressNever ResponseCompression = "never" CompressAlways ResponseCompression = "always" CompressNotJSON ResponseCompression = "not JSON" )
Possible values for ResponseCompression.
type Server ¶
type Server struct { // AccessControl, if not nil, is a callback that is invoked per request to // determine what CORS access control headers, if any, should be added to // the response. // // It controls what cross-origin RPC calls are allowed from a browser and what // responses cross-origin clients may see. Does not effect calls made by // non-browser RPC clients. // // This callback is invoked for each CORS pre-flight request as well as for // actual RPC requests. It can make its decision based on the origin of // the request. // // If nil, the server will not send any CORS headers and the browser will // use the default "same-origin" policy. // // See https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS for the general // overview of CORS policies. AccessControl func(ctx context.Context, origin string) AccessControlDecision // HackFixFieldMasksForJSON indicates whether to attempt a workaround for // https://github.com/golang/protobuf/issues/745 when the request has // Content-Type: application/json. This hack is scheduled for removal. // TODO(crbug/1082369): Remove this workaround once field masks can be decoded. HackFixFieldMasksForJSON bool // UnaryServerInterceptor provides a hook to intercept the execution of // a unary RPC on the server. It is the responsibility of the interceptor to // invoke handler to complete the RPC. UnaryServerInterceptor grpc.UnaryServerInterceptor // ResponseCompression controls how the server compresses responses. // // It applies only to eligible responses. A response is eligible for // compression if the client sends "Accept-Encoding: gzip" request header // (default for all Go clients, including the standard pRPC client) and // the response size is larger than a certain threshold. // // If ResponseCompression is CompressNever or CompressDefault, responses are // never compressed. This is a conservative default (since the server // generally doesn't know the trade off between CPU and network in the // environment it runs in and assumes network is cheaper; in particular this // situation is realized when the server is running on the localhost network). // // If ResponseCompression is CompressAlways, eligible responses are always // compressed. // // If ResponseCompression is CompressNotJSON, only eligible responses that // do **not** use JSON encoding will be compressed. JSON responses are left // uncompressed at this layer. This is primarily added for the Appengine // environment: GAE runtime compresses JSON responses on its own already, // presumably using a more efficient compressor (C++ and all). Note that the // client will see all eligible responses compressed (JSON ones will be // compressed by the GAE, and non-JSON ones will be compressed by the pRPC // server). // // A compressed response is accompanied by "Content-Encoding: gzip" response // header, telling the client that it should decompress it. // // The request compression is configured independently on the client. The // server always accepts compressed requests. ResponseCompression ResponseCompression // MaxRequestSize is how many bytes the server will read from the request body // before rejecting it as too big. // // This is a protection against OOMing the server by big requests. // // When using request compression the limit is checked twice: when reading // the original compressed request, and when decompressing it. Checking the // decompressed request size is done to to prevent attacks like "zip bombs" // where very small payloads decompress to enormous buffer sizes. // // Note that some environments impose their own request size limits. In // particular, AppEngine and Cloud Run (but only when using HTTP/1), have // a 32MiB request size limit. // // If <= 0, DefaultMaxRequestSize will be used. MaxRequestSize int // contains filtered or unexported fields }
Server is a pRPC server to serve RPC requests. Zero value is valid.
func (*Server) InstallHandlers ¶
func (s *Server) InstallHandlers(r *router.Router, base router.MiddlewareChain)
InstallHandlers installs HTTP handlers at /prpc/:service/:method.
See https://godoc.org/go.chromium.org/luci/grpc/prpc#hdr-Protocol for pRPC protocol.
Assumes incoming requests are not authenticated yet. Authentication should be performed by an interceptor (just like when using a gRPC server).
func (*Server) RegisterOverride ¶
RegisterOverride registers an overriding function.
Panics if an override for the given service method is already registered.
func (*Server) RegisterService ¶
func (s *Server) RegisterService(desc *grpc.ServiceDesc, impl any)
RegisterService registers a service implementation. Called from the generated code.
desc must contain description of the service, its message types and all transitive dependencies.
Panics if a service of the same name is already registered.
func (*Server) ServiceNames ¶
ServiceNames returns a sorted list of full names of all registered services.