protovalidate

README

protovalidate

protovalidate is a Go library designed to validate Protobuf messages at runtime based on user-defined validation constraints. Powered by Google's Common Expression Language (CEL), it provides a flexible and efficient foundation for defining and evaluating custom validation rules. The primary goal of protovalidate is to help developers ensure data consistency and integrity across the network without requiring generated code.

Installation

Requires the go toolchain (≥ v1.18)

To install the package, use the go get command from within your Go module:

go get github.com/bufbuild/protovalidate/go/v2

Import the package into your Go project:

import "github.com/bufbuild/protovalidate/go/v2"

Remember to always check for the latest version of protovalidate on the project's GitHub releases page to ensure you're using the most up-to-date version.

Usage

For API-specific details, you can refer to the official pkg.go.dev documentation which provides in-depth information on the library's API, functions, types, and source files. This can be particularly useful for understanding the lower-level workings of protovalidate and how to leverage its full potential.

Example

package main

import (
  "fmt"
  pb "github.com/path/to/generated/protos"
  "github.com/bufbuild/protovalidate/go/v2"
)

func main() {
  msg := &pb.Person{
    Id:    1000, 
    Email: "example@bufbuild.com", 
    Name:  "Protobuf",
    Home: &example.Person_Location{
      Lat: 37.7, 
      Lng: -122.4,
    },
  }

  v, err := protovalidate.New()
  if err != nil {
    fmt.Println("failed to initialize validator:", err)
  }

  if err = v.Validate(msg); err != nil {
    fmt.Println("validation failed:", err)
  } else {
    fmt.Println("validation succeeded")
  }
}

Implementing validation constraints

Validation constraints are defined directly within .proto files. Documentation for adding constraints can be found in the root README and the comprehensive docs.

The protovalidate package assumes the constraint extensions are imported into the protoc-gen-go generated code via buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go.

Buf managed mode

If you are using Buf managed mode to augment Go code generation, ensure that the protovalidate module is excluded in your buf.gen.yaml:

version: v1
# <snip>
managed:
  enabled: true
  go_package_prefix:
    except:
      - buf.build/bufbuild/protovalidate
# <snip>

Lazy mode

protovalidate defaults to lazily construct validation logic for Protobuf message types the first time they are encountered. A validator's internal cache can be pre-warmed with the WithMessages or WithDescriptors options during initialization:

validator, err := protovalidate.New(
  protovalidate.WithMessages(
    &pb.MyFoo{}, 
    &pb.MyBar{}, 
  ),
)

Lazy mode requires usage of a mutex to keep the validator thread-safe, which results in about 50% of CPU time spent obtaining a read lock. While performance is sub-microsecond, the mutex overhead can be further reduced by disabling lazy mode with the WithDisableLazy option. Note that all expected messages must be provided during initialization of the validator:

validator, err := protovalidate.New(
  protovalidate.WithDisableLazy(true),
  protovalidate.WithMessages(
    &pb.MyFoo{},
    &pb.MyBar{},
  ),
)

Support legacy protoc-gen-validate constraints

The protovalidate module comes with a legacy package which adds opt-in support for existing protoc-gen-validate constraints. Provide thelegacy.WithLegacySupport option when initializing the validator:

validator, err := protovalidate.New(
  legacy.WithLegacySupport(legacy.ModeMerge),
)

protoc-gen-validate code generation is not used by protovalidate. The legacy package assumes the protoc-gen-validate extensions are imported into the protoc-gen-go generated code via github.com/envoyproxy/protoc-gen-validate/validate.

A migration tool is also available to incrementally upgrade legacy constraints in .proto files.

Performance

Benchmarks are provided to test a variety of use-cases. Generally, after the initial cold start, validation on a message is sub-microsecond and only allocates in the event of a validation error.

[circa 15 May 2023]
goos: darwin
goarch: arm64
pkg: github.com/bufbuild/protovalidate/go/v2
BenchmarkValidator
BenchmarkValidator/ColdStart
BenchmarkValidator/ColdStart-10         	    4372	    276457 ns/op	  470780 B/op	    9255 allocs/op
BenchmarkValidator/Lazy/Valid
BenchmarkValidator/Lazy/Valid-10        	 9022392	     134.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkValidator/Lazy/Invalid
BenchmarkValidator/Lazy/Invalid-10      	 3416996	     355.9 ns/op	     632 B/op	      14 allocs/op
BenchmarkValidator/Lazy/FailFast
BenchmarkValidator/Lazy/FailFast-10     	 6751131	     172.6 ns/op	     168 B/op	       3 allocs/op
BenchmarkValidator/PreWarmed/Valid
BenchmarkValidator/PreWarmed/Valid-10   	17557560	     69.10 ns/op	       0 B/op	       0 allocs/op
BenchmarkValidator/PreWarmed/Invalid
BenchmarkValidator/PreWarmed/Invalid-10 	 3621961	     332.9 ns/op	     632 B/op	      14 allocs/op
BenchmarkValidator/PreWarmed/FailFast
BenchmarkValidator/PreWarmed/FailFast-10	13960359	     92.22 ns/op	     168 B/op	       3 allocs/op
PASS

Documentation

Overview

Example

validator, err := New()
if err != nil {
	log.Fatal(err)
}

person := &pb.Person{
	Id:    1234,
	Email: "protovalidate@buf.build",
	Name:  "Buf Build",
	Home: &pb.Coordinates{
		Lat: 27.380583333333334,
		Lng: 33.631838888888886,
	},
}

err = validator.Validate(person)
fmt.Println("valid:", err)

person.Email = "not an email"
err = validator.Validate(person)
fmt.Println("invalid:", err)

Output:

valid: <nil>
invalid: validation error:
 - email: value must be a valid email address [string.email]

Index

• type CompilationError
• type RuntimeError
• type StandardConstraintInterceptor
• type StandardConstraintResolver
• type ValidationError
• type Validator
  • func New(options ...ValidatorOption) (*Validator, error)
  • func (v *Validator) Validate(msg proto.Message) error
• type ValidatorOption
  • func WithDescriptors(descriptors ...protoreflect.MessageDescriptor) ValidatorOption
  • func WithDisableLazy(disable bool) ValidatorOption
  • func WithFailFast(failFast bool) ValidatorOption
  • func WithMessages(messages ...proto.Message) ValidatorOption
  • func WithStandardConstraintInterceptor(interceptor StandardConstraintInterceptor) ValidatorOption
  • func WithUTC(useUTC bool) ValidatorOption

Examples

• Package
• ValidationError
• WithDescriptors
• WithDisableLazy
• WithFailFast
• WithMessages

Types

type CompilationError

type CompilationError = errors.CompilationError

A CompilationError is returned if a CEL expression cannot be compiled & type-checked or if invalid standard constraints are applied to a field.

type RuntimeError

type RuntimeError = errors.RuntimeError

A RuntimeError is returned if a valid CEL expression evaluation is terminated, typically due to an unknown or mismatched type.

type StandardConstraintInterceptor

type StandardConstraintInterceptor func(res StandardConstraintResolver) StandardConstraintResolver

StandardConstraintInterceptor can be provided to WithStandardConstraintInterceptor to allow modifying a StandardConstraintResolver.

type StandardConstraintResolver

type StandardConstraintResolver interface {
	ResolveMessageConstraints(desc protoreflect.MessageDescriptor) *validate.MessageConstraints
	ResolveOneofConstraints(desc protoreflect.OneofDescriptor) *validate.OneofConstraints
	ResolveFieldConstraints(desc protoreflect.FieldDescriptor) *validate.FieldConstraints
}

StandardConstraintResolver is responsible for resolving the standard constraints from the provided protoreflect.Descriptor. The default resolver can be intercepted and modified using WithStandardConstraintInterceptor.

type ValidationError

type ValidationError = errors.ValidationError

A ValidationError is returned if one or more constraints on a message are violated. This error type can be converted into a validate.Violations message via ToProto.

err = validator.Validate(msg)
var valErr *ValidationError
if ok := errors.As(err, &valErr); ok {
  pb := valErr.ToProto()
  // ...
}

Example

validator, err := New()
if err != nil {
	log.Fatal(err)
}

loc := &pb.Coordinates{Lat: 999.999}
err = validator.Validate(loc)
var valErr *ValidationError
if ok := errors.As(err, &valErr); ok {
	msg := valErr.ToProto()
	fmt.Println(msg.Violations[0].FieldPath, msg.Violations[0].ConstraintId)
}

Output:

lat double.gte_lte

type Validator

type Validator struct {
	// contains filtered or unexported fields
}

Validator performs validation on any proto.Message values. The Validator is safe for concurrent use.

func New

func New(options ...ValidatorOption) (*Validator, error)

New creates a Validator with the given options. An error may occur in setting up the CEL execution environment if the configuration is invalid. See the individual ValidatorOption for how they impact the fallibility of New.

func (*Validator) Validate

func (v *Validator) Validate(msg proto.Message) error

Validate checks that message satisfies its constraints. Constraints are defined within the Protobuf file as options from the buf.validate package. An error is returned if the constraints are violated (ValidationError), the evaluation logic for the message cannot be built (CompilationError), or there is a type error when attempting to evaluate a CEL expression associated with the message (RuntimeError).

type ValidatorOption

type ValidatorOption func(*config)

A ValidatorOption modifies the default configuration of a Validator. See the individual options for their defaults and affects on the fallibility of configuring a Validator.

func WithDescriptors

func WithDescriptors(descriptors ...protoreflect.MessageDescriptor) ValidatorOption

WithDescriptors allows warming up the Validator with message descriptors that are expected to be validated. Messages included transitively (i.e., fields with message values) are automatically handled.

Example

pbType, err := protoregistry.GlobalTypes.FindMessageByName("tests.example.v1.Person")
if err != nil {
	log.Fatal(err)
}

validator, err := New(
	WithDescriptors(
		pbType.Descriptor(),
	),
)
if err != nil {
	log.Fatal(err)
}

person := &pb.Person{
	Id:    1234,
	Email: "protovalidate@buf.build",
	Name:  "Protocol Buffer",
}
err = validator.Validate(person)
fmt.Println(err)

Output:

<nil>

func WithDisableLazy

func WithDisableLazy(disable bool) ValidatorOption

WithDisableLazy prevents the Validator from lazily building validation logic for a message it has not encountered before. Disabling lazy logic additionally eliminates any internal locking as the validator becomes read-only.

Note: All expected messages must be provided by WithMessages or WithDescriptors during initialization.

Example

person := &pb.Person{
	Id:    1234,
	Email: "protovalidate@buf.build",
	Name:  "Buf Build",
	Home: &pb.Coordinates{
		Lat: 27.380583333333334,
		Lng: 33.631838888888886,
	},
}

validator, err := New(
	WithMessages(&pb.Coordinates{}),
	WithDisableLazy(true),
)
if err != nil {
	log.Fatal(err)
}

err = validator.Validate(person.Home)
fmt.Println("person.Home:", err)
err = validator.Validate(person)
fmt.Println("person:", err)

Output:

person.Home: <nil>
person: compilation error: no evaluator available for tests.example.v1.Person

func WithFailFast

func WithFailFast(failFast bool) ValidatorOption

WithFailFast specifies whether validation should fail on the first constraint violation encountered or if all violations should be accumulated. By default, all violations are accumulated.

Example

loc := &pb.Coordinates{Lat: 999.999, Lng: -999.999}

validator, err := New()
if err != nil {
	log.Fatal(err)
}
err = validator.Validate(loc)
fmt.Println("default:", err)

validator, err = New(WithFailFast(true))
if err != nil {
	log.Fatal(err)
}
err = validator.Validate(loc)
fmt.Println("fail fast:", err)

Output:

default: validation error:
 - lat: value must be greater than or equal to -90 and less than or equal to 90 [double.gte_lte]
 - lng: value must be greater than or equal to -180 and less than or equal to 180 [double.gte_lte]
fail fast: validation error:
 - lat: value must be greater than or equal to -90 and less than or equal to 90 [double.gte_lte]

func WithMessages

func WithMessages(messages ...proto.Message) ValidatorOption

WithMessages allows warming up the Validator with messages that are expected to be validated. Messages included transitively (i.e., fields with message values) are automatically handled.

Example

validator, err := New(
	WithMessages(&pb.Person{}),
)
if err != nil {
	log.Fatal(err)
}

person := &pb.Person{
	Id:    1234,
	Email: "protovalidate@buf.build",
	Name:  "Protocol Buffer",
}
err = validator.Validate(person)
fmt.Println(err)

Output:

<nil>

func WithStandardConstraintInterceptor

func WithStandardConstraintInterceptor(interceptor StandardConstraintInterceptor) ValidatorOption

WithStandardConstraintInterceptor allows intercepting the StandardConstraintResolver used by the Validator to modify or replace it.

func WithUTC

func WithUTC(useUTC bool) ValidatorOption

WithUTC specifies whether timestamp operations should use UTC or the OS's local timezone for timestamp related values. By default, the local timezone is used.