protocol

package
v2.0.207+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 20, 2019 License: MIT Imports: 33 Imported by: 0

Documentation

Overview

Package protocol defines the core Gazette datamodel, validation behaviors, and gRPC APIs which are shared across clients and broker servers. Datamodel types and APIs are implemented as generated protobuf messages and stubs, typically extended with additional parsing, validation, and shared implementation behaviors. A central goal of this package to be highly exacting in defining allowed "shapes" that types & messages may take (through implementations of the Validator interface), providing strong guarantees to brokers and clients that messages are well-formed without need for additional ad-hoc, repetitive checks (which often become a maintenance burden).

Index

Constants

View Source 
const DispatcherGRPCBalancerName = "protocolDispatcher"

DispatcherGRPCBalancerName is the client-side dispatcher's registered gRPC balancer. To utilize client-side dispatching, the service endpoint should be dialed with grpc.WithBalancerName(protocol.DispatcherGRPCBalancerName).

Variables

View Source 
var (
	ErrInvalidLengthProtocol = fmt.Errorf("proto: negative length found during unmarshaling")
	ErrIntOverflowProtocol   = fmt.Errorf("proto: integer overflow")
)
View Source 
var CompressionCodec_name = map[int32]string{
	0: "INVALID",
	1: "NONE",
	2: "GZIP",
	3: "ZSTANDARD",
	4: "SNAPPY",
	5: "GZIP_OFFLOAD_DECOMPRESSION",
}
View Source 
var CompressionCodec_value = map[string]int32{
	"INVALID":                    0,
	"NONE":                       1,
	"GZIP":                       2,
	"ZSTANDARD":                  3,
	"SNAPPY":                     4,
	"GZIP_OFFLOAD_DECOMPRESSION": 5,
}
View Source 
var JournalSpec_Flag_name = map[int32]string{
	0: "NOT_SPECIFIED",
	1: "O_RDONLY",
	2: "O_WRONLY",
	4: "O_RDWR",
}
View Source 
var JournalSpec_Flag_value = map[string]int32{
	"NOT_SPECIFIED": 0,
	"O_RDONLY":      1,
	"O_WRONLY":      2,
	"O_RDWR":        4,
}
View Source 
var Status_name = map[int32]string{
	0:  "OK",
	1:  "JOURNAL_NOT_FOUND",
	2:  "NO_JOURNAL_PRIMARY_BROKER",
	3:  "NOT_JOURNAL_PRIMARY_BROKER",
	5:  "NOT_JOURNAL_BROKER",
	4:  "INSUFFICIENT_JOURNAL_BROKERS",
	6:  "OFFSET_NOT_YET_AVAILABLE",
	7:  "WRONG_ROUTE",
	8:  "FRAGMENT_MISMATCH",
	9:  "ETCD_TRANSACTION_FAILED",
	10: "NOT_ALLOWED",
	11: "WRONG_APPEND_OFFSET",
	12: "INDEX_HAS_GREATER_OFFSET",
}
View Source 
var Status_value = map[string]int32{
	"OK":                           0,
	"JOURNAL_NOT_FOUND":            1,
	"NO_JOURNAL_PRIMARY_BROKER":    2,
	"NOT_JOURNAL_PRIMARY_BROKER":   3,
	"NOT_JOURNAL_BROKER":           5,
	"INSUFFICIENT_JOURNAL_BROKERS": 4,
	"OFFSET_NOT_YET_AVAILABLE":     6,
	"WRONG_ROUTE":                  7,
	"FRAGMENT_MISMATCH":            8,
	"ETCD_TRANSACTION_FAILED":      9,
	"NOT_ALLOWED":                  10,
	"WRONG_APPEND_OFFSET":          11,
	"INDEX_HAS_GREATER_OFFSET":     12,
}

Functions

func ExtendContext 

func ExtendContext(err error, format string, args ...interface{}) error

ExtendContext type-checks |err| to a *ValidationError, and if matched extends it with |context|. In all cases the value of |err| is returned.

func NewValidationError 

func NewValidationError(format string, args ...interface{}) error

NewValidationError parallels fmt.Errorf to returns a new ValidationError instance.

func RegisterGRPCDispatcher 

func RegisterGRPCDispatcher(localZone string)

RegisterGRPCDispatcher registers the dispatcher balancer with gRPC. It should be called once at program startup. The supplied |localZone| is used to prefer intra-zone (over inter-zone) members where able.

func RegisterJournalServer 

func RegisterJournalServer(s *grpc.Server, srv JournalServer)

func ValidateSingleValueLabels 

func ValidateSingleValueLabels(m LabelSet) error

ValidateSingleValueLabels compares the LabelSet to labels.SingleValueLabels, and returns an error if any labels have multiple values.

func ValidateToken 

func ValidateToken(n string, min, max int) error

ValidateToken ensures the string consists only of |tokenAlphabet| characters, and is of length |min| <= len(n) <= |max|.

func WithDispatchDefault 

func WithDispatchDefault(ctx context.Context) context.Context

WithDispatchDefault attaches a Route and ProcessSpec_ID which indicate that the RPC should be dispatched to the default service address.

func WithDispatchItemRoute 

func WithDispatchItemRoute(ctx context.Context, dr DispatchRouter, item string, requirePrimary bool) context.Context

WithDispatchItemRoute uses the DispatchRouter to resolve |item| to a Route and ProcessSpec_ID, which are in-turn attached to the Context and returned for dispatcher's use.

func WithDispatchRoute 

func WithDispatchRoute(ctx context.Context, rt Route, id ProcessSpec_ID) context.Context

WithDispatchRoute attaches a Route and optional ProcessSpec_ID to a Context passed to a gRPC RPC call. If ProcessSpec_ID is non-zero valued, the RPC is dispatched to the specified member. Otherwise, the RPC is dispatched to a Route member, preferring:

  • A member not having a currently-broken network connection (eg, due to a stale Route or network split).
  • A member which is in the same zone as the caller (potentially reducing network traffic costs.
  • A member having a Ready connection (potentially reducing latency).

Types

type AppendRequest 

type AppendRequest struct {
	// Header is attached by a proxying broker peer to the first AppendRequest message.
	Header *Header `protobuf:"bytes,1,opt,name=header" json:"header,omitempty"`
	// Journal to be appended to.
	Journal Journal `protobuf:"bytes,2,opt,name=journal,proto3,casttype=Journal" json:"journal,omitempty"`
	// If do_not_proxy is true, the broker will not proxy the append if it is
	// not the current primary.
	DoNotProxy bool `protobuf:"varint,3,opt,name=do_not_proxy,json=doNotProxy,proto3" json:"do_not_proxy,omitempty"`
	// Journal offset at which the append should begin. Most clients should leave
	// at zero, which uses the broker's tracked offset. The append offset must be
	// one greater than furthest written offset of the journal, or
	// WRONG_APPEND_OFFSET is returned.
	Offset int64 `protobuf:"varint,5,opt,name=offset,proto3" json:"offset,omitempty"`
	// Content chunks to be appended. Immediately prior to closing the stream,
	// the client must send an empty chunk (eg, zero-valued AppendRequest) to
	// indicate the Append should be committed. Absence of this empty chunk
	// prior to EOF is interpreted by the broker as a rollback of the Append.
	Content []byte `protobuf:"bytes,4,opt,name=content,proto3" json:"content,omitempty"`
}

func (*AppendRequest) Descriptor 

func (*AppendRequest) Descriptor() ([]byte, []int)

func (*AppendRequest) Marshal 

func (m *AppendRequest) Marshal() (dAtA []byte, err error)

func (*AppendRequest) MarshalTo 

func (m *AppendRequest) MarshalTo(dAtA []byte) (int, error)

func (*AppendRequest) ProtoMessage 

func (*AppendRequest) ProtoMessage()

func (*AppendRequest) ProtoSize 

func (m *AppendRequest) ProtoSize() (n int)

func (*AppendRequest) Reset 

func (m *AppendRequest) Reset()

func (*AppendRequest) String 

func (m *AppendRequest) String() string

func (*AppendRequest) Unmarshal 

func (m *AppendRequest) Unmarshal(dAtA []byte) error

func (*AppendRequest) Validate 

func (m *AppendRequest) Validate() error

Validate returns an error if the AppendRequest is not well-formed.

func (*AppendRequest) XXX_DiscardUnknown 

func (m *AppendRequest) XXX_DiscardUnknown()

func (*AppendRequest) XXX_Marshal 

func (m *AppendRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*AppendRequest) XXX_Merge 

func (dst *AppendRequest) XXX_Merge(src proto.Message)

func (*AppendRequest) XXX_Size 

func (m *AppendRequest) XXX_Size() int

func (*AppendRequest) XXX_Unmarshal 

func (m *AppendRequest) XXX_Unmarshal(b []byte) error

type AppendResponse 

type AppendResponse struct {
	// Status of the Append RPC.
	Status Status `protobuf:"varint,1,opt,name=status,proto3,enum=protocol.Status" json:"status,omitempty"`
	// Header of the response.
	Header Header `protobuf:"bytes,2,opt,name=header" json:"header"`
	// If status is OK, then |commit| is the Fragment which places the
	// committed Append content within the Journal.
	Commit *Fragment `protobuf:"bytes,3,opt,name=commit" json:"commit,omitempty"`
}

func (*AppendResponse) Descriptor 

func (*AppendResponse) Descriptor() ([]byte, []int)

func (*AppendResponse) Marshal 

func (m *AppendResponse) Marshal() (dAtA []byte, err error)

func (*AppendResponse) MarshalTo 

func (m *AppendResponse) MarshalTo(dAtA []byte) (int, error)

func (*AppendResponse) ProtoMessage 

func (*AppendResponse) ProtoMessage()

func (*AppendResponse) ProtoSize 

func (m *AppendResponse) ProtoSize() (n int)

func (*AppendResponse) Reset 

func (m *AppendResponse) Reset()

func (*AppendResponse) String 

func (m *AppendResponse) String() string

func (*AppendResponse) Unmarshal 

func (m *AppendResponse) Unmarshal(dAtA []byte) error

func (*AppendResponse) Validate 

func (m *AppendResponse) Validate() error

Validate returns an error if the AppendResponse is not well-formed.

func (*AppendResponse) XXX_DiscardUnknown 

func (m *AppendResponse) XXX_DiscardUnknown()

func (*AppendResponse) XXX_Marshal 

func (m *AppendResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*AppendResponse) XXX_Merge 

func (dst *AppendResponse) XXX_Merge(src proto.Message)

func (*AppendResponse) XXX_Size 

func (m *AppendResponse) XXX_Size() int

func (*AppendResponse) XXX_Unmarshal 

func (m *AppendResponse) XXX_Unmarshal(b []byte) error

type ApplyRequest 

type ApplyRequest struct {
	Changes []ApplyRequest_Change `protobuf:"bytes,1,rep,name=changes" json:"changes"`
}

func (*ApplyRequest) Descriptor 

func (*ApplyRequest) Descriptor() ([]byte, []int)

func (*ApplyRequest) Marshal 

func (m *ApplyRequest) Marshal() (dAtA []byte, err error)

func (*ApplyRequest) MarshalTo 

func (m *ApplyRequest) MarshalTo(dAtA []byte) (int, error)

func (*ApplyRequest) ProtoMessage 

func (*ApplyRequest) ProtoMessage()

func (*ApplyRequest) ProtoSize 

func (m *ApplyRequest) ProtoSize() (n int)

func (*ApplyRequest) Reset 

func (m *ApplyRequest) Reset()

func (*ApplyRequest) String 

func (m *ApplyRequest) String() string

func (*ApplyRequest) Unmarshal 

func (m *ApplyRequest) Unmarshal(dAtA []byte) error

func (*ApplyRequest) Validate 

func (m *ApplyRequest) Validate() error

func (*ApplyRequest) XXX_DiscardUnknown 

func (m *ApplyRequest) XXX_DiscardUnknown()

func (*ApplyRequest) XXX_Marshal 

func (m *ApplyRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ApplyRequest) XXX_Merge 

func (dst *ApplyRequest) XXX_Merge(src proto.Message)

func (*ApplyRequest) XXX_Size 

func (m *ApplyRequest) XXX_Size() int

func (*ApplyRequest) XXX_Unmarshal 

func (m *ApplyRequest) XXX_Unmarshal(b []byte) error

type ApplyRequest_Change 

type ApplyRequest_Change struct {
	// Expected ModRevision of the current JournalSpec. If the Journal is being
	// created, expect_mod_revision is zero.
	ExpectModRevision int64 `protobuf:"varint,1,opt,name=expect_mod_revision,json=expectModRevision,proto3" json:"expect_mod_revision,omitempty"`
	// JournalSpec to be updated (if expect_mod_revision > 0) or created
	// (if expect_mod_revision == 0).
	Upsert *JournalSpec `protobuf:"bytes,2,opt,name=upsert" json:"upsert,omitempty"`
	// Journal to be deleted. expect_mod_revision must not be zero.
	Delete Journal `protobuf:"bytes,3,opt,name=delete,proto3,casttype=Journal" json:"delete,omitempty"`
}

Change defines an insertion, update, or deletion to be applied to the set of JournalSpecs. Exactly one of |upsert| or |delete| must be set.

func (*ApplyRequest_Change) Descriptor 

func (*ApplyRequest_Change) Descriptor() ([]byte, []int)

func (*ApplyRequest_Change) Marshal 

func (m *ApplyRequest_Change) Marshal() (dAtA []byte, err error)

func (*ApplyRequest_Change) MarshalTo 

func (m *ApplyRequest_Change) MarshalTo(dAtA []byte) (int, error)

func (*ApplyRequest_Change) ProtoMessage 

func (*ApplyRequest_Change) ProtoMessage()

func (*ApplyRequest_Change) ProtoSize 

func (m *ApplyRequest_Change) ProtoSize() (n int)

func (*ApplyRequest_Change) Reset 

func (m *ApplyRequest_Change) Reset()

func (*ApplyRequest_Change) String 

func (m *ApplyRequest_Change) String() string

func (*ApplyRequest_Change) Unmarshal 

func (m *ApplyRequest_Change) Unmarshal(dAtA []byte) error

func (*ApplyRequest_Change) Validate 

func (m *ApplyRequest_Change) Validate() error

func (*ApplyRequest_Change) XXX_DiscardUnknown 

func (m *ApplyRequest_Change) XXX_DiscardUnknown()

func (*ApplyRequest_Change) XXX_Marshal 

func (m *ApplyRequest_Change) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ApplyRequest_Change) XXX_Merge 

func (dst *ApplyRequest_Change) XXX_Merge(src proto.Message)

func (*ApplyRequest_Change) XXX_Size 

func (m *ApplyRequest_Change) XXX_Size() int

func (*ApplyRequest_Change) XXX_Unmarshal 

func (m *ApplyRequest_Change) XXX_Unmarshal(b []byte) error

type ApplyResponse 

type ApplyResponse struct {
	// Status of the Apply RPC.
	Status Status `protobuf:"varint,1,opt,name=status,proto3,enum=protocol.Status" json:"status,omitempty"`
	// Header of the response.
	Header Header `protobuf:"bytes,2,opt,name=header" json:"header"`
}

func (*ApplyResponse) Descriptor 

func (*ApplyResponse) Descriptor() ([]byte, []int)

func (*ApplyResponse) Marshal 

func (m *ApplyResponse) Marshal() (dAtA []byte, err error)

func (*ApplyResponse) MarshalTo 

func (m *ApplyResponse) MarshalTo(dAtA []byte) (int, error)

func (*ApplyResponse) ProtoMessage 

func (*ApplyResponse) ProtoMessage()

func (*ApplyResponse) ProtoSize 

func (m *ApplyResponse) ProtoSize() (n int)

func (*ApplyResponse) Reset 

func (m *ApplyResponse) Reset()

func (*ApplyResponse) String 

func (m *ApplyResponse) String() string

func (*ApplyResponse) Unmarshal 

func (m *ApplyResponse) Unmarshal(dAtA []byte) error

func (*ApplyResponse) Validate 

func (m *ApplyResponse) Validate() error

func (*ApplyResponse) XXX_DiscardUnknown 

func (m *ApplyResponse) XXX_DiscardUnknown()

func (*ApplyResponse) XXX_Marshal 

func (m *ApplyResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ApplyResponse) XXX_Merge 

func (dst *ApplyResponse) XXX_Merge(src proto.Message)

func (*ApplyResponse) XXX_Size 

func (m *ApplyResponse) XXX_Size() int

func (*ApplyResponse) XXX_Unmarshal 

func (m *ApplyResponse) XXX_Unmarshal(b []byte) error

type BrokerSpec 

type BrokerSpec struct {
	// ProcessSpec of the broker.
	ProcessSpec `protobuf:"bytes,1,opt,name=process_spec,json=processSpec,embedded=process_spec" json:"process_spec" yaml:",inline"`
	// Maximum number of assigned Journal replicas.
	JournalLimit uint32 `protobuf:"varint,2,opt,name=journal_limit,json=journalLimit,proto3" json:"journal_limit,omitempty"`
}

BrokerSpec describes a Gazette broker and its configuration.

func (*BrokerSpec) Descriptor 

func (*BrokerSpec) Descriptor() ([]byte, []int)

func (*BrokerSpec) ItemLimit 

func (m *BrokerSpec) ItemLimit() int

v3_allocator.MemberValue implementation.

func (*BrokerSpec) Marshal 

func (m *BrokerSpec) Marshal() (dAtA []byte, err error)

func (*BrokerSpec) MarshalString 

func (m *BrokerSpec) MarshalString() string

MarshalString returns the marshaled encoding of the JournalSpec as a string.

func (*BrokerSpec) MarshalTo 

func (m *BrokerSpec) MarshalTo(dAtA []byte) (int, error)

func (*BrokerSpec) ProtoMessage 

func (*BrokerSpec) ProtoMessage()

func (*BrokerSpec) ProtoSize 

func (m *BrokerSpec) ProtoSize() (n int)

func (*BrokerSpec) Reset 

func (m *BrokerSpec) Reset()

func (*BrokerSpec) String 

func (m *BrokerSpec) String() string

func (*BrokerSpec) Unmarshal 

func (m *BrokerSpec) Unmarshal(dAtA []byte) error

func (*BrokerSpec) Validate 

func (m *BrokerSpec) Validate() error

Validate returns an error if the BrokerSpec is not well-formed.

func (*BrokerSpec) XXX_DiscardUnknown 

func (m *BrokerSpec) XXX_DiscardUnknown()

func (*BrokerSpec) XXX_Marshal 

func (m *BrokerSpec) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*BrokerSpec) XXX_Merge 

func (dst *BrokerSpec) XXX_Merge(src proto.Message)

func (*BrokerSpec) XXX_Size 

func (m *BrokerSpec) XXX_Size() int

func (*BrokerSpec) XXX_Unmarshal 

func (m *BrokerSpec) XXX_Unmarshal(b []byte) error

func (*BrokerSpec) ZeroLimit 

func (m *BrokerSpec) ZeroLimit()

ZeroLimit zeros the BrokerSpec JournalLimit.

type CompressionCodec 

type CompressionCodec int32

CompressionCode defines codecs known to Gazette. 

const (
	// INVALID is the zero-valued CompressionCodec, and is not a valid codec.
	CompressionCodec_INVALID CompressionCodec = 0
	// NONE encodes Fragments without any applied compression, with default suffix ".raw".
	CompressionCodec_NONE CompressionCodec = 1
	// GZIP encodes Fragments using the Gzip library, with default suffix ".gz".
	CompressionCodec_GZIP CompressionCodec = 2
	// ZSTANDARD encodes Fragments using the ZStandard library, with default suffix ".zst".
	CompressionCodec_ZSTANDARD CompressionCodec = 3
	// SNAPPY encodes Fragments using the Snappy library, with default suffix ".sz".
	CompressionCodec_SNAPPY CompressionCodec = 4
	// GZIP_OFFLOAD_DECOMPRESSION is the GZIP codec with additional behavior
	// around reads and writes to remote Fragment stores, designed to offload
	// the work of decompression onto compatible stores. Specifically:
	//  * Fragments are written with a "Content-Encoding: gzip" header.
	//  * Client read requests are made with "Accept-Encoding: identity".
	// This can be helpful in contexts where reader IO bandwidth to the storage
	// API is unconstrained, as the cost of decompression is offloaded to the
	// store and CPU-intensive batch readers may receive a parallelism benefit.
	// While this codec may provide substantial read-time performance improvements,
	// it is an advanced configuration and the "Content-Encoding" header handling
	// can be subtle and sometimes confusing. It uses the default suffix ".gzod".
	CompressionCodec_GZIP_OFFLOAD_DECOMPRESSION CompressionCodec = 5
)

func CompressionCodecFromExtension 

func CompressionCodecFromExtension(ext string) (CompressionCodec, error)

CompressionCodecFromExtension matches a file extension to its corresponding CompressionCodec.

func (CompressionCodec) EnumDescriptor 

func (CompressionCodec) EnumDescriptor() ([]byte, []int)

func (CompressionCodec) MarshalYAML 

func (m CompressionCodec) MarshalYAML() (interface{}, error)

func (CompressionCodec) String 

func (x CompressionCodec) String() string

func (CompressionCodec) ToExtension 

func (m CompressionCodec) ToExtension() string

ToExtension returns the file extension of the CompressionCodec.

func (*CompressionCodec) UnmarshalYAML 

func (m *CompressionCodec) UnmarshalYAML(unmarshal func(interface{}) error) error

func (CompressionCodec) Validate 

func (m CompressionCodec) Validate() error

Validate returns an error if the CompressionCodec is not well-formed.

type DispatchRouter 

type DispatchRouter interface {
	// Route an |item| to a Route, which may be empty if the Route is unknown.
	Route(ctx context.Context, item string) Route
	// UpdateRoute for |item|. A nil |route| is treated as an invalidation.
	UpdateRoute(item string, route *Route)
	// IsNoopRouter returns true if Route is a no-op.
	IsNoopRouter() bool
}

DispatchRouter routes item to Routes, and observes item Routes.

type Endpoint 

type Endpoint string

Endpoint defines an accessible service address. It is a URL, where the scheme defines the network transport and semantics of the host, path, and query components. At present, supported schemes are:

func (Endpoint) URL 

func (ep Endpoint) URL() *url.URL

URL returns the Endpoint as a URL. The Endpoint must Validate, or URL panics.

func (Endpoint) Validate 

func (ep Endpoint) Validate() error

Validate returns an error if the Endpoint is not well-formed.

type Fragment 

type Fragment struct {
	// Journal of the Fragment.
	Journal Journal `protobuf:"bytes,1,opt,name=journal,proto3,casttype=Journal" json:"journal,omitempty"`
	// Begin (inclusive) and end (exclusive) offset of the Fragment within the Journal.
	Begin int64 `protobuf:"varint,2,opt,name=begin,proto3" json:"begin,omitempty"`
	End   int64 `protobuf:"varint,3,opt,name=end,proto3" json:"end,omitempty"`
	// SHA1 sum of the Fragment's content.
	Sum SHA1Sum `protobuf:"bytes,4,opt,name=sum" json:"sum"`
	// Codec with which the Fragment's content is compressed.
	CompressionCodec CompressionCodec `` /* 141-byte string literal not displayed */
	// Fragment store which backs the Fragment. Empty if the Fragment has yet to
	// be persisted and is still local to a Broker.
	BackingStore FragmentStore `protobuf:"bytes,6,opt,name=backing_store,json=backingStore,proto3,casttype=FragmentStore" json:"backing_store,omitempty"`
	// Modification timestamp of the Fragment within the backing store, represented as seconds
	// since the epoch.
	ModTime int64 `protobuf:"varint,7,opt,name=mod_time,json=modTime,proto3" json:"mod_time,omitempty"`
}

Fragment is a content-addressed description of a contiguous Journal span, defined by the [begin, end) offset range covered by the Fragment and the SHA1 sum of the corresponding Journal content.

func ParseContentName 

func ParseContentName(journal Journal, name string) (Fragment, error)

ParseContentName parses a Journal and ContentName into a Fragment, or returns an error.

func ParseContentPath 

func ParseContentPath(p string) (Fragment, error)

ParseContentPath parses a ContentPath into a Fragment, or returns an error.

func (*Fragment) ContentLength 

func (m *Fragment) ContentLength() int64

ContentLength returns the number of content bytes contained in this Fragment. If compression is used, this will differ from the file size of the Fragment.

func (*Fragment) ContentName 

func (m *Fragment) ContentName() string

ContentName returns the content-addressed base file name of this Fragment.

func (*Fragment) ContentPath 

func (m *Fragment) ContentPath() string

ContentPath returns the content-addressed path of this Fragment.

func (*Fragment) Descriptor 

func (*Fragment) Descriptor() ([]byte, []int)

func (*Fragment) Marshal 

func (m *Fragment) Marshal() (dAtA []byte, err error)

func (*Fragment) MarshalTo 

func (m *Fragment) MarshalTo(dAtA []byte) (int, error)

func (*Fragment) ProtoMessage 

func (*Fragment) ProtoMessage()

func (*Fragment) ProtoSize 

func (m *Fragment) ProtoSize() (n int)

func (*Fragment) Reset 

func (m *Fragment) Reset()

func (*Fragment) String 

func (m *Fragment) String() string

func (*Fragment) Unmarshal 

func (m *Fragment) Unmarshal(dAtA []byte) error

func (*Fragment) Validate 

func (m *Fragment) Validate() error

Validate returns an error if the Fragment is not well-formed.

func (*Fragment) XXX_DiscardUnknown 

func (m *Fragment) XXX_DiscardUnknown()

func (*Fragment) XXX_Marshal 

func (m *Fragment) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Fragment) XXX_Merge 

func (dst *Fragment) XXX_Merge(src proto.Message)

func (*Fragment) XXX_Size 

func (m *Fragment) XXX_Size() int

func (*Fragment) XXX_Unmarshal 

func (m *Fragment) XXX_Unmarshal(b []byte) error

type FragmentStore 

type FragmentStore string

FragmentStore defines a storage backend base path for Journal Fragments. It is a URL, where the scheme defines the storage backend service. As FragmentStores "root" remote storage locations of fragments, their path component must end in a trailing slash.

FragmentStore implementations may support additional configuration which can be declared via URL query arguments. The meaning of these query arguments and values are specific to the store in question; consult the store implementation to see properties available for configuration.

Currently supported schemes are `gs` for Google Cloud Storage, `s3` for Amazon S3, and `file` for a local file-system / NFS mount. Eg:

  • s3://bucket-name/a/sub-path/?profile=a-shared-credentials-profile
  • gs://bucket-name/a/sub-path/?
  • file:///a/local/volume/mount

func (FragmentStore) URL 

func (fs FragmentStore) URL() *url.URL

URL returns the FragmentStore as a URL. The FragmentStore must Validate, or URL panics.

func (FragmentStore) Validate 

func (fs FragmentStore) Validate() error

Validate returns an error if the FragmentStore is not well-formed.

type FragmentsRequest 

type FragmentsRequest struct {
	// Header is attached by a proxying broker peer.
	Header *Header `protobuf:"bytes,1,opt,name=header" json:"header,omitempty"`
	// Journal to be read.
	Journal Journal `protobuf:"bytes,2,opt,name=journal,proto3,casttype=Journal" json:"journal,omitempty"`
	// BeginModTime is an optional field specifying an inclusive lower bound on
	// the modification timestamp for a fragment to be returned. The timestamp is
	// represented as seconds since the epoch.
	BeginModTime int64 `protobuf:"varint,3,opt,name=begin_mod_time,json=beginModTime,proto3" json:"begin_mod_time,omitempty"`
	// EndModTime is an optional field specifying an exclusive upper bound on
	// the modification timestamp for a fragment to be returned. The timestamp is
	// represented as seconds since the epoch.
	EndModTime int64 `protobuf:"varint,4,opt,name=end_mod_time,json=endModTime,proto3" json:"end_mod_time,omitempty"`
	// The NextPageToke value returned from a previous, continued FragmentsRequest, if any.
	NextPageToken int64 `protobuf:"varint,5,opt,name=next_page_token,json=nextPageToken,proto3" json:"next_page_token,omitempty"`
	// PageLimit is an optional fieled specifying how many fragments to be returned
	// with the response. The default value for PageLimit is 1000.
	PageLimit int32 `protobuf:"varint,6,opt,name=page_limit,json=pageLimit,proto3" json:"page_limit,omitempty"`
	// SignatureTTL requests that a temporary signed GET URL be returned with each
	// response Fragment, valid for |signatureTTL|.
	SignatureTTL *time.Duration `protobuf:"bytes,7,opt,name=signatureTTL,stdduration" json:"signatureTTL,omitempty"`
	// If do_not_proxy is true, the broker will not proxy the request to another broker on the client's behalf.
	DoNotProxy bool `protobuf:"varint,8,opt,name=do_not_proxy,json=doNotProxy,proto3" json:"do_not_proxy,omitempty"`
}

func (*FragmentsRequest) Descriptor 

func (*FragmentsRequest) Descriptor() ([]byte, []int)

func (*FragmentsRequest) Marshal 

func (m *FragmentsRequest) Marshal() (dAtA []byte, err error)

func (*FragmentsRequest) MarshalTo 

func (m *FragmentsRequest) MarshalTo(dAtA []byte) (int, error)

func (*FragmentsRequest) ProtoMessage 

func (*FragmentsRequest) ProtoMessage()

func (*FragmentsRequest) ProtoSize 

func (m *FragmentsRequest) ProtoSize() (n int)

func (*FragmentsRequest) Reset 

func (m *FragmentsRequest) Reset()

func (*FragmentsRequest) String 

func (m *FragmentsRequest) String() string

func (*FragmentsRequest) Unmarshal 

func (m *FragmentsRequest) Unmarshal(dAtA []byte) error

func (*FragmentsRequest) Validate 

func (m *FragmentsRequest) Validate() error

func (*FragmentsRequest) XXX_DiscardUnknown 

func (m *FragmentsRequest) XXX_DiscardUnknown()

func (*FragmentsRequest) XXX_Marshal 

func (m *FragmentsRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*FragmentsRequest) XXX_Merge 

func (dst *FragmentsRequest) XXX_Merge(src proto.Message)

func (*FragmentsRequest) XXX_Size 

func (m *FragmentsRequest) XXX_Size() int

func (*FragmentsRequest) XXX_Unmarshal 

func (m *FragmentsRequest) XXX_Unmarshal(b []byte) error

type FragmentsResponse 

type FragmentsResponse struct {
	// Status of the Apply RPC.
	Status Status `protobuf:"varint,1,opt,name=status,proto3,enum=protocol.Status" json:"status,omitempty"`
	// Header of the response.
	Header    Header                        `protobuf:"bytes,2,opt,name=header" json:"header"`
	Fragments []FragmentsResponse__Fragment `protobuf:"bytes,3,rep,name=fragments" json:"fragments"`
	// The NextPageToke value to be returned on subsequent Fragments requests. If the value is
	// zero then there are no more fragments to be returned for this page.
	NextPageToken int64 `protobuf:"varint,4,opt,name=next_page_token,json=nextPageToken,proto3" json:"next_page_token,omitempty"`
}

func (*FragmentsResponse) Descriptor 

func (*FragmentsResponse) Descriptor() ([]byte, []int)

func (*FragmentsResponse) Marshal 

func (m *FragmentsResponse) Marshal() (dAtA []byte, err error)

func (*FragmentsResponse) MarshalTo 

func (m *FragmentsResponse) MarshalTo(dAtA []byte) (int, error)

func (*FragmentsResponse) ProtoMessage 

func (*FragmentsResponse) ProtoMessage()

func (*FragmentsResponse) ProtoSize 

func (m *FragmentsResponse) ProtoSize() (n int)

func (*FragmentsResponse) Reset 

func (m *FragmentsResponse) Reset()

func (*FragmentsResponse) String 

func (m *FragmentsResponse) String() string

func (*FragmentsResponse) Unmarshal 

func (m *FragmentsResponse) Unmarshal(dAtA []byte) error

func (*FragmentsResponse) Validate 

func (m *FragmentsResponse) Validate() error

func (*FragmentsResponse) XXX_DiscardUnknown 

func (m *FragmentsResponse) XXX_DiscardUnknown()

func (*FragmentsResponse) XXX_Marshal 

func (m *FragmentsResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*FragmentsResponse) XXX_Merge 

func (dst *FragmentsResponse) XXX_Merge(src proto.Message)

func (*FragmentsResponse) XXX_Size 

func (m *FragmentsResponse) XXX_Size() int

func (*FragmentsResponse) XXX_Unmarshal 

func (m *FragmentsResponse) XXX_Unmarshal(b []byte) error

type FragmentsResponse__Fragment 

type FragmentsResponse__Fragment struct {
	Spec Fragment `protobuf:"bytes,1,opt,name=spec" json:"spec"`
	// SignedURL is a temporary URL at which a direct GET of the Fragment may
	// be issued, signed by the broker's credentials. Set only if the request
	// specified a SignatureTTL.
	SignedUrl string `protobuf:"bytes,2,opt,name=signed_url,json=signedUrl,proto3" json:"signed_url,omitempty"`
}

Fragments of the Response.

func (*FragmentsResponse__Fragment) Descriptor 

func (*FragmentsResponse__Fragment) Descriptor() ([]byte, []int)

func (*FragmentsResponse__Fragment) Marshal 

func (m *FragmentsResponse__Fragment) Marshal() (dAtA []byte, err error)

func (*FragmentsResponse__Fragment) MarshalTo 

func (m *FragmentsResponse__Fragment) MarshalTo(dAtA []byte) (int, error)

func (*FragmentsResponse__Fragment) ProtoMessage 

func (*FragmentsResponse__Fragment) ProtoMessage()

func (*FragmentsResponse__Fragment) ProtoSize 

func (m *FragmentsResponse__Fragment) ProtoSize() (n int)

func (*FragmentsResponse__Fragment) Reset 

func (m *FragmentsResponse__Fragment) Reset()

func (*FragmentsResponse__Fragment) String 

func (m *FragmentsResponse__Fragment) String() string

func (*FragmentsResponse__Fragment) Unmarshal 

func (m *FragmentsResponse__Fragment) Unmarshal(dAtA []byte) error

func (*FragmentsResponse__Fragment) Validate 

func (m *FragmentsResponse__Fragment) Validate() error

func (*FragmentsResponse__Fragment) XXX_DiscardUnknown 

func (m *FragmentsResponse__Fragment) XXX_DiscardUnknown()

func (*FragmentsResponse__Fragment) XXX_Marshal 

func (m *FragmentsResponse__Fragment) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*FragmentsResponse__Fragment) XXX_Merge 

func (dst *FragmentsResponse__Fragment) XXX_Merge(src proto.Message)

func (*FragmentsResponse__Fragment) XXX_Size 

func (m *FragmentsResponse__Fragment) XXX_Size() int

func (*FragmentsResponse__Fragment) XXX_Unmarshal 

func (m *FragmentsResponse__Fragment) XXX_Unmarshal(b []byte) error
type Header struct {
	// ID of the process responsible for request processing. May be empty iff
	// Header is being used within a proxied request, and that request may be
	// dispatched to any member of the Route.
	ProcessId ProcessSpec_ID `protobuf:"bytes,1,opt,name=process_id,json=processId" json:"process_id"`
	// Route of processes specifically responsible for this RPC, or an empty Route
	// if any process is capable of serving the RPC.
	Route Route       `protobuf:"bytes,2,opt,name=route" json:"route"`
	Etcd  Header_Etcd `protobuf:"bytes,3,opt,name=etcd" json:"etcd"`
}

Header captures metadata such as the process responsible for processing an RPC, and its effective Etcd state.

func NewUnroutedHeader 

func NewUnroutedHeader(s *allocator.State) (hdr Header)

NewUnroutedHeader returns a Header with its ProcessId and Etcd fields derived from the v3_allocator.State, and Route left as zero-valued. It is a helper for APIs which do not utilize item resolution but still return Headers (eg, List and Update).

func (*Header) Descriptor 

func (*Header) Descriptor() ([]byte, []int)

func (*Header) Marshal 

func (m *Header) Marshal() (dAtA []byte, err error)

func (*Header) MarshalTo 

func (m *Header) MarshalTo(dAtA []byte) (int, error)

func (*Header) ProtoMessage 

func (*Header) ProtoMessage()

func (*Header) ProtoSize 

func (m *Header) ProtoSize() (n int)

func (*Header) Reset 

func (m *Header) Reset()

func (*Header) String 

func (m *Header) String() string

func (*Header) Unmarshal 

func (m *Header) Unmarshal(dAtA []byte) error

func (Header) Validate 

func (m Header) Validate() error

Validate returns an error if the Header is not well-formed.

func (*Header) XXX_DiscardUnknown 

func (m *Header) XXX_DiscardUnknown()

func (*Header) XXX_Marshal 

func (m *Header) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Header) XXX_Merge 

func (dst *Header) XXX_Merge(src proto.Message)

func (*Header) XXX_Size 

func (m *Header) XXX_Size() int

func (*Header) XXX_Unmarshal 

func (m *Header) XXX_Unmarshal(b []byte) error

type Header_Etcd 

type Header_Etcd struct {
	// cluster_id is the ID of the cluster.
	ClusterId uint64 `protobuf:"varint,1,opt,name=cluster_id,json=clusterId,proto3" json:"cluster_id,omitempty"`
	// member_id is the ID of the member.
	MemberId uint64 `protobuf:"varint,2,opt,name=member_id,json=memberId,proto3" json:"member_id,omitempty"`
	// revision is the Etcd key-value store revision when the request was applied.
	Revision int64 `protobuf:"varint,3,opt,name=revision,proto3" json:"revision,omitempty"`
	// raft_term is the raft term when the request was applied.
	RaftTerm uint64 `protobuf:"varint,4,opt,name=raft_term,json=raftTerm,proto3" json:"raft_term,omitempty"`
}

Etcd represents the effective Etcd MVCC state under which a Gazette broker is operating in its processing of requests and responses. Its inclusion allows brokers to reason about relative "happened before" Revision ordering of apparent routing conflicts in proxied or replicated requests, as well as enabling sanity checks over equality of Etcd ClusterId (and precluding, for example, split-brain scenarios where different brokers are backed by different Etcd clusters). Etcd is kept in sync with etcdserverpb.ResponseHeader.

func FromEtcdResponseHeader 

func FromEtcdResponseHeader(h epb.ResponseHeader) Header_Etcd

FromEtcdResponseHeader converts an etcd ResponseHeader to an equivalent Header_Etcd.

func (*Header_Etcd) Descriptor 

func (*Header_Etcd) Descriptor() ([]byte, []int)

func (*Header_Etcd) Marshal 

func (m *Header_Etcd) Marshal() (dAtA []byte, err error)

func (*Header_Etcd) MarshalTo 

func (m *Header_Etcd) MarshalTo(dAtA []byte) (int, error)

func (*Header_Etcd) ProtoMessage 

func (*Header_Etcd) ProtoMessage()

func (*Header_Etcd) ProtoSize 

func (m *Header_Etcd) ProtoSize() (n int)

func (*Header_Etcd) Reset 

func (m *Header_Etcd) Reset()

func (*Header_Etcd) String 

func (m *Header_Etcd) String() string

func (*Header_Etcd) Unmarshal 

func (m *Header_Etcd) Unmarshal(dAtA []byte) error

func (Header_Etcd) Validate 

func (m Header_Etcd) Validate() error

Validate returns an error if the Header_Etcd is not well-formed.

func (*Header_Etcd) XXX_DiscardUnknown 

func (m *Header_Etcd) XXX_DiscardUnknown()

func (*Header_Etcd) XXX_Marshal 

func (m *Header_Etcd) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Header_Etcd) XXX_Merge 

func (dst *Header_Etcd) XXX_Merge(src proto.Message)

func (*Header_Etcd) XXX_Size 

func (m *Header_Etcd) XXX_Size() int

func (*Header_Etcd) XXX_Unmarshal 

func (m *Header_Etcd) XXX_Unmarshal(b []byte) error

type Journal 

type Journal string

Journal uniquely identifies a journal brokered by Gazette. By convention, journals are named using a forward-slash notation which captures their hierarchical relationships into organizations, topics and partitions. For example, a Journal might be: "company-journals/interesting-topic/part-1234"

func (Journal) String 

func (n Journal) String() string

String returns the Journal as a string.

func (Journal) Validate 

func (n Journal) Validate() error

Validate returns an error if the Journal is not well-formed. It must be of the base64 alphabet, a clean path (as defined by path.Clean), and must not begin with a '/'.

type JournalClient 

type JournalClient interface {
	// List Journals, their JournalSpecs and current Routes.
	List(ctx context.Context, in *ListRequest, opts ...grpc.CallOption) (*ListResponse, error)
	// Apply changes to the collection of Journals managed by the brokers.
	Apply(ctx context.Context, in *ApplyRequest, opts ...grpc.CallOption) (*ApplyResponse, error)
	// Read from a specific Journal.
	Read(ctx context.Context, in *ReadRequest, opts ...grpc.CallOption) (Journal_ReadClient, error)
	// Append content to a specific Journal.
	Append(ctx context.Context, opts ...grpc.CallOption) (Journal_AppendClient, error)
	// Replicate appended content of a Journal. Replicate is used between broker
	// peers in the course of processing Append transactions, but is not intended
	// for direct use by clients.
	Replicate(ctx context.Context, opts ...grpc.CallOption) (Journal_ReplicateClient, error)
	// List Fragments of a Journal.
	ListFragments(ctx context.Context, in *FragmentsRequest, opts ...grpc.CallOption) (*FragmentsResponse, error)
}

JournalClient is the client API for Journal service.

For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.

func NewJournalClient 

func NewJournalClient(cc *grpc.ClientConn) JournalClient

type JournalServer 

type JournalServer interface {
	// List Journals, their JournalSpecs and current Routes.
	List(context.Context, *ListRequest) (*ListResponse, error)
	// Apply changes to the collection of Journals managed by the brokers.
	Apply(context.Context, *ApplyRequest) (*ApplyResponse, error)
	// Read from a specific Journal.
	Read(*ReadRequest, Journal_ReadServer) error
	// Append content to a specific Journal.
	Append(Journal_AppendServer) error
	// Replicate appended content of a Journal. Replicate is used between broker
	// peers in the course of processing Append transactions, but is not intended
	// for direct use by clients.
	Replicate(Journal_ReplicateServer) error
	// List Fragments of a Journal.
	ListFragments(context.Context, *FragmentsRequest) (*FragmentsResponse, error)
}

JournalServer is the server API for Journal service.

type JournalSpec 

type JournalSpec struct {
	// Name of the Journal.
	Name Journal `protobuf:"bytes,1,opt,name=name,proto3,casttype=Journal" json:"name,omitempty" yaml:",omitempty"`
	// Desired replication of this Journal. This defines the Journal's tolerance
	// to broker failures before data loss can occur (eg, a replication factor
	// of three means two failures are tolerated).
	Replication int32 `protobuf:"varint,2,opt,name=replication,proto3" json:"replication,omitempty" yaml:",omitempty"`
	// User-defined Labels of this JournalSpec. Two label names are reserved
	// and may not be used within a JournalSpec's Labels: "name" and "prefix".
	LabelSet `protobuf:"bytes,3,opt,name=labels,embedded=labels" json:"labels" yaml:",omitempty,inline"`
	Fragment JournalSpec_Fragment `protobuf:"bytes,4,opt,name=fragment" json:"fragment" yaml:",omitempty"`
	// Flags of the Journal, as a combination of Flag enum values. The Flag enum
	// not used directly, as protobuf enums do not allow for or'ed bitfields.
	Flags JournalSpec_Flag `protobuf:"varint,6,opt,name=flags,proto3,casttype=JournalSpec_Flag" json:"flags,omitempty" yaml:",omitempty"`
}

JournalSpec describes a Journal and its configuration.

func IntersectJournalSpecs 

func IntersectJournalSpecs(a, b JournalSpec) JournalSpec

IntersectJournalSpecs returns a JournalSpec having a non-zero-valued field for each field value which is shared between |a| and |b|.

func SubtractJournalSpecs 

func SubtractJournalSpecs(a, b JournalSpec) JournalSpec

SubtractJournalSpecs returns a JournalSpec derived from |a| but having a zero-valued field for each field which is matched by |b|.

func UnionJournalSpecs 

func UnionJournalSpecs(a, b JournalSpec) JournalSpec

UnionJournalSpecs returns a JournalSpec combining all non-zero-valued fields across |a| and |b|. Where both |a| and |b| provide a non-zero value for a field, the value of |a| is retained.

func (*JournalSpec) Descriptor 

func (*JournalSpec) Descriptor() ([]byte, []int)

func (*JournalSpec) DesiredReplication 

func (m *JournalSpec) DesiredReplication() int

DesiredReplication returns the configured Replication of the spec. It implements allocator.ItemValue.

func (*JournalSpec) IsConsistent 

func (m *JournalSpec) IsConsistent(_ keyspace.KeyValue, assignments keyspace.KeyValues) bool

IsConsistent returns true if the Route stored under each of |assignments| agrees with the Route implied by the |assignments| keys. It implements allocator.ItemValue.

func (*JournalSpec) Marshal 

func (m *JournalSpec) Marshal() (dAtA []byte, err error)

func (*JournalSpec) MarshalString 

func (m *JournalSpec) MarshalString() string

MarshalString returns the marshaled encoding of the JournalSpec as a string.

func (*JournalSpec) MarshalTo 

func (m *JournalSpec) MarshalTo(dAtA []byte) (int, error)

func (*JournalSpec) ProtoMessage 

func (*JournalSpec) ProtoMessage()

func (*JournalSpec) ProtoSize 

func (m *JournalSpec) ProtoSize() (n int)

func (*JournalSpec) Reset 

func (m *JournalSpec) Reset()

func (*JournalSpec) String 

func (m *JournalSpec) String() string

func (*JournalSpec) Unmarshal 

func (m *JournalSpec) Unmarshal(dAtA []byte) error

func (*JournalSpec) Validate 

func (m *JournalSpec) Validate() error

Validate returns an error if the JournalSpec is not well-formed.

func (*JournalSpec) XXX_DiscardUnknown 

func (m *JournalSpec) XXX_DiscardUnknown()

func (*JournalSpec) XXX_Marshal 

func (m *JournalSpec) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*JournalSpec) XXX_Merge 

func (dst *JournalSpec) XXX_Merge(src proto.Message)

func (*JournalSpec) XXX_Size 

func (m *JournalSpec) XXX_Size() int

func (*JournalSpec) XXX_Unmarshal 

func (m *JournalSpec) XXX_Unmarshal(b []byte) error

type JournalSpec_Flag 

type JournalSpec_Flag int32

Flags define Journal IO control behaviors. Where possible, flags are named after an equivalent POSIX flag. 

const (
	// NOT_SPECIFIED is considered as equivalent to O_RDWR by the broker. When
	// JournalSpecs are union'ed (eg, by the `journalspace` pkg), NOT_SPECIFIED
	// is considered as unset relative to any other non-zero Flag value.
	JournalSpec_NOT_SPECIFIED JournalSpec_Flag = 0
	// The Journal is available for reads (only).
	JournalSpec_O_RDONLY JournalSpec_Flag = 1
	// The Journal is available for writes (only).
	JournalSpec_O_WRONLY JournalSpec_Flag = 2
	// The Journal may be used for reads or writes.
	JournalSpec_O_RDWR JournalSpec_Flag = 4
)

func (JournalSpec_Flag) EnumDescriptor 

func (JournalSpec_Flag) EnumDescriptor() ([]byte, []int)

func (JournalSpec_Flag) MarshalYAML 

func (x JournalSpec_Flag) MarshalYAML() (interface{}, error)

MarshalYAML maps the JournalSpec_Flag to a YAML value.

func (JournalSpec_Flag) MayRead 

func (x JournalSpec_Flag) MayRead() bool

MayRead returns whether reads are permitted.

func (JournalSpec_Flag) MayWrite 

func (x JournalSpec_Flag) MayWrite() bool

MayWrite returns whether writes are permitted.

func (JournalSpec_Flag) String 

func (x JournalSpec_Flag) String() string

func (*JournalSpec_Flag) UnmarshalYAML 

func (x *JournalSpec_Flag) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML maps a YAML integer directly to a Flag value, or a YAML string to a Flag with corresponding enum name.

func (JournalSpec_Flag) Validate 

func (x JournalSpec_Flag) Validate() error

Validate returns an error if the JournalSpec_Flag is malformed.

type JournalSpec_Fragment 

type JournalSpec_Fragment struct {
	// Target content length of each Fragment. In normal operation after Fragments
	// reach at least this length, they will be closed and new ones begun. Note
	// lengths may be smaller at times (eg, due to changes in Journal routing
	// topology). Content length differs from Fragment file size, in that the
	// former reflects uncompressed bytes.
	Length int64 `protobuf:"varint,1,opt,name=length,proto3" json:"length,omitempty" yaml:",omitempty"`
	// Codec used to compress Journal Fragments.
	CompressionCodec CompressionCodec `` /* 176-byte string literal not displayed */
	// Storage backend base path for this Journal's Fragments. Must be in URL
	// form, with the choice of backend defined by the scheme. The full path of
	// a Journal's Fragment is derived by joining the fragment_store path with
	// the Fragment's ContentPath. Eg, given a fragment_store of
	// "s3://My-AWS-bucket/a/prefix" and a JournalSpec of name "my/journal",
	// a complete Fragment path might be:
	//   "s3://My-AWS-bucket/a/prefix/my/journal/000123-000456-789abcdef.gzip
	//
	// Multiple fragment_stores may be specified, in which case the Journal's
	// Fragments are the union of all Fragments present across all stores, and
	// new Fragments always persist to the first specified store. This can be
	// helpful in performing incremental migrations, where new Journal content is
	// written to the new store, while content in the old store remains available
	// (and, depending on fragment_retention or recovery log pruning, may
	// eventually be removed).
	//
	// If no fragment_stores are specified, the Journal is still useable but will
	// not persist Fragments to any a backing fragment store. This allows for
	// real-time streaming use cases where reads of historical data are not needed.
	Stores []FragmentStore `protobuf:"bytes,3,rep,name=stores,casttype=FragmentStore" json:"stores,omitempty" yaml:",omitempty"`
	// Interval of time between refreshes of remote Fragment listings from
	// configured fragment_stores.
	RefreshInterval time.Duration `` /* 135-byte string literal not displayed */
	// Retention duration for historical Fragments of this Journal within the
	// Fragment stores. If less than or equal to zero, Fragments are retained
	// indefinetely.
	Retention time.Duration `protobuf:"bytes,5,opt,name=retention,stdduration" json:"retention" yaml:",omitempty"`
	// Flush interval defines a UTC time segment, since epoch time,
	// after which a spool must be flushed to the FragmentStore.
	FlushInterval time.Duration `` /* 127-byte string literal not displayed */
}

Fragment is JournalSpec configuration which pertains to the creation, persistence, and indexing of the Journal's Fragments.

func (*JournalSpec_Fragment) Descriptor 

func (*JournalSpec_Fragment) Descriptor() ([]byte, []int)

func (*JournalSpec_Fragment) Marshal 

func (m *JournalSpec_Fragment) Marshal() (dAtA []byte, err error)

func (*JournalSpec_Fragment) MarshalTo 

func (m *JournalSpec_Fragment) MarshalTo(dAtA []byte) (int, error)

func (*JournalSpec_Fragment) ProtoMessage 

func (*JournalSpec_Fragment) ProtoMessage()

func (*JournalSpec_Fragment) ProtoSize 

func (m *JournalSpec_Fragment) ProtoSize() (n int)

func (*JournalSpec_Fragment) Reset 

func (m *JournalSpec_Fragment) Reset()

func (*JournalSpec_Fragment) String 

func (m *JournalSpec_Fragment) String() string

func (*JournalSpec_Fragment) Unmarshal 

func (m *JournalSpec_Fragment) Unmarshal(dAtA []byte) error

func (*JournalSpec_Fragment) Validate 

func (m *JournalSpec_Fragment) Validate() error

Validate returns an error if the JournalSpec_Fragment is not well-formed.

func (*JournalSpec_Fragment) XXX_DiscardUnknown 

func (m *JournalSpec_Fragment) XXX_DiscardUnknown()

func (*JournalSpec_Fragment) XXX_Marshal 

func (m *JournalSpec_Fragment) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*JournalSpec_Fragment) XXX_Merge 

func (dst *JournalSpec_Fragment) XXX_Merge(src proto.Message)

func (*JournalSpec_Fragment) XXX_Size 

func (m *JournalSpec_Fragment) XXX_Size() int

func (*JournalSpec_Fragment) XXX_Unmarshal 

func (m *JournalSpec_Fragment) XXX_Unmarshal(b []byte) error

type Journal_AppendClient 

type Journal_AppendClient interface {
	Send(*AppendRequest) error
	CloseAndRecv() (*AppendResponse, error)
	grpc.ClientStream
}

type Journal_AppendServer 

type Journal_AppendServer interface {
	SendAndClose(*AppendResponse) error
	Recv() (*AppendRequest, error)
	grpc.ServerStream
}

type Journal_ReadClient 

type Journal_ReadClient interface {
	Recv() (*ReadResponse, error)
	grpc.ClientStream
}

type Journal_ReadServer 

type Journal_ReadServer interface {
	Send(*ReadResponse) error
	grpc.ServerStream
}

type Journal_ReplicateClient 

type Journal_ReplicateClient interface {
	Send(*ReplicateRequest) error
	Recv() (*ReplicateResponse, error)
	grpc.ClientStream
}

type Journal_ReplicateServer 

type Journal_ReplicateServer interface {
	Send(*ReplicateResponse) error
	Recv() (*ReplicateRequest, error)
	grpc.ServerStream
}

type Label 

type Label struct {
	Name  string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
	Value string `protobuf:"bytes,2,opt,name=value,proto3" json:"value,omitempty"`
}

Label defines a key & value pair which can be attached to entities like JournalSpecs and BrokerSpecs. Labels may be used to provide identifying attributes which do not directly imply semantics to the core system, but are meaningful to users or for higher-level Gazette tools.

func (*Label) Descriptor 

func (*Label) Descriptor() ([]byte, []int)

func (*Label) Marshal 

func (m *Label) Marshal() (dAtA []byte, err error)

func (*Label) MarshalTo 

func (m *Label) MarshalTo(dAtA []byte) (int, error)

func (*Label) ProtoMessage 

func (*Label) ProtoMessage()

func (*Label) ProtoSize 

func (m *Label) ProtoSize() (n int)

func (*Label) Reset 

func (m *Label) Reset()

func (*Label) String 

func (m *Label) String() string

func (*Label) Unmarshal 

func (m *Label) Unmarshal(dAtA []byte) error

func (Label) Validate 

func (m Label) Validate() error

Validate returns an error if the Label is not well-formed.

func (*Label) XXX_DiscardUnknown 

func (m *Label) XXX_DiscardUnknown()

func (*Label) XXX_Marshal 

func (m *Label) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Label) XXX_Merge 

func (dst *Label) XXX_Merge(src proto.Message)

func (*Label) XXX_Size 

func (m *Label) XXX_Size() int

func (*Label) XXX_Unmarshal 

func (m *Label) XXX_Unmarshal(b []byte) error

type LabelSelector 

type LabelSelector struct {
	// Include is Labels which must be matched for a LabelSet to be selected. If
	// empty, all Labels are included. An include Label with empty ("") value is
	// matched by a Label of the same name having any value.
	Include LabelSet `protobuf:"bytes,1,opt,name=include" json:"include"`
	// Exclude is Labels which cannot be matched for a LabelSet to be selected. If
	// empty, no Labels are excluded. An exclude Label with empty ("") value
	// excludes a Label of the same name having any value.
	Exclude LabelSet `protobuf:"bytes,2,opt,name=exclude" json:"exclude"`
}

LabelSelector defines a filter over LabelSets.

func ParseLabelSelector 

func ParseLabelSelector(s string) (LabelSelector, error)

ParseLabelSelector parses a LabelSelector string. Selector strings are composed of a comma-separate list of selector expressions. Allowed expression types are equality, in-equality, set membership, set exclusion, existence, and non-existence. Eg:

  • "foo = bar" requires that label "foo" be present with value "bar"
  • "foo != bar" requires that label "foo" not be present with value "bar"
  • "foo" requires that label "foo" be present (with any value).
  • "!foo" requires that label "foo" not be present.
  • "foo in (bar,baz)" requires that "foo" be present with either "bar" or "baz".
  • "foo notin (bar,baz)" requires that "foo", if present, not have value "bar" or "baz".

Additional examples of composite expressions:

  • "topic in (topic/one, topic/two), prefix=/my/journal/prefix"
  • "env in (production, qa), tier not in (frontend,backend), partition"

ParseLabelSelector is invariant to _reasonable_ spacing: eg, "not in" and "notin" may be used interchangeably, as may "==" and "=", with or without single spacing on either side.

func (*LabelSelector) Descriptor 

func (*LabelSelector) Descriptor() ([]byte, []int)

func (*LabelSelector) Marshal 

func (m *LabelSelector) Marshal() (dAtA []byte, err error)

func (*LabelSelector) MarshalTo 

func (m *LabelSelector) MarshalTo(dAtA []byte) (int, error)

func (LabelSelector) Matches 

func (m LabelSelector) Matches(s LabelSet) bool

Matches returns whether the LabelSet is matched by the LabelSelector.

func (*LabelSelector) ProtoMessage 

func (*LabelSelector) ProtoMessage()

func (*LabelSelector) ProtoSize 

func (m *LabelSelector) ProtoSize() (n int)

func (*LabelSelector) Reset 

func (m *LabelSelector) Reset()

func (LabelSelector) String 

func (s LabelSelector) String() string

String returns a canonical string representation of the LabelSelector.

func (*LabelSelector) Unmarshal 

func (m *LabelSelector) Unmarshal(dAtA []byte) error

func (LabelSelector) Validate 

func (m LabelSelector) Validate() error

Validate returns an error if the LabelSelector is not well-formed.

func (*LabelSelector) XXX_DiscardUnknown 

func (m *LabelSelector) XXX_DiscardUnknown()

func (*LabelSelector) XXX_Marshal 

func (m *LabelSelector) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*LabelSelector) XXX_Merge 

func (dst *LabelSelector) XXX_Merge(src proto.Message)

func (*LabelSelector) XXX_Size 

func (m *LabelSelector) XXX_Size() int

func (*LabelSelector) XXX_Unmarshal 

func (m *LabelSelector) XXX_Unmarshal(b []byte) error

type LabelSet 

type LabelSet struct {
	// Labels of the set. Instances must be unique and sorted over (Name, Value).
	Labels []Label `protobuf:"bytes,1,rep,name=labels" json:"labels" yaml:",omitempty"`
}

LabelSet is a collection of labels and their values.

func ExtractJournalSpecMetaLabels 

func ExtractJournalSpecMetaLabels(spec *JournalSpec, out LabelSet) LabelSet

ExtractJournalSpecMetaLabels adds to the LabelSet a singular label "name", with value of the JournalSpec Name, and multi-label "prefix", having a value for each path component prefix of Name.

func IntersectLabelSets 

func IntersectLabelSets(lhs, rhs, out LabelSet) LabelSet

IntersectLabelSets returns the LabelSet having all labels present in both |lhs| and |rhs| with matched values.

func MustLabelSet 

func MustLabelSet(nv ...string) (set LabelSet)

MustLabelSet is a convenience for constructing LabelSets from a sequence of Name, Value arguments. The result LabelSet must Validate or MustLabelSet panics.

func SubtractLabelSet 

func SubtractLabelSet(lhs, rhs, out LabelSet) LabelSet

SubtractLabelSets returns the LabelSet having labels in |lhs| which are not present in |rhs| with matched values.

func UnionLabelSets 

func UnionLabelSets(lhs, rhs, out LabelSet) LabelSet

UnionLabelSets returns the LabelSet having all labels present in either |lhs| or |rhs|. Where both |lhs| and |rhs| have values for a label, those of |lhs| are preferred.

func (*LabelSet) AddValue 

func (m *LabelSet) AddValue(name, value string)

AddValue adds Label |name| with |value|, retaining any existing Labels |name|.

func (*LabelSet) Descriptor 

func (*LabelSet) Descriptor() ([]byte, []int)

func (*LabelSet) Marshal 

func (m *LabelSet) Marshal() (dAtA []byte, err error)

func (*LabelSet) MarshalTo 

func (m *LabelSet) MarshalTo(dAtA []byte) (int, error)

func (*LabelSet) ProtoMessage 

func (*LabelSet) ProtoMessage()

func (*LabelSet) ProtoSize 

func (m *LabelSet) ProtoSize() (n int)

func (*LabelSet) Remove 

func (m *LabelSet) Remove(name string)

Remove all instances of Labels |name|.

func (*LabelSet) Reset 

func (m *LabelSet) Reset()

func (*LabelSet) SetValue 

func (m *LabelSet) SetValue(name, value string)

SetValue adds Label |name| with |value|, replacing any existing Labels |name|.

func (*LabelSet) String 

func (m *LabelSet) String() string

func (*LabelSet) Unmarshal 

func (m *LabelSet) Unmarshal(dAtA []byte) error

func (LabelSet) Validate 

func (m LabelSet) Validate() error

Validate returns an error if the LabelSet is not well-formed.

func (LabelSet) ValueOf 

func (m LabelSet) ValueOf(name string) string

ValueOf returns the first value of Label |name|, or "" if it doesn't exist in the LabelSet.

func (LabelSet) ValuesOf 

func (m LabelSet) ValuesOf(name string) (values []string)

ValuesOf returns the values of Label |name|, or nil if it doesn't exist in the LabelSet.

func (*LabelSet) XXX_DiscardUnknown 

func (m *LabelSet) XXX_DiscardUnknown()

func (*LabelSet) XXX_Marshal 

func (m *LabelSet) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*LabelSet) XXX_Merge 

func (dst *LabelSet) XXX_Merge(src proto.Message)

func (*LabelSet) XXX_Size 

func (m *LabelSet) XXX_Size() int

func (*LabelSet) XXX_Unmarshal 

func (m *LabelSet) XXX_Unmarshal(b []byte) error

type ListRequest 

type ListRequest struct {
	// Selector optionally refines the set of journals which will be enumerated.
	// If zero-valued, all journals are returned. Otherwise, only JournalSpecs
	// matching the LabelSelector will be returned. Two meta-labels "name" and
	// "prefix" are additionally supported by the selector, where:
	//   * name=examples/a-name will match a JournalSpec with Name "examples/a-name"
	//   * prefix=examples/ will match any JournalSpec having prefix "examples/".
	//     The prefix Label value must end in '/'.
	Selector LabelSelector `protobuf:"bytes,1,opt,name=selector" json:"selector"`
	// Maximum number of journals to return in a ListResponse.
	// This field is optional, and the broker may enforce its own limit.
	PageLimit uint32 `protobuf:"varint,2,opt,name=page_limit,json=pageLimit,proto3" json:"page_limit,omitempty"`
	// A pagination token returned by a previous call to List, that indicates
	// where this request should continue from. Optional.
	PageToken string `protobuf:"bytes,3,opt,name=page_token,json=pageToken,proto3" json:"page_token,omitempty"`
}

func (*ListRequest) Descriptor 

func (*ListRequest) Descriptor() ([]byte, []int)

func (*ListRequest) Marshal 

func (m *ListRequest) Marshal() (dAtA []byte, err error)

func (*ListRequest) MarshalTo 

func (m *ListRequest) MarshalTo(dAtA []byte) (int, error)

func (*ListRequest) ProtoMessage 

func (*ListRequest) ProtoMessage()

func (*ListRequest) ProtoSize 

func (m *ListRequest) ProtoSize() (n int)

func (*ListRequest) Reset 

func (m *ListRequest) Reset()

func (*ListRequest) String 

func (m *ListRequest) String() string

func (*ListRequest) Unmarshal 

func (m *ListRequest) Unmarshal(dAtA []byte) error

func (*ListRequest) Validate 

func (m *ListRequest) Validate() error

func (*ListRequest) XXX_DiscardUnknown 

func (m *ListRequest) XXX_DiscardUnknown()

func (*ListRequest) XXX_Marshal 

func (m *ListRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ListRequest) XXX_Merge 

func (dst *ListRequest) XXX_Merge(src proto.Message)

func (*ListRequest) XXX_Size 

func (m *ListRequest) XXX_Size() int

func (*ListRequest) XXX_Unmarshal 

func (m *ListRequest) XXX_Unmarshal(b []byte) error

type ListResponse 

type ListResponse struct {
	// Status of the List RPC.
	Status Status `protobuf:"varint,1,opt,name=status,proto3,enum=protocol.Status" json:"status,omitempty"`
	// Header of the response.
	Header   Header                 `protobuf:"bytes,2,opt,name=header" json:"header"`
	Journals []ListResponse_Journal `protobuf:"bytes,3,rep,name=journals" json:"journals"`
	// A pagination token which indicates where the next request should continue
	// from. Empty if and only if this ListResponse completes the listing.
	NextPageToken string `protobuf:"bytes,4,opt,name=next_page_token,json=nextPageToken,proto3" json:"next_page_token,omitempty"`
}

func (*ListResponse) Descriptor 

func (*ListResponse) Descriptor() ([]byte, []int)

func (*ListResponse) Marshal 

func (m *ListResponse) Marshal() (dAtA []byte, err error)

func (*ListResponse) MarshalTo 

func (m *ListResponse) MarshalTo(dAtA []byte) (int, error)

func (*ListResponse) ProtoMessage 

func (*ListResponse) ProtoMessage()

func (*ListResponse) ProtoSize 

func (m *ListResponse) ProtoSize() (n int)

func (*ListResponse) Reset 

func (m *ListResponse) Reset()

func (*ListResponse) String 

func (m *ListResponse) String() string

func (*ListResponse) Unmarshal 

func (m *ListResponse) Unmarshal(dAtA []byte) error

func (*ListResponse) Validate 

func (m *ListResponse) Validate() error

func (*ListResponse) XXX_DiscardUnknown 

func (m *ListResponse) XXX_DiscardUnknown()

func (*ListResponse) XXX_Marshal 

func (m *ListResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ListResponse) XXX_Merge 

func (dst *ListResponse) XXX_Merge(src proto.Message)

func (*ListResponse) XXX_Size 

func (m *ListResponse) XXX_Size() int

func (*ListResponse) XXX_Unmarshal 

func (m *ListResponse) XXX_Unmarshal(b []byte) error

type ListResponse_Journal 

type ListResponse_Journal struct {
	Spec JournalSpec `protobuf:"bytes,1,opt,name=spec" json:"spec"`
	// Current ModRevision of the JournalSpec.
	ModRevision int64 `protobuf:"varint,2,opt,name=mod_revision,json=modRevision,proto3" json:"mod_revision,omitempty"`
	// Route of the journal, including endpoints.
	Route Route `protobuf:"bytes,3,opt,name=route" json:"route"`
}

Journals of the response.

func (*ListResponse_Journal) Descriptor 

func (*ListResponse_Journal) Descriptor() ([]byte, []int)

func (*ListResponse_Journal) Marshal 

func (m *ListResponse_Journal) Marshal() (dAtA []byte, err error)

func (*ListResponse_Journal) MarshalTo 

func (m *ListResponse_Journal) MarshalTo(dAtA []byte) (int, error)

func (*ListResponse_Journal) ProtoMessage 

func (*ListResponse_Journal) ProtoMessage()

func (*ListResponse_Journal) ProtoSize 

func (m *ListResponse_Journal) ProtoSize() (n int)

func (*ListResponse_Journal) Reset 

func (m *ListResponse_Journal) Reset()

func (*ListResponse_Journal) String 

func (m *ListResponse_Journal) String() string

func (*ListResponse_Journal) Unmarshal 

func (m *ListResponse_Journal) Unmarshal(dAtA []byte) error

func (*ListResponse_Journal) Validate 

func (m *ListResponse_Journal) Validate() error

func (*ListResponse_Journal) XXX_DiscardUnknown 

func (m *ListResponse_Journal) XXX_DiscardUnknown()

func (*ListResponse_Journal) XXX_Marshal 

func (m *ListResponse_Journal) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ListResponse_Journal) XXX_Merge 

func (dst *ListResponse_Journal) XXX_Merge(src proto.Message)

func (*ListResponse_Journal) XXX_Size 

func (m *ListResponse_Journal) XXX_Size() int

func (*ListResponse_Journal) XXX_Unmarshal 

func (m *ListResponse_Journal) XXX_Unmarshal(b []byte) error

type NoopDispatchRouter 

type NoopDispatchRouter struct{}

NoopDispatchRouter is a DispatchRouter which doesn't route.

func (NoopDispatchRouter) IsNoopRouter 

func (NoopDispatchRouter) IsNoopRouter() bool

func (NoopDispatchRouter) Route 

func (NoopDispatchRouter) Route(context.Context, string) Route

func (NoopDispatchRouter) UpdateRoute 

func (NoopDispatchRouter) UpdateRoute(string, *Route)

type ProcessSpec 

type ProcessSpec struct {
	Id ProcessSpec_ID `protobuf:"bytes,1,opt,name=id" json:"id"`
	// Advertised URL of the process.
	Endpoint Endpoint `protobuf:"bytes,2,opt,name=endpoint,proto3,casttype=Endpoint" json:"endpoint,omitempty"`
}

ProcessSpec describes a uniquely identified process and its addressable endpoint.

func (*ProcessSpec) Descriptor 

func (*ProcessSpec) Descriptor() ([]byte, []int)

func (*ProcessSpec) GetEndpoint 

func (m *ProcessSpec) GetEndpoint() Endpoint

func (*ProcessSpec) GetId 

func (m *ProcessSpec) GetId() ProcessSpec_ID

func (*ProcessSpec) Marshal 

func (m *ProcessSpec) Marshal() (dAtA []byte, err error)

func (*ProcessSpec) MarshalTo 

func (m *ProcessSpec) MarshalTo(dAtA []byte) (int, error)

func (*ProcessSpec) ProtoMessage 

func (*ProcessSpec) ProtoMessage()

func (*ProcessSpec) ProtoSize 

func (m *ProcessSpec) ProtoSize() (n int)

func (*ProcessSpec) Reset 

func (m *ProcessSpec) Reset()

func (*ProcessSpec) String 

func (m *ProcessSpec) String() string

func (*ProcessSpec) Unmarshal 

func (m *ProcessSpec) Unmarshal(dAtA []byte) error

func (*ProcessSpec) Validate 

func (m *ProcessSpec) Validate() error

Validate returns an error if the ProcessSpec is not well-formed.

func (*ProcessSpec) XXX_DiscardUnknown 

func (m *ProcessSpec) XXX_DiscardUnknown()

func (*ProcessSpec) XXX_Marshal 

func (m *ProcessSpec) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ProcessSpec) XXX_Merge 

func (dst *ProcessSpec) XXX_Merge(src proto.Message)

func (*ProcessSpec) XXX_Size 

func (m *ProcessSpec) XXX_Size() int

func (*ProcessSpec) XXX_Unmarshal 

func (m *ProcessSpec) XXX_Unmarshal(b []byte) error

type ProcessSpec_ID 

type ProcessSpec_ID struct {
	// "Zone" in which the process is running. Zones may be AWS, Azure, or Google
	// Cloud Platform zone identifiers, or rack locations within a colo, or
	// given some other custom meaning. Gazette will replicate across multiple
	// zones, and seeks to minimize traffic which must cross zones (for example,
	// by proxying reads to a broker in the current zone).
	Zone string `protobuf:"bytes,1,opt,name=zone,proto3" json:"zone,omitempty"`
	// Unique suffix of the process within |zone|. It is permissible for a
	// suffix value to repeat across zones, but never within zones. In practice,
	// it's recommended to use a FQDN, Kubernetes Pod name, or comparable unique
	// and self-describing value as the ID suffix.
	Suffix string `protobuf:"bytes,2,opt,name=suffix,proto3" json:"suffix,omitempty"`
}

ID composes a zone and a suffix to uniquely identify a ProcessSpec.

func (*ProcessSpec_ID) Descriptor 

func (*ProcessSpec_ID) Descriptor() ([]byte, []int)

func (ProcessSpec_ID) Less 

func (m ProcessSpec_ID) Less(other ProcessSpec_ID) bool

Less returns whether the ProcessSpec_ID is less than the argument BrokerSpec_ID, under (Zone, Suffix) ordering.

func (*ProcessSpec_ID) Marshal 

func (m *ProcessSpec_ID) Marshal() (dAtA []byte, err error)

func (*ProcessSpec_ID) MarshalTo 

func (m *ProcessSpec_ID) MarshalTo(dAtA []byte) (int, error)

func (*ProcessSpec_ID) ProtoMessage 

func (*ProcessSpec_ID) ProtoMessage()

func (*ProcessSpec_ID) ProtoSize 

func (m *ProcessSpec_ID) ProtoSize() (n int)

func (*ProcessSpec_ID) Reset 

func (m *ProcessSpec_ID) Reset()

func (*ProcessSpec_ID) String 

func (m *ProcessSpec_ID) String() string

func (*ProcessSpec_ID) Unmarshal 

func (m *ProcessSpec_ID) Unmarshal(dAtA []byte) error

func (ProcessSpec_ID) Validate 

func (m ProcessSpec_ID) Validate() error

Validate returns an error if the ProcessSpec_ID is not well-formed.

func (*ProcessSpec_ID) XXX_DiscardUnknown 

func (m *ProcessSpec_ID) XXX_DiscardUnknown()

func (*ProcessSpec_ID) XXX_Marshal 

func (m *ProcessSpec_ID) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ProcessSpec_ID) XXX_Merge 

func (dst *ProcessSpec_ID) XXX_Merge(src proto.Message)

func (*ProcessSpec_ID) XXX_Size 

func (m *ProcessSpec_ID) XXX_Size() int

func (*ProcessSpec_ID) XXX_Unmarshal 

func (m *ProcessSpec_ID) XXX_Unmarshal(b []byte) error

type ReadRequest 

type ReadRequest struct {
	// Header is attached by a proxying broker peer.
	Header *Header `protobuf:"bytes,1,opt,name=header" json:"header,omitempty"`
	// Journal to be read.
	Journal Journal `protobuf:"bytes,2,opt,name=journal,proto3,casttype=Journal" json:"journal,omitempty"`
	// Desired offset to begin reading from. Value -1 has special handling, where
	// the read is performed from the current write head. All other positive
	// values specify a desired exact byte offset to read from. If the offset is
	// not available (eg, because it represents a portion of Journal which has
	// been permanently deleted), the broker will return the next available
	// offset. Callers should therefore always inspect the ReadResponse offset.
	Offset int64 `protobuf:"varint,3,opt,name=offset,proto3" json:"offset,omitempty"`
	// Whether the operation should block until content becomes available.
	// OFFSET_NOT_YET_AVAILABLE is returned if a non-blocking read has no ready content.
	Block bool `protobuf:"varint,4,opt,name=block,proto3" json:"block,omitempty"`
	// If do_not_proxy is true, the broker will not proxy the read to another
	// broker, or open and proxy a remote Fragment on the client's behalf.
	DoNotProxy bool `protobuf:"varint,5,opt,name=do_not_proxy,json=doNotProxy,proto3" json:"do_not_proxy,omitempty"`
	// If metadata_only is true, the broker will respond with Journal and
	// Fragment metadata but not content.
	MetadataOnly bool `protobuf:"varint,6,opt,name=metadata_only,json=metadataOnly,proto3" json:"metadata_only,omitempty"`
}

func (*ReadRequest) Descriptor 

func (*ReadRequest) Descriptor() ([]byte, []int)

func (*ReadRequest) Marshal 

func (m *ReadRequest) Marshal() (dAtA []byte, err error)

func (*ReadRequest) MarshalTo 

func (m *ReadRequest) MarshalTo(dAtA []byte) (int, error)

func (*ReadRequest) ProtoMessage 

func (*ReadRequest) ProtoMessage()

func (*ReadRequest) ProtoSize 

func (m *ReadRequest) ProtoSize() (n int)

func (*ReadRequest) Reset 

func (m *ReadRequest) Reset()

func (*ReadRequest) String 

func (m *ReadRequest) String() string

func (*ReadRequest) Unmarshal 

func (m *ReadRequest) Unmarshal(dAtA []byte) error

func (*ReadRequest) Validate 

func (m *ReadRequest) Validate() error

Validate returns an error if the ReadRequest is not well-formed.

func (*ReadRequest) XXX_DiscardUnknown 

func (m *ReadRequest) XXX_DiscardUnknown()

func (*ReadRequest) XXX_Marshal 

func (m *ReadRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ReadRequest) XXX_Merge 

func (dst *ReadRequest) XXX_Merge(src proto.Message)

func (*ReadRequest) XXX_Size 

func (m *ReadRequest) XXX_Size() int

func (*ReadRequest) XXX_Unmarshal 

func (m *ReadRequest) XXX_Unmarshal(b []byte) error

type ReadResponse 

type ReadResponse struct {
	// Status of the Read RPC.
	Status Status `protobuf:"varint,1,opt,name=status,proto3,enum=protocol.Status" json:"status,omitempty"`
	// Header of the response. Accompanies the first ReadResponse of the response stream.
	Header *Header `protobuf:"bytes,2,opt,name=header" json:"header,omitempty"`
	// The effective offset of the read. See ReadRequest offset.
	Offset int64 `protobuf:"varint,3,opt,name=offset,proto3" json:"offset,omitempty"`
	// The offset to next be written, by the next append transaction served by
	// broker. In other words, the last offset through which content is
	// available to be read from the Journal. This is a metadata field and will
	// not be returned with a content response.
	WriteHead int64 `protobuf:"varint,4,opt,name=write_head,json=writeHead,proto3" json:"write_head,omitempty"`
	// Fragment to which the offset was mapped. This is a metadata field and will
	// not be returned with a content response.
	Fragment *Fragment `protobuf:"bytes,5,opt,name=fragment" json:"fragment,omitempty"`
	// If Fragment is remote, a URL from which it may be directly read.
	FragmentUrl string `protobuf:"bytes,6,opt,name=fragment_url,json=fragmentUrl,proto3" json:"fragment_url,omitempty"`
	// Content chunks of the read.
	Content []byte `protobuf:"bytes,7,opt,name=content,proto3" json:"content,omitempty"`
}

func (*ReadResponse) Descriptor 

func (*ReadResponse) Descriptor() ([]byte, []int)

func (*ReadResponse) Marshal 

func (m *ReadResponse) Marshal() (dAtA []byte, err error)

func (*ReadResponse) MarshalTo 

func (m *ReadResponse) MarshalTo(dAtA []byte) (int, error)

func (*ReadResponse) ProtoMessage 

func (*ReadResponse) ProtoMessage()

func (*ReadResponse) ProtoSize 

func (m *ReadResponse) ProtoSize() (n int)

func (*ReadResponse) Reset 

func (m *ReadResponse) Reset()

func (*ReadResponse) String 

func (m *ReadResponse) String() string

func (*ReadResponse) Unmarshal 

func (m *ReadResponse) Unmarshal(dAtA []byte) error

func (*ReadResponse) Validate 

func (m *ReadResponse) Validate() error

Validate returns an error if the ReadResponse is not well-formed.

func (*ReadResponse) XXX_DiscardUnknown 

func (m *ReadResponse) XXX_DiscardUnknown()

func (*ReadResponse) XXX_Marshal 

func (m *ReadResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ReadResponse) XXX_Merge 

func (dst *ReadResponse) XXX_Merge(src proto.Message)

func (*ReadResponse) XXX_Size 

func (m *ReadResponse) XXX_Size() int

func (*ReadResponse) XXX_Unmarshal 

func (m *ReadResponse) XXX_Unmarshal(b []byte) error

type ReplicateRequest 

type ReplicateRequest struct {
	// Header defines the primary broker, Route, and Etcd Revision under which
	// this Replicate stream is being established. Each replication peer
	// independently inspects and verifies the current Journal Route topology.
	Header *Header `protobuf:"bytes,1,opt,name=header" json:"header,omitempty"`
	// Journal to be replicated to.
	Journal Journal `protobuf:"bytes,2,opt,name=journal,proto3,casttype=Journal" json:"journal,omitempty"`
	// Proposed Fragment to commit. Also verified by each replica.
	Proposal *Fragment `protobuf:"bytes,3,opt,name=proposal" json:"proposal,omitempty"`
	// Content to be replicated.
	Content []byte `protobuf:"bytes,4,opt,name=content,proto3" json:"content,omitempty"`
	// Delta offset of |content| relative to current Fragment |end|.
	ContentDelta int64 `protobuf:"varint,5,opt,name=content_delta,json=contentDelta,proto3" json:"content_delta,omitempty"`
	// Acknowledge requests that the peer send an acknowledging ReplicateResponse
	// on successful application of the ReplicateRequest.
	Acknowledge bool `protobuf:"varint,6,opt,name=acknowledge,proto3" json:"acknowledge,omitempty"`
}

func (*ReplicateRequest) Descriptor 

func (*ReplicateRequest) Descriptor() ([]byte, []int)

func (*ReplicateRequest) Marshal 

func (m *ReplicateRequest) Marshal() (dAtA []byte, err error)

func (*ReplicateRequest) MarshalTo 

func (m *ReplicateRequest) MarshalTo(dAtA []byte) (int, error)

func (*ReplicateRequest) ProtoMessage 

func (*ReplicateRequest) ProtoMessage()

func (*ReplicateRequest) ProtoSize 

func (m *ReplicateRequest) ProtoSize() (n int)

func (*ReplicateRequest) Reset 

func (m *ReplicateRequest) Reset()

func (*ReplicateRequest) String 

func (m *ReplicateRequest) String() string

func (*ReplicateRequest) Unmarshal 

func (m *ReplicateRequest) Unmarshal(dAtA []byte) error

func (*ReplicateRequest) Validate 

func (m *ReplicateRequest) Validate() error

Validate returns an error if the ReplicateRequest is not well-formed.

func (*ReplicateRequest) XXX_DiscardUnknown 

func (m *ReplicateRequest) XXX_DiscardUnknown()

func (*ReplicateRequest) XXX_Marshal 

func (m *ReplicateRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ReplicateRequest) XXX_Merge 

func (dst *ReplicateRequest) XXX_Merge(src proto.Message)

func (*ReplicateRequest) XXX_Size 

func (m *ReplicateRequest) XXX_Size() int

func (*ReplicateRequest) XXX_Unmarshal 

func (m *ReplicateRequest) XXX_Unmarshal(b []byte) error

type ReplicateResponse 

type ReplicateResponse struct {
	// Status of the Replicate RPC.
	Status Status `protobuf:"varint,1,opt,name=status,proto3,enum=protocol.Status" json:"status,omitempty"`
	// Header of the response. Accompanies the first ReplicateResponse of the response stream.
	Header *Header `protobuf:"bytes,2,opt,name=header" json:"header,omitempty"`
	// If status is FRAGMENT_MISMATCH, then |fragment| is the replica's
	// Fragment at the current Journal head, which was found to be inconsistent
	// with the request |proposal| Fragment.
	Fragment *Fragment `protobuf:"bytes,3,opt,name=fragment" json:"fragment,omitempty"`
}

func (*ReplicateResponse) Descriptor 

func (*ReplicateResponse) Descriptor() ([]byte, []int)

func (*ReplicateResponse) Marshal 

func (m *ReplicateResponse) Marshal() (dAtA []byte, err error)

func (*ReplicateResponse) MarshalTo 

func (m *ReplicateResponse) MarshalTo(dAtA []byte) (int, error)

func (*ReplicateResponse) ProtoMessage 

func (*ReplicateResponse) ProtoMessage()

func (*ReplicateResponse) ProtoSize 

func (m *ReplicateResponse) ProtoSize() (n int)

func (*ReplicateResponse) Reset 

func (m *ReplicateResponse) Reset()

func (*ReplicateResponse) String 

func (m *ReplicateResponse) String() string

func (*ReplicateResponse) Unmarshal 

func (m *ReplicateResponse) Unmarshal(dAtA []byte) error

func (*ReplicateResponse) Validate 

func (m *ReplicateResponse) Validate() error

Validate returns an error if the ReplicateResponse is not well-formed.

func (*ReplicateResponse) XXX_DiscardUnknown 

func (m *ReplicateResponse) XXX_DiscardUnknown()

func (*ReplicateResponse) XXX_Marshal 

func (m *ReplicateResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ReplicateResponse) XXX_Merge 

func (dst *ReplicateResponse) XXX_Merge(src proto.Message)

func (*ReplicateResponse) XXX_Size 

func (m *ReplicateResponse) XXX_Size() int

func (*ReplicateResponse) XXX_Unmarshal 

func (m *ReplicateResponse) XXX_Unmarshal(b []byte) error

type Route 

type Route struct {
	// Members of the Route, ordered on ascending ProcessSpec.ID (zone, suffix).
	Members []ProcessSpec_ID `protobuf:"bytes,1,rep,name=members" json:"members"`
	// Index of the ProcessSpec serving as primary within |members|,
	// or -1 of no member is currently primary.
	Primary int32 `protobuf:"varint,2,opt,name=primary,proto3" json:"primary,omitempty"`
	// Endpoints of each Route member. If not empty, |endpoints| has the same
	// length and order as |members|, and captures the endpoint of each one.
	Endpoints []Endpoint `protobuf:"bytes,3,rep,name=endpoints,casttype=Endpoint" json:"endpoints,omitempty"`
}

Route captures the current topology of an item and the processes serving it.

func (*Route) AttachEndpoints 

func (m *Route) AttachEndpoints(ks *keyspace.KeySpace)

AttachEndpoints maps Route members through the KeySpace to their respective specs, and attaches the associated Endpoint of each to the Route. KeySpace must already be read-locked.

func (Route) Copy 

func (m Route) Copy() Route

Copy returns a deep copy of the Route.

func (*Route) Descriptor 

func (*Route) Descriptor() ([]byte, []int)

func (Route) Equivalent 

func (m Route) Equivalent(other *Route) bool

Equivalent returns true if the Routes have equivalent broker Names, Zones, and current Primary. It does not compare broker Endpoints.

func (*Route) Init 

func (m *Route) Init(assignments keyspace.KeyValues)

Initialize Route with the provided allocator Assignments.

func (*Route) Marshal 

func (m *Route) Marshal() (dAtA []byte, err error)

func (Route) MarshalString 

func (m Route) MarshalString() string

MarshalString returns the marshaled encoding of the Route as a string.

func (*Route) MarshalTo 

func (m *Route) MarshalTo(dAtA []byte) (int, error)

func (*Route) ProtoMessage 

func (*Route) ProtoMessage()

func (*Route) ProtoSize 

func (m *Route) ProtoSize() (n int)

func (*Route) Reset 

func (m *Route) Reset()

func (*Route) String 

func (m *Route) String() string

func (*Route) Unmarshal 

func (m *Route) Unmarshal(dAtA []byte) error

func (Route) Validate 

func (m Route) Validate() error

Validate returns an error if the Route is not well-formed.

func (*Route) XXX_DiscardUnknown 

func (m *Route) XXX_DiscardUnknown()

func (*Route) XXX_Marshal 

func (m *Route) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Route) XXX_Merge 

func (dst *Route) XXX_Merge(src proto.Message)

func (*Route) XXX_Size 

func (m *Route) XXX_Size() int

func (*Route) XXX_Unmarshal 

func (m *Route) XXX_Unmarshal(b []byte) error

type RoutedJournalClient 

type RoutedJournalClient interface {
	JournalClient
	DispatchRouter
}

RoutedJournalClient composes a JournalClient and DispatchRouter.

func NewRoutedJournalClient 

func NewRoutedJournalClient(jc JournalClient, dr DispatchRouter) RoutedJournalClient

NewRoutedJournalClient composes a JournalClient and DispatchRouter.

type SHA1Sum 

type SHA1Sum struct {
	Part1 uint64 `protobuf:"fixed64,1,opt,name=part1,proto3" json:"part1,omitempty"`
	Part2 uint64 `protobuf:"fixed64,2,opt,name=part2,proto3" json:"part2,omitempty"`
	Part3 uint32 `protobuf:"fixed32,3,opt,name=part3,proto3" json:"part3,omitempty"`
}

SHA1Sum is a 160-bit SHA1 digest.

func SHA1SumFromDigest 

func SHA1SumFromDigest(r []byte) SHA1Sum

SHA1SumFromDigest converts SHA1 sum in digest form into a SHA1Sum. |r| must have the length of a SHA1 digest (20 bytes), or it panics.

func SHA1SumOf 

func SHA1SumOf(str string) SHA1Sum

SHA1SumOf SHA1 sums |str| and returns a SHA1Sum.

func (*SHA1Sum) Descriptor 

func (*SHA1Sum) Descriptor() ([]byte, []int)

func (SHA1Sum) IsZero 

func (m SHA1Sum) IsZero() bool

IsZero returns whether this SHA1Sum is zero-valued. As a special case, Fragments having no content are consistently mapped to the zero-valued SHA1Sum (rather than SHA1 of "", which is da39a3ee5e6b4b0d3255bfef95601890afd80709).

func (*SHA1Sum) Marshal 

func (m *SHA1Sum) Marshal() (dAtA []byte, err error)

func (*SHA1Sum) MarshalTo 

func (m *SHA1Sum) MarshalTo(dAtA []byte) (int, error)

func (*SHA1Sum) ProtoMessage 

func (*SHA1Sum) ProtoMessage()

func (*SHA1Sum) ProtoSize 

func (m *SHA1Sum) ProtoSize() (n int)

func (*SHA1Sum) Reset 

func (m *SHA1Sum) Reset()

func (*SHA1Sum) String 

func (m *SHA1Sum) String() string

func (SHA1Sum) ToDigest 

func (m SHA1Sum) ToDigest() (r [20]byte)

ToDigest converts the SHA1Sum to a flat, fixed-size array.

func (*SHA1Sum) Unmarshal 

func (m *SHA1Sum) Unmarshal(dAtA []byte) error

func (*SHA1Sum) XXX_DiscardUnknown 

func (m *SHA1Sum) XXX_DiscardUnknown()

func (*SHA1Sum) XXX_Marshal 

func (m *SHA1Sum) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*SHA1Sum) XXX_Merge 

func (dst *SHA1Sum) XXX_Merge(src proto.Message)

func (*SHA1Sum) XXX_Size 

func (m *SHA1Sum) XXX_Size() int

func (*SHA1Sum) XXX_Unmarshal 

func (m *SHA1Sum) XXX_Unmarshal(b []byte) error

type Status 

type Status int32

Status is a response status code, used universally across Gazette RPC APIs. 

const (
	Status_OK Status = 0
	// The named journal does not exist.
	Status_JOURNAL_NOT_FOUND Status = 1
	// There is no current primary broker for the journal. This is a temporary
	// condition which should quickly resolve, assuming sufficient broker capacity.
	Status_NO_JOURNAL_PRIMARY_BROKER Status = 2
	// The present broker is not the assigned primary broker for the journal.
	Status_NOT_JOURNAL_PRIMARY_BROKER Status = 3
	// The present broker is not an assigned broker for the journal.
	Status_NOT_JOURNAL_BROKER Status = 5
	// There are an insufficient number of assigned brokers for the journal
	// to meet its required replication.
	Status_INSUFFICIENT_JOURNAL_BROKERS Status = 4
	// The requested offset is not yet available. This indicates either that the
	// offset has not yet been written, or that the broker is not yet aware of a
	// written fragment covering the offset. Returned only by non-blocking reads.
	Status_OFFSET_NOT_YET_AVAILABLE Status = 6
	// The peer disagrees with the Route accompanying a ReplicateRequest.
	Status_WRONG_ROUTE Status = 7
	// The peer disagrees with the Fragment proposal accompanying a ReplicateRequest.
	Status_FRAGMENT_MISMATCH Status = 8
	// The Etcd transaction failed. Returned by Update RPC when an
	// expect_mod_revision of the UpdateRequest differs from the current
	// ModRevision of the JournalSpec within the store.
	Status_ETCD_TRANSACTION_FAILED Status = 9
	// A disallowed journal access was attempted (eg, a write where the
	// journal disables writes, or read where journals disable reads).
	Status_NOT_ALLOWED Status = 10
	// The Append is refused because its requested offset is not equal
	// to the furthest written offset of the journal.
	Status_WRONG_APPEND_OFFSET Status = 11
	// The Append is refused because the replication pipeline tracks a smaller
	// journal offset than that of the remote fragment index. This indicates
	// that journal replication consistency has been lost in the past, due to
	// too many broker or Etcd failures.
	Status_INDEX_HAS_GREATER_OFFSET Status = 12
)

func (Status) EnumDescriptor 

func (Status) EnumDescriptor() ([]byte, []int)

func (Status) String 

func (x Status) String() string

func (Status) Validate 

func (x Status) Validate() error

type ValidationError 

type ValidationError struct {
	Context []string
	Err     error
}

ValidationError is an error implementation which captures its validation context.

func (*ValidationError) Error 

func (ve *ValidationError) Error() string

Error implements the error interface.

type Validator 

type Validator interface {
	Validate() error
}

Validator is a type able to validate itself. Validate inspects the type for syntactic or semantic issues, and returns a descriptive error if any violations are encountered. It is recommended that Validate return instances of ValidationError where possible, which enables tracking nested contexts.

Source Files

View all Source files

Directories

Path Synopsis
Package journalspace provides mechanisms for mapping a collection of JournalSpecs into a minimally-described hierarchical structure, and for mapping back again.
 Package journalspace provides mechanisms for mapping a collection of JournalSpecs into a minimally-described hierarchical structure, and for mapping back again.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.