README
¶
Gazette V1 is deprecated and will be removed, but remains at the repository root for migration compatibility. See `Gazette V2 <v2/>`_ for the current version of gazette (under the ``v2`` directory).
Documentation
¶
Overview ¶
Gazette is a distributed byte-stream transaction engine. It models a concept of "journals": infinite length, append-only files.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
gazconsumer
Given a list of -prefix arguments, expand those prefixes to a list of Etcd paths to Gazette consumers and compare the state of the consumer to the state of the source journals to determine how much data backlog exists for each consumer.
|
Given a list of -prefix arguments, expand those prefixes to a list of Etcd paths to Gazette consumers and compare the state of the consumer to the state of the source journals to determine how much data backlog exists for each consumer. |
|
gazretention
gazretention is used to expose fragments from cloud storage that we no longer wish to retain.
|
gazretention is used to expose fragments from cloud storage that we no longer wish to retain. |
|
examples
|
|
|
stream-sum/summer
summer is a consumer plugin (eg, should be built with `go build --buildmode=plugin`).
|
summer is a consumer plugin (eg, should be built with `go build --buildmode=plugin`). |
|
word-count/counter
counter is a consumer plugin (eg, should be built with `go build --buildmode=plugin`).
|
counter is a consumer plugin (eg, should be built with `go build --buildmode=plugin`). |
|
word-count/shuffler
shuffler is a consumer plugin (eg, should be built with `go build --buildmode=plugin`).
|
shuffler is a consumer plugin (eg, should be built with `go build --buildmode=plugin`). |
|
pkg
|
|
|
async
Package async implements a simple Promise API.
|
Package async implements a simple Promise API. |
|
consensus
Package consensus provides building blocks for solving difficult distributed consensus-related problems atop Etcd, both within and across processes.
|
Package consensus provides building blocks for solving difficult distributed consensus-related problems atop Etcd, both within and across processes. |
|
consumer
Code generated by mockery v1.0.0 Code generated by mockery v1.0.0 Package consumer is a generated protocol buffer package.
|
Code generated by mockery v1.0.0 Code generated by mockery v1.0.0 Package consumer is a generated protocol buffer package. |
|
envflag
Package envflag implements support for setting flags through environment variables.
|
Package envflag implements support for setting flags through environment variables. |
|
envflagfactory
Package envflagfactory contains common env flag definitions used across gazette tools.
|
Package envflagfactory contains common env flag definitions used across gazette tools. |
|
gazette
Package gazette contains server components tied to the service lifetime, and clients.
|
Package gazette contains server components tied to the service lifetime, and clients. |
|
journal
Package journal contains all runtime components for Gazette journals, including Fragment & Spool for journal content, Head (serving replications), Tail (reads), and Broker (for brokering new writes).
|
Package journal contains all runtime components for Gazette journals, including Fragment & Spool for journal content, Head (serving replications), Tail (reads), and Broker (for brokering new writes). |
|
mainboilerplate
Package mainboilerplate contains shared boilerplate for this project's programs.
|
Package mainboilerplate contains shared boilerplate for this project's programs. |
|
recoverylog
Package recoverylog is a generated protocol buffer package.
|
Package recoverylog is a generated protocol buffer package. |
|
topic
Package topic is a client library for topic descriptions and partitioned writes.
|
Package topic is a client library for topic descriptions and partitioned writes. |
|
v2
|
|
|
cmd/gazctl/editor
Package editor provides utilities for editing and re-editing text files in an editor until the changes are accepted or the user aborts the attempt.
|
Package editor provides utilities for editing and re-editing text files in an editor until the changes are accepted or the user aborts the attempt. |
|
examples/stream-sum
Package stream_sum is an example application consisting of three stages: 1) A `chunker` job randomly generates a number of unique "streams", with stream content emitted across a number of interleaved data chunks.
|
Package stream_sum is an example application consisting of three stages: 1) A `chunker` job randomly generates a number of unique "streams", with stream content emitted across a number of interleaved data chunks. |
|
examples/stream-sum/summer
Package summer runs the stream_sum.Summer consumer.
|
Package summer runs the stream_sum.Summer consumer. |
|
examples/word-count
Package word_count is an example application which provides a gRPC API for publishing texts and querying running counts of NGrams extracted from previously published texts.
|
Package word_count is an example application which provides a gRPC API for publishing texts and querying running counts of NGrams extracted from previously published texts. |
|
examples/word-count/counter
Package counter runs the word_count.Counter consumer.
|
Package counter runs the word_count.Counter consumer. |
|
pkg/allocator
Package allocator implements a distributed algorithm for assigning a number of "Items" across a number of "Members", where each Member runs an instance of the Allocator.
|
Package allocator implements a distributed algorithm for assigning a number of "Items" across a number of "Members", where each Member runs an instance of the Allocator. |
|
pkg/allocator/push_relabel
Package push_relabel implements a greedy variant of the push/relabel algorithm.
|
Package push_relabel implements a greedy variant of the push/relabel algorithm. |
|
pkg/broker
Package broker implements the broker runtime and protocol.JournalServer APIs (Read, Append, Replicate, List, Apply).
|
Package broker implements the broker runtime and protocol.JournalServer APIs (Read, Append, Replicate, List, Apply). |
|
pkg/brokertest
Package brokertest provides utilities for testing components requiring a live Gazette broker.
|
Package brokertest provides utilities for testing components requiring a live Gazette broker. |
|
pkg/client
Package client provides implementation for clients of the broker API.
|
Package client provides implementation for clients of the broker API. |
|
pkg/consumertest
Package consumertest provides utilities for in-process unit testing of Gazette consumer applications.
|
Package consumertest provides utilities for in-process unit testing of Gazette consumer applications. |
|
pkg/etcdtest
Package etcdtest provides test support for obtaining a client to an Etcd server.
|
Package etcdtest provides test support for obtaining a client to an Etcd server. |
|
pkg/fragment
Package fragment is concerned with the mapping of journal offsets to protocol.Fragments, to corresponding local or remote journal content.
|
Package fragment is concerned with the mapping of journal offsets to protocol.Fragments, to corresponding local or remote journal content. |
|
pkg/keyspace
Package keyspace implements an efficient mechanism to mirror a decoded Etcd key/value space into a local KeySpace, which may be kept updated via a long- lived Watch operation.
|
Package keyspace implements an efficient mechanism to mirror a decoded Etcd key/value space into a local KeySpace, which may be kept updated via a long- lived Watch operation. |
|
pkg/mainboilerplate/runconsumer
Package runconsumer extends consumer.Application with support for configuration and application initialization.
|
Package runconsumer extends consumer.Application with support for configuration and application initialization. |
|
pkg/message
Package message defines a Message interface and Envelope type, and provides a Framing interface and implementations.
|
Package message defines a Message interface and Envelope type, and provides a Framing interface and implementations. |
|
pkg/protocol
Package protocol defines the core Gazette datamodel, validation behaviors, and gRPC APIs which are shared across clients and broker servers.
|
Package protocol defines the core Gazette datamodel, validation behaviors, and gRPC APIs which are shared across clients and broker servers. |
|
pkg/protocol/journalspace
Package journalspace provides mechanisms for mapping a collection of JournalSpecs into a minimally-described hierarchical structure, and for mapping back again.
|
Package journalspace provides mechanisms for mapping a collection of JournalSpecs into a minimally-described hierarchical structure, and for mapping back again. |
Click to show internal directories.
Click to hide internal directories.