Documentation
¶
Overview ¶
Package riegeli implements a Reader and Writer for the Riegeli records format.
C++ implementation: https://github.com/google/riegeli Format spec: https://github.com/google/riegeli/blob/master/doc/riegeli_records_file_format.md
Index ¶
Constants ¶
const ( DefaultChunkSize uint64 = 1 << 20 DefaultBrotliLevel = 9 DefaultZSTDLevel = 9 )
Defaults for the WriterOptions.
Variables ¶
var DefaultCompression = BrotliCompression(DefaultBrotliLevel)
DefaultCompression is the default Compression for the WriterOptions.
Functions ¶
This section is empty.
Types ¶
type CompressionType ¶
CompressionType is the type of compression used for encoding Riegeli chunks.
var ( // NoCompression indicates that no compression will be used to encode chunks. NoCompression CompressionType = &compressionLevel{noCompression, 0} // SnappyCompression indicates to use Snappy compression. SnappyCompression CompressionType = &compressionLevel{snappyCompression, 0} )
func BrotliCompression ¶
func BrotliCompression(level int) CompressionType
BrotliCompression returns a CompressionType for Brotli compression with the given quality level. If level < 0 || level > 11, then the DefaultBrotliLevel will be used.
func ZSTDCompression ¶
func ZSTDCompression(level int) CompressionType
ZSTDCompression returns a CompressionType for zstd compression with the given compression level. If level < 0 || level > 22 (outside of the levels specified by the zstdlib spec), then the DefaultZSTDLevel will be used.
type ReadSeeker ¶
type ReadSeeker interface {
Reader
// Seek interprets pos as an offset to a record within the Riegeli file. pos
// must be between 0 and the file's size. If pos is between records, Seek will
// position the reader to the next record in the file.
Seek(pos int64) error
// SeekToRecord seeks to the given RecordPosition.
SeekToRecord(pos RecordPosition) error
}
ReadSeeker is a Riegeli records file reader able to seek to arbitrary positions.
func NewReadSeeker ¶
func NewReadSeeker(r io.ReadSeeker) ReadSeeker
NewReadSeeker returns a Riegeli ReadSeeker for r.
type Reader ¶
type Reader interface {
// RecordsMetadata returns the optional metadata from the underlying Riegeli
// file. If not found, an empty RecordsMetadata is returned and err == nil.
RecordsMetadata() (*rmpb.RecordsMetadata, error)
// Next reads and returns the next Riegeli record from the underlying io.Reader.
Next() ([]byte, error)
// NextProto reads, unmarshals, and returns the next proto.Message from the
// underlying io.Reader.
NextProto(msg proto.Message) error
// Position returns the current position of the Reader.
Position() (RecordPosition, error)
}
Reader is a sequential Riegeli records file reader.
type RecordPosition ¶
type RecordPosition struct {
// ChunkBegin is the starting offset of a chunk within a Riegeli file.
ChunkBegin int64
// RecordIndex is the index of a record within the chunk starting at
// ChunkBegin.
RecordIndex int64
}
A RecordPosition is a pointer to the starting offset of a record within a Riegeli file.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer is a Riegeli records file writer.
func NewWriter ¶
func NewWriter(w io.Writer, opts *WriterOptions) *Writer
NewWriter returns a Riegeli Writer for a new Riegeli file to be written to w.
func NewWriterAt ¶
func NewWriterAt(w io.Writer, pos int, opts *WriterOptions) *Writer
NewWriterAt returns a Riegeli Writer at the given byte offset within w.
func (*Writer) Close ¶
Close releases all resources associated with Writer. Any buffered records will be flushed before releasing any resources.
func (*Writer) Position ¶ added in v0.0.31
func (w *Writer) Position() RecordPosition
Position returns the current position of the Writer.
type WriterOptions ¶
type WriterOptions struct {
// Desired uncompressed size of a chunk which groups records.
ChunkSize uint64
// Compression is the type of compression used for encoding chunks.
Compression CompressionType
// Transpose determines whether Protocol Buffer messages have their component
// key-value entries encoded in separate buffers for better compression.
Transpose bool
}
WriterOptions customizes the behavior of a Riegeli Writer.
func ParseOptions ¶ added in v0.0.29
func ParseOptions(s string) (*WriterOptions, error)
ParseOptions decodes a WriterOptions from text:
options ::= option? ("," option?)*
option ::=
"default" |
"transpose" (":" ("true" | "false"))? |
"uncompressed" |
"brotli" (":" brotli_level)? |
"zstd" (":" zstd_level)? |
"chunk_size" ":" chunk_size
brotli_level ::= integer 0..11 (default 9)
zstd_level ::= integer 0..22 (default 9)
chunk_size ::= positive integer
func (*WriterOptions) String ¶ added in v0.0.29
func (o *WriterOptions) String() string
String encodes the WriterOptions as text.
Source Files
Directories