Documentation ¶
Index ¶
Constants ¶
const ( // CompressionNullLabel is used when OCF blocks are not compressed. CompressionNullLabel = "null" // CompressionDeflateLabel is used when OCF blocks are compressed using the // deflate algorithm. CompressionDeflateLabel = "deflate" // CompressionSnappyLabel is used when OCF blocks are compressed using the // snappy algorithm. CompressionSnappyLabel = "snappy" )
Variables ¶
var ( // MaxBlockCount is the maximum number of data items allowed in a single // binary block that will be decoded from a binary stream. This check is to // ensure decoding binary data will not cause the library to over allocate // RAM, potentially creating a denial of service on the system. // // If a particular application needs to decode binary Avro data that // potentially has more data items in a single block, then this variable may // be modified at your discretion. MaxBlockCount = int64(math.MaxInt32) // MaxBlockSize is the maximum number of bytes that will be allocated for a // single block of data items when decoding from a binary stream. This check // is to ensure decoding binary data will not cause the library to over // allocate RAM, potentially creating a denial of service on the system. // // If a particular application needs to decode binary Avro data that // potentially has more bytes in a single block, then this variable may be // modified at your discretion. MaxBlockSize = int64(math.MaxInt32) )
Functions ¶
Types ¶
type BinaryCoder ¶
type BinaryCoder interface { BinaryDecoder BinaryEncoder }
BinaryCoder interface describes types that expose both the BinaryDecode and the BinaryEncode methods.
type BinaryDecoder ¶
BinaryDecoder interface describes types that expose the BinaryDecode method.
type BinaryEncoder ¶
BinaryEncoder interface describes types that expose the BinaryEncode method.
type Codec ¶ added in v0.0.6
type Codec struct {
// contains filtered or unexported fields
}
Codec stores function pointers for encoding and decoding Avro blobs according to their defined specification. Their state is created during initialization, but then never modified, so the same Codec may be safely used in multiple go routines to encode and or decode different Avro streams concurrently.
func (Codec) BinaryDecode ¶ added in v0.0.6
BinaryDecode decodes the provided byte slice in accordance with the Codec's Avro schema. On success, it returns the decoded value, along with a new byte slice with the decoded bytes consumed. In other words, when decoding an Avro int that happens to take 3 bytes, the returned byte slice will be like the original byte slice, but with the first three bytes removed. On error, it returns the original byte slice without any bytes consumed and the error.
func (Codec) BinaryEncode ¶ added in v0.0.6
BinaryEncode encodes the provided datum value in accordance with the Codec's Avro schema. It takes a byte slice to which to append the encoded bytes. On success, it returns the new byte slice with the appended byte slice. On error, it returns the original byte slice without any encoded bytes.
func (Codec) TextDecode ¶ added in v0.1.0
TextDecode decodes the provided byte slice in accordance with the Codec's Avro schema. On success, it returns the decoded value, along with a new byte slice with the decoded bytes consumed. In other words, when decoding an Avro int that happens to take 3 bytes, the returned byte slice will be like the original byte slice, but with the first three bytes removed. On error, it returns the original byte slice without any bytes consumed and the error.
func (Codec) TextEncode ¶ added in v0.1.0
TextEncode encodes the provided datum value in accordance with the Codec's Avro schema. It takes a byte slice to which to append the encoded bytes. On success, it returns the new byte slice with the appended byte slice. On error, it returns the original byte slice without any encoded bytes.
type Compression ¶ added in v0.0.8
type Compression uint8
Compression are values used to specify compression algorithm used to compress and decompress Avro Object Container File (OCF) streams.
const ( // CompressionNull is used when OCF blocks are not compressed. CompressionNull Compression = iota // CompressionDeflate is used when OCF blocks are compressed using the // deflate algorithm. CompressionDeflate // CompressionSnappy is used when OCF blocks are compressed using the snappy // algorithm. CompressionSnappy )
type ErrInvalidName ¶
type ErrInvalidName struct {
Message string
}
ErrInvalidName is the error returned when one or more parts of an Avro name is invalid.
func (ErrInvalidName) Error ¶
func (e ErrInvalidName) Error() string
type OCFReader ¶ added in v0.0.8
type OCFReader struct {
// contains filtered or unexported fields
}
OCFReader structure is used to read Object Container Files (OCF).
func NewOCFReader ¶ added in v0.0.8
NewOCFReader initializes and returns a new structure used to read an Avro Object Container File (OCF).
func example(ior io.Reader) error { ocfr, err := goavro.NewOCFReader(ior) if err != nil { return err } for ocfr.Scan() { datum, err := ocfr.Read() if err != nil { return err } fmt.Println(datum) } return ocfr.Err() }
func (*OCFReader) Err ¶ added in v0.0.8
Err returns the last error encountered while reading the OCF file. It does not reset the read error.
func (*OCFReader) Read ¶ added in v0.0.8
Read consumes one data item from the Avro OCF stream and returns it. Read is designed to be called only once after each invocation of the Scan method. See the documentation for goavro.NewOCFReader for an example of how to use Read.
func (*OCFReader) Scan ¶ added in v0.0.8
Scan returns true when there is at least one more data item to be read from the Avro OCF. Scan ought to be called prior to calling the Read method each time the Read method is invoked. See the documentation for goavro.NewOCFReader for an example of how to use Scan.
type OCFWriter ¶ added in v0.0.8
type OCFWriter struct {
// contains filtered or unexported fields
}
OCFWriter is used to create an Avro Object Container File (OCF).
func NewOCFWriter ¶ added in v0.0.8
func NewOCFWriter(config OCFWriterConfig) (*OCFWriter, error)
NewOCFWriter returns a newly created OCFWriter which may be used to create an Avro Object Container File (OCF).
type OCFWriterConfig ¶ added in v0.0.8
type OCFWriterConfig struct { W io.Writer // W specifies the io.Writer to send the encode the data, (required). Schema string // Schema specifies the Avro schema for the data to be encoded, (required). Compression Compression // Codec specifies the compression codec used, (optional). If omitted, defaults to "null" codec. }
OCFWriterConfig is used to specify creation parameters for OCFWriter.
type TextCoder ¶ added in v0.1.0
type TextCoder interface { TextDecoder TextEncoder }
TextCoder interface describes types that expose both the TextDecode and the TextEncode methods.
type TextDecoder ¶ added in v0.1.0
TextDecoder interface describes types that expose the TextDecode method.
type TextEncoder ¶ added in v0.1.0
TextEncoder interface describes types that expose the TextEncode method.