datacodec

package
v0.0.0-...-605a850 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 3, 2024 License: Apache-2.0 Imports: 14 Imported by: 5

Documentation

Overview

Package datacodec contains functionality to encode and decode CQL data.

This package is not used anywhere in this library, but can be used by client code to encode and decode actual CQL values sent to or received from any Cassandra-compatible backend.

`datacodec.Codec` is its main interface, and it has two main methods: 

// Encode encodes the given source into dest. The parameter source must be a value of a supported Go type for the
// CQL type being encoded, or a pointer thereto; a nil value is encoded as a CQL NULL.
Encode(source interface{}, version primitive.ProtocolVersion) (dest []byte, err error)

// Decode decodes the given source into dest. The parameter dest must be a pointer to a supported Go type for
// the CQL type being decoded; it cannot be nil. If return parameter wasNull is true, then the decoded value was a
// NULL, in which case the actual value stored in dest will be set to its zero value.
Decode(source []byte, dest interface{}, version primitive.ProtocolVersion) (wasNull bool, err error)

Obtaining a codec

It is not required to implement the datacodec.Codec interface unless a custom encoding/decoding scheme must be used. The NewCodec function can be used to obtain a codec for a all valid CQL types, including complex ones (lists, sets, maps, tuples and user-defined types).

Simple CQL types also have a global codec available; for example the datacodec.Varchar codec can be used to decode and encode to CQL varchar.

Codecs for complex types can be obtained through constructor functions:

  • NewList
  • NewSet
  • NewMap
  • NewTuple
  • NewUserDefined

Using a codec

Codecs accept a wide variety of inputs, but they have a "preferred" Go type that is the ideal representation of the CQL type in Go. In the table below, preferred types are listed first. 

CQL type              | Accepted Go types (1)                           | Notes
bigint, counter       | int64, *int64                                   |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  |
                      | *big.Int                                        |
                      | string, *string                                 | formatted and parsed as base 10 number
blob                  | []byte, *[]byte                                 |
                      | string, *string                                 |
boolean               | bool, *bool                                     |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  | zero=false, other=true
date                  | time.Time, *time.Time                           | time at start of day (in UTC); clock part set to zero
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  | days since Unix epoch
                      | string, *string                                 | parsed according to layout, default is "2006-01-02"
decimal               | CqlDecimal, *CqlDecimal                         |
double                | float64, *float64                               |
                      | float32, *float32                               |
                      | *big.Float                                      |
duration              | CqlDuration, *CqlDuration                       |
float                 | float32, *float32                               |
                      | float64, *float64                               |
inet                  | net.IP, *net.IP                                 |
                      | []byte, *[]byte                                 |
                      | string, *string                                 | parsed with net.ParseIP
int                   | int32, *int32                                   |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  |
                      | string, *string                                 | formatted and parsed as base 10 number
list, set             | any compatible slice                            | element types must match
                      | any compatible array                            | element types must match
map                   | any compatible map                              | key and value types must match
smallint              | int16, *int16                                   |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  |
                      | string, *string                                 | formatted and parsed as base 10 number
time                  | time.Duration, *time.Duration                   |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  | nanoseconds since start of day
                      | time.Time, *time.Time                           | time since start of day (in UTC); date part set to zero
                      | string, *string                                 | parsed according to layout, default is "15:04:05.999999999"
timestamp             | time.Time, *time.Time                           |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  | milliseconds since Unix epoch
                      | string, *string                                 | parsed according to layout and location, defaults are "2006-01-02T15:04:05.999999999-07:00" and UTC
tinyint               | int8, *int8                                     |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  |
                      | string, *string                                 | formatted and parsed as base 10 number
tuple                 | any compatible slice                            | slice size and elements must match
                      | any compatible array                            | array size and elements must match
                      | any compatible struct                           | fields must be exported and are marshaled in order of declaration
user-defined type     | any compatible map                              | map key must be string
                      | any compatible struct                           | fields must be exported; field names match case-insensitively (2)
                      | any compatible slice                            | slice size and elements must match
                      | any compatible array                            | array size and elements must match
uuid, timeuuid        | primitive.UUID, *primitive.UUID                 |
                      | [16]byte, []byte                                | raw UUID bytes, length must be 16 bytes
                      | string, *string                                 | hex representation, parsed with primitive.ParseUuid
varchar, text, ascii  | string, *string                                 |
                      | []byte, *[]byte, []rune, *[]rune                |
varint                | big.Int, *big.Int                               |
                      | int[64-8], *int[64-8], uint[64-8], *uint[64-8]  |
                      | string, *string                                 | formatted and parsed as base 10 number

(1) types listed on the first line are the preferred types for each CQL type. Note that non-pointer types are only accepted when encoding, never when decoding. (2) when mapping structs to user-defined types, the "cassandra" field tag can be used to override the corresponding field name.

In addition to the accepted types above, all codecs also accept interface{} when encoding and *interface{} when decoding. When encoding an interface{} value, the actual runtime value stored in the variable must be of an accepted type. When decoding to *interface{}, the codec will use the preferred type to decode, then store its value in the target variable; if the decoded value was NULL, the target will be set to nil.

Encoding data

Sources can be passed by value or by reference, unless specified otherwise in the table above. Nils are encoded as CQL NULLs; encoding such a value is generally a no-op and returns a nil []byte.

Decoding data

Destination values must be passed by reference, see examples below. This is also valid for slices and maps, and is required for the value to be addressable (i.e. settable), and for the changes applied to the value to be visible once the method returns.

When a CQL NULL is decoded, the passed `dest` variable is set to its zero value. Depending on the variable type, the zero value may or may not be distinguishable from the CQL zero value for that type. For example, if a CQL bigint was NULL and `dest` is `int64`, then `dest` will be set to `0`, effectively making this indistinguishable from a non-NULL bigint value of `0`. This is why the return parameter `wasNull` is used for: if `wasNull` is true then the decoded value was a CQL NULL.

A typical invocation of `Codec.Decode` is as follows: 

 source := []byte{...}
 var value int64
 wasNull, err := datacodec.Bigint.Decode(source, &value, primitive.ProtocolVersion5)
 if err != nil {
	  fmt.Println("Decoding failed: ", err)
 } else if wasNull {
	  fmt.Println("CQL value was NULL")
 } else {
	  fmt.Println("CQL value was:", value)
 }

Decoding a complex CQL type, such as a list, is not very different: 

 source := []byte{...}
 var value []int
 listOfInt := datatype.NewListType(datatype.Int)
 listOfIntCodec, _ := datacodec.NewList(listOfInt)
 wasNull, err := listOfIntCodec.Decode(source, &value, primitive.ProtocolVersion5)
 if err != nil {
	  fmt.Println("Decoding failed: ", err)
 } else if wasNull {
	  fmt.Println("CQL value was NULL")
 } else {
	  fmt.Println("CQL value was:", value)
 }

Index

Constants

View Source 
const (
	TimeLayoutDefault = "15:04:05.999999999"

	// TimeMaxDuration is the maximum duration that can be stored in a CQL time value. Any int64 value that is lesser
	// than zero or grater than this value will be rejected.
	TimeMaxDuration = 24*time.Hour - 1
)
View Source 
const DateLayoutDefault = "2006-01-02"
View Source 
const TimestampLayoutDefault = "2006-01-02T15:04:05.999999999-07:00"

Variables

View Source 
var (
	// DateMin is the minimum representable CQL date: -5877641-06-23; it corresponds to math.MinInt32 days before the
	// Epoch.
	DateMin = ConvertEpochDaysToTime(math.MinInt32)

	// DateMax is the maximum representable CQL date: 5881580-07-11; it corresponds to math.MaxInt32 days after the
	// Epoch.
	DateMax = ConvertEpochDaysToTime(math.MaxInt32)
)
View Source 
var (
	// TimestampMin is the minimum representable CQL timestamp: -292275055-05-16 16:47:04.192 UTC.
	TimestampMin = time.Unix(-9223372036854776, 192_000_000).UTC()

	// TimestampMax is the maximum representable CQL timestamp: +292278994-08-17 07:12:55.807 UTC.
	TimestampMax = time.Unix(9223372036854775, 807_000_000).UTC()
)
View Source 
var Date = NewDate(DateLayoutDefault)

Date is a codec for the CQL date type with default layout. Its preferred Go type is time.Time, but it can encode from and decode to string and to and from most numeric types as well. When encoding from and decoding to time.Time, only the date part is considered, the clock part is ignored. Also note that all time.Time values are normalized to UTC before encoding and after decoding. Also note that not all time.Time values can be converted to CQL dates: the valid range is from -5877641-06-23 UTC to 5881580-07-11 UTC inclusive; these values can be obtained with ConvertEpochDaysToTime(math.MinInt32) and ConvertEpochDaysToTime(math.MaxInt32). When encoding from and decoding to numeric types, the numeric value represents the number of days since the Epoch. Note that a better representation for the CQL date type can be found in the civil package from cloud.google.com, see https://pkg.go.dev/cloud.google.com/go/civil.

View Source 
var ErrConversionNotSupported = errors.New("conversion not supported")
View Source 
var ErrDestinationTypeNotSupported = errors.New("destination type not supported")
View Source 
var ErrNilDataType = errors.New("data type is nil")
View Source 
var ErrNilDestination = errors.New("destination is nil")
View Source 
var ErrPointerTypeExpected = errors.New("destination is not pointer")
View Source 
var ErrSourceTypeNotSupported = errors.New("source type not supported")
View Source 
var PassThrough = Blob

PassThrough is another name for the Blob codec.

View Source 
var Time = NewTime(TimeLayoutDefault)

Time is a codec for the CQL time type with default layout. Its preferred Go type is time.Duration, but it can encode from and decode to time.Time, string and to most numeric types as well. When encoding from and decoding to time.Duration, the duration must be >= 0 and <= TimeMaxDuration, otherwise an error is returned. When encoding from and decoding to time.Time, only the clock part is considered, the date part is ignored. Also note that all time.Time values are normalized to UTC before encoding and after decoding. When encoding from and decoding to numeric types, the numeric value represents the number of nanoseconds since the beginning of the day. Note that a better representation for the CQL date type can be found in the civil package from cloud.google.com, see https://pkg.go.dev/cloud.google.com/go/civil.

View Source 
var Timestamp = NewTimestamp(TimestampLayoutDefault, time.UTC)

Timestamp is the default codec for the CQL timestamp type. Its preferred Go type is time.Time, but it can encode from and decode to string and to most numeric types as well. Note that not all time.Time values can be converted to CQL timestamp: the valid range is from TimestampMin to TimestampMax inclusive. When encoding from and decoding to numeric types, the numeric value is supposed to be the number of milliseconds since the Epoch. The layout is Cassandra's preferred layout for CQL timestamp literals and is ISO-8601-compatible.

Functions

func ConvertDurationToNanosOfDay 

func ConvertDurationToNanosOfDay(d time.Duration) (int64, error)

ConvertDurationToNanosOfDay is a function that converts from a time.Duration into nanos since the beginning of the day. An error is returned if the given time value is outside the valid range for CQL time values: from 0 to TimeMaxDuration inclusive.

func ConvertEpochDaysToTime 

func ConvertEpochDaysToTime(days int32) time.Time

ConvertEpochDaysToTime is a function that converts from days since the Epoch into a time.Time in UTC. The returned time will have its clock part set to zero. ConvertEpochDaysToTime(math.MinInt32) returns the minimum valid CQL date: -5877641-06-23 UTC. ConvertEpochDaysToTime(math.MaxInt32) returns the maximum valid CQL date: 5881580-07-11 UTC.

func ConvertEpochMillisToTime 

func ConvertEpochMillisToTime(millis int64) time.Time

ConvertEpochMillisToTime is a function that converts from milliseconds since the Epoch into a time.Time. The returned time will be in UTC.

func ConvertNanosOfDayToDuration 

func ConvertNanosOfDayToDuration(nanos int64) (time.Duration, error)

ConvertNanosOfDayToDuration is a function that converts from nanos since the beginning of the day into a time.Duration. An error is returned if the given time value is outside the valid range for CQL time values: from 0 to TimeMaxDuration inclusive.

func ConvertNanosOfDayToTime 

func ConvertNanosOfDayToTime(nanos int64) (time.Time, error)

ConvertNanosOfDayToTime is a function that converts from nanos since the beginning of the day into a time.Time in UTC. The returned time will have its date part set to 0001-01-01 and its time zone will be UTC. An error is returned if the given time value is outside the valid range for CQL time values: from 0 to TimeMaxDuration inclusive.

func ConvertTimeToEpochDays 

func ConvertTimeToEpochDays(t time.Time) (int32, error)

ConvertTimeToEpochDays is a function that converts from a time.Time into days since the Epoch. The given time is normalized to UTC before the computation. An error is returned if the given time value is outside the valid range for CQL date values: from -5877641-06-23 UTC to 5881580-07-11 UTC inclusive.

func ConvertTimeToEpochMillis 

func ConvertTimeToEpochMillis(t time.Time) (int64, error)

ConvertTimeToEpochMillis is a function that converts from a time.Time into milliseconds since the Epoch. An error is returned if the given time value cannot be converted to milliseconds since the Epoch; convertible values range from TimestampMin to TimestampMax inclusive.

func ConvertTimeToNanosOfDay 

func ConvertTimeToNanosOfDay(t time.Time) int64

ConvertTimeToNanosOfDay is a function that converts from a time.Time into nanos since the beginning of the day. The given time is normalized to UTC before the computation.

func PreferredGoType 

func PreferredGoType(dt datatype.DataType) (reflect.Type, error)

PreferredGoType returns the best matching Go type for the given data type; e.g. for the CQL type varchar it returns string, and for the CQL type timestamp it returns time.Time. Note that this function avoids pointer types unless they are absolutely required, e.g. for the CQL type varint this function returns *big.Int since the usage of the value type big.Int is not recommended.

Types

type Codec 

type Codec interface {
	Encoder
	Decoder

	DataType() datatype.DataType
}

Codec is a codec for a specific CQL type. 

var Ascii Codec = &stringCodec{datatype.Ascii}

Ascii is a codec for the CQL ascii type. Its preferred Go type is string, but it can encode from and decode to []byte and []rune as well. The returned codec does not actually enforce that all strings are valid ASCII; it's the caller's responsibility to ensure that they are valid. 

var Bigint Codec = &bigintCodec{dataType: datatype.Bigint}

Bigint is a codec for the CQL bigint type. Its preferred Go type is int64, but it can encode from and decode to most numeric types, including big.Int. Note: contrary to what the name similarity suggests, bigint codecs cannot handle all possible big.Int values; the best CQL type for handling big.Int is varint, not bigint. 

var Blob Codec = &blobCodec{dataType: datatype.Blob}

Blob is a codec for the CQL blob type. Its preferred Go type is []byte, but it can encode from and decode to string as well. When given a []byte source or destination, the encoding and decoding operations are actually no-ops, that is: the []byte value is passed along as is. 

var Boolean Codec = &booleanCodec{}

Boolean is a codec for the CQL boolean type. Its preferred Go type is bool, but it can encode from and decode to most numerical types as well (using the general convention: 0 = false, any other value = true). 

var Counter Codec = &bigintCodec{dataType: datatype.Counter}

Counter is a codec for the CQL counter type. Its preferred Go type is int64, but it can encode from and decode to most numeric types, including big.Int. Note: contrary to what the name similarity suggests, bigint codecs cannot handle all possible big.Int values; the best CQL type for handling big.Int is varint, not bigint. 

var Decimal Codec = &decimalCodec{}

Decimal is a codec for the CQL decimal type. There is no built-in representation of arbitrary-precision decimal values in Go's standard library. This is why this codec can only encode from and decode to CqlDecimal. 

var Double Codec = &doubleCodec{}

Double is a codec for the CQL double type. Its preferred Go type is float64, but it can encode from and decode to most floating-point types, including big.Float. 

var Duration Codec = &durationCodec{}

Duration is a codec for the CQL duration type, introduced in protocol v5. There is no built-in representation of arbitrary-precision duration values in Go's standard library. This is why this codec can only encode from and decode to CqlDuration. 

var Float Codec = &floatCodec{}

Float is a codec for the CQL float type. Its preferred Go type is float32, but it can encode from and decode to float64 as well. 

var Inet Codec = &inetCodec{}

Inet is a codec for the CQL inet type. Its preferred Go type is net.IP but it can encode from and decode to []byte as well. 

var Int Codec = &intCodec{}

Int is a codec for the CQL int type. Its preferred Go type is int32, but it can encode from and decode to most numeric types. 

var Smallint Codec = &smallintCodec{}

Smallint is a codec for the CQL smallint type. Its preferred Go type is int16, but it can encode from and decode to most numeric types. 

var Timeuuid Codec = &uuidCodec{dataType: datatype.Timeuuid}

Timeuuid is a codec for the CQL timeuuid type. Out of better options available in Go's standard library, its preferred Go type is primitive.Uuid, but it can encode from and decode to []byte, [16]byte and string as well. This codec does not actually enforce that user-provided UUIDs are time UUIDs; it is functionally equivalent to the codecs returned by NewUuid. When dealing with UUIDs in Go, consider using a high-level library such as Google's uuid package: https://pkg.go.dev/github.com/google/uuid. 

var Tinyint Codec = &tinyintCodec{}

Tinyint is a codec for the CQL tinyint type. Its preferred Go type is int8, but it can encode from and decode to most numeric types. 

var Uuid Codec = &uuidCodec{dataType: datatype.Uuid}

Uuid is a codec for the CQL uuid type. Out of better options available in Go's standard library, its preferred Go type is primitive.Uuid, but it can encode from and decode to []byte, [16]byte and string as well. When dealing with UUIDs in Go, consider using a high-level library such as Google's uuid package: https://pkg.go.dev/github.com/google/uuid. 

var Varchar Codec = &stringCodec{datatype.Varchar}

Varchar is a codec for the CQL varchar (or text) type. Its preferred Go type is string, but it can encode from and decode to []byte and []rune as well. 

var Varint Codec = &varintCodec{}

Varint is a codec for the CQL varint type, a type that can handle arbitrary-length integers. Its preferred Go type is big.Int, but it can encode from and decode to most numeric types.

func NewCodec 

func NewCodec(dt datatype.DataType) (Codec, error)

NewCodec creates a new codec for the given data type. For simple CQL types, this function actually returns one of the existing singletons. For complex CQL types, it delegates to one of the constructor functions available: NewList, NewSet, NewMap, NewTuple, NewUserDefined and NewCustom.

func NewCustom 

func NewCustom(customType *datatype.Custom) Codec

NewCustom returns a codec for the CQL custom type. Its preferred Go type is []byte, but it can encode from and decode to string as well. This codec is identical to the Blob codec.

func NewDate 

func NewDate(layout string) Codec

NewDate creates a new codec for the CQL date type, with the given layout. The Layout is used only when encoding from or decoding to string; it is ignored otherwise. See NewDate for important notes on accepted types.

func NewList 

func NewList(dataType *datatype.List) (Codec, error)

func NewMap 

func NewMap(dataType *datatype.Map) (Codec, error)

func NewSet 

func NewSet(dataType *datatype.Set) (Codec, error)

func NewTime 

func NewTime(layout string) Codec

NewTime creates a new codec for CQL time type, with the given layout. The Layout is used only when encoding from or decoding to string; it is ignored otherwise. See NewTime for important notes on accepted types.

func NewTimestamp 

func NewTimestamp(layout string, location *time.Location) Codec

NewTimestamp creates a new codec for CQL timestamp values, with the given layout and location. The Layout is used only when encoding from or decoding to string; it is ignored otherwise. The location is only useful if the layout does not include any time zone, in which case the time zone is assumed to be in the given location.

func NewTuple 

func NewTuple(tupleType *datatype.Tuple) (Codec, error)

func NewUserDefined 

func NewUserDefined(dataType *datatype.UserDefined) (Codec, error)

type CqlDecimal 

type CqlDecimal struct {

	// Unscaled is a big.Int representing the unscaled decimal value.
	Unscaled *big.Int

	// Scale is the decimal value scale.
	Scale int32
}

CqlDecimal is the poor man's representation in Go of a CQL decimal value, since there is no built-in representation of arbitrary-precision decimal values in Go's standard library. Note that this value is pretty useless as is. It's highly recommended converting this value to some other type using a dedicated library. The most popular one is: https://pkg.go.dev/github.com/ericlagergren/decimal/v3. The zero value of a CqlDecimal is encoded as zero, with zero scale.

type CqlDuration 

type CqlDuration struct {
	Months int32
	Days   int32
	Nanos  time.Duration
}

CqlDuration is a CQL type introduced in protocol v5. A duration can either be positive or negative. If a duration is positive all the integers must be positive or zero. If a duration is negative all the numbers must be negative or zero.

type Decoder 

type Decoder interface {

	// Decode decodes the given source into dest. The parameter dest must be a pointer to a supported Go type for
	// the CQL type being decoded; it cannot be nil. If return parameter wasNull is true, then the decoded value was a
	// NULL, in which case the actual value stored in dest will be set to its zero value.
	Decode(source []byte, dest interface{}, version primitive.ProtocolVersion) (wasNull bool, err error)
}

type Encoder 

type Encoder interface {

	// Encode encodes the given source into dest. The parameter source must be a value of a supported Go type for the
	// CQL type being encoded, or a pointer thereto; a nil value is encoded as a CQL NULL.
	Encode(source interface{}, version primitive.ProtocolVersion) (dest []byte, err error)
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.