datatype

Documentation

Overview

Package datatype contains necessary logic to sanitise a JSON object coming from a reader. This package is subjected to change.

Index

• Constants
• Variables
• type ByteType
  • func (b ByteType) Equal(other DataType) bool
  • func (b ByteType) String() string
• type Container
  • func NewContainer(list []DataType) *Container
  • func (c *Container) Add(d ...DataType)
  • func (c *Container) Error() error
  • func (c *Container) Len() int
  • func (c *Container) List() []DataType
  • func (c *Container) String(timestamp time.Time) string
• type DataContainer
  • func JobResultDataTypes(r io.Reader, mapper Mapper) DataContainer
• type DataType
  • func FromJason(key string, value jason.Value) (DataType, error)
• type FloatListType
  • func (fl FloatListType) Equal(other DataType) bool
  • func (fl FloatListType) String() string
• type FloatType
  • func (f FloatType) Equal(other DataType) bool
  • func (f FloatType) String() string
• type GCListType
  • func (flt GCListType) Equal(other DataType) bool
  • func (flt GCListType) String() string
• type KiloByteType
  • func (k KiloByteType) Equal(other DataType) bool
  • func (k KiloByteType) String() string
• type MapConvert
  • func DefaultMapper() *MapConvert
  • func MapsFromViper(v *viper.Viper) *MapConvert
  • func (m *MapConvert) Values(prefix string, values map[string]*jason.Value) []DataType
• type MapConvertMock
  • func (m *MapConvertMock) Values(prefix string, values map[string]*jason.Value) []DataType
• type Mapper
• type MegaByteType
  • func (m MegaByteType) Equal(other DataType) bool
  • func (m MegaByteType) String() string
• type MemTypeMock
  • func (m MemTypeMock) IsByte() bool
  • func (m MemTypeMock) IsKiloByte() bool
  • func (m MemTypeMock) IsMegaByte() bool
• type StringType
  • func (s StringType) Equal(other DataType) bool
  • func (s StringType) String() string

Examples

• FromJason (FloatType)
• FromJason (MalformedInput)
• FromJason (StringType)

Constants

const (
	// BYTE ..
	BYTE = 1.0
	// KILOBYTE ..
	KILOBYTE = 1024 * BYTE
	// MEGABYTE ..
	MEGABYTE = 1024 * KILOBYTE
)

Variables

var ErrUnidentifiedJason = errors.New("unidentified jason value")

ErrUnidentifiedJason .

Types

type ByteType

type ByteType struct {
	Key   string
	Value float64
}

ByteType represents a pair of key values in which the value represents bytes It converts the value to MB

func (ByteType) Equal

func (b ByteType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal

func (ByteType) String

func (b ByteType) String() string

String satisfies the Stringer interface

type Container

type Container struct {
	Err error
	// contains filtered or unexported fields
}

Container satisfies the DataContainer and error interfaces

func NewContainer

func NewContainer(list []DataType) *Container

NewContainer returns a new container

func (*Container) Add

func (c *Container) Add(d ...DataType)

Add adds to the list

func (*Container) Error

func (c *Container) Error() error

Error returns the error

func (*Container) Len

func (c *Container) Len() int

Len returns the length of the data

func (*Container) List

func (c *Container) List() []DataType

List returns the data The error is not provided here, please check the Err value

func (*Container) String

func (c *Container) String(timestamp time.Time) string

String prepends the timestamp to the list, and generates a json object suitable for recording into a document store. TODO: also provide JSONDeocder encoder TODO: a lot of data doesn't require processing, use them as they are

type DataContainer

type DataContainer interface {
	// Returns the list. You should not update this list as it is a shared list and anyone can read from it.
	// If you append to this list, there is a chance you are not referring to the same underlying array in memory.
	List() []DataType
	Len() int

	String(timestamp time.Time) string
	// Returns the Err value
	Error() error
}

DataContainer is an interface for holding a list of DataType. I'm aware of the container/list package, which is awesome, but I needed to guard this with a mutex. To iterate over this container:

for i := 0; i < d.Len(); i++ {
    item := d.Get(i)
}

func JobResultDataTypes

func JobResultDataTypes(r io.Reader, mapper Mapper) DataContainer

JobResultDataTypes generates a list of DataType and puts them inside the DataContainer TODO: bypass the operation that won't be converted in any way. They are not supposed to be read and converted back. TODO: this operation can happen only once. Lazy load the thing.

type DataType

type DataType interface {
	fmt.Stringer

	// Equal compares both keys and values and returns true if they are equal
	Equal(DataType) bool
}

DataType implements Stringer and Marshal/Unmarshal

func FromJason

func FromJason(key string, value jason.Value) (DataType, error)

FromJason returns an instance of DataType from jason value

Example (FloatType)

package main

import (
	"fmt"
	"reflect"

	"github.com/antonholmquist/jason"
	"github.com/arsham/expvastic/datatype"
)

func main() {
	j, _ := jason.NewValueFromBytes([]byte("666.6"))
	result, err := datatype.FromJason("some float", *j)
	fmt.Printf("error: %v\n", err)
	fmt.Printf("Type: %v\n", reflect.TypeOf(result))
	r := result.(*datatype.FloatType)
	fmt.Printf("Result key: %s\n", r.Key)
	fmt.Printf("Result value: %f\n", r.Value)
	fmt.Printf("String representation: %s\n", result.String())

}

Output:

error: <nil>
Type: *datatype.FloatType
Result key: some float
Result value: 666.600000
String representation: "some float":666.600000

Example (MalformedInput)

package main

import (
	"fmt"
	"reflect"

	"github.com/antonholmquist/jason"
	"github.com/arsham/expvastic/datatype"
)

func main() {
	j, _ := jason.NewValueFromBytes([]byte(`{malformed object}`))
	result, err := datatype.FromJason("ignored", *j)
	fmt.Printf("error: %v\n", err)
	fmt.Printf("Type: %v\n", reflect.TypeOf(result))
}

Output:

error: unidentified jason value
Type: <nil>

Example (StringType)

package main

import (
	"fmt"
	"reflect"

	"github.com/antonholmquist/jason"
	"github.com/arsham/expvastic/datatype"
)

func main() {
	j, _ := jason.NewValueFromBytes([]byte(`"some string"`))
	result, err := datatype.FromJason("string key", *j)
	fmt.Printf("error: %v\n", err)
	fmt.Printf("Type: %v\n", reflect.TypeOf(result))
	r := result.(*datatype.StringType)
	fmt.Printf("Result key: %s\n", r.Key)
	fmt.Printf("Result value: %s\n", r.Value)
	fmt.Printf("String representation: %s\n", result.String())
}

Output:

error: <nil>
Type: *datatype.StringType
Result key: string key
Result value: some string
String representation: "string key":"some string"

type FloatListType

type FloatListType struct {
	Key   string
	Value []float64
}

FloatListType represents a pair of key values that the value is a list of floats

func (FloatListType) Equal

func (fl FloatListType) Equal(other DataType) bool

Equal compares both keys and all values and returns true if they are equal. The values are checked in an unordered fashion.

func (FloatListType) String

func (fl FloatListType) String() string

String satisfies the Stringer interface

type FloatType

type FloatType struct {
	Key   string
	Value float64
}

FloatType represents a pair of key values that the value is a float64

func (FloatType) Equal

func (f FloatType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal Not implemented

func (FloatType) String

func (f FloatType) String() string

String satisfies the Stringer interface

type GCListType

type GCListType struct {
	Key   string
	Value []uint64
}

GCListType represents a pair of key values of GC list info

func (GCListType) Equal

func (flt GCListType) Equal(other DataType) bool

Equal is not implemented. You should iterate and check yourself. Equal compares both keys and values and returns true if they are equal

func (GCListType) String

func (flt GCListType) String() string

String satisfies the Stringer interface

type KiloByteType

type KiloByteType struct {
	Key   string
	Value float64
}

KiloByteType represents a pair of key values in which the value represents bytes It converts the value to MB

func (KiloByteType) Equal

func (k KiloByteType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal

func (KiloByteType) String

func (k KiloByteType) String() string

String satisfies the Stringer interface

type MapConvert

type MapConvert struct {
	// contains filtered or unexported fields
}

MapConvert can produce output from the defined types.

func DefaultMapper

func DefaultMapper() *MapConvert

DefaultMapper returns a MapConvert object that is populated by the default mappings.

func MapsFromViper

func MapsFromViper(v *viper.Viper) *MapConvert

MapsFromViper reads from the map file and produces functions for conversion used in type decoder. It first reads from the default settings defined in the maps.yml in the same folder, then overrides with the user specified mappings.

func (*MapConvert) Values

func (m *MapConvert) Values(prefix string, values map[string]*jason.Value) []DataType

Values returns a slice of DataTypes based on the given name/value inputs. It flattens the float list values, therefore you will get multiple values per input. If the name is found in memory_bytes map, it will return one of those, otherwise it will return a FloatType or StringType if can convert. It will return nil if the value is not one of above.

type MapConvertMock

type MapConvertMock struct {
	GCTypes         []string
	MemoryTypes     map[string]MemTypeMock
	ValuesFunc      func(prefix string, values map[string]*jason.Value) []DataType
	DefaultCovertor Mapper
}

MapConvertMock is the mocked version of MapConvert

func (*MapConvertMock) Values

func (m *MapConvertMock) Values(prefix string, values map[string]*jason.Value) []DataType

Values calls the ValuesFunc if exists, otherwise returns nil

type Mapper

type Mapper interface {
	// Values closes the channel once all input has been exhausted.
	Values(prefix string, values map[string]*jason.Value) []DataType
}

Mapper generates DataTypes based on the given name/value inputs.

type MegaByteType

type MegaByteType struct {
	Key   string
	Value float64
}

MegaByteType represents a pair of key values in which the value represents bytes It converts the value to MB

func (MegaByteType) Equal

func (m MegaByteType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal

func (MegaByteType) String

func (m MegaByteType) String() string

String satisfies the Stringer interface

type MemTypeMock

type MemTypeMock struct {
	// contains filtered or unexported fields
}

MemTypeMock is the mocked version of memType

func (MemTypeMock) IsByte

func (m MemTypeMock) IsByte() bool

func (MemTypeMock) IsKiloByte

func (m MemTypeMock) IsKiloByte() bool

func (MemTypeMock) IsMegaByte

func (m MemTypeMock) IsMegaByte() bool

type StringType

type StringType struct {
	Key   string
	Value string
}

StringType represents a pair of key values that the value is a string

func (StringType) Equal

func (s StringType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal

func (StringType) String

func (s StringType) String() string

String satisfies the Stringer interface