data

package
v0.37.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 22, 2024 License: Apache-2.0 Imports: 13 Imported by: 58

README

Data utilities

Expandable Map & Collection

Expandable structure enable nested data structure to substitution source for other data structure or text. Data substitution expression starts with $ sign, you can use path where dot or [index] allows access data sub node.

ExpandAsText

ExpandAsText expands any expression that has satisfied dependencies, meaning only expression that path is present can be expanded, otherwise expression is left unchanged. In case when UDF is used, expression is expanded if UDF does not return an error.

Usage:

    aMap := Map(map[string]interface{}{
		"key1": 1,
		"key2": map[string]interface{}{
			"subKey1":10,
			"subKey2":20,
		},
		"key3": "subKey2",
		"array": []interface{}{
			111, 222, 333,
		},
		"slice": []interface{}{
			map[string]interface{}{
				"attr1":111,
				"attr2":222,
			},
		},
	})
	expandedText := aMap.ExpandAsText(`1: $key1, 
2: ${array[2]}  
3: $key2.subKey1 
4: $key2[$key3] ${slice[0].attr1}  
5: ${(key1 + 1) * 3} 
6: $abc
7: end
`)
	
	
/* expands to 
1: 1, 
2: 333  
3: 10 
4: 20 111  
5: 6 
6: $abc
7: end
*/

Expand arbitrary data structure

    aMap := Map(map[string]interface{}{
		"key1": 1,
		"key2": map[string]interface{}{
			"subKey1":10,
			"subKey2":20,
		},
		"key3": "subKey2",
		"array": []interface{}{
			111, 222, 333,
		},
		"slice": []interface{}{
			map[string]interface{}{
				"attr1":111,
				"attr2":222,
			},
		},
	})
    
    data := map[string]interface{}{
    	"k1": "$key1",
    	"k2":"$array",
    	"k3":"$key2",
    }
    expanded := aMap.Expand(data)
    /* expands to
        map[string]interface{}{
        	"k1": 1,
        	"k2": []interface{}{111, 222, 333},
        	"k3": map[string]interface{}{
                "subKey1":10,
                "subKey2":20,
            },
        }
    */

UDF expandable User defined function

You can add dynamic data substitution by registering function in top level map.

type Udf func(interface{}, Map) (interface{}, error)

        aMap: data.Map(map[string]interface{}{
            "dailyCap":   100,
            "overallCap": 2,
            "AsFloat": func(source interface{}, state Map) (interface{}, error) {
                return toolbox.AsFloat(source), nil
            },
        })

        expanded := aMap.Expand("$AsFloat($dailyCap)")
        //expands to actual float: 100.0

Predefined UDF

Compacted slice

Using a generic data structure in a form []map[string]interface{} is extremely memory inefficient, CompactedSlice addresses the memory inefficiency by storing the new item values in a slice and by mapping corresponding fields to the item slice index positions. On top of that any neighboring nil values can be compacted too.

Usage


    collection := NewCompactedSlice(true, true)

    for i := 0;i<10;i++ {
        collection.Add(map[string]interface{}{
            "f1":  i+1,
            "f12": i+10,
            "f15": i*20,
            "f20": i+4,
            "f11": nil,
            "f12": nil,
            "f13": nil,
            "f14": "",
        })
	}
    
    collection.Range(func(data interface{}) (bool, error) {
        actual = append(actual, toolbox.AsMap(data))
        return true, nil
    })

Documentation

Index

Constants

View Source 
const (
	UDFKey = "_udf"
)

Variables

This section is empty.

Functions

func ExtractPath 

func ExtractPath(expression string) string

func Parse 

func Parse(expression string, handler func(expression string, isUDF bool, argument interface{}) (interface{}, bool)) interface{}

Types

type Collection 

type Collection []interface{}

Collection represents a slice of interface{} (generic type)

func NewCollection 

func NewCollection() *Collection

NewCollection creates a new collection and returns a pointer

func (*Collection) PadWithMap 

func (s *Collection) PadWithMap(size int)

PadWithMap creates missing elements with a map

func (*Collection) Push 

func (s *Collection) Push(value interface{})

Push appends provided value to the slice

func (*Collection) Range 

func (s *Collection) Range(handler func(item interface{}, index int) (bool, error)) error

Range iterates over every item in this collection as long as handler returns true. Handler takes an index and index of the slice element.

func (*Collection) RangeInt 

func (s *Collection) RangeInt(handler func(item interface{}, index int) (bool, error)) error

RangeMap iterates every int item in this collection as long as handler returns true. Handler takes an index and index of the slice element

func (*Collection) RangeMap 

func (s *Collection) RangeMap(handler func(item Map, index int) (bool, error)) error

RangeMap iterates every map item in this collection as long as handler returns true. Handler takes an index and index of the slice element

func (*Collection) RangeString 

func (s *Collection) RangeString(handler func(item interface{}, index int) (bool, error)) error

RangeMap iterates every string item in this collection as long as handler returns true. Handler takes an index and index of the slice element

func (*Collection) String 

func (s *Collection) String() string

String returns a string representation of this collection

type CompactedSlice 

type CompactedSlice struct {
	RawEncoding bool
	// contains filtered or unexported fields
}

CompactedSlice represented a compacted slice to represent object collection

func NewCompactedSlice 

func NewCompactedSlice(omitEmpty, compressNils bool) *CompactedSlice

NewCompactedSlice create new compacted slice

func (*CompactedSlice) Add 

func (s *CompactedSlice) Add(data map[string]interface{})

Add adds data to a collection

func (*CompactedSlice) Fields added in v0.18.0 

func (s *CompactedSlice) Fields() []*Field

func (*CompactedSlice) Iterator added in v0.6.6 

func (s *CompactedSlice) Iterator() toolbox.Iterator

Iterator returns a slice iterator

func (CompactedSlice) MarshalJSON added in v0.29.2 

func (d CompactedSlice) MarshalJSON() ([]byte, error)

func (*CompactedSlice) Range 

func (s *CompactedSlice) Range(handler func(item interface{}) (bool, error)) error

Range iterate over slice, and remove processed data from the compacted slice

func (*CompactedSlice) Ranger added in v0.18.0 

func (s *CompactedSlice) Ranger() toolbox.Ranger

Ranger moves data from slice to ranger

func (*CompactedSlice) Size 

func (s *CompactedSlice) Size() int

Size returns size of collection

func (*CompactedSlice) SortedIterator added in v0.6.6 

func (s *CompactedSlice) SortedIterator(indexBy []string) (toolbox.Iterator, error)

SortedIterator returns sorted iterator

func (*CompactedSlice) SortedRange added in v0.6.6 

func (s *CompactedSlice) SortedRange(indexBy []string, handler func(item interface{}) (bool, error)) error

SortedRange sort collection by supplied index and then call for each item supplied handler callback

type Field added in v0.18.0 

type Field struct {
	Name string
	Type reflect.Type
	// contains filtered or unexported fields
}

type Map 

type Map map[string]interface{}

Map types is an alias type to map[string]interface{} with extended behaviours

func NewMap 

func NewMap() Map

NewMap creates a new instance of a map.

func (*Map) Apply 

func (s *Map) Apply(source map[string]interface{})

Apply copies all elements of provided map to this map.

func (*Map) AsEncodableMap 

func (s *Map) AsEncodableMap() map[string]interface{}

Returns a map that can be encoded by json or other decoders. Since a map can store a function, it filters out any non marshalable types.

func (*Map) Clone 

func (s *Map) Clone() Map

Clones create a clone of this map.

func (*Map) Delete 

func (s *Map) Delete(keys ...string)

Delete removes the supplied keys, it supports key of path expression with dot i.e. request.method

func (*Map) Expand 

func (s *Map) Expand(source interface{}) interface{}

Expand expands provided value of any type with dollar sign expression/

func (*Map) ExpandAsText 

func (s *Map) ExpandAsText(text string) string

ExpandAsText expands all matching expressions starting with dollar sign ($)

func (*Map) ExpandWithoutUDF added in v0.35.0 

func (s *Map) ExpandWithoutUDF(text string) string

ExpandWithoutUDF expands text like ExpandAsText but keeps quotes for function arguments

func (*Map) Get 

func (s *Map) Get(key string) interface{}

Get returns a value for provided key

func (*Map) GetBoolean 

func (s *Map) GetBoolean(key string) bool

GetBoolean returns value for provided key as boolean.

func (*Map) GetCollection 

func (s *Map) GetCollection(key string) *Collection

GetCollection returns value for provided key as collection pointer.

func (*Map) GetFloat 

func (s *Map) GetFloat(key string) float64

GetFloat returns value for provided key as float64.

func (*Map) GetInt 

func (s *Map) GetInt(key string) int

GetInt returns value for provided key as int.

func (*Map) GetMap 

func (s *Map) GetMap(key string) Map

GetMap returns value for provided key as map.

func (*Map) GetString 

func (s *Map) GetString(key string) string

GetString returns value for provided key as string.

func (*Map) GetValue 

func (s *Map) GetValue(expr string) (interface{}, bool)

GetValue returns value for provided expression. The expression uses dot (.) to denote nested data structure. The following expression as supported

  1. <-key shift
  2. ++key pre increment
  3. key++ post increment
  4. $key reference access

func (*Map) Has 

func (s *Map) Has(key string) bool

Has returns true if the provided key is present

func (*Map) Put 

func (s *Map) Put(key string, value interface{})

Put puts key value into the map.

func (*Map) Range 

func (s *Map) Range(callback func(k string, v interface{}) (bool, error)) error

Range iterates every key, value pair of this map, calling supplied callback as long it does return true.

func (*Map) Replace 

func (s *Map) Replace(key, val string)

Replace replaces supplied key/path with corresponding value

func (*Map) SetValue 

func (s *Map) SetValue(expr string, value interface{})

Set value sets value in the map for the supplied expression. The expression uses dot (.) to denote nested data structure. For instance the following expression key1.subKey1.attr1 Would take or create map for key1, followied bu takeing or creating antother map for subKey1 to set attr1 key with provided value The following expression as supported

  1. $key reference - the final key is determined from key's content.
  2. ->key push expression to append value at the end of the slice

type Udf 

type Udf func(interface{}, Map) (interface{}, error)

Udf represents a user defined function used to transform data.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.