keyval

README

package keyval

Package keyval provides a convenient method handling data in a key/value format.

Features of the keyval package

The package revolves around the KeyValue map which maps the keys to the values. The map can be created by reading the keyvals from a file or from two slices of strings, one being keys the other values. The file format has the form:

<key>: <value(s)>

When reading from a file, values can cross multiple lines in the file. Both inline and standalone comments in the keyval file are supported. Comments use the Go // syntax.

Values are stored in a struct that converts the value(s) into all the types the value supports. These can be:

• string
• int
• float64
• date (time.Time)
• []string
• []int
• []float64
• []time.Time

The struct includes a BestType field that is the "best" type the value can be. The order of precedence, in decreasing order, is:

• date (time.Time)
• int
• float64
• string

Note that slices take precedence over unary types.

Duplicate keys are allowed. If duplicates are detected, a "count" is appended to the key, starting with "1". Duplicates are numbered in the order they are found in the file. The above can cause problems if you intend to have "key", "key" and another key called "key1" -- so beware.

If the value can be parsed as a slice, leading and trailing spaces are removed after the string is split into a slice. The default delimiter for slices is ",". If you have dates like "January 2, 2000", you'll need to change it to something else.

There is one special key: include. The value associated with this key is a file name. The kevvals from the specified file are loaded when the "include" key is encountered.

There are functions to check whether required keys are present and whether extra keys are present. There is also a validation function: CheckLegals. See the example.

Date formats that are accepted are:

"20060102"
"01/02/2006"
"1/2/2006"
"January 2, 2006"
"Jan 2, 2006"

Documentation

Overview

Package keyval provides a convenient method handling data in a key/value format.

Features of the keyval package

The package revolves around the Keyval map which maps the keys to the values. The map can be created by reading the keyvals from a file or from two slices of strings, one being keys the other values. The file format has the form:

<key>: <value(s)>

When reading from a file, values can cross multiple lines in the file. Both inline and standalone comments in the keyval file are supported. Comments use the Go // syntax.

Values are stored in a struct that converts the value(s) into all the types the value supports. These can be:

• string
• int
• float64
• date (time.Time)
• []string
• []int
• []float64
• []time.Time

The struct includes a BestType field that is the "best" type the value can be. The order of precedence, in decreasing order, is:

• date (time.Time)
• int
• float64
• string

Note that slices take precedence over unary types.

Duplicate keys are allowed. If duplicates are detected, a "count" is appended to the key, starting with "1". Duplicates are numbered in the order they are found in the file. The above can cause problems if you intend to have "key", "key" *and* another key called "key1" -- so beware.

If the value can be parsed as a slice, leading and trailing spaces are removed after the string is split into a slice. The default delimiter for slices is ",". If you have dates like "January 2, 2000", you'll need to change it to something else.

There is one special key: include. The value associated with this key is a file name. The kevvals from the specified file are loaded when the "include" key is encountered.

There are functions to check whether required keys are present and whether extra keys are present. There is also a validation function: CheckLegals. See the example.

Date formats that are accepted are:

"20060102"
"01/02/2006"
"1/2/2006"
"January 2, 2006"
"Jan 2, 2006"

Index

• Variables
• func BuildLegals(legalKeys string) (keys, field, val []string)
• func CheckLegals(kv KeyVal, legalKeys string) error
• func CleanString(str, cutSet string) string
• func ReadKV2Slc(specFile string) (keys, vals []string, err error)
• type DataType
  • func (i DataType) String() string
• type KeyVal
  • func ProcessKVs(keys, vals []string) (kv KeyVal, err error)
  • func ReadKV(specFile string) (keyval KeyVal, err error)
  • func (kv KeyVal) Get(key string) *Value
  • func (kv KeyVal) GetBest(key string) (data any, datatype DataType)
  • func (kv KeyVal) GetMultiple(root string) []*Value
  • func (kv KeyVal) GetMultipleTrim(root string) []string
  • func (kv KeyVal) GetTrim(key string) string
  • func (kv KeyVal) Missing(needles string) (missing []string)
  • func (kv KeyVal) Present(needles string) (present []string)
  • func (kv KeyVal) Unknown(universe string) (novel []string)
• type Value
  • func Populate(valStr string) *Value

Examples

• CheckLegals
• ReadKV2Slc

Variables

var (
	KVDelim   = ":"  // KVDelim is the delimiter that separates the key from the value
	ListDelim = ","  // ListDelim separates list (slice) elements in the value.
	LineEOL   = "\n" // FileEOF is the end-of-line character
)

Functions

func BuildLegals

func BuildLegals(legalKeys string) (keys, field, val []string)

BuildLegals takes the string in legal.txt returning 3 slices. The first is the target key, the second is a category and the third is the value. The format for the string is: key:required-<yes/no> key:type-<string/int/float> key:multiples-<yes/no> key:requires-<another key name>

Only the first two are required.

func CheckLegals

func CheckLegals(kv KeyVal, legalKeys string) error

CheckLegals builds the legal keys, types and "required" then checks kv against this. CheckLegals returns the first error it finds in this order:

• missing required key
• bad value
• unknown keys

If you don't care about extra keys, you can just ignore the last error.

Example

This example shows how to check a keyval passes QA.

// legalDefs defines a universe of 4 keys.
// The presence of other keys will cause an error.
// If key1 or key2 must be there.
// If key3 is there, then key4 must also be there.
// keys must be of type int.
// key2 may have multiple entries, but the others may not.
const legalDefs = `
key1:required-yes
key1:type-string
key1:values-yes,no

key2:required-yes
key2:type-string
key2:multiple-yes

key3:required-no
key3:type-int
key3:requires-key4

key4:required-no
key4:type-string`

keys := []string{"key1", "key2", "key2", "key3", "key4"}
vals := []string{"yes", "first", "second", "42", "meaning"}

// After processing, key2 will be represented as key21 and key22.
keyval, err := ProcessKVs(keys, vals)
if err != nil {
	panic(err)
}

if e := CheckLegals(keyval, legalDefs); e != nil {
	panic(e)
}

fmt.Println("everything is good")

// let's see what a type error looks like.
keyval["key3"] = Populate("oh oh")
if e := CheckLegals(keyval, legalDefs); e != nil {
	fmt.Println(e)
}

keyval["key3"] = Populate("42")
delete(keyval, "key4")
if e := CheckLegals(keyval, legalDefs); e != nil {
	fmt.Println(e)
}

keyval["key4"] = Populate("I'm back")
keyval["key5"] = Populate("I'm extra")

if e := CheckLegals(keyval, legalDefs); e != nil {
	fmt.Println(e)
}

Output:

everything is good
value to key key3 must be integer
missing required key key4
unknown key(s): [key5]

func CleanString

func CleanString(str, cutSet string) string

CleanString removes all the characters in cutSet from str

func ReadKV2Slc

func ReadKV2Slc(specFile string) (keys, vals []string, err error)

ReadKV2Slc reads the specFile and returns the key/vals as two slices of strings. These can be processed into a KeyVal by ProcessKVs.

Example

This example shows the result of reading the specs1.txt file located in the data directory of this package.

dataPath := os.Getenv("data")
fileName := dataPath + "/specs1.txt"
ListDelim = ","

var (
	key, val []string
	kv       KeyVal
	e        error
)

// instead of these statements, we could use ReadKV(fileName)
if key, val, e = ReadKV2Slc(fileName); e != nil {
	panic(e)
}

kv, e = ProcessKVs(key, val)
if e != nil {
	panic(e)
}

choose := []string{"a", "b", "c", "d", "e", "f"}

for ind := 0; ind < len(choose); ind++ {
	k := choose[ind]
	v := kv[k]
	fmt.Println(k)
	fmt.Println("string: ", v.AsString)
	if v.AsInt != nil {
		fmt.Println("int: ", *v.AsInt)
	}
	if v.AsFloat != nil {
		fmt.Println("float: ", *v.AsFloat)
	}
	if v.AsSliceS != nil {
		fmt.Println("slice: ", v.AsSliceS)
	}
	fmt.Println("best: ", v.BestType)
	fmt.Println()
}

Output:

a
string:  hello
slice:  [hello]
best:  String

b
string:  a,b,c, d,e,f
slice:  [a b c d e f]
best:  SliceStr

c
string:  1
int:  1
float:  1
slice:  [1]
best:  Int

d
string:  3.2
float:  3.2
slice:  [3.2]
best:  Float

e
string:  1,2,3,4
slice:  [1 2 3 4]
best:  SliceInt

f
string:  1.1, 3,4,5,8.9
slice:  [1.1 3 4 5 8.9]
best:  SliceFloat

Types

type DataType

type DataType int

DataType is used to identify the "best" data type of the value. The decreasing order of precedence is:

• slices
• unary types

Within each of these types, the order is:

• int
• float
• string

const (
	String DataType = 0 + iota
	Float
	Int
	Date
	SliceStr
	SliceFloat
	SliceInt
	SliceDate
	InValid
)

func (DataType) String

func (i DataType) String() string

type KeyVal

type KeyVal map[string]*Value

KeyVal holds the map representation of the keyval file.

func ProcessKVs

func ProcessKVs(keys, vals []string) (kv KeyVal, err error)

ProcessKVs process keys and vals as two slices of string. It returns a KeyVal.

func ReadKV

func ReadKV(specFile string) (keyval KeyVal, err error)

ReadKV reads a key/val set from specFile and returns KeyVal

func (KeyVal) Get

func (kv KeyVal) Get(key string) *Value

Get returns a value. Nil is returned if the "want" DataType is not a legal type.

func (KeyVal) GetBest

func (kv KeyVal) GetBest(key string) (data any, datatype DataType)

GetBest returns the Value element of the BestType along with what that type is.

func (KeyVal) GetMultiple

func (kv KeyVal) GetMultiple(root string) []*Value

GetMultiple retrieves all the Values that start with root that have duplicate keys. The actual keys would be "root"1, "root"2, .... The keys are returned in order.

func (KeyVal) GetMultipleTrim

func (kv KeyVal) GetMultipleTrim(root string) []string

GetMultipleTrim returns a multiple key as a trimmed string slice

func (KeyVal) GetTrim

func (kv KeyVal) GetTrim(key string) string

GetTrim returns the key as a string with spaces trimmed

func (KeyVal) Missing

func (kv KeyVal) Missing(needles string) (missing []string)

Missing returns a slice of needles that are not keys in kv. needles is a comma-separated list of keys to look for. returns nil if all needles are present.

func (KeyVal) Present

func (kv KeyVal) Present(needles string) (present []string)

Present returns the keys in needles that are in kv.

func (KeyVal) Unknown

func (kv KeyVal) Unknown(universe string) (novel []string)

Unknown returns the keys in kv that are not in universe. universe is a comma-separated string that has the universe of known keys. returns nil if all keys in kv are in universe. Any entry in universe that ends in * is treated as a wildcard

type Value

type Value struct {
	AsString string
	AsInt    *int
	AsFloat  *float64
	AsDate   *time.Time
	AsSliceS []string
	AsSliceI []int
	AsSliceF []float64
	AsSliceD []time.Time
	BestType DataType
}

The Value struct holds the val part of the keyval. All legal elements are populated.

func Populate

func Populate(valStr string) *Value

Populate populates all the legal values that valStr can accommodate. The AsString field is always populated. The BestType is set using the order of precedence described under the type DataType.