README
¶
smapping
Golang structs generic mapping.
Version Limit
To support nesting object conversion, the lowest Golang version supported is 1.12.0.
To support smapping.SQLScan, the lowest Golang version supported is 1.13.0.
Table of Contents
At Glimpse
What?
A library to provide a mapped structure generically/dynamically.
Who?
Anyone who has to work with large structure.
Why?
Scalability and Flexibility.
When?
At the runtime.
Where?
In users code.
How?
By converting into smapping.Mapped which alias for map[string]interface{},
users can iterate the struct arbitarily with reflect package.
Motivation
Working with between struct, and json with Golang has various
degree of difficulty.
The thing that makes difficult is that sometimes we get arbitrary json
or have to make json with arbitrary fields. Sometime we also need to have
a different field names, extracting specific fields, working with same
structure with different domain fields name etc.
In order to answer those flexibility, we map the object struct to the more general data structure as table/map.
Table/Map is the data structure which ubiquitous after list, which in turn table/map can be represented as list of pair values (In Golang we can't have it because there's no tuple data type, tuple is limited as return values).
Object can be represented as table/map dynamically just like in JavaScript/EcmaScript which object is behaving like table and in Lua with its metatable. By some extent we can represent the JSON as table too.
In this library, we provide the mechanism to smoothly map the object
representation back-and-forth without having the boilerplate of
type-checking one by one by hand. Type-checking by hand is certainly
seems easier when the domain set is small, but it soon becomes
unbearable as the structure and/or architecure dynamically changed
because of newer insight and information. Hence in Who section
mentioned this library is for anyone who has to work with large
domain set.
Except for type smapping.Mapped as alias, we don't provide others
type struct currently as each operation doesn't need to keep the
internal state so each operation is transparent and almost functional
(almost functional because we modify the struct fields values instead of
returning the new struct itself, but this is only trade-off because Golang
doesn't have type-parameter which known as generic).
Since v0.1.10, we added the MapEncoder and
MapDecoder interfaces for users to have custom conversion
for custom and self-defined struct.
Install
go get -u github.com/Kirill-Znamenskiy/smapping
Examples
Basic usage examples
Below example are basic representation how we can work with smapping.
Several examples are converged into single runnable example for the ease
of reusing the same structure definition and its various tags.
Refer this example to get a glimpse of how to do things. Afterward,
users can creatively use to accomplish what they're wanting to do
with the provided flexibility.
package main
import (
"encoding/json"
"fmt"
"github.com/Kirill-Znamenskiy/smapping"
)
type Source struct {
Label string `json:"label"`
Info string `json:"info"`
Version int `json:"version"`
}
type Sink struct {
Label string
Info string
}
type HereticSink struct {
NahLabel string `json:"label"`
HahaInfo string `json:"info"`
Version string `json:"heretic_version"`
}
type DifferentOneField struct {
Name string `json:"name"`
Label string `json:"label"`
Code string `json:"code"`
Private string `json:"private" api:"internal"`
}
func main() {
source := Source{
Label: "source",
Info: "the origin",
Version: 1,
}
fmt.Println("source:", source)
mapped := smapping.MapFields(source)
fmt.Println("mapped:", mapped)
sink := Sink{}
err := smapping.FillStruct(&sink, mapped)
if err != nil {
panic(err)
}
fmt.Println("sink:", sink)
maptags := smapping.MapTags(source, "json")
fmt.Println("maptags:", maptags)
hereticsink := HereticSink{}
err = smapping.FillStructByTags(&hereticsink, maptags, "json")
if err != nil {
panic(err)
}
fmt.Println("heretic sink:", hereticsink)
fmt.Println("=============")
recvjson := []byte(`{"name": "bella", "label": "balle", "code": "albel", "private": "allbe"}`)
dof := DifferentOneField{}
_ = json.Unmarshal(recvjson, &dof)
fmt.Println("unmarshaled struct:", dof)
marshaljson, _ := json.Marshal(dof)
fmt.Println("marshal back:", string(marshaljson))
// What we want actually "internal" instead of "private" field
// we use the api tags on to make the json
apijson, _ := json.Marshal(smapping.MapTagsWithDefault(dof, "api", "json"))
fmt.Println("api marshal:", string(apijson))
fmt.Println("=============")
// This time is the reverse, we receive "internal" field when
// we need to receive "private" field to match our json tag field
respjson := []byte(`{"name": "bella", "label": "balle", "code": "albel", "internal": "allbe"}`)
respdof := DifferentOneField{}
_ = json.Unmarshal(respjson, &respdof)
fmt.Println("unmarshal resp:", respdof)
// to get that, we should put convert the json to Mapped first
jsonmapped := smapping.Mapped{}
_ = json.Unmarshal(respjson, &jsonmapped)
// now we fill our struct respdof
_ = smapping.FillStructByTags(&respdof, jsonmapped, "api")
fmt.Println("full resp:", respdof)
returnback, _ := json.Marshal(respdof)
fmt.Println("marshal resp back:", string(returnback))
// first we unmarshal respdof, we didn't get the "private" field
// but after our mapping, we get "internal" field value and
// simply marshaling back to `returnback`
}
Nested object example
This example illustrates how we map back-and-forth even with deep
nested object structure. The ability to map nested objects is to
creatively change its representation whether to flatten all tagged
field name even though the inner struct representation is nested.
Regardless of the usage (whether to flatten the representation) or
just simply fetching and remapping into different domain name set,
the ability to map the nested object is necessary.
type RefLevel3 struct {
What string `json:"finally"`
}
type Level2 struct {
*RefLevel3 `json:"ref_level3"`
}
type Level1 struct {
Level2 `json:"level2"`
}
type TopLayer struct {
Level1 `json:"level1"`
}
type MadNest struct {
TopLayer `json:"top"`
}
var madnestStruct MadNest = MadNest{
TopLayer: TopLayer{
Level1: Level1{
Level2: Level2{
RefLevel3: &RefLevel3{
What: "matryoska",
},
},
},
},
}
func main() {
// since we're targeting the same MadNest, both of functions will yield
// same result hence this unified example/test.
var madnestObj MadNest
var err error
testByTags := true
if testByTags {
madnestMap := smapping.MapTags(madnestStruct, "json")
err = smapping.FillStructByTags(&madnestObj, madnestMap, "json")
} else {
madnestMap := smapping.MapFields(madnestStruct)
err = smapping.FillStruct(&madnestObj)
}
if err != nil {
fmt.Printf("%s", err.Error())
return
}
// the result should yield as intented value.
if madnestObj.TopLayer.Level1.Level2.RefLevel3.What != "matryoska" {
fmt.Printf("Error: expected \"matroska\" got \"%s\"", madnestObj.Level1.Level2.RefLevel3.What)
}
}
SQLScan usage example
This example, we're using sqlite3 as the database, we add a convenience
feature for any struct/type that implements Scan method as smapping.SQLScanner.
Keep in mind this is quite different with sql.Scanner that's also requiring
the type/struct to implement Scan method. The difference here, smapping.SQLScanner
receiving variable arguments of interface{} as values' placeholder while sql.Scanner
is only receive a single interface{} argument as source. smapping.SQLScan is
working for Scan literally after we've gotten the *sql.Row or *sql.Rows.
package main
import (
"database/sql"
"encoding/json"
"fmt"
"github.com/Kirill-Znamenskiy/smapping"
_ "github.com/mattn/go-sqlite3"
)
type book struct {
Author author `json:"author"`
}
type author struct {
Num int `json:"num"`
ID sql.NullString `json:"id"`
Name sql.NullString `json:"name"`
}
func (a author) MarshalJSON() ([]byte, error) {
mapres := map[string]interface{}{}
if !a.ID.Valid {
//if a.ID == nil || !a.ID.Valid {
mapres["id"] = nil
} else {
mapres["id"] = a.ID.String
}
//if a.Name == nil || !a.Name.Valid {
if !a.Name.Valid {
mapres["name"] = nil
} else {
mapres["name"] = a.Name.String
}
mapres["num"] = a.Num
return json.Marshal(mapres)
}
func getAuthor(db *sql.DB, id string) author {
res := author{}
err := db.QueryRow("select * from author where id = ?", id).
Scan(&res.Num, &res.ID, &res.Name)
if err != nil {
panic(err)
}
return res
}
func getAuthor12(db *sql.DB, id string) author {
result := author{}
fields := []string{"num", "id", "name"}
err := smapping.SQLScan(
db.QueryRow("select * from author where id = ?", id),
&result,
"json",
fields...)
if err != nil {
panic(err)
}
return result
}
func getAuthor13(db *sql.DB, id string) author {
result := author{}
fields := []string{"num", "name"}
err := smapping.SQLScan(
db.QueryRow("select num, name from author where id = ?", id),
&result,
"json",
fields...)
if err != nil {
panic(err)
}
return result
}
func getAllAuthor(db *sql.DB) []author {
result := []author{}
rows, err := db.Query("select * from author")
if err != nil {
panic(err)
}
for rows.Next() {
res := author{}
if err := smapping.SQLScan(rows, &res, "json"); err != nil {
fmt.Println("error scan:", err)
break
}
result = append(result, res)
}
return result
}
func main() {
db, err := sql.Open("sqlite3", "./dummy.db")
if err != nil {
panic(err)
}
defer db.Close()
_, err = db.Exec(`
drop table if exists author;
create table author(num integer primary key autoincrement, id text, name text);
insert into author(id, name) values
('id1', 'name1'),
('this-nil', null);`)
if err != nil {
panic(err)
}
//auth1 := author{ID: &sql.NullString{String: "id1"}}
auth1 := author{ID: sql.NullString{String: "id1"}}
auth1 = getAuthor(db, auth1.ID.String)
fmt.Println("auth1:", auth1)
jsonbyte, _ := json.Marshal(auth1)
fmt.Println("json auth1:", string(jsonbyte))
b1 := book{Author: auth1}
fmt.Println(b1)
jbook1, _ := json.Marshal(b1)
fmt.Println("json book1:", string(jbook1))
auth2 := getAuthor(db, "this-nil")
fmt.Println("auth2:", auth2)
jbyte, _ := json.Marshal(auth2)
fmt.Println("json auth2:", string(jbyte))
b2 := book{Author: auth2}
fmt.Println("book2:", b2)
jbook2, _ := json.Marshal(b2)
fmt.Println("json book2:", string(jbook2))
fmt.Println("author12:", getAuthor12(db, auth1.ID.String))
fmt.Println("author13:", getAuthor13(db, auth1.ID.String))
fmt.Println("all author1:", getAllAuthor(db))
}
Omit fields example
Often we need to reuse the same object with exception a field or two. With smapping it's possible to generate
map with custom tag. However having different tag would be too much of manual work.
In this example, we'll see how to exclude using the delete keyword.
package main
import (
"github.com/Kirill-Znamenskiy/smapping"
)
type Struct struct {
Field1 int `json:"field1"`
Field2 string `json:"field2"`
RequestOnly string `json:"input"`
ResponseOnly string `jsoN:"output"`
}
func main() {
s := Struct{
Field1: 5,
Field2: "555",
RequestOnly: "vanish later",
ResponseOnly: "still available",
}
m := smapping.MapTags(s, "json")
_, ok := m["input"]
if !ok {
panic("key 'input' should be still available")
}
delete(m, "input")
_, ok = m["input"]
if ok {
panic("key 'input' should be not available")
}
}
LICENSE
MIT
Documentation
¶
Index ¶
- func FillStruct(obj interface{}, mapped Mapped) error
- func FillStructByTags(obj interface{}, mapped Mapped, tagname string) error
- func FillStructDeflate(obj interface{}, mapped Mapped, tagname string) error
- func SQLScan(row SQLScanner, obj interface{}, tag string, x ...string) error
- func SetFieldFromTag(obj interface{}, tagName, tagValue string, value interface{}, ...) (bool, error)
- type MapDecoder
- type MapEncoder
- type Mapped
- type SQLScanner
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func FillStruct ¶
FillStruct acts just like “json.Unmarshal“ but works with “Mapped“ instead of bytes of char that made from “json“.
Example ¶
mapped := MapFields(&sourceobj)
sinked := sink{}
err := FillStruct(&sinked, mapped)
if err != nil {
panic(err)
}
fmt.Println(sinked)
Output: {source the origin}
func FillStructByTags ¶
FillStructByTags fills the field that has tagname and tagvalue instead of Mapped key name.
Example ¶
maptags := MapTags(&sourceobj, "json")
for k, v := range maptags {
if vt, ok := v.(time.Time); ok {
fmt.Printf("maptags[%s]: %s\n", k, vt.Format(time.RFC3339))
} else {
fmt.Printf("maptags[%s]: %v\n", k, v)
}
}
diffsink := differentSink{}
err := FillStructByTags(&diffsink, maptags, "json")
if err != nil {
panic(err)
}
fmt.Println(diffsink.DiffLabel)
fmt.Println(diffsink.NiceInfo)
fmt.Println(diffsink.Version)
fmt.Println(diffsink.Toki)
fmt.Println(*diffsink.Addr)
Output: maptags[label]: source maptags[info]: the origin maptags[version]: 1 maptags[tomare]: 2000-01-01T00:00:00Z maptags[address]: hello異世界 source the origin 0001-01-01 00:00:00 +0000 UTC hello異世界
func FillStructDeflate ¶
FillStructDeflate fills the nested object from flat map. This works by filling outer struct first and then checking its subsequent object fields.
Example (FromJson) ¶
type (
nest2 struct {
N2FieldFloat float64 `json:"nested2_flt"`
N2FieldStr string `json:"nested2_str"`
}
nest1 struct {
N1FieldFloat float64 `json:"nested1_flt"`
N1FieldStr string `json:"nested1_str"`
Nest2 *nest2 `json:"nested2"`
}
outerobj struct {
FieldFloat float64 `json:"field_flt"`
FieldStr string `json:"field_str"`
Nest1 nest1 `json:"nested1"`
}
)
rawb := `
{
"field_str": "555",
"field_flt": 5,
"nested1_flt": 515,
"nested1_str": "515",
"nested2_flt": 525,
"nested2_str": "525"
}`
var m map[string]interface{}
fmt.Println(json.Unmarshal([]byte(rawb), &m))
var tgt outerobj
fmt.Println("error result fill struct deflate:", FillStructDeflate(&tgt, m, "json"))
fmt.Printf("%#v\n", tgt.FieldFloat)
fmt.Printf("%#v\n", tgt.FieldStr)
fmt.Printf("%#v\n", tgt.Nest1.N1FieldFloat)
fmt.Printf("%#v\n", tgt.Nest1.N1FieldStr)
fmt.Printf("%#v\n", tgt.Nest1.Nest2.N2FieldFloat)
fmt.Printf("%#v\n", tgt.Nest1.Nest2.N2FieldStr)
Output: <nil> error result fill struct deflate: <nil> 5 "555" 515 "515" 525 "525"
func SQLScan ¶
func SQLScan(row SQLScanner, obj interface{}, tag string, x ...string) error
SQLScan is the function that will map scanning object based on provided field name or field tagged string. The tags can receive the empty string "" and then it will map the field name by default.
Example (AllFields) ¶
currtime := time.Now()
dr := createDummyRow(currtime)
result := dummyValues{}
if err := SQLScan(dr, &result, ""); err != nil {
fmt.Println("Error happened!")
return
}
fmt.Printf("NullString is Valid? %t\n", result.NullString.Valid)
fmt.Printf("result.NullString is %s\n", result.NullString.String)
fmt.Printf("NullTime is Valid? %t\n", result.NullTime.Valid)
fmt.Printf("result.NullTime.Time.Equal(dr.Values.NullTime.Time)? %t\n",
result.NullTime.Time.Equal(dr.Values.NullTime.Time))
fmt.Printf("result.Uint64 == %d\n", result.Uint64)
Output: NullString is Valid? true result.NullString is hello 異世界 NullTime is Valid? true result.NullTime.Time.Equal(dr.Values.NullTime.Time)? true result.Uint64 == 5
Example (SuppliedFields) ¶
currtime := time.Now()
dr := createDummyRow(currtime)
result := dummyValues{}
if err := SQLScan(dr, &result,
"", /* This is the tag, since we don't have so put it empty
to match the field name */
/* Below arguments are variadic and we only take several
fields from all available dummyValues */
"Int32", "Uint64", "Bool", "Bytes",
"NullString", "NullTime"); err != nil {
fmt.Println("Error happened!")
return
}
fmt.Printf("NullString is Valid? %t\n", result.NullString.Valid)
fmt.Printf("NullTime is Valid? %t\n", result.NullTime.Valid)
fmt.Printf("result.NullTime.Time.Equal(dr.Values.NullTime.Time)? %t\n",
result.NullTime.Time.Equal(dr.Values.NullTime.Time))
fmt.Printf("result.Uint64 == %d\n", result.Uint64)
Output: NullString is Valid? true NullTime is Valid? true result.NullTime.Time.Equal(dr.Values.NullTime.Time)? true result.Uint64 == 5
func SetFieldFromTag ¶
Types ¶
type MapDecoder ¶
type MapDecoder interface {
MapDecode(interface{}) error
}
Example ¶
var err error
var smap map[string]interface{}
type MapDecoderExample struct {
Data sometype `json:"data"`
}
smap = map[string]interface{}{"data": "hello"}
var s MapDecoderExample
err = FillStructByTags(&s, smap, "json")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(s.Data.Value)
type map2 struct {
Data *sometype `json:"data"`
}
smap = map[string]interface{}{"data": "hello2"}
var s2 map2
// s2 := map2{Data: &sometype{}}
err = FillStructByTags(&s2, smap, "json")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(s2.Data.Value)
Output: hello hello2
type MapEncoder ¶
type MapEncoder interface {
MapEncode() (interface{}, error)
}
Example ¶
type MapEncoderExample struct {
Data sometype `json:"data"`
}
type map2 struct {
Data *sometype `json:"data"`
}
s := MapEncoderExample{Data: sometype{Value: "hello"}}
smap := MapTags(&s, "json")
str, ok := smap["data"].(string)
if !ok {
fmt.Println("Not ok!")
return
}
fmt.Println(str)
s2 := map2{Data: &sometype{Value: "hello2"}}
smap = MapTags(&s2, "json")
str, ok = smap["data"].(string)
if !ok {
fmt.Println("Not ok!")
return
}
fmt.Println(str)
Output: hello hello2
type Mapped ¶
type Mapped map[string]interface{}
Mapped simply an alias
func MapFields ¶
func MapFields(x interface{}) Mapped
MapFields maps between struct to mapped interfaces{}. The argument must be (zero or minterface{} pointers to) struct or else it will be ignored. Now it's implemented as MapTags with empty tag "".
Only map the exported fields.
Example ¶
mapped := MapFields(sourceobj) printIfNotExists(mapped, "Label", "Info", "Version")
Output:
func MapTags ¶
MapTags maps the tag value of defined field tag name. This enable various field extraction that will be mapped to mapped interfaces{}.
Example (Basic) ¶
ptrSourceObj := &sourceobj maptags := MapTags(&ptrSourceObj, "json") printIfNotExists(maptags, "label", "info", "version")
Output:
Example (Nested) ¶
nestedSource := differentSourceSink{
Source: sourceobj,
DiffSink: differentSink{
DiffLabel: "nested diff",
NiceInfo: "nested info",
Version: "next version",
Toki: toki,
Addr: &hello,
},
}
// this part illustrates that MapTags or any other Map function
// accept arbitrary pointers to struct.
ptrNestedSource := &nestedSource
ptr2NestedSource := &ptrNestedSource
ptr3NestedSource := &ptr2NestedSource
nestedMap := MapTags(&ptr3NestedSource, "json")
for k, v := range nestedMap {
fmt.Println("top key:", k)
for kk, vv := range v.(Mapped) {
if vtime, ok := vv.(time.Time); ok {
fmt.Println(" nested:", kk, vtime.Format(time.RFC3339))
} else {
fmt.Println(" nested:", kk, vv)
}
}
fmt.Println()
}
Output: top key: source nested: label source nested: info the origin nested: version 1 nested: tomare 2000-01-01T00:00:00Z nested: address hello異世界 top key: differentSink nested: label nested diff nested: info nested info nested: unversion next version nested: doki 2000-01-01T00:00:00Z nested: address hello異世界
Example (TwoTags) ¶
general := generalFields{
Name: "duran",
Rank: "private",
Code: 1337,
nickname: "drone",
}
mapjson := MapTags(&general, "json")
printIfNotExists(mapjson, "name", "rank", "code")
mapapi := MapTags(&general, "api")
printIfNotExists(mapapi, "general_name", "general_rank", "general_code")
Output:
func MapTagsFlatten ¶
MapTagsFlatten is to flatten mapped object with specific tag. The limitation of this flattening that it can't have duplicate tag name and it will give incorrect result because the older value will be written with newer map field value.
Example ¶
type (
Last struct {
Final string `json:"final"`
Destination string
}
Lv3 struct {
Lv3Str string `json:"lv3str"`
*Last `json:"last"`
Lv3Dummy string
}
Lv2 struct {
Lv2Str string `json:"lv2str"`
Lv3 `json:"lv3"`
Lv2Dummy string
}
Lv1 struct {
Lv2
Lv1Str string `json:"lv1str"`
Lv1Dummy string
}
)
obj := Lv1{
Lv1Str: "level 1 string",
Lv1Dummy: "baka",
Lv2: Lv2{
Lv2Dummy: "bakabaka",
Lv2Str: "level 2 string",
Lv3: Lv3{
Lv3Dummy: "bakabakka",
Lv3Str: "level 3 string",
Last: &Last{
Final: "destination",
Destination: "overloop",
},
},
},
}
for k, v := range MapTagsFlatten(&obj, "json") {
fmt.Printf("key: %s, value: %v\n", k, v)
}
Output: key: final, value: destination key: lv1str, value: level 1 string key: lv2str, value: level 2 string key: lv3str, value: level 3 string
func MapTagsWithDefault ¶
MapTagsWithDefault maps the tag with optional fallback tags. This to enable tag differences when there are only few difference with the default “json“ tag.
Example ¶
type higherCommon struct {
General generalFields `json:"general"`
Communality string `json:"common"`
Available bool `json:"available" api:"is_available"`
}
rawjson := []byte(`{
"general": {
name: "duran",
rank: "private",
code: 1337,
},
"common": "rare",
"available": true
}`)
hc := higherCommon{}
_ = json.Unmarshal(rawjson, &hc)
maptags := MapTagsWithDefault(&hc, "api", "json")
printIfNotExists(maptags, "available")
Output: available : not exists
type SQLScanner ¶
type SQLScanner interface {
Scan(dest ...interface{}) error
}
SQLScanner is the interface that dictate any type that implement Scan method to be compatible with sql.Row Scan method.
Source Files