portal

README

戳我看中文介绍

What's portal?

It's a lightweight package which simplifies Go object serialization. Inspired by marshmallow, but with concurrency builtin for better performance.

portal can be used to serialize app-level objects to specified objects (schema structs). The serialized objects can be rendered to any standard formats like JSON for an HTTP API. Most importantly, if some fields of a schema have different data sources, portal could spawn several goroutines to retrieve fields' data concurrently.

Note that unlike marshmallow, portal only focuses on object serialization. So, if you want to validate struct fields, please refer to go-playground/validator or asaskevich/govalidator.

Features

1. Can be configured to fill data into multiple fields concurrently.
2. Support flexible field filtering for any (nested) schemas.
3. Automatically convert field data to the expected field type, no more boilerplate code.
4. Simple and clean API.

Install

get get -u github.com/ifaceless/portal

Quickstart

Full example can be found here.

Model Definitions

CLICK HERE | model.go

type NotificationModel struct {
	ID      int
	Title   string
	Content string
}

type UserModel struct {
	ID int
}

func (u *UserModel) Fullname() string {
	return fmt.Sprintf("user:%d", u.ID)
}

func (u *UserModel) Notifications() (result []*NotificationModel) {
	for i := 0; i < 1; i++ {
		result = append(result, &NotificationModel{
			ID:      i,
			Title:   fmt.Sprintf("title_%d", i),
			Content: fmt.Sprintf("content_%d", i),
		})
	}
	return
}

type TaskModel struct {
	ID     int
	UserID int
	Title  string
}

func (t *TaskModel) User() *UserModel {
	return &UserModel{t.UserID}
}

Schema Definitions

CLICK HERE | schema.go

type NotiSchema struct {
	ID      string `json:"id,omitempty"`
	Title   string `json:"title,omitempty"`
	Content string `json:"content,omitempty"`
}

type UserSchema struct {
	ID                   string        `json:"id,omitempty"`
	// Get user name from `UserModel.Fullname()`
	Name                 string        `json:"name,omitempty" portal:"attr:Fullname"`
	Notifications        []*NotiSchema `json:"notifications,omitempty" portal:"nested"`
	AnotherNotifications []*NotiSchema `json:"another_notifications,omitempty" portal:"nested;attr:Notifications"`
}

type TaskSchema struct {
	ID          string      `json:"id,omitempty"`
	Title       string      `json:"title,omitempty"`
	Description string      `json:"description,omitempty" portal:"meth:GetDescription"`
	// UserSchema is a nested schema
	User        *UserSchema `json:"user,omitempty" portal:"nested"`
	// We just want `Name` field for `SimpleUser`.
	// Besides, the data source is the same with `UserSchema`
	SimpleUser  *UserSchema `json:"simple_user,omitempty" portal:"nested;only:Name;attr:User"`
}

func (ts *TaskSchema) GetDescription(model *model.TaskModel) string {
	return "Custom description"
}

Serialization Examples

package main

import (
	"encoding/json"
	"github.com/ifaceless/portal"
)

func main() {
	// log debug info
	portal.SetDebug(true)
	// set max worker pool size
	portal.SetMaxPoolSize(1024)
	// make sure to clean up.
	defer portal.CleanUp()

	// write to a specified task schema
	var taskSchema schema.TaskSchema
	portal.Dump(&taskSchema, &taskModel)
	// data: {"id":"1","title":"Finish your jobs.","description":"Custom description","user":{"id":"1","name":"user:1","notifications":[{"id":"0","title":"title_0","content":"content_0"}],"another_notifications":[{"id":"0","title":"title_0","content":"content_0"}]},"simple_user":{"name":"user:1"}}
	data, _ := json.Marshal(taskSchema)

	// select specified fields
	portal.Dump(&taskSchema, &taskModel, portal.Only("Title", "SimpleUser"))
	// data: {"title":"Finish your jobs.","simple_user":{"name":"user:1"}}
	data, _ := json.Marshal(taskSchema)
	
	// select fields with alias defined in the json tag.
	// actually, the default alias tag is `json`, `portal.FieldAliasMapTagName("json")` is optional.
	portal.Dump(&taskSchema, &taskModel, portal.Only("title", "SimpleUser"), portal.FieldAliasMapTagName("json"))
	// data: {"title":"Finish your jobs.","simple_user":{"name":"user:1"}}
	data, _ := json.Marshal(taskSchema)

	// you can keep any fields for any nested schemas
	// multiple fields are separated with ','
	// nested fields are wrapped with '[' and ']'
	portal.Dump(&taskSchema, &taskModel, portal.Only("ID", "User[ID,Notifications[ID],AnotherNotifications[Title]]", "SimpleUser"))
	// data: {"id":"1","user":{"id":"1","notifications":[{"id":"0"}],"another_notifications":[{"title":"title_0"}]},"simple_user":{"name":"user:1"}}
	data, _ := json.Marshal(taskSchema)

	// ignore specified fields
	portal.Dump(&taskSchema, &taskModel, portal.Exclude("Description", "ID", "User[Name,Notifications[ID,Content],AnotherNotifications], SimpleUser"))
	// data: {"title":"Finish your jobs.","user":{"id":"1","notifications":[{"title":"title_0"}]}}
	data, _ := json.Marshal(taskSchema)

	// dump multiple tasks
	var taskSchemas []schema.TaskSchema
	portal.Dump(&taskSchemas, &taskModels, portal.Only("ID", "Title", "User[Name]"))
	// data: [{"id":"0","title":"Task #1","user":{"name":"user:100"}},{"id":"1","title":"Task #2","user":{"name":"user:101"}}]
	data, _ := json.Marshal(taskSchema)
}

To learn more about portal, please read the User Guide~

Concurrency Strategy

1. Any fields tagged with portal:"async" will be serialized asynchronously.
2. When dumping to multiple schemas, portal will do it concurrently if any fields in the schema are tagged with protal:"async".
3. You can always disable concurrency strategy with option portal.DisableConcurrency().

Cache Strategy

1. Cache is implemented in the field level when portal.SetCache(portal.DefaultCache) is configured.
2. Cache will be disabled by tagging the fields with portal:"disablecache" or by defining a PortalDisableCache() bool meth for the schema struct, or by a portal.DisableCache() option setting while dumping.
3. Cache is available for one schema's one time dump, after the dump, the cache will be invalidated.

Core APIs

func New(opts ...Option) (*Chell, error)
func Dump(dst, src interface{}, opts ...Option) error 
func DumpWithContext(ctx context.Context, dst, src interface{}, opts ...Option)
func SetDebug(v bool)
func SetMaxPoolSize(size int)
func CleanUp()

More Field Types

• field.Timestamp: time.Time <-> unix timestamp.
• field.UpperString: convert string to upper case.
• field.LowerString: convert string to lower case.

License

portal is licensed under the MIT license. Please feel free and have fun~

Documentation

Index

• Variables
• func CleanUp()
• func CustomFieldTagMap(in map[string]string) option
• func DisableCache() option
• func DisableConcurrency() option
• func Dump(dst, src interface{}, opts ...option) error
• func DumpWithContext(ctx context.Context, dst, src interface{}, opts ...option) error
• func Exclude(fields ...string) option
• func FieldAliasMapTagName(tag string) option
• func Only(fields ...string) option
• func SetCache(c Cacher)
• func SetDebug(v bool)
• func SetMaxPoolSize(size int)
• type Cacher
• type Chell
  • func New(opts ...option) (*Chell, error)
  • func (c *Chell) Dump(dst, src interface{}) error
  • func (c *Chell) DumpWithContext(ctx context.Context, dst, src interface{}) error
  • func (c *Chell) SetExcludeFields(fields ...string) error
  • func (c *Chell) SetOnlyFields(fields ...string) error
• type ErrNil
  • func (e *ErrNil) Error() string
• type MapCache
  • func (m *MapCache) Get(_ context.Context, key interface{}) (interface{}, error)
  • func (m *MapCache) Set(_ context.Context, key, value interface{}) error
• type ValueSetter
• type Valuer

Variables

var (
	DefaultCache = newMapCache()
)

Functions

func CleanUp

func CleanUp()

CleanUp releases the global worker pool. You should call this function only once before the main goroutine exits.

func CustomFieldTagMap

func CustomFieldTagMap(in map[string]string) option

CustomFieldTagMap sets custom tag for each field. It will override the default tag settings defined in your struct. The key should be: `<StructName>.<FieldName>`

func DisableCache

func DisableCache() option

DisableCache disables cache strategy

func DisableConcurrency

func DisableConcurrency() option

DisableConcurrency disables concurrency strategy.

func Dump

func Dump(dst, src interface{}, opts ...option) error

Dump dumps src data to dst. You can filter fields with optional config `portal.Only` or `portal.Exclude`.

func DumpWithContext

func DumpWithContext(ctx context.Context, dst, src interface{}, opts ...option) error

DumpWithContext dumps src data to dst with an extra context param. You can filter fields with optional config `portal.Only` or `portal.Exclude`.

func Exclude

func Exclude(fields ...string) option

Exclude specifies the fields to exclude. Examples: ``` c := New(Exclude("A")) // exclude field A c := New(Exclude("A[B,C]")) // exclude field B and C of the nested struct A, but other fields of struct A are still selected. ```

func FieldAliasMapTagName

func FieldAliasMapTagName(tag string) option

FieldAliasMapTagName sets the tag name (e.g. `yaml`, `json`) to parse alias of a field name. Example: ```

struct Schema {
    ID   int `json:"id"`
}

// portal parses the json tag, and maps `id` -> `ID`. ```

func Only

func Only(fields ...string) option

Only specifies the fields to keep. Examples: ``` c := New(Only("A")) // keep field A only c := New("A[B,C]") // // keep field B and C of the nested struct A ```

func SetCache

func SetCache(c Cacher)

SetCache enable cache strategy

func SetDebug

func SetDebug(v bool)

func SetMaxPoolSize

func SetMaxPoolSize(size int)

SetMaxPoolSize limits the capacity of all worker pools.

Types

type Cacher

type Cacher interface {
	Set(ctx context.Context, key interface{}, value interface{}) error
	Get(ctx context.Context, key interface{}) (interface{}, error)
}

type Chell

type Chell struct {
	// contains filtered or unexported fields
}

Chell manages the dumping state.

func New

func New(opts ...option) (*Chell, error)

New creates a new Chell instance with a worker pool waiting to be feed. It's highly recommended to call function `portal.Dump()` or `portal.DumpWithContext()` directly.

func (*Chell) Dump

func (c *Chell) Dump(dst, src interface{}) error

Dump dumps src data to dst. You can filter fields with optional config `portal.Only` or `portal.Exclude`.

func (*Chell) DumpWithContext

func (c *Chell) DumpWithContext(ctx context.Context, dst, src interface{}) error

DumpWithContext dumps src data to dst with an extra context param. You can filter fields with optional config `portal.Only` or `portal.Exclude`.

func (*Chell) SetExcludeFields

func (c *Chell) SetExcludeFields(fields ...string) error

SetOnlyFields specifies the fields to exclude. Examples: ``` c := New() c.SetExcludeFields("A") // exclude field A c.SetExcludeFields("A[B,C]") // exclude field B and C of the nested struct A, but other fields of struct A are still selected. ```

func (*Chell) SetOnlyFields

func (c *Chell) SetOnlyFields(fields ...string) error

SetOnlyFields specifies the fields to keep. Examples: ``` c := New() c.SetOnlyFields("A") // keep field A only c.SetOnlyFields("A[B,C]") // keep field B and C of the nested struct A ```

type ErrNil

type ErrNil struct{}

func (*ErrNil) Error

func (e *ErrNil) Error() string

type MapCache

type MapCache struct {
	// contains filtered or unexported fields
}

func (*MapCache) Get

func (m *MapCache) Get(_ context.Context, key interface{}) (interface{}, error)

func (*MapCache) Set

func (m *MapCache) Set(_ context.Context, key, value interface{}) error

type ValueSetter

type ValueSetter interface {
	SetValue(v interface{}) error
}

type Valuer

type Valuer interface {
	Value() (interface{}, error)
}