form

README

Package form

Package form Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values.

It has the following features:

• Supports map of almost all types.
• Supports both Numbered and Normal arrays eg. "Array[0]" and just "Array" with multiple values passed.
• Array honours the specified index. eg. if "Array[2]" is the only Array value passed down, it will be put at index 2; if array isn't big enough it will be expanded.
• Only creates objects as necessary eg. if no array or map values are passed down, the array and map are left as their default values in the struct.
• Allows for Custom Type registration.
• Handles time.Time using RFC3339 time format by default, but can easily be changed by registering a Custom Type, see below.
• Handles Encoding & Decoding of almost all Go types eg. can Decode into struct, array, map, int... and Encode a struct, array, map, int...

Common Questions

• Does it support encoding.TextUnmarshaler? No because TextUnmarshaler only accepts []byte but posted values can have multiple values, so is not suitable.

Supported Types ( out of the box )

• string
• bool
• int, int8, int16, int32, int64
• uint, uint8, uint16, uint32, uint64
• float32, float64
• struct and anonymous struct
• interface{}
• time.Time - by default using RFC3339
• a pointer to one of the above types
• slice, array
• map
• custom types can override any of the above types
• many other types may be supported inherently

NOTE: map, struct and slice nesting are ad infinitum.

Installation

Use go get.

go get github.com/go-playground/form

Then import the form package into your own code.

import "github.com/go-playground/form"

Usage

• Use symbol . for separating fields/structs. (eg. structfield.field)
• Use [index or key] for access to index of a slice/array or key for map. (eg. arrayfield[0], mapfield[keyvalue])

<form method="POST">
  <input type="text" name="Name" value="joeybloggs"/>
  <input type="text" name="Age" value="3"/>
  <input type="text" name="Gender" value="Male"/>
  <input type="text" name="Address[0].Name" value="26 Here Blvd."/>
  <input type="text" name="Address[0].Phone" value="9(999)999-9999"/>
  <input type="text" name="Address[1].Name" value="26 There Blvd."/>
  <input type="text" name="Address[1].Phone" value="1(111)111-1111"/>
  <input type="text" name="active" value="true"/>
  <input type="text" name="MapExample[key]" value="value"/>
  <input type="text" name="NestedMap[key][key]" value="value"/>
  <input type="text" name="NestedArray[0][0]" value="value"/>
  <input type="submit"/>
</form>

Examples

Decoding

package main

import (
	"fmt"
	"log"
	"net/url"

	"github.com/go-playground/form"
)

// Address contains address information
type Address struct {
	Name  string
	Phone string
}

// User contains user information
type User struct {
	Name        string
	Age         uint8
	Gender      string
	Address     []Address
	Active      bool `form:"active"`
	MapExample  map[string]string
	NestedMap   map[string]map[string]string
	NestedArray [][]string
}

// use a single instance of Decoder, it caches struct info
var decoder *form.Decoder

func main() {
	decoder = form.NewDecoder()

	// this simulates the results of http.Request's ParseForm() function
	values := parseForm()

	var user User

	// must pass a pointer
	err := decoder.Decode(&user, values)
	if err != nil {
		log.Panic(err)
	}

	fmt.Printf("%#v\n", user)
}

// this simulates the results of http.Request's ParseForm() function
func parseForm() url.Values {
	return url.Values{
		"Name":                []string{"joeybloggs"},
		"Age":                 []string{"3"},
		"Gender":              []string{"Male"},
		"Address[0].Name":     []string{"26 Here Blvd."},
		"Address[0].Phone":    []string{"9(999)999-9999"},
		"Address[1].Name":     []string{"26 There Blvd."},
		"Address[1].Phone":    []string{"1(111)111-1111"},
		"active":              []string{"true"},
		"MapExample[key]":     []string{"value"},
		"NestedMap[key][key]": []string{"value"},
		"NestedArray[0][0]":   []string{"value"},
	}
}

Encoding

package main

import (
	"fmt"
	"log"

	"github.com/go-playground/form"
)

// Address contains address information
type Address struct {
	Name  string
	Phone string
}

// User contains user information
type User struct {
	Name        string
	Age         uint8
	Gender      string
	Address     []Address
	Active      bool `form:"active"`
	MapExample  map[string]string
	NestedMap   map[string]map[string]string
	NestedArray [][]string
}

// use a single instance of Encoder, it caches struct info
var encoder *form.Encoder

func main() {
	encoder = form.NewEncoder()

	user := User{
		Name:   "joeybloggs",
		Age:    3,
		Gender: "Male",
		Address: []Address{
			{Name: "26 Here Blvd.", Phone: "9(999)999-9999"},
			{Name: "26 There Blvd.", Phone: "1(111)111-1111"},
		},
		Active:      true,
		MapExample:  map[string]string{"key": "value"},
		NestedMap:   map[string]map[string]string{"key": {"key": "value"}},
		NestedArray: [][]string{{"value"}},
	}

	// must pass a pointer
	values, err := encoder.Encode(&user)
	if err != nil {
		log.Panic(err)
	}

	fmt.Printf("%#v\n", values)
}

Registering Custom Types

Decoder

decoder.RegisterCustomTypeFunc(func(vals []string) (interface{}, error) {
		return time.Parse("2006-01-02", vals[0])
	}, time.Time{})

ADDITIONAL: if a struct type is registered, the function will only be called if a url.Value exists for the struct and not just the struct fields eg. url.Values{"User":"Name%3Djoeybloggs"} will call the custom type function with 'User' as the type, however url.Values{"User.Name":"joeybloggs"} will not.

Encoder

encoder.RegisterCustomTypeFunc(func(x interface{}) ([]string, error) {
		return []string{x.(time.Time).Format("2006-01-02")}, nil
	}, time.Time{})

Ignoring Fields

you can tell form to ignore fields using - in the tag

type MyStruct struct {
    Field string `form:"-"`
}

Omitempty

you can tell form to omit empty fields using ,omitempty or FieldName,omitempty in the tag

type MyStruct struct {
    Field  string `form:",omitempty"`
	Field2 string `form:"CustomFieldName,omitempty"`
}

Notes

To maximize compatibility with other systems the Encoder attempts to avoid using array indexes in url.Values if at all possible.

eg.

// A struct field of
Field []string{"1", "2", "3"}

// will be output a url.Value as
"Field": []string{"1", "2", "3"}

and not
"Field[0]": []string{"1"}
"Field[1]": []string{"2"}
"Field[2]": []string{"3"}

// however there are times where it is unavoidable, like with pointers
i := int(1)
Field []*string{nil, nil, &i}

// to avoid index 1 and 2 must use index
"Field[2]": []string{"1"}

Benchmarks

Run on MacBook Pro (15-inch, 2017) using go version go1.9 darwin/amd64

NOTE: the 1 allocation and B/op in the first 4 decodes is actually the struct allocating when passing it in, so primitives are actually zero allocation.

go test -run=NONE -bench=. -benchmem=true
goos: darwin
goarch: amd64
pkg: github.com/go-playground/form/benchmarks

BenchmarkSimpleUserDecodeStruct-8                             	 5000000	       243 ns/op	      64 B/op	       1 allocs/op
BenchmarkSimpleUserDecodeStructParallel-8                     	20000000	        72.4 ns/op	      64 B/op	       1 allocs/op
BenchmarkSimpleUserEncodeStruct-8                             	 2000000	       683 ns/op	     485 B/op	      10 allocs/op
BenchmarkSimpleUserEncodeStructParallel-8                     	10000000	       205 ns/op	     485 B/op	      10 allocs/op
BenchmarkPrimitivesDecodeStructAllPrimitivesTypes-8           	 2000000	       739 ns/op	      96 B/op	       1 allocs/op
BenchmarkPrimitivesDecodeStructAllPrimitivesTypesParallel-8   	10000000	       214 ns/op	      96 B/op	       1 allocs/op
BenchmarkPrimitivesEncodeStructAllPrimitivesTypes-8           	  500000	      3608 ns/op	    2977 B/op	      36 allocs/op
BenchmarkPrimitivesEncodeStructAllPrimitivesTypesParallel-8   	 1000000	      1013 ns/op	    2978 B/op	      36 allocs/op
BenchmarkComplexArrayDecodeStructAllTypes-8                   	  100000	     13358 ns/op	    2249 B/op	     121 allocs/op
BenchmarkComplexArrayDecodeStructAllTypesParallel-8           	  500000	      3656 ns/op	    2249 B/op	     121 allocs/op
BenchmarkComplexArrayEncodeStructAllTypes-8                   	  100000	     11574 ns/op	    7112 B/op	     104 allocs/op
BenchmarkComplexArrayEncodeStructAllTypesParallel-8           	  500000	      3278 ns/op	    7112 B/op	     104 allocs/op
BenchmarkComplexMapDecodeStructAllTypes-8                     	  100000	     18945 ns/op	    5305 B/op	     130 allocs/op
BenchmarkComplexMapDecodeStructAllTypesParallel-8             	  300000	      5213 ns/op	    5308 B/op	     130 allocs/op
BenchmarkComplexMapEncodeStructAllTypes-8                     	  100000	     12177 ns/op	    6972 B/op	     129 allocs/op
BenchmarkComplexMapEncodeStructAllTypesParallel-8             	  500000	      3409 ns/op	    6971 B/op	     129 allocs/op
BenchmarkDecodeNestedStruct-8                                 	  500000	      2642 ns/op	     384 B/op	      14 allocs/op
BenchmarkDecodeNestedStructParallel-8                         	 2000000	       760 ns/op	     384 B/op	      14 allocs/op
BenchmarkEncodeNestedStruct-8                                 	 1000000	      1686 ns/op	     693 B/op	      16 allocs/op
BenchmarkEncodeNestedStructParallel-8                         	 3000000	       468 ns/op	     693 B/op	      16 allocs/op

Competitor benchmarks can be found here

Complimentary Software

Here is a list of software that compliments using this library post decoding.

• Validator - Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving.
• Conform - Trims, sanitizes & scrubs data based on struct tags.

Package Versioning

I'm jumping on the vendoring bandwagon, you should vendor this package as I will not be creating different version with gopkg.in like allot of my other libraries.

Why? because my time is spread pretty thin maintaining all of the libraries I have + LIFE, it is so freeing not to worry about it and will help me keep pouring out bigger and better things for you the community.

License

Distributed under MIT License, please see license file in code for more details.

Documentation

Overview

Package form Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values.

It has the following features:

• Primitives types cause zero allocations.
• Supports map of almost all types.
• Supports both Numbered and Normal arrays eg. "Array[0]" and just "Array" with multiple values passed.
• Array honours the specified index. eg. if "Array[2]" is the only Array value passed down, it will be put at index 2; if array isn't big enough it will be expanded.
• Only creates objects as necessary eg. if no `array` or `map` values are passed down, the `array` and `map` are left as their default values in the struct.
• Allows for Custom Type registration.
• Handles time.Time using RFC3339 time format by default, but can easily be changed by registering a Custom Type, see below.
• Handles Encoding & Decoding of almost all Go types eg. can Decode into struct, array, map, int... and Encode a struct, array, map, int...

Common Questions

Questions

Does it support encoding.TextUnmarshaler?
No because TextUnmarshaler only accepts []byte but posted values can have
multiple values, so is not suitable.

Supported Types

out of the box supported types

• string

• bool

• int, int8, int16, int32, int64

• uint, uint8, uint16, uint32, uint64

• float32, float64

• struct and anonymous struct

• interface{}

• time.Time` - by default using RFC3339

• a `pointer` to one of the above types

• slice, array

• map

• `custom types` can override any of the above types

• many other types may be supported inherently (eg. bson.ObjectId is type ObjectId string, which will get populated by the string type

  **NOTE**: map, struct and slice nesting are ad infinitum.

Usage

symbols

• Use symbol `.` for separating fields/structs. (eg. `structfield.field`)
• Use `[index or key]` for access to index of a slice/array or key for map. (eg. `arrayfield[0]`, `mapfield[keyvalue]`)

html

<form method="POST">
    <input type="text" name="Name" value="joeybloggs"/>
    <input type="text" name="Age" value="3"/>
    <input type="text" name="Gender" value="Male"/>
    <input type="text" name="Address[0].Name" value="26 Here Blvd."/>
    <input type="text" name="Address[0].Phone" value="9(999)999-9999"/>
    <input type="text" name="Address[1].Name" value="26 There Blvd."/>
    <input type="text" name="Address[1].Phone" value="1(111)111-1111"/>
    <input type="text" name="active" value="true"/>
    <input type="text" name="MapExample[key]" value="value"/>
    <input type="text" name="NestedMap[key][key]" value="value"/>
    <input type="text" name="NestedArray[0][0]" value="value"/>
    <input type="submit"/>
</form>

Example

example decoding the above HTML

package main

import (
    "fmt"
    "log"
    "net/url"

    "github.com/go-playground/form"
)

// Address contains address information
type Address struct {
    Name  string
    Phone string
}

// User contains user information
type User struct {
    Name        string
    Age         uint8
    Gender      string
    Address     []Address
    Active      bool `form:"active"`
    MapExample  map[string]string
    NestedMap   map[string]map[string]string
    NestedArray [][]string
}

// use a single instance of Decoder, it caches struct info
var decoder *form.Decoder

func main() {
    decoder = form.NewDecoder()

    // this simulates the results of http.Request's ParseForm() function
    values := parseForm()

    var user User

    // must pass a pointer
    err := decoder.Decode(&user, values)
    if err != nil {
        log.Panic(err)
    }

    fmt.Printf("%#v\n", user)
}

// this simulates the results of http.Request's ParseForm() function
func parseForm() url.Values {
    return url.Values{
        "Name":                []string{"joeybloggs"},
        "Age":                 []string{"3"},
        "Gender":              []string{"Male"},
        "Address[0].Name":     []string{"26 Here Blvd."},
        "Address[0].Phone":    []string{"9(999)999-9999"},
        "Address[1].Name":     []string{"26 There Blvd."},
        "Address[1].Phone":    []string{"1(111)111-1111"},
        "active":              []string{"true"},
        "MapExample[key]":     []string{"value"},
        "NestedMap[key][key]": []string{"value"},
        "NestedArray[0][0]":   []string{"value"},
    }
}

example encoding

package main

import (
    "fmt"
    "log"

    "github.com/go-playground/form"
)

// Address contains address information
type Address struct {
    Name  string
    Phone string
}

// User contains user information
type User struct {
    Name        string
    Age         uint8
    Gender      string
    Address     []Address
    Active      bool `form:"active"`
    MapExample  map[string]string
    NestedMap   map[string]map[string]string
    NestedArray [][]string
}

// use a single instance of Encoder, it caches struct info
var encoder *form.Encoder

func main() {
    encoder = form.NewEncoder()

    user := User{
        Name:   "joeybloggs",
        Age:    3,
        Gender: "Male",
        Address: []Address{
            {Name: "26 Here Blvd.", Phone: "9(999)999-9999"},
            {Name: "26 There Blvd.", Phone: "1(111)111-1111"},
        },
        Active:      true,
        MapExample:  map[string]string{"key": "value"},
        NestedMap:   map[string]map[string]string{"key": {"key": "value"}},
        NestedArray: [][]string{{"value"}},
    }

    // must pass a pointer
    values, err := encoder.Encode(&user)
    if err != nil {
        log.Panic(err)
    }

    fmt.Printf("%#v\n", values)
}

Registering Custom Types

Decoder

decoder.RegisterCustomTypeFunc(func(vals []string) (interface{}, error) {
        return time.Parse("2006-01-02", vals[0])
    }, time.Time{})

ADDITIONAL: if a struct type is registered, the function will only be called
if a url.Value exists for the struct and not just the struct fields
eg. url.Values{"User":"Name%3Djoeybloggs"} will call the custom type function
with 'User' as the type, however url.Values{"User.Name":"joeybloggs"} will not.

Encoder

encoder.RegisterCustomTypeFunc(func(x interface{}) ([]string, error) {
        return []string{x.(time.Time).Format("2006-01-02")}, nil
    }, time.Time{})

Ignoring Fields

you can tell form to ignore fields using `-` in the tag

type MyStruct struct {
    Field string `form:"-"`
}

Omitempty

you can tell form to omit empty fields using `,omitempty` or `FieldName,omitempty` in the tag

type MyStruct struct {
    Field  string `form:",omitempty"`
    Field2 string `form:"CustomFieldName,omitempty"`
}

Notes

To maximize compatibility with other systems the Encoder attempts to avoid using array indexes in url.Values if at all possible.

eg.

// A struct field of
Field []string{"1", "2", "3"}

// will be output a url.Value as
"Field": []string{"1", "2", "3"}

and not
"Field[0]": []string{"1"}
"Field[1]": []string{"2"}
"Field[2]": []string{"3"}

// however there are times where it is unavoidable, like with pointers
i := int(1)
Field []*string{nil, nil, &i}

// to avoid index 1 and 2 must use index
"Field[2]": []string{"1"}

Index

• func ExtractType(current reflect.Value) (reflect.Value, reflect.Kind)
• type AnonymousMode
• type DecodeCustomTypeFunc
• type DecodeErrors
  • func (d DecodeErrors) Error() string
• type Decoder
  • func NewDecoder() *Decoder
  • func (d *Decoder) Decode(v interface{}, values url.Values) (err error)
  • func (d *Decoder) RegisterCustomTypeFunc(fn DecodeCustomTypeFunc, types ...interface{})
  • func (d *Decoder) RegisterTagNameFunc(fn TagNameFunc)
  • func (d *Decoder) SetMaxArraySize(size uint)
  • func (d *Decoder) SetMode(mode Mode)
  • func (d *Decoder) SetTagName(tagName string)
• type EncodeCustomTypeFunc
• type EncodeErrors
  • func (e EncodeErrors) Error() string
• type Encoder
  • func NewEncoder() *Encoder
  • func (e *Encoder) Encode(v interface{}) (values url.Values, err error)
  • func (e *Encoder) RegisterCustomTypeFunc(fn EncodeCustomTypeFunc, types ...interface{})
  • func (e *Encoder) RegisterTagNameFunc(fn TagNameFunc)
  • func (e *Encoder) SetAnonymousMode(mode AnonymousMode)
  • func (e *Encoder) SetMode(mode Mode)
  • func (e *Encoder) SetTagName(tagName string)
• type InvalidDecoderError
  • func (e *InvalidDecoderError) Error() string
• type InvalidEncodeError
  • func (e *InvalidEncodeError) Error() string
• type Mode
• type TagNameFunc

Functions

func ExtractType

func ExtractType(current reflect.Value) (reflect.Value, reflect.Kind)

ExtractType gets the actual underlying type of field value. it is exposed for use within you Custom Functions

Types

type AnonymousMode

type AnonymousMode uint8

AnonymousMode specifies how data should be rolled up or separated from anonymous structs

const (
	// AnonymousEmbed embeds anonymous data when encoding
	// eg. type A struct { Field string }
	//     type B struct { A, Field string }
	//     encode results: url.Values{"Field":[]string{"B FieldVal", "A FieldVal"}}
	AnonymousEmbed AnonymousMode = iota

	// AnonymousSeparate does not embed anonymous data when encoding
	// eg. type A struct { Field string }
	//     type B struct { A, Field string }
	//     encode results: url.Values{"Field":[]string{"B FieldVal"}, "A.Field":[]string{"A FieldVal"}}
	AnonymousSeparate
)

type DecodeCustomTypeFunc

type DecodeCustomTypeFunc func([]string) (interface{}, error)

DecodeCustomTypeFunc allows for registering/overriding types to be parsed.

type DecodeErrors

type DecodeErrors map[string]error

DecodeErrors is a map of errors encountered during form decoding

func (DecodeErrors) Error

func (d DecodeErrors) Error() string

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder is the main decode instance

func NewDecoder

func NewDecoder() *Decoder

NewDecoder creates a new decoder instance with sane defaults

func (*Decoder) Decode

func (d *Decoder) Decode(v interface{}, values url.Values) (err error)

Decode parses the given values and sets the corresponding struct and/or type values

Decode returns an InvalidDecoderError if interface passed is invalid.

func (*Decoder) RegisterCustomTypeFunc

func (d *Decoder) RegisterCustomTypeFunc(fn DecodeCustomTypeFunc, types ...interface{})

RegisterCustomTypeFunc registers a CustomTypeFunc against a number of types. NOTE: This method is not thread-safe it is intended that these all be registered prior to any parsing

ADDITIONAL: if a struct type is registered, the function will only be called if a url.Value exists for the struct and not just the struct fields eg. url.Values{"User":"Name%3Djoeybloggs"} will call the custom type function with `User` as the type, however url.Values{"User.Name":"joeybloggs"} will not.

func (*Decoder) RegisterTagNameFunc

func (d *Decoder) RegisterTagNameFunc(fn TagNameFunc)

RegisterTagNameFunc registers a custom tag name parser function NOTE: This method is not thread-safe it is intended that these all be registered prior to any parsing

ADDITIONAL: once a custom function has been registered the default, or custom set, tag name is ignored and relies 100% on the function for the name data. The return value WILL BE CACHED and so return value must be consistent.

func (*Decoder) SetMaxArraySize

func (d *Decoder) SetMaxArraySize(size uint)

SetMaxArraySize sets maximum array size that can be created. This limit is for the array indexing this library supports to avoid potential DOS or man-in-the-middle attacks using an unusually high number. DEFAULT: 10000

func (*Decoder) SetMode

func (d *Decoder) SetMode(mode Mode)

SetMode sets the mode the decoder should run Default is ModeImplicit

func (*Decoder) SetTagName

func (d *Decoder) SetTagName(tagName string)

SetTagName sets the given tag name to be used by the decoder. Default is "form"

type EncodeCustomTypeFunc

type EncodeCustomTypeFunc func(x interface{}) ([]string, error)

EncodeCustomTypeFunc allows for registering/overriding types to be parsed.

type EncodeErrors

type EncodeErrors map[string]error

EncodeErrors is a map of errors encountered during form encoding

func (EncodeErrors) Error

func (e EncodeErrors) Error() string

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder is the main encode instance

func NewEncoder

func NewEncoder() *Encoder

NewEncoder creates a new encoder instance with sane defaults

func (*Encoder) Encode

func (e *Encoder) Encode(v interface{}) (values url.Values, err error)

Encode encodes the given values and sets the corresponding struct values

func (*Encoder) RegisterCustomTypeFunc

func (e *Encoder) RegisterCustomTypeFunc(fn EncodeCustomTypeFunc, types ...interface{})

RegisterCustomTypeFunc registers a CustomTypeFunc against a number of types NOTE: this method is not thread-safe it is intended that these all be registered prior to any parsing

func (*Encoder) RegisterTagNameFunc

func (e *Encoder) RegisterTagNameFunc(fn TagNameFunc)

RegisterTagNameFunc registers a custom tag name parser function NOTE: This method is not thread-safe it is intended that these all be registered prior to any parsing

ADDITIONAL: once a custom function has been registered the default, or custom set, tag name is ignored and relies 100% on the function for the name data. The return value WILL BE CACHED and so return value must be consistent.

func (*Encoder) SetAnonymousMode

func (e *Encoder) SetAnonymousMode(mode AnonymousMode)

SetAnonymousMode sets the mode the encoder should run Default is AnonymousEmbed

func (*Encoder) SetMode

func (e *Encoder) SetMode(mode Mode)

SetMode sets the mode the encoder should run Default is ModeImplicit

func (*Encoder) SetTagName

func (e *Encoder) SetTagName(tagName string)

SetTagName sets the given tag name to be used by the encoder. Default is "form"

type InvalidDecoderError

type InvalidDecoderError struct {
	Type reflect.Type
}

An InvalidDecoderError describes an invalid argument passed to Decode. (The argument passed to Decode must be a non-nil pointer.)

func (*InvalidDecoderError) Error

func (e *InvalidDecoderError) Error() string

type InvalidEncodeError

type InvalidEncodeError struct {
	Type reflect.Type
}

An InvalidEncodeError describes an invalid argument passed to Encode.

func (*InvalidEncodeError) Error

func (e *InvalidEncodeError) Error() string

type Mode

type Mode uint8

Mode specifies which mode the form decoder is to run

const (

	// ModeImplicit tries to parse values for all
	// fields that do not have an ignore '-' tag
	ModeImplicit Mode = iota

	// ModeExplicit only parses values for field with a field tag
	// and that tag is not the ignore '-' tag
	ModeExplicit
)

type TagNameFunc

type TagNameFunc func(field reflect.StructField) string

TagNameFunc allows for adding of a custom tag name parser