MessagePack Code Generator
This is a code generation tool and serialization library for MesssagePack. It is targeted at the go generate
tool. You can read more about MessagePack in the wiki, or at msgpack.org.
Why?
Quickstart
Note: you need at least go 1.3 to compile this package, and at least go 1.4 to use go generate
.
In a source file, include the following directive:
//go:generate msgp
The msgp
command will generate serialization methods for all exported struct
definitions in the file. You will need to include that directive in every file that contains structs that
need code generation.
You can read more about the code generation options here.
Use
Field names can be set in much the same way as the encoding/json
package. For example:
type Person struct {
Name string `msg:"name"`
Address string `msg:"address"`
Age int `msg:"age"`
Hidden string `msg:"-"` // this field is ignored
unexported bool // this field is also ignored
}
By default, the code generator will satisfy msgp.Sizer
, msgp.Encodable
, msgp.Decodable
,
msgp.Marshaler
, and msgp.Unmarshaler
. Carefully-designed applications can use these methods to do
marshalling/unmarshalling with zero allocations.
While msgp.Marshaler
and msgp.Unmarshaler
are quite similar to the standard library's
json.Marshaler
and json.Unmarshaler
, msgp.Encodable
and msgp.Decodable
are useful for
stream serialization. (*msgp.Writer
and *msgp.Reader
are essentially protocol-aware versions
of *bufio.Writer
and *bufio.Reader
, respectively.)
Features
- Extremely fast generated code
- JSON interoperability (see
msgp.CopyToJSON() and msgp.UnmarshalAsJSON()
)
- Support for embedded fields, anonymous structs, and multi-field inline declarations
- Identifier resolution (see below)
- Native support for Go's
time.Time
, complex64
, and complex128
types
- Generation of both
[]byte
-oriented and io.Reader/io.Writer
-oriented methods
- Support for arbitrary type system extensions
- Preprocessor directives
Because of (limited) identifier resolution, the code generator will still yield the
correct code for the following struct declaration:
const Eight = 8
type MyInt int
type Data []byte
type Struct struct {
Which map[string]*MyInt `msg:"which"`
Other Data `msg:"other"`
Nums [Eight]float64 `msg:"nums"`
}
As long as the declarations of MyInt
and Data
are in the same file as Struct
, the parser will figure out that
MyInt
is really an int
, and Data
is really just a []byte
. The constant Eight
does not have to be in the
same file as the struct definition, but it does have to be in the same package (as the generated code will simply
use Eight
as a literal under the assumption that the compiler will figure out what it is.) Note that this only works for "base" types (no composite types, although []byte
is supported as a special case.) Unresolved identifiers are (optimistically)
assumed to be struct definitions in other files. (The parser will spit out warnings about unresolved identifiers.)
Extensions
MessagePack supports defining your own types through "extensions," which are just a tuple of
the data "type" (int8
) and the raw binary. You can see a worked example in the wiki.
Status
Alpha. I will break stuff. There is an open milestone for Beta stability (targeted for January.) Only the /msgp
sub-directory will have a stability guarantee.
You can read more about how msgp
maps MessagePack types onto Go types in the wiki.
Here some of the known limitations/restrictions:
- All fields of a struct that are not Go built-ins are assumed (optimistically) to have been seen by the code generator in another file. The generator will output a warning if it can't resolve an identifier in the file, or if it ignores an exported field. The generated code will fail to compile if you encounter this issue, so it shouldn't catch you by surprise.
- Like most serializers,
chan
and func
fields are ignored, as well as non-exported fields.
- Methods are only generated for
struct
definitions. Chances are that we will keep things this way.
- Encoding of
interface{}
is limited to built-ins or types that have explicit encoding methods.
- Maps must have
string
keys. This is intentional (as it preserves JSON interop.) Although non-string map keys are not forbidden by the MessagePack standard, many serializers impose this restriction. (It also means any well-formed struct
can be de-serialized into a map[string]interface{}
.) The only exception to this rule is that the deserializers will allow you to read map keys encoded as bin
types, due to the fact that some legacy encodings permitted this. (However, those values will still be cast to Go string
s, and they will be converted to str
types when re-encoded. It is the responsibility of the user to ensure that map keys are UTF-8 safe in this case.) The same rules hold true for JSON translation.
- All variable-length objects (maps, strings, arrays, extensions, etc.) cannot have more than
(1<<32)-1
elements.
If the output compiles, then there's a pretty good chance things are fine. (Plus, we generate tests for you.) Please, please, please file an issue if you think the generator is writing broken code.
If you like benchmarks, see here. In general, you pay a performance penalty using this code generator over the gogoprotobuf (protocol buffers) generator, but you get substantially better JSON interop and dedicated io.Reader/Writer
methods, along with the ability to use Go as your schema language.
As one might expect, the generated methods that deal with []byte
are faster, but the io.Reader/Writer
methods are generally more memory-efficient for large (> 2KB) objects. As always, benchmark for your particular use case.