README
¶
Colfer 
Colfer is a binary serialization format optimized for speed and size.
The project's compiler colf(1) generates source code from schema definitions
to marshal and unmarshall data structures.
This is free and unencumbered software released into the public domain. The format is inspired by Protocol Buffer.
Language Support
- C, ISO/IEC 9899:2011 compliant a.k.a. C11, C++ compatible
- Go, a.k.a. golang
- Java, Android compatible
- JavaScript, a.k.a. ECMAScript, NodeJS compatible
Features
- Simple and straightforward in use
- No dependencies other than the core library
- Both faster and smaller than: Protocol Buffers, FlatBuffers and MessagePack
- Robust including size protection
- Maximum of 127 fields per data structure
- No support for enumerations
- Framed; suitable for concatenation/streaming
TODO's
- Rust and Python
- List support for integers and timestamps
Use
Download a prebuilt compiler
or run go get -u github.com/pascaldekloe/colfer/cmd/colf to make one yourself.
Without arguments the command prints its manual.
NAME
colf — compile Colfer schemas
SYNOPSIS
colf [ options ] language [ file ... ]
DESCRIPTION
Generates source code for a language. The options are: C, Go,
Java and JavaScript.
The file operands specify the input. Directories are scanned for
files with the colf extension. If file is absent, colf includes
the working directory.
A package can have multiple schema files.
OPTIONS
-b directory
Use a specific destination base directory. (default ".")
-f Normalizes schemas on the fly.
-l expression
Sets the default upper limit for the number of elements in a
list. The expression is applied to the target language under
the name ColferListMax. (default "64 * 1024")
-p prefix
Adds a package prefix. Use slash as a separator when nesting.
-s expression
Sets the default upper limit for serial byte sizes. The
expression is applied to the target language under the name
ColferSizeMax. (default "16 * 1024 * 1024")
-v Enables verbose reporting to the standard error.
-x class
Makes all generated classes extend a super class. Use slash as
a package separator. Java only.
EXIT STATUS
The command exits 0 on succes, 1 on compilation failure and 2
when invoked without arguments.
EXAMPLES
Compile ./io.colf with compact limits as C:
colf -b src -s 2048 -l 96 C io.colf
Compile ./api/*.colf in package com.example as Java:
colf -p com/example -x com/example/Parent Java api
BUGS
Report bugs at https://github.com/pascaldekloe/colfer/issues
Text validation is not part of the marshalling and unmarshalling
process. C and Go just pass any malformed UTF-8 characters. Java
and JavaScript replace unmappable content with the '?' character
(ASCII 63).
SEE ALSO
protoc(1)
It is recommended to commit the generated source code to the respective version control.
Alternatively, you may use the Maven plugin.
<plugin>
<groupId>net.quies.colfer</groupId>
<artifactId>colfer-maven-plugin</artifactId>
<version>1.11</version>
<configuration>
<packagePrefix>com/example</packagePrefix>
</configuration>
</plugin>
Schema
Data structures are defined in .colf files. The format is quite conventional.
// Package demo offers a demonstration.
// These comment lines will end up in the generated code.
package demo
// Course is the grounds where the game of golf is played.
type course struct {
ID uint64
name text
holes []hole
image binary
tags []text
}
type hole struct {
// Lat is the latitude of the cup.
lat float64
// Lon is the longitude of the cup.
lon float64
// Par is the difficulty index.
par uint8
// Water marks the presence of water.
water bool
// Sand marks the presence of sand.
sand bool
}
See what the generated code looks like in C, Go, Java or JavaScript.
The following table shows how Colfer data types are applied per language.
| Colfer | C | Go | Java | JavaScript |
|---|---|---|---|---|
| bool | char | bool | boolean | Boolean |
| uint8 | uint8_t | uint8 | byte † | Number |
| uint16 | uint16_t | uint16 | short † | Number |
| uint32 | uint32_t | uint32 | int † | Number |
| uint64 | uint64_t | uint64 | long † | Number ‡ |
| int32 | int32_t | int32 | int | Number |
| int64 | int64_t | int64 | long | Number ‡ |
| float32 | float | float32 | float | Number |
| float64 | double | float64 | double | Number |
| timestamp | timespec | time.Time †† | time.Instant | Date + Number |
| text | const char* + size_t | string | String †‡ | String †‡ |
| binary | uint8_t* + size_t | []byte | byte[] | Uint8Array |
| list | * + size_t | slice | array | Array |
- † signed representation of unsigned data, i.e. may overflow to negative.
- ‡ range limited to (1 - 2⁵³, 2⁵³ - 1)
- †† timezone not preserved
- †‡ characters limited by UTF-16 (
U+0000,U+10FFFF)
Lists may contain floating points, text, binaries or data structures.
Compatibility
Name changes do not affect the serialization format. Deprecated fields should be renamed to clearly discourage their use. For backwards compatibility new fields must be added to the end of colfer structs. Thus the number of fields can be seen as the schema version.
Performance
Colfer aims to be the fastest and the smallest format while still resilient to malicious input. See the benchmark wiki for a comparison. Suboptimal performance is treated like a bug.
Documentation
¶
Overview ¶
Package colfer provides schema definitions.
Index ¶
- func Format(file string) (changed bool, err error)
- func GenerateC(basedir string, packages Packages) error
- func GenerateECMA(basedir string, packages Packages) error
- func GenerateGo(basedir string, packages Packages) error
- func GenerateJava(basedir string, packages Packages) error
- func IsCKeyword(s string) bool
- func IsECMAKeyword(s string) bool
- func IsJavaKeyword(s string) bool
- type Field
- type Package
- type Packages
- type Struct
- func (s *Struct) DocText(indent string) string
- func (s *Struct) HasBinary() bool
- func (s *Struct) HasBinaryList() bool
- func (s *Struct) HasFloat() bool
- func (s *Struct) HasList() bool
- func (s *Struct) HasText() bool
- func (s *Struct) HasTimestamp() bool
- func (s *Struct) NameTitle() string
- func (s *Struct) String() string
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Format ¶
Format normalizes the file's content. The content of file is expected to be syntactically correct.
func GenerateECMA ¶
GenerateECMA writes the code into file "Colfer.js".
func GenerateGo ¶
GenerateGo writes the code into file "Colfer.go".
func GenerateJava ¶
GenerateJava writes the code into the respective ".java" files.
func IsCKeyword ¶
IsCKeyword returs whether s is a reserved word in C code.
func IsECMAKeyword ¶
IsECMAKeyword returs whether s is a reserved word in ECMAScript code.
func IsJavaKeyword ¶
IsJavaKeyword returs whether s is a reserved word in Java code.
Types ¶
type Field ¶
type Field struct {
// Struct is the parent.
Struct *Struct
// Index is the Struct.Fields position.
Index int
// Name is the identification token.
Name string
// NameNative is the language specific Name.
NameNative string
// Docs are the documentation texts.
Docs []string
// Type is the datatype.
Type string
// TypeNative is the language specific Type.
TypeNative string
// TypeRef is the Colfer data structure reference.
TypeRef *Struct
// TypeList flags whether the datatype is a list.
TypeList bool
}
Field is a Struct member definition.
type Package ¶
type Package struct {
// Name is the identification token.
Name string
// NameNative is the language specific Name.
NameNative string
// Docs are the documentation texts.
Docs []string
// Structs are the type definitions.
Structs []*Struct
// SchemaFiles are the source filenames.
SchemaFiles []string
// SizeMax is the uper limit expression.
SizeMax string
// ListMax is the uper limit expression.
ListMax string
// SuperClass is the fully qualified path.
SuperClass string
// SuperClassNative is the language specific SuperClass.
SuperClassNative string
}
Package is a named definition bundle.
func ParseFiles ¶
ParseFiles returns the schema definitions.
func (*Package) HasTimestamp ¶
HasTimestamp returns whether p has one or more timestamp fields.
func (*Package) SchemaFileList ¶
SchemaFileList returns a listing text.
type Packages ¶ added in v1.7.1
type Packages []*Package
func (Packages) HasTimestamp ¶ added in v1.7.1
HasTimestamp returns whether any of the packages has one or more timestamp fields.
type Struct ¶
type Struct struct {
Pkg *Package
// Name is the identification token.
Name string
// NameNative is the language specific Name.
NameNative string
// Docs are the documentation texts.
Docs []string
// Fields are the elements in order of appearance.
Fields []*Field
// SchemaFile is the source filename.
SchemaFile string
}
Struct is a data structure definition.
func (*Struct) HasBinaryList ¶
HasBinaryList returns whether s has one or more binary list fields.
func (*Struct) HasTimestamp ¶
HasTimestamp returns whether s has one or more timestamp fields.
Source Files
Directories