README
¶
State Encoding and Decoding
The state package implements the encoding and decoding of data structures for
go_stateify. This package is designed for use cases other than the standard
encoding packages, e.g. gob and json. Principally:
-
This package operates on complex object graphs and accurately serializes and restores all relationships. That is, you can have things like: intrusive pointers, cycles, and pointer chains of arbitrary depths. These are not handled appropriately by existing encoders. This is not an implementation flaw: the formats themselves are not capable of representing these graphs, as they can only generate directed trees.
-
This package allows installing order-dependent load callbacks and then resolves that graph at load time, with cycle detection. Similarly, there is no analogous feature possible in the standard encoders.
-
This package handles the resolution of interfaces, based on a registered type name. For interface objects type information is saved in the serialized format. This is generally true for
gobas well, but it works differently.
Here's an overview of how encoding and decoding works.
Encoding
Encoding produces a statefile, which contains a list of chunks of the form
(header, payload). The payload can either be some raw data, or a series of
encoded wire objects representing some object graph. All encoded objects are
defined in the wire subpackage.
Encoding of an object graph begins with encodeState.Save.
1. Memory Map & Encoding
To discover relationships between potentially interdependent data structures
(for example, a struct may contain pointers to members of other data
structures), the encoder first walks the object graph and constructs a memory
map of the objects in the input graph. As this walk progresses, objects are
queued in the pending list and items are placed on the deferred list as they
are discovered. No single object will be encoded multiple times, but the
discovered relationships between objects may change as more parts of the overall
object graph are discovered.
The encoder starts at the root object and recursively visits all reachable
objects, recording the address ranges containing the underlying data for each
object. This is stored as a segment set (addrSet), mapping address ranges to
the of the object occupying the range; see encodeState.values. Note that there
is special handling for zero-sized types and map objects during this process.
Additionally, the encoder assigns each object a unique identifier which is used
to indicate relationships between objects in the statefile; see objectID in
encode.go.
2. Type Serialization
The enoder will subsequently serialize all information about discovered types, including field names. These are used during decoding to reconcile these types with other internally registered types.
3. Object Serialization
With a full address map, and all objects correctly encoded, all object encodings
are serialized. The assigned objectIDs aren't explicitly encoded in the
statefile. The order of object messages in the stream determine their IDs.
Example
Given the following data structure definitions:
type system struct {
o *outer
i *inner
}
type outer struct {
a int64
cn *container
}
type container struct {
n uint64
elem *inner
}
type inner struct {
c container
x, y uint64
}
Initialized like this:
o := outer{
a: 10,
cn: nil,
}
i := inner{
x: 20,
y: 30,
c: container{},
}
s := system{
o: &o,
i: &i,
}
o.cn = &i.c
o.cn.elem = &i
Encoding will produce an object stream like this:
g0r1 = struct{
i: g0r3,
o: g0r2,
}
g0r2 = struct{
a: 10,
cn: g0r3.c,
}
g0r3 = struct{
c: struct{
elem: g0r3,
n: 0u,
},
x: 20u,
y: 30u,
}
Note how g0r3.c is correctly encoded as the underlying container object for
inner.c, and how the pointer from outer.cn points to it, despite system.i
being discovered after the pointer to it in system.o.cn. Also note that
decoding isn't strictly reliant on the order of encoded object stream, as long
as the relationship between objects are correctly encoded.
Decoding
Decoding reads the statefile and reconstructs the object graph. Decoding begins
in decodeState.Load. Decoding is performed in a single pass over the object
stream in the statefile, and a subsequent pass over all deserialized objects is
done to fire off all loading callbacks in the correctly defined order. Note that
introducing cycles is possible here, but these are detected and an error will be
returned.
Decoding is relatively straight forward. For most primitive values, the decoder
constructs an appropriate object and fills it with the values encoded in the
statefile. Pointers need special handling, as they must point to a value
allocated elsewhere. When values are constructed, the decoder indexes them by
their objectIDs in decodeState.objectsByID. The target of pointers are
resolved by searching for the target in this index by their objectID; see
decodeState.register. For pointers to values inside another value (fields in a
pointer, elements of an array), the decoder uses the accessor path to walk to
the appropriate location; see walkChild.
Documentation
¶
Overview ¶
Package state provides functionality related to saving and loading object graphs. For most types, it provides a set of default saving / loading logic that will be invoked automatically if custom logic is not defined.
Kind Support ---- ------- Bool default Int default Int8 default Int16 default Int32 default Int64 default Uint default Uint8 default Uint16 default Uint32 default Uint64 default Float32 default Float64 default Complex64 default Complex128 default Array default Chan custom Func custom Interface default Map default Ptr default Slice default String default Struct custom (*) Unless zero-sized. UnsafePointer custom
See README.md for an overview of how encoding and decoding works.
Index ¶
- func Failf(fmtStr string, v ...any)
- func IsZeroValue(val any) bool
- func ReadHeader(r *wire.Reader) (length uint64, object bool, err error)
- func Register(t Type)
- func Release()
- func WriteHeader(w *wire.Writer, length uint64, object bool) error
- type ErrState
- type SaverLoader
- type Sink
- type Source
- type Stats
- type Type
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Failf ¶
Failf is a wrapper around panic that should be used to generate errors that can be caught during saving and loading.
func IsZeroValue ¶
IsZeroValue checks if the given value is the zero value.
This function is used by the stateify tool.
func ReadHeader ¶
ReadHeader reads an object header.
Each object written to the statefile is prefixed with a header. See WriteHeader for more information; these functions are exported to allow non-state writes to the file to play nice with debugging tools.
func Register ¶
func Register(t Type)
Register registers a type.
This must be called on init and only done once.
func Release ¶
func Release()
Release releases references to global type databases. Must only be called in contexts where they will definitely never be used, in order to save memory.
func WriteHeader ¶
WriteHeader writes a header.
Each object written to the statefile should be prefixed with a header. In order to generate statefiles that play nicely with debugging tools, raw writes should be prefixed with a header with object set to false and the appropriate length. This will allow tools to skip these regions.
Types ¶
type ErrState ¶
type ErrState struct {
// contains filtered or unexported fields
}
ErrState is returned when an error is encountered during encode/decode.
type SaverLoader ¶
type SaverLoader interface {
// StateSave saves the state of the object to the given Map.
StateSave(Sink)
// StateLoad loads the state of the object.
StateLoad(context.Context, Source)
}
SaverLoader must be implemented by struct types.
type Sink ¶
type Sink struct {
// contains filtered or unexported fields
}
Sink is used for Type.StateSave.
func (Sink) Save ¶
Save adds the given object to the map.
You should pass always pointers to the object you are saving. For example:
type X struct {
A int
B *int
}
func (x *X) StateTypeInfo(m Sink) state.TypeInfo {
return state.TypeInfo{
Name: "pkg.X",
Fields: []string{
"A",
"B",
},
}
}
func (x *X) StateSave(m Sink) {
m.Save(0, &x.A) // Field is A.
m.Save(1, &x.B) // Field is B.
}
func (x *X) StateLoad(m Source) {
m.Load(0, &x.A) // Field is A.
m.Load(1, &x.B) // Field is B.
}
func (Sink) SaveValue ¶
SaveValue adds the given object value to the map.
This should be used for values where pointers are not available, or casts are required during Save/Load.
For example, if we want to cast external package type P.Foo to int64:
func (x *X) StateSave(m Sink) {
m.SaveValue(0, "A", int64(x.A))
}
func (x *X) StateLoad(m Source) {
m.LoadValue(0, new(int64), func(x any) {
x.A = P.Foo(x.(int64))
})
}
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source is used for Type.StateLoad.
func (Source) AfterLoad ¶
func (s Source) AfterLoad(fn func())
AfterLoad schedules a function execution when all objects have been allocated and their automated loading and customized load logic have been executed. fn will not be executed until all of current object's dependencies' AfterLoad() logic, if exist, have been executed.
func (Source) Load ¶
Load loads the given object passed as a pointer..
See Sink.Save for an example.
type Stats ¶
type Stats struct {
// contains filtered or unexported fields
}
Stats tracks encode / decode timing.
This currently provides a meaningful String function and no other way to extract stats about individual types.
All exported receivers accept nil.
type Type ¶
type Type interface {
// StateTypeName returns the type's name.
//
// This is used for matching type information during encoding and
// decoding, as well as dynamic interface dispatch. This should be
// globally unique.
StateTypeName() string
// StateFields returns information about the type.
//
// Fields is the set of fields for the object. Calls to Sink.Save and
// Source.Load must be made in-order with respect to these fields.
//
// This will be called at most once per serialization.
StateFields() []string
}
Type is an interface that must be implemented by Struct objects. This allows these objects to be serialized while minimizing runtime reflection required.
All these methods can be automatically generated by the go_statify tool.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package pretty is a pretty-printer for state streams.
|
Package pretty is a pretty-printer for state streams. |
|
Package statefile defines the state file data stream.
|
Package statefile defines the state file data stream. |
|
Package tests tests the state packages.
|
Package tests tests the state packages. |
|
Package wire contains a few basic types that can be composed to serialize graph information for the state package.
|
Package wire contains a few basic types that can be composed to serialize graph information for the state package. |