structex

package module
v1.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2021 License: MIT Imports: 9 Imported by: 5

README

STRUCTure EXtensions

structex provides annotation rules that extend Go structures for implementation of encoding and decoding of byte backed data frames.

structex aims to make it easier to represent industry standard data formats that can be directly embedded in code.

Example

Given the first three bytes T10 SCSI Primary Commands definition of a SCSI INQUIRY data

A structex annotated struct type declaration might be

package main

import (
    "fmt"

    "github.com/HewlettPackard/structex"
)

type SCSI_Standard_Inquiry struct {
    PeripheralDeviceType uint8 `bitfield:"5"`          // Byte 0
    PeripheralQualifier  uint8 `bitfield:"3"`
    reserved0            uint8 `bitfield:"6,reserved"` // Byte 1
    LU_Cong              uint8 `bitfield:"1"`
    RMB                  uint8 `bitfield:"1"`
    Version              uint8                         // Byte 2
}

func main() {
    var inquiry = new(SCSI_Standard_Inquiry)

    // Perform IOCTL on device, returns byte reader to byte response

    if err := structex.Decode(byteReader, inquiry); err != nil {
        panic("a problem")
    }

    fmt.Printf("SCSI Peripheral Device Type: %d\n", inquiry.PeripheralDeviceType)
    fmt.Printf("SCSI Peripheral Qualifier: %d\n", inquiry.PeripheralQualifier)
    fmt.Printf("Inquiry Version: %#02x\n", inquiry.Version)
    // etc
}

Annotation Format

Endianness

Little-endian (default) or big-endian tags support for integer and unsigned integer types little:""

big:""

Bitfield

Bitfields define a structure field with an explicit size in bits. They are analogous to bit fields in the C specification.

bitfield:"[size][,reserved]"

`size`:     Specifies the size, in bits, of the field.

`reserved`: Optional modifier that specifies the field contains reserved
            bits and should be encoded as zeros.
Self-Described Layout

Many industry standards support dynamically sized return fields where the data layout is self described by other fields. To support such formats two annotations are provided.

sizeof:"[name][,relative]"

name        Specifies that the field describes the size of `name` within the
            structure. Should be used with slices.

            During decoding, if field is non-zero, the field's value is
            used to limit the number elements in the array or slice of
            name `name`.

relative    Optional modifier that specifies the value describing the size of
            `name` is relative to the field offset within the structure.
            This is often used in T10.org documentation

countof:"[name]"

name        Specifies that the value describes the count of elements in
            the `name` field.

            During decoding, if field is non-zero, the field's value is
            used to limit the number elements in the array or slice of
            name `name`.
Truncation

Structex expects sufficient data for decoding the desired structure. When data structures are used to define a maximum size of the response buffer, you can use the truncate tag on an arry or slice to permit structex to truncate the returned data with what is provided by the source buffer. If truncate is not specified and the provided buffer is of smaller size than the data structure, an io.EOF error is expected.

truncate:""

Alignment

Annotations can specified the byte-alignment requirement for structure fields. Analogous to the alignas specifier in C. Can only be applied to non-bitfield structure fields.

align:"[value]"

value       An integer value specifying the byte alignment of the field.
            Invalid non-zero alignments panic.

Full Tags

The tags documented above are abbreviated for ease of use; if desired, the full tag format is supported. This provides clarity that tags are part of the structex package, but it means typing more and using every sort of quote and backtic at your disposal.

Examples

`structex:"little"`
`structex:"big"`
`structex:"bitfield='3'"`
`structex:"bitfield='3,reserved'"`
`structex:"countOf='D'"`
`structex:"sizeOf='E'"`
`structex:"sizeOf='F,relative'"`
`structex:"align='8'"`
`structex:"truncate"`

Performance

structex places code readability ahead of any pack/unpack performance. Reflection is used to analyze the structure definitions, which itself carries heavy overhead. Parsing the tags also takes time.

For those looking for more performant code, consider code generation or a cache based solutions

Documentation

Overview

Copyright 2021 Hewlett Packard Enterprise Development LP

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Index

Constants

This section is empty.

Variables

View Source 
var (
	CannotDeductSliceLengthError = errors.New("Cannot duduct slice length")
)

Functions

func Decode 

func Decode(reader io.ByteReader, s interface{}) error

Decode reads data from a ByteReader into provided annotated structure.

Deserialization occurs according to the annotations in the structure which take several options:

Bitfields: 

Bitfields define a structure field with an explicit size in bits. They are
analogous to bit fields in the C specification. Un

`bitfield:[size][,reserved]`

size       Specifies the size, in bits, of the field.

reserved   Optional modifier that specifies the field contains reserved
           bits and should be encoded as zeros.

Dynamic Layouts: 

Many industry standards support dynamically sized return fields where the
data layout is self described by other fields. To support such formats
two annotations are provided.

`sizeOf:"[Field][,relative]"`

Field		Specifies that the field describes the size of Field within the
			structure.

			During decoding, if field is non-zero, the field's value is
			used to limit the number elements in the array or slice of
			name Field.

relative	Optional modifier that specifies the field value describing
			the size of Field is relative to the field offset within
			the structure. This is often used in T10.org documentation

`countOf:"[Field]"`

Field		Specifies that the field describes the count of elements in
			Field.

			During decoding, if field is non-zero, the field's value is
			used to limit the number elements in the array or slice of
			name Field.

Alignment: 

Annotations can specified the byte-alignment requirement for structure
fields. Analogous to the alignas specifier in C. Can only be applied
to non-bitfield structure fields.

`align:"[value]"`

value		An integer value specifying the byte alignment of the field.
			Invalid non-zero alignments panic.

func DecodeByteBuffer 

func DecodeByteBuffer(b *bytes.Buffer, s interface{}) error

DecodeByteBuffer takes a raw byte buffer and unpacks the buffer into the provided structure. Unused bytes do not cause an error.

func Encode 

func Encode(writer io.ByteWriter, s interface{}) error

Encode serializes the data structure defined by 's' into the available io.ByteWriter stream. Annotation rules are as defined in the Decode function.

func EncodeByteBuffer added in v1.0.1 

func EncodeByteBuffer(s interface{}) ([]byte, error)

EncodeByteBuffer serializes the provided data structure 's' into a new byte buffer. Bytes are packed according to the annotation rules defined for 's'.

func Size 

func Size(s interface{}) (uint64, error)

Size returns the size of the structure after considering all annotation rules. Annotation rules are defined in the Decode function.

Size cannot determine the length of Slice types unless a layout annotation exists (`sizeOf` or `countOf`) that contains a non-zero value.

Types

type Buffer 

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer forms the basis of the storage area used for Pack and Unpack operations.

func NewBuffer 

func NewBuffer(s interface{}) *Buffer

NewBuffer returns a new Buffer for making Pack and Unpack operations ahead of the file Write and after file Read operations, respectfully.

The general pattern is to declare an annotated structure and fill in the desired fields. For example 

    type Test struct {
	    Param1 uint32 `bitfield:"5"`
	    Param2 uint32 `bitfield:"3"`
    }

would declare variable test with parameters 

var test Test
test.Param1 = 3
test.Param2 = 1

A buffer is allocated of appropriate size using the NewBuffer function 

b := structex.NewBuffer(test)

for which the structure is packed into 

structex.Pack(b, test)

The Buffer can then be used for a File Write operations 

f, err := os.Open("path/to/file")
nbytes, err := f.Write(b.Bytes())

or used for File Read operations 

nbytes, err := f.Read(b.Bytes())

for which the Buffer can feed into Unpack to decode the available bytes 

var unpacked = new (Test)
err := structex.Unpack(b, unpacked)

func (*Buffer) Bytes 

func (buf *Buffer) Bytes() []byte

Bytes returns the raw bytes forming the basis of the Buffer

func (*Buffer) DebugDump 

func (buf *Buffer) DebugDump()

DebugDump will print the Buffer in raw byte format, 16 bytes per line. Useful for debugging structure packing and unpacking.

func (*Buffer) ReadByte 

func (buf *Buffer) ReadByte() (byte, error)

ReadByte implements the io.Reader requirement of the Buffer

func (*Buffer) Reset 

func (buf *Buffer) Reset()

Reset will clear the buffer for reuse

func (*Buffer) WriteByte 

func (buf *Buffer) WriteByte(b byte) error

WriteByte implements the io.Writer requirement for the Buffer

type TaggingError 

type TaggingError struct {
	// contains filtered or unexported fields
}

A TaggingError occurs when the pack/unpack routines have detected the structure field annotation does not match what is expected in the data stream.

func (*TaggingError) Error 

func (e *TaggingError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.