README
¶
Fixed width file parser (decoder) for GO (golang)
Updated library derived from Oleg Lobanov's fwencoder with some aspects of Ian Lopshire's go-fixedwidth
Ths version currently only decodes but has a few additional features.
- It supports the TextMarshaler/TextUnmarshaler interface
- It allows multiple calls to the decoder by allowing a pointer to a struct to be passed to it as well as a slice.
- It's slightly faster because it caches conversion functions
- It supports arbitrary record endings and field conversions
- It allows the headers to be predefined by the caller
- It does not support JSON decoding for complex data structures.
- Encoding is also unsupported.
This library is using to parse fixed-width table data like:
Name Address Postcode Phone Credit Limit Birthday
Evan Whitehouse V4560 Camel Back Road 3122 (918) 605-5383 1000000.5 19870101
Chuck Norris P.O. Box 872 77868 (713) 868-6003 10909300 19651203
Install
To install the library use the following command:
$ go get -u github.com/goslogan/fw
Simple example
Parsing data from io.Reader:
type Person struct {
Name string
Address string
Postcode int
Phone string
CreditLimit float64 `json:"Credit Limit"`
Bday time.Time `column:"Birthday" format:"20060102"`
}
input, _ := os.ReadFile("/path/to/file")
defer f.Close
var people []Person
err := fw.Unmarshal(input, &people)
Documentation
¶
Overview ¶
Package fw defines a model for converting fixed width input data into Redis structs.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Unmarshal ¶
Unmarshal decodes a buffer into the array or structed pointed to by v If v is not an array only the first record will be read
Example ¶
source := []byte("name dob \nPeter 2008-10-11\nNicki 1987-01-28")
type Person struct {
Name string `column:"name"`
DOB time.Time `column:"dob" format:"2006-01-02"`
}
people := []Person{}
err := Unmarshal(source, &people)
if err != nil {
panic(err)
}
fmt.Printf("Number of records: %d\n", len(people))
for _, person := range people {
fmt.Printf("%s=%s\n", person.Name, person.DOB.Format(time.RFC822))
}
Output: Number of records: 2 Peter=11 Oct 08 00:00 UTC Nicki=28 Jan 87 00:00 UTC
func UnmarshalReader ¶
UnmarshalReader decodes an io.Reader into the array or structed pointed to by v If v is not an array only the first record will be read
Types ¶
type CastingError ¶ added in v0.1.1
type CastingError struct {
Value string
Err error
Field reflect.StructField
}
func (*CastingError) Error ¶ added in v0.1.1
func (err *CastingError) Error() string
type Decoder ¶
type Decoder struct {
RecordTerminator []byte // RecordTerminator identifies the sequence of bytes used to indicate end of record (default is "\n")
FieldSeparator string // FieldSeparator is used to identify the characters between fields and also to trim those characters. It's used as part of a regular expression (default is a space)
SkipFirstRecord bool // SkipFirstRecord defines whether the first line should be ignored.
// By default, it is not skipped. If SetColumns is called, headers will be skipped.
// It may then be desirable to reset it. If SetColumns has been called, the headers
// will be read and discarded if SkipFirstRecord is true
IgnoreEmptyRecords bool // IgnoreEmptyRecores can be set to true to so that empty records
// will not cause an invalid record length error
SkipLengthCheck bool // SkipLengthCheck can be set to true to allow records to have a different
// contains filtered or unexported fields
}
A Decoder reads and decodes fixed width data from an input stream. The caller can either define field sizes directly via Decoder.SetHeaders or they can be read from the first line of input.
Annotations ¶
Structs are annotated with the name of the input field/column with the column annotation. Referencing a column which does not exist will cause the field to be silently ignored during processing. Given the range of date/time formats in data, time.Time fields are supported additionally by the format annotation which allows the template for time.ParseDate to be provided.
Usable target structures ¶
The data structure passed to Decoder.Decode or Unmarshal must be a pointer to an existing slice or a pointer to a struct. If a slice is provided, it must contain structs or pointers to structs. It can be empty. Data is appended to the slice.
All basic go data types are supported automatically. As mentioned above time.Time is supported explicitly. Any other data type must support the encoding.TextUnmarshaler interface. Any other data type will cause an error to be returned.
Example ¶
source := []byte("name dob \nPeter 2008-10-11\nNicki 1987-01-28")
type Person struct {
Name string `column:"name"`
DOB time.Time `column:"dob" format:"2006-01-02"`
}
person := Person{}
decoder := NewDecoder(bytes.NewBuffer(source))
err := decoder.Decode(&person)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", person)
Output: {Name:Peter DOB:2008-10-11 00:00:00 +0000 UTC}
Example (Explicit) ¶
source := []byte("Peter 2008-10-11\nNicki 1987-01-28")
type Person struct {
Name string `column:"name"`
DOB time.Time `column:"dob" format:"2006-01-02"`
}
columns := map[string][]int{"name": {0, 8}, "dob": {8, 18}}
person := Person{}
decoder := NewDecoder(bytes.NewBuffer(source))
decoder.SetHeaders(columns)
err := decoder.Decode(&person)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", person)
Output: {Name:Peter DOB:2008-10-11 00:00:00 +0000 UTC}
func NewDecoder ¶
NewDecoder returns a new decoder that reads from r.
func (*Decoder) Decode ¶
Decode reads from its input and stores the decoded data to the value pointed to by v. v may point to a struct or a slice of structs (or pointers to structs)
Currently, the maximum decodable line length is bufio.MaxScanTokenSize-1. ErrTooLong is returned if a line is encountered that too long to decode.
func (*Decoder) SetHeaders ¶
SetHeaders overrides any headers parsed from the first line of input. If decoder.SetHeaders is called , decoder.SkipFirstRecord is set to false. If decoder.SkipFirstRecord is then set to true, the first line will be read but not parsed
type InvalidInputError ¶ added in v0.1.1
An InvalidInputError is returned when the input to Decode is not usable
func (*InvalidInputError) Error ¶ added in v0.1.1
func (err *InvalidInputError) Error() string
type InvalidLengthError ¶ added in v0.1.1
type InvalidLengthError struct {
Headers map[string][]int
Line string
LineNum int
HeadersLength int
}
An InvalidLengthError describes the state of decoding when a data record does not have the same length as the headers indicated.
func (*InvalidLengthError) Error ¶ added in v0.1.1
func (err *InvalidLengthError) Error() string
type InvalidTypeError ¶ added in v0.1.1
type InvalidTypeError struct {
Field reflect.StructField
}
func (*InvalidTypeError) Error ¶ added in v0.1.1
func (err *InvalidTypeError) Error() string
type InvalidUnmarshalError ¶
An InvalidUnmarshalError describes an invalid argument passed to Unmarshal. (The argument to Unmarshal must be a non-nil pointer.)
func (*InvalidUnmarshalError) Error ¶
func (err *InvalidUnmarshalError) Error() string
type OverflowError ¶ added in v0.1.1
type OverflowError struct {
Value interface{}
Field reflect.StructField
}
func (*OverflowError) Error ¶ added in v0.1.1
func (err *OverflowError) Error() string
Source Files