Documentation ¶
Overview ¶
Package csvutil provides fast and idiomatic mapping between CSV and Go values.
This package does not provide a CSV parser itself, it is based on the Reader and Writer interfaces which are implemented by eg. std csv package. This gives a possibility of choosing any other CSV writer or reader which may be more performant.
Index ¶
- Variables
- func Header(v interface{}, tag string) ([]string, error)
- func Marshal(v interface{}) ([]byte, error)
- func Unmarshal(data []byte, v interface{}) error
- type Decoder
- type Encoder
- type InvalidDecodeError
- type InvalidEncodeError
- type InvalidMarshalError
- type InvalidUnmarshalError
- type Marshaler
- type MarshalerError
- type Reader
- type UnmarshalTypeError
- type Unmarshaler
- type UnsupportedTypeError
- type Writer
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrFieldCount = errors.New("wrong number of fields in record")
ErrFieldCount is returned when header's length doesn't match the length of the read record.
Functions ¶
func Header ¶ added in v1.1.0
Header scans the provided struct type and generates a CSV header for it.
Field names are written in the same order as struct fields are defined. Embedded struct's fields are treated as if they were part of the outer struct. Fields that are embedded types and that are tagged are treated like any other field.
Unexported fields and fields with tag "-" are ignored.
Tagged fields have the priority over non tagged fields with the same name.
Following the Go visibility rules if there are multiple fields with the same name (tagged or not tagged) on the same level and choice between them is ambiguous, then all these fields will be ignored.
It is a good practice to call Header once for each type. The suitable place for calling it is init function. Look at Decoder.DecodingDataWithNoHeader example.
If tag is left empty the default "csv" will be used.
Header will return UnsupportedTypeError if the provided value is nil or is not a struct.
Example ¶
package main import ( "fmt" "log" "github.com/jszwec/csvutil" ) func main() { type User struct { ID int Name string Age int `csv:",omitempty"` State int `csv:"-"` City string ZIP string `csv:"zip_code"` } header, err := csvutil.Header(User{}, "csv") if err != nil { log.Fatal(err) } fmt.Println(header) }
Output: [ID Name Age City zip_code]
func Marshal ¶
Marshal returns the CSV encoding of slice v. If v is not a slice or elements are not structs then Marshal returns InvalidMarshalError.
Marshal uses the std encoding/csv.Writer with its default settings for csv encoding.
Marshal will always encode the CSV header even for the empty slice.
For the exact encoding rules look at Encoder.Encode method.
Example ¶
package main import ( "fmt" "github.com/jszwec/csvutil" ) func main() { type Address struct { City string Country string } type User struct { Name string Address Age int `csv:"age,omitempty"` } users := []User{ {Name: "John", Address: Address{"Boston", "USA"}, Age: 26}, {Name: "Bob", Address: Address{"LA", "USA"}, Age: 27}, {Name: "Alice", Address: Address{"SF", "USA"}}, } b, err := csvutil.Marshal(users) if err != nil { fmt.Println("error:", err) } fmt.Println(string(b)) }
Output: Name,City,Country,age John,Boston,USA,26 Bob,LA,USA,27 Alice,SF,USA,
Example (CustomMarshalCSV) ¶
package main import ( "fmt" "github.com/jszwec/csvutil" ) type Status uint8 const ( Unknown = iota Success Failure ) func (s Status) MarshalCSV() ([]byte, error) { switch s { case Success: return []byte("success"), nil case Failure: return []byte("failure"), nil default: return []byte("unknown"), nil } } type Job struct { ID int Status Status } func main() { jobs := []Job{ {1, Success}, {2, Failure}, } b, err := csvutil.Marshal(jobs) if err != nil { fmt.Println("error:", err) } fmt.Println(string(b)) }
Output: ID,Status 1,success 2,failure
func Unmarshal ¶
Unmarshal parses the CSV-encoded data and stores the result in the slice pointed to by v. If v is nil or not a pointer to a slice, Unmarshal returns an InvalidUnmarshalError.
Unmarshal uses the std encoding/csv.Reader for parsing and csvutil.Decoder for populating the struct elements in the provided slice. For exact decoding rules look at the Decoder's documentation.
The first line in data is treated as a header. Decoder will use it to map csv columns to struct's fields.
In case of success the provided slice will be reinitialized and its content fully replaced with decoded data.
Example ¶
package main import ( "fmt" "github.com/jszwec/csvutil" ) func main() { var csvInput = []byte(` name,age jacek,26 john,27`, ) type User struct { Name string `csv:"name"` Age int `csv:"age"` } var users []User if err := csvutil.Unmarshal(csvInput, &users); err != nil { fmt.Println("error:", err) } fmt.Printf("%+v", users) }
Output: [{Name:jacek Age:26} {Name:john Age:27}]
Types ¶
type Decoder ¶
type Decoder struct { // Tag defines which key in the struct field's tag to scan for names and // options (Default: 'csv'). Tag string // contains filtered or unexported fields }
A Decoder reads and decodes string records into structs.
Example (CustomUnmarshalCSV) ¶
package main import ( "fmt" "strconv" "github.com/jszwec/csvutil" ) type Bar int func (b *Bar) UnmarshalCSV(data []byte) error { n, err := strconv.Atoi(string(data)) *b = Bar(n) return err } type Foo struct { Int int `csv:"int"` Bar Bar `csv:"bar"` } func main() { var csvInput = []byte(` int,bar 5,10 6,11`) var foos []Foo if err := csvutil.Unmarshal(csvInput, &foos); err != nil { fmt.Println("error:", err) } fmt.Printf("%+v", foos) }
Output: [{Int:5 Bar:10} {Int:6 Bar:11}]
Example (DecodeEmbedded) ¶
package main import ( "encoding/csv" "fmt" "io" "log" "strings" "github.com/jszwec/csvutil" ) func main() { type Address struct { ID int `csv:"id"` // same field as in User - this one will be empty City string `csv:"city"` State string `csv:"state"` } type User struct { Address ID int `csv:"id"` // same field as in Address - this one wins Name string `csv:"name"` Age int `csv:"age"` } csvReader := csv.NewReader(strings.NewReader( "id,name,age,city,state\n" + "1,alice,25,la,ca\n" + "2,bob,30,ny,ny")) dec, err := csvutil.NewDecoder(csvReader) if err != nil { log.Fatal(err) } var users []User for { var u User if err := dec.Decode(&u); err == io.EOF { break } else if err != nil { log.Fatal(err) } users = append(users, u) } fmt.Println(users) }
Output: [{{0 la ca} 1 alice 25} {{0 ny ny} 2 bob 30}]
Example (DecodingDataWithNoHeader) ¶
package main import ( "bytes" "encoding/csv" "fmt" "io" "log" "github.com/jszwec/csvutil" ) type User struct { ID int Name string Age int `csv:",omitempty"` State int `csv:"-"` City string ZIP string `csv:"zip_code"` } var userHeader []string func init() { h, err := csvutil.Header(User{}, "csv") if err != nil { log.Fatal(err) } userHeader = h } func main() { data := []byte(` 1,John,27,la,90005 2,Bob,,ny,10005`) r := csv.NewReader(bytes.NewReader(data)) dec, err := csvutil.NewDecoder(r, userHeader...) if err != nil { log.Fatal(err) } var users []User for { var u User if err := dec.Decode(&u); err == io.EOF { break } else if err != nil { log.Fatal(err) } users = append(users, u) } fmt.Printf("%+v", users) }
Output: [{ID:1 Name:John Age:27 State:0 City:la ZIP:90005} {ID:2 Name:Bob Age:0 State:0 City:ny ZIP:10005}]
func NewDecoder ¶
NewDecoder returns a new decoder that reads from r.
Decoder will match struct fields according to the given header.
If header is empty NewDecoder will read one line and treat it as a header.
Records coming from r must be of the same length as the header.
NewDecoder may return io.EOF if there is no data in r and no header was provided by the caller.
func (*Decoder) Decode ¶
Decode reads the next string record from its input and stores it in the value pointed to by v which must be a non-nil struct pointer.
Decode matches all exported struct fields based on the header. Struct fields can be adjusted by using tags.
The "omitempty" option specifies that the field should be omitted from the decoding if record's field is an empty string.
Examples of struct field tags and their meanings:
// Decode matches this field with "myName" header column. Field int `csv:"myName"` // Decode matches this field with "Field" header column. Field int // Decode matches this field with "myName" header column and decoding is not // called if record's field is an empty string. Field int `csv:"myName,omitempty"` // Decode matches this field with "Field" header column and decoding is not // called if record's field is an empty string. Field int `csv:",omitempty"` // Decode ignores this field. Field int `csv:"-"`
By default decode looks for "csv" tag, but this can be changed by setting Decoder.Tag field.
To Decode into a custom type v must implement csvutil.Unmarshaler or encoding.TextUnmarshaler.
Anonymous struct fields with tags are treated like normal fields and they must implement csvutil.Unmarshaler or encoding.TextUnmarshaler.
Anonymous struct fields without tags are populated just as if they were part of the main struct. However, fields in the main struct have bigger priority and they are populated first. If main struct and anonymous struct field have the same fields, the main struct's fields will be populated.
Fields of type []byte expect the data to be base64 encoded strings.
Float fields are decoded to NaN if a string value is 'NaN'. This check is case insensitive.
Example ¶
package main import ( "encoding/csv" "fmt" "io" "log" "strings" "github.com/jszwec/csvutil" ) func main() { type User struct { ID *int `csv:"id,omitempty"` Name string `csv:"name"` City string `csv:"city"` Age int `csv:"age"` } csvReader := csv.NewReader(strings.NewReader(` id,name,age,city ,alice,25,la ,bob,30,ny`)) dec, err := csvutil.NewDecoder(csvReader) if err != nil { log.Fatal(err) } var users []User for { var u User if err := dec.Decode(&u); err == io.EOF { break } else if err != nil { log.Fatal(err) } users = append(users, u) } fmt.Println(users) }
Output: [{<nil> alice la 25} {<nil> bob ny 30}]
func (*Decoder) Header ¶
Header returns the first line that came from the reader, or returns the defined header by the caller.
func (*Decoder) Record ¶
Record returns the most recently read record. The slice is valid until the next call to Decode.
func (*Decoder) Unused ¶
Unused returns a list of column indexes that were not used during decoding due to lack of matching struct field.
Example ¶
package main import ( "encoding/csv" "fmt" "io" "log" "strings" "github.com/jszwec/csvutil" ) func main() { type User struct { Name string `csv:"name"` City string `csv:"city"` Age int `csv:"age"` OtherData map[string]string `csv:"-"` } csvReader := csv.NewReader(strings.NewReader(` name,age,city,zip alice,25,la,90005 bob,30,ny,10005`)) dec, err := csvutil.NewDecoder(csvReader) if err != nil { log.Fatal(err) } header := dec.Header() var users []User for { u := User{OtherData: make(map[string]string)} if err := dec.Decode(&u); err == io.EOF { break } else if err != nil { log.Fatal(err) } for _, i := range dec.Unused() { u.OtherData[header[i]] = dec.Record()[i] } users = append(users, u) } fmt.Println(users) }
Output: [{alice la 25 map[zip:90005]} {bob ny 30 map[zip:10005]}]
type Encoder ¶
type Encoder struct { // Tag defines which key in the struct field's tag to scan for names and // options (Default: 'csv'). Tag string // If AutoHeader is true, a struct header is encoded during the first call // to Encode automatically (Default: true). AutoHeader bool // contains filtered or unexported fields }
Encoder writes structs CSV representations to the output stream.
func NewEncoder ¶
NewEncoder returns a new encoder that writes to w.
func (*Encoder) Encode ¶
Encode writes the CSV encoding of v to the output stream. The provided argument v must be a non-nil struct.
Only the exported fields will be encoded.
First call to Encode will write a header unless EncodeHeader was called first or AutoHeader is false. Header names can be customized by using tags ('csv' by default), otherwise original Field names are used.
Header and fields are written in the same order as struct fields are defined. Embedded struct's fields are treated as if they were part of the outer struct. Fields that are embedded types and that are tagged are treated like any other field, but they have to implement Marshaler or encoding.TextMarshaler interfaces.
Marshaler interface has the priority over encoding.TextMarshaler.
Tagged fields have the priority over non tagged fields with the same name.
Following the Go visibility rules if there are multiple fields with the same name (tagged or not tagged) on the same level and choice between them is ambiguous, then all these fields will be ignored.
Nil values will be encoded as empty strings. Same will happen if 'omitempty' tag is set, and the value is a default value like 0, false or nil interface.
Bool types are encoded as 'true' or 'false'.
Float types are encoded using strconv.FormatFloat with precision -1 and 'G' format. NaN values are encoded as 'NaN' string.
Fields of type []byte are being encoded as base64-encoded strings.
Fields can be excluded from encoding by using '-' tag option.
Examples of struct tags:
// Field appears as 'myName' header in CSV encoding. Field int `csv:"myName"` // Field appears as 'Field' header in CSV encoding. Field int // Field appears as 'myName' header in CSV encoding and is an empty string // if Field is 0. Field int `csv:"myName,omitempty"` // Field appears as 'Field' header in CSV encoding and is an empty string // if Field is 0. Field int `csv:",omitempty"` // Encode ignores this field. Field int `csv:"-"`
Encode doesn't flush data. The caller is responsible for calling Flush() if the used Writer supports it.
Example ¶
package main import ( "bytes" "encoding/csv" "fmt" "github.com/jszwec/csvutil" ) func main() { type Address struct { City string Country string } type User struct { Name string Address Age int `csv:"age,omitempty"` } users := []User{ {Name: "John", Address: Address{"Boston", "USA"}, Age: 26}, {Name: "Bob", Address: Address{"LA", "USA"}, Age: 27}, {Name: "Alice", Address: Address{"SF", "USA"}}, } var buf bytes.Buffer w := csv.NewWriter(&buf) enc := csvutil.NewEncoder(w) for _, u := range users { if err := enc.Encode(u); err != nil { fmt.Println("error:", err) } } w.Flush() if err := w.Error(); err != nil { fmt.Println("error:", err) } fmt.Println(buf.String()) }
Output: Name,City,Country,age John,Boston,USA,26 Bob,LA,USA,27 Alice,SF,USA,
func (*Encoder) EncodeHeader ¶ added in v1.1.0
EncodeHeader writes the CSV header of the provided struct value to the output stream. The provided argument v must be a struct value.
The first Encode method call will not write header if EncodeHeader was called before it. This method can be called in cases when a data set could be empty, but header is desired.
EncodeHeader is like Header function, but it works with the Encoder and writes directly to the output stream. Look at Header documentation for the exact header encoding rules.
Example ¶
package main import ( "bytes" "encoding/csv" "fmt" "github.com/jszwec/csvutil" ) func main() { type User struct { Name string Age int `csv:"age,omitempty"` } var buf bytes.Buffer w := csv.NewWriter(&buf) enc := csvutil.NewEncoder(w) if err := enc.EncodeHeader(User{}); err != nil { fmt.Println("error:", err) } w.Flush() if err := w.Error(); err != nil { fmt.Println("error:", err) } fmt.Println(buf.String()) }
Output: Name,age
type InvalidDecodeError ¶
An InvalidDecodeError describes an invalid argument passed to Decode. (The argument to Decode must be a non-nil struct pointer)
func (*InvalidDecodeError) Error ¶
func (e *InvalidDecodeError) Error() string
type InvalidEncodeError ¶
InvalidEncodeError is returned by Encode when the provided value was invalid.
func (*InvalidEncodeError) Error ¶
func (e *InvalidEncodeError) Error() string
type InvalidMarshalError ¶
InvalidMarshalError is returned by Marshal when the provided value was invalid.
func (*InvalidMarshalError) Error ¶
func (e *InvalidMarshalError) Error() string
type InvalidUnmarshalError ¶
An InvalidUnmarshalError describes an invalid argument passed to Unmarshal. (The argument to Unmarshal must be a non-nil slice of structs pointer)
func (*InvalidUnmarshalError) Error ¶
func (e *InvalidUnmarshalError) Error() string
type Marshaler ¶
Marshaler is the interface implemented by types that can marshal themselves into valid string.
type MarshalerError ¶
MarshalerError is returned by Encoder when MarshalCSV or MarshalText returned an error.
func (*MarshalerError) Error ¶
func (e *MarshalerError) Error() string
type Reader ¶
Reader provides the interface for reading a single CSV record.
If there is no data left to be read, Read returns (nil, io.EOF).
It is implemented by csv.Reader.
type UnmarshalTypeError ¶
type UnmarshalTypeError struct { Value string // string value Type reflect.Type // type of Go value it could not be assigned to }
An UnmarshalTypeError describes a string value that was not appropriate for a value of a specific Go type.
func (*UnmarshalTypeError) Error ¶
func (e *UnmarshalTypeError) Error() string
type Unmarshaler ¶
Unmarshaler is the interface implemented by types that can unmarshal a single record's field description of themselves.
type UnsupportedTypeError ¶
An UnsupportedTypeError is returned when attempting to encode or decode a value of an unsupported type.
func (*UnsupportedTypeError) Error ¶
func (e *UnsupportedTypeError) Error() string