struct2csv

README

struct2csv

Create either a CSV file or [][]string from a slice of structs.

About

Struct2csv takes struct(s) and transforms them into CSV which can be either written to a provide writer with encoding/csv or returned as [][]string using Marshal.

The field names are used as the first row of the csv data. Use of field tags for csv header row column names is supported. By default, struct2csv uses the looks for field tags for csv. It can be configured to use the values of other field tags, e.g. yaml or json, instead, or, to ignore field tags.

Each struct in the provided slice becomes its own row. Slices are represented as a single column, with each element separated by a comma. Maps are represented as a single column; entries become a comma separated list of key:value pairs. Embedded struct fields become their own columns unless they are part of a slice or map. More complex types use separators, "(" and ")" by default, to group subtypes: e.g. a field with the type map[string][]string will have an output similar to key1: (key1, slice, values).

The separators to use can be set with encoder.SetSeparators(begin, end). Passing empty strings, "", will result in no separators being used. The separators are used for composite types with lists.

Only exported fields become part of the csv data, unless the field's struct tag is -. Some types, like channels and funcs, are skipped.

If a non-struct Kind is received, an error will occur. If a non-slice is passed to Marshal or WriteStructs, an error will be returned.

Usage

Using Directly

A new encoder can be created with the New() func. This will return a new encoder initalized with default value. The encoder can be configured using it's exposed methods. When using the encoder, all data is returned as []string or [][]string values. It is your responsibility to encode it to CSV using encoding/csv.

Extract data from a slice of structs:

data := []MyStruct{MyStruct{}, MyStruct{}}
enc := struct2csv.New()
rows, err := enc.Marshal(data)
if err != nil {
        // handle error
}

Extract data from a slice of structs; one at a time:

data := []MyStruct{MyStruct{}, MyStruct{}}
enc := struct2csv.New()
var rows [][]string
// get the column names first
colhdrs, err := enc.GetHeaders(data[0])
if err != nil {
        // handle error
}
rows = append(rows, colhdrs)
// get the data from each struct
for _, v := range data {
        row, err := enc.GetRow(v)
        if err != nil {
                // handle error
        }
        rows = append(rows, row)
}

Using with the Writer

A writer can be created by passing an io.Writer to NewWriter(w) and using it's methods. The Writer wraps encoding/csv's Writer and struct2csv's Encoder.

This Writer exposes csv,Writer's methods using wrapper methods. Struct2csv's Writer has additional methods for configuring the encoder and working with Structs.

Create CSV from a slice of structs:

data := []MyStruct{MyStruct{}, MyStruct{}}
buff := &bytes.Buffer{}
w := struct2csv.NewWriter(buff)
err := w.WriteStructs(data)
if err != nil {
        // handle error
}

Extract data from a slice of structs; one at a time:

data := []MyStruct{MyStruct{}, MyStruct{}}
buff := &bytes.Buffer{}
w := struct2csv.NewWriter(buff)
// set the column names first
err := w.WriteColNames(data[0])
if err != nil {
        // handle error
}
// get the data from each struct
for _, v := range data {
        err = w.Write
        if err != nil {
                // handle error
        }
        rows = append(rows, row)
}
// must flush the writer
w.Flush()
fmt.Println(buff.String())

Configuration of an Encoder

By default, an encoder will use tag fields with the tag csv, if they exist, as the column header value for a field. If such a tag does not exist, the column name will be used. The encoder will also use ( and ) as its begin and end separator values.

The separator values can be changed with the Encoder.SetSeparators(beginSep, endSep) method. If the separators are set to "", an empty string, nothing will be used. This mainly applies to lists.

The tag that the encoder uses can be changed by calling Encoder.SetTag(value).

Tags can be ignored by calling Encoder.SetUseTag(false). This will result in the struct field names being used as the colmn header values.

Supported types

The following reflect.Kind are supported:

Bool
Int
Int8
Int16
Int32
Int64
Uint
Uint8
Uint16
Uint32
Uint64
Float32
Float64
Complex64
Complex128
Array
Map
Ptr
Slice
String
Struct

The following reflect.Kind are not supported, or have not yet been implemented. Any fields using any of these kinds will be ignored. If a map uses any of these Kinds in either its key or value, it will be ignored.

Chan
Func
Uintptr
Interface
UnsafePointer

Embedded types

If a type is embedded, any exported fields within that struct become their own columns with the field name being the column name, unless a field tag has been defined. The name of the embedded struct does not become part of the column header name.

Maps, Slices, and Arrays

Map

Maps are a single column in the resulting CSV as maps can have a variable number of elements and there is no way to account for this within CSV. Each map element becomes a key:value pair with each element seperated by a ,. Keys are sorted.

map[string]string{"Douglas Adams": "Hitchhiker's Guide to the Galaxy", "William Gibson": "Neuromancer"}

becomes:

Douglas Adams:Hitchhiker's Guide to the Galaxy, William Gibson:Neuromancer

If the map's value is a composite type, the values of the composite type become a comma separated list surrounded by ().

map[string][]string{
        "William Gibson": []string{"Neuromancer" ,"Count Zero", "Mona Lisa Overdrive"},
        "Frank Herbert": []string{"Destination Void", "Jesus Incident", "Lazurus Effect"},
}

becomes:

Frank Herbert:(Destination Void, Jesus Incident, Lazurus Effect),William Gibson:(Neuromancer, Count Zero, Mona Lisa Overdrive)

Slices and Arrays

Slices and arrays are a single column in the resulting CSV as slices can have a variable number of elements and there is no way to account for this within CSV. Arrays are treated the same as slices. Slices become a comma separated list of values.

Structs

Struct fields become their own column. If the struct is embedded, only its field name is used for the column name. This may lead to some ambiguity in column names. Options to either prefix the embedded struct's field name with the struct name, or with the full path to the struct, in the case of deeply nested embedded structs may be added in the future (pull requests supporting this are also welcome!) If the struct is part of a composite type, like a map or slice, it will be part of that column with its data nested, using separators as appropriate.

Custom Handler for Structs

You can configure custom handlers for a struct:

type DateStruct struct {
    Name  string
    Start time.Time `json:"start" csv:"Start" handler:"ConvertTime"`
    End   time.Time `json:"end" csv:"End" handler:"ConvertTime"`
}

func (DateStruct) ConvertTime(t time.Time) string {
    return t.UTC().String()
}

func main() {
    date := DateStruct{
        Name: "test",
        Start: time.Now(),
        End: time.Now().Add(time.Hour * 24),
    }
    
    enc := New()
    enc.SetHandlerTag("handler")
    row, err := enc.GetRow(date)
}

The ConvertTime handler of the struct (not the field) will be called with the field's value.

To enhance the flexibility:

• multiple handlers per struct are supported (just one ConvertTime in the example)
• you need to explicitly set the handler-tag-name of the encoder (handler in the example) to avoid unexpected interference

All handlers need to implement an interface like this:

func (s S) MyHandler(v interface{}) string

It needs to return a string, which will be used as the csv-value for the field.

Pointers and nils

Pointers are dereferenced. Struct field types using multiple, consecutive pointers, e.g. **string, are not supported. Struct fields with composite types support mulitple, non-consecutive pointers, for whatever reason, e.g. *[]*string, *map[*string]*[]*string, are supported.

A nil result in an empty string, regardless of its type.

Header row

It is possible to get the header row for a struct by calling the GetHeaders func with the struct from which you want the column names. The names are returned as a []string.

Data of a single struct

It is possible to get the data from a single struct by calling the GetStructData func with the struct from which you want the column data. The data is returned as a []string.

TODO

• Add option to add names of embedded structs to the column header for its fields.
• Add support for interface{}

Documentation

Overview

Package struct2csv creates slices of strings out of struct fields. Struct2csv can work with either a struct or a slice of structs, The data can be returned as []string, in the case of a single struct, or [][]string, in the case of a slice of structs, by using struct2csv's encoder and `Marshal`.

For writing struct data directly to a CSV file, a writer can be used. `NewWriter` accepts an `io.Writer` and uses it to create a `csv.Writer`, which is used for writing the generated data. The `csv.Writer`'s exported fields are exposed as methods.

Fields of Kind Uintptr, Unsafepointer, Chan, Func, and Interface are not supported.

Unexported fields will be ignored.

Field tags are supported. Struct2csv will look for field tags matching `csv`, unless it's configured to either ignore field tags or use different field tag.

Index

• Variables
• type Encoder
  • func New() *Encoder
  • func (e *Encoder) ColNames() []string
  • func (e *Encoder) GetColNames(v interface{}) ([]string, error)
  • func (e *Encoder) GetRow(v interface{}) ([]string, error)
  • func (e *Encoder) Marshal(v interface{}) ([][]string, error)
  • func (e *Encoder) SetBase(i int)
  • func (e *Encoder) SetHandlerTag(handlerTag string)
  • func (e *Encoder) SetSeparators(beg, end string)
  • func (e *Encoder) SetTag(s string)
  • func (e *Encoder) SetUseTags(b bool)
• type StructRequiredError
  • func (e StructRequiredError) Error() string
• type StructSliceError
  • func (e StructSliceError) Error() string
• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (w *Writer) ColNames() []string
  • func (w *Writer) Comma() rune
  • func (w *Writer) Error() error
  • func (w *Writer) Flush()
  • func (w *Writer) Rows() int
  • func (w *Writer) SetBase(i int)
  • func (w *Writer) SetComma(r rune)
  • func (w *Writer) SetSeparators(beg, end string)
  • func (w *Writer) SetTag(s string)
  • func (w *Writer) SetUseCRLF(b bool)
  • func (w *Writer) SetUseTags(b bool)
  • func (w *Writer) UseCRLF() bool
  • func (w *Writer) Write(row []string) error
  • func (w *Writer) WriteAll(data [][]string) error
  • func (w *Writer) WriteColNames(st interface{}) error
  • func (w *Writer) WriteStruct(st interface{}) error
  • func (w *Writer) WriteStructs(st interface{}) error

Variables

var (
	// ErrNilSlice occurs when the slice of structs to encode is nil.
	ErrNilSlice = errors.New("struct2csv: the slice of structs was nil")
	// ErrEmptySlice occurs when the slice of structs to encode is empty.
	ErrEmptySlice = errors.New("struct2csv: the slice of structs was empty")
)

Types

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder handles encoding of a CSV from a struct.

func New

func New() *Encoder

New returns an initialized Encoder.

func (*Encoder) ColNames

func (e *Encoder) ColNames() []string

ColNames returns the encoder's saved column names as a copy. The colNames field must be populated before using this.

func (*Encoder) GetColNames

func (e *Encoder) GetColNames(v interface{}) ([]string, error)

GetColNames get's the column names from the received struct. If the interface is not a struct, an error will occur.

Field tags are supported. By default, the column names will be the value of the `csv` tag, if any. This can be changed with the SetTag(newTag) func; e.g. `json` to use JSON tags. Use of field tags can be toggled with the the SetUseTag(bool) func. If use of field tags is set to FALSE, the field's name will be used.

func (*Encoder) GetRow

func (e *Encoder) GetRow(v interface{}) ([]string, error)

GetRow get's the data from the passed struct. This only operates on single structs. If you wish to transmogrify everything at once, use Encoder.Marshal([]T).

func (*Encoder) Marshal

func (e *Encoder) Marshal(v interface{}) ([][]string, error)

Marshal takes a slice of structs and returns a [][]byte representing CSV data. Each field in the struct results in a column. Fields that are slices are stored in a single column as a comma separated list of values. Fields that are maps are stored in a single column as a comma separted list of key:value pairs.

If the passed data isn't a slice of structs an error will be returned.

func (*Encoder) SetBase

func (e *Encoder) SetBase(i int)

SetBase sets the base for strings.FormatUint. By default, this is 10. Set the base if another base should be used for formatting uint values.

Base 2 is the minimum value; anything less will be set to two.

func (*Encoder) SetHandlerTag

func (e *Encoder) SetHandlerTag(handlerTag string)

func (*Encoder) SetSeparators

func (e *Encoder) SetSeparators(beg, end string)

SetSeparators sets the begin and end separator values for lists. Setting the separators to "", empty strings, results in no separators being added. By default, "(" and ")" are used as the begin and end separators,

func (*Encoder) SetTag

func (e *Encoder) SetTag(s string)

SetTag sets the tag that the Encoder should use for header (column) names. By default, this is set to 'csv'. If the received value is an empty string, nothing will be done

func (*Encoder) SetUseTags

func (e *Encoder) SetUseTags(b bool)

SetUseTags sets whether or not tags should be used for header (column) names.

type StructRequiredError

type StructRequiredError struct {
	// contains filtered or unexported fields
}

A StructRequiredError is returned when a non-struct type is received.

func (StructRequiredError) Error

func (e StructRequiredError) Error() string

type StructSliceError

type StructSliceError struct {
	// contains filtered or unexported fields
}

A StructSliceError is returned when an interface that isn't a slice of type struct is received.

func (StructSliceError) Error

func (e StructSliceError) Error() string

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

A Writer writes structs to a CSV encoded file. This wraps both `csv.Writer` and this package's `Encoder`.

func NewWriter

func NewWriter(w io.Writer) *Writer

NewWriter returns a new Writer that write to w.

func (*Writer) ColNames

func (w *Writer) ColNames() []string

ColNames returns a copy of the encoder's cached column names

func (*Writer) Comma

func (w *Writer) Comma() rune

Comma is the field delimiter, set to '.'

func (*Writer) Error

func (w *Writer) Error() error

Error reports an error that has occurred during a previous Write or Flush

func (*Writer) Flush

func (w *Writer) Flush()

Flush writes any buffered data to the underlying io.Writer.

func (*Writer) Rows

func (w *Writer) Rows() int

Rows returns the number of CSV rows created. This includes the header row.

func (*Writer) SetBase

func (w *Writer) SetBase(i int)

SetBase set's the base for _uint_ values; mainly used for `strings.FormatUint()`. By default, this is set to 10, for base 10 numbering. Any base value < 2 will be set to 2, binary.

func (*Writer) SetComma

func (w *Writer) SetComma(r rune)

SetComma takes the passed rune and uses it to set the field delimiter for CSV fields.

func (*Writer) SetSeparators

func (w *Writer) SetSeparators(beg, end string)

SetSeparators sets the begin and end separator values for lists, which default to `(` and `)`. Empty strings result in no separators being used.

func (*Writer) SetTag

func (w *Writer) SetTag(s string)

SetTag set's the tag value to match on for a struct's field tags.

func (*Writer) SetUseCRLF

func (w *Writer) SetUseCRLF(b bool)

SetUseCRLF set's the csv'writer's UseCRLF field

func (*Writer) SetUseTags

func (w *Writer) SetUseTags(b bool)

SetUseTags set's whether or not field tag values should be checked. If field tags are not being checked, the field name will be used for the column name.

func (*Writer) UseCRLF

func (w *Writer) UseCRLF() bool

UseCRLF exposes the csv writer's UseCRLF field.

func (*Writer) Write

func (w *Writer) Write(row []string) error

Write takes a slice of strings and writes them as a single CSV record.

func (*Writer) WriteAll

func (w *Writer) WriteAll(data [][]string) error

WriteAll writes multiple CSV records, a two-d slice of strings, `[][]string` to w using Write and then calls Flush.

func (*Writer) WriteColNames

func (w *Writer) WriteColNames(st interface{}) error

WriteColNames writes out the column names of the CSV field.

func (*Writer) WriteStruct

func (w *Writer) WriteStruct(st interface{}) error

WriteStruct takes a struct, marshals it to CSV and writhes the CSV record to the writer

func (*Writer) WriteStructs

func (w *Writer) WriteStructs(st interface{}) error

WriteStructs takes a slice of structs and writes them as CSV records. This includes writing out the column names as the first row. When done, Flush is called.