csv

README

csv-go

Why does this exist?

I am tired of rewriting this over and over to cover edge cases where other language standard csv implementations have assertions on the format and formatting I cannot guarantee are valid for a given file and how it was constructed. I've written variations that cover far fewer concerns over the years, and I figured I'll make a superset of one that does everything I need. Feel free to use however you may wish.

Documentation

Index

• Variables
• type BufferedReader
• type Reader
  • func NewReader(options ...ReaderOption) (*Reader, error)
  • func (r *Reader) Close() error
  • func (r *Reader) Err() error
  • func (r *Reader) IntoIter() iter.Seq[[]string]
  • func (r *Reader) Row() []string
  • func (r *Reader) Scan() bool
• type ReaderOption
• type ReaderOptions
  • func ReaderOpts() ReaderOptions
  • func (ReaderOptions) BorrowRow(b bool) ReaderOption
  • func (ReaderOptions) Comment(r rune) ReaderOption
  • func (ReaderOptions) CommentsAllowedAfterStartOfRecords(b bool) ReaderOption
  • func (ReaderOptions) DiscoverRecordSeparator(b bool) ReaderOption
  • func (ReaderOptions) ErrorOnNewlineInUnquotedField(b bool) ReaderOption
  • func (ReaderOptions) ErrorOnNoByteOrderMarker(b bool) ReaderOption
  • func (ReaderOptions) ErrorOnNoRows(b bool) ReaderOption
  • func (ReaderOptions) ErrorOnQuotesInUnquotedField(b bool) ReaderOption
  • func (ReaderOptions) Escape(r rune) ReaderOption
  • func (ReaderOptions) ExpectHeaders(h []string) ReaderOption
  • func (ReaderOptions) FieldSeparator(r rune) ReaderOption
  • func (ReaderOptions) NumFields(n int) ReaderOption
  • func (ReaderOptions) Quote(r rune) ReaderOption
  • func (ReaderOptions) Reader(r io.Reader) ReaderOption
  • func (ReaderOptions) RecordSeparator(s string) ReaderOption
  • func (ReaderOptions) RemoveByteOrderMarker(b bool) ReaderOption
  • func (ReaderOptions) RemoveHeaderRow(b bool) ReaderOption
  • func (ReaderOptions) TerminalRecordSeparatorEmitsRecord(b bool) ReaderOption
  • func (ReaderOptions) TrimHeaders(b bool) ReaderOption

Variables

var (
	// classifications
	ErrIO              = errors.New("io error")
	ErrParsing         = errors.New("parsing error")
	ErrFieldCount      = errors.New("field count error")
	ErrBadConfig       = errors.New("bad config")
	ErrBadReadRuneImpl = errors.New("bad ReadRune implementation")
	// instances
	ErrTooManyFields               = errors.New("too many fields")
	ErrNotEnoughFields             = errors.New("not enough fields")
	ErrReaderClosed                = errors.New("reader closed")
	ErrUnexpectedHeaderRowContents = errors.New("header row values do not match expectations")
	ErrBadRecordSeparator          = errors.New("record separator can only be one valid utf8 rune long or \"\\r\\n\"")
	ErrIncompleteQuotedField       = fmt.Errorf("incomplete quoted field: %w", io.ErrUnexpectedEOF)
	ErrQuoteInUnquotedField        = fmt.Errorf("quote found in unquoted field")
	ErrInvalidQuotedFieldEnding    = errors.New("unexpected character found after end of quoted field") // expecting field separator, record separator, quote char, or end of file if field count matches expectations
	ErrNoHeaderRow                 = fmt.Errorf("no header row: %w", io.ErrUnexpectedEOF)
	ErrNoRows                      = fmt.Errorf("no rows: %w", io.ErrUnexpectedEOF)
	ErrNoByteOrderMarker           = errors.New("no byte order marker")
	ErrNilReader                   = errors.New("nil reader")
	ErrInvalidEscapeInQuotedField  = errors.New("invalid escape sequence in quoted field")
	ErrNewlineInUnquotedField      = errors.New("newline rune found in unquoted field")
	ErrUnexpectedQuoteAfterField   = errors.New("unexpected quote after quoted+escaped field")
	ErrBadUnreadRuneImpl           = errors.New("UnreadRune failed")
	ErrUnsafeCRFileEnd             = fmt.Errorf("ended in a carriage return which must be quoted when record separator is CRLF: %w", io.ErrUnexpectedEOF)
	// ReadByte should never fail because we're always preceding this call with UnreadRune
	//
	// it could happen if someone is trying to read concurrently or made their own bad buffered reader implementation
	ErrBadReadByteImpl = errors.New("ReadByte failed")
)

Types

type BufferedReader

type BufferedReader interface {
	io.Reader
	ReadRune() (r rune, size int, err error)
	UnreadRune() error
	ReadByte() (byte, error)
}

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

func NewReader

func NewReader(options ...ReaderOption) (*Reader, error)

NewReader creates a new instance of a CSV reader which is not safe for concurrent reads.

func (*Reader) Close

func (r *Reader) Close() error

Close should be called after reading all rows successfully from the underlying reader and checking the result of r.Err().

Close currently always returns nil, but in the future it may not. It is not a substitute for checking r.Err().

Should any configuration options require post-flight checks they will be implemented here.

It will never attempt to close the underlying reader.

func (*Reader) Err

func (r *Reader) Err() error

func (*Reader) IntoIter

func (r *Reader) IntoIter() iter.Seq[[]string]

IntoIter converts the reader state into an iterator. Calling this method more than once returns the same iterator instance.

If the reader is configured with BorrowRow(true) then the resulting slice and field strings are only valid to use up until the next iteration and should not be saved to persistent memory.

It is best practice to check if Err() returns a non-nil error after fully traversing this iterator.

This is just a syntactic sugar method to work with range statements in go1.23 and later.

func (*Reader) Row

func (r *Reader) Row() []string

Row returns a slice of strings that represents a row of a dataset.

Row only returns valid results after a call to Scan() return true. For efficiency reasons this method should not be called more than once between calls to Scan().

If the reader is configured with BorrowRow(true) then the resulting slice and field strings are only valid to use up until the next call to Scan and should not be saved to persistent memory.

func (*Reader) Scan

func (r *Reader) Scan() bool

type ReaderOption

type ReaderOption func(*rCfg)

type ReaderOptions

type ReaderOptions struct{}

ReaderOptions should never be instantiated manually

Instead call ReaderOpts()

This is only exported to allow godocs to discover the exported methods.

ReaderOptions will never have exported members and the zero value is not part of the semver guarantee. Instantiate it incorrectly at your own peril.

Calling the function is a nop that is compiled away anyways, you will not optimize anything at all. Use ReaderOpts()!

func ReaderOpts

func ReaderOpts() ReaderOptions

func (ReaderOptions) BorrowRow

func (ReaderOptions) BorrowRow(b bool) ReaderOption

BorrowRow alters the row function to return the underlying string slice every time it is called rather than a copy.

Only set to true if the returned row slice and field strings within it are never used after the next call to Scan. Consider copying the slice and at least copy the strings within it via strings.Copy().

Please consider this to be a micro optimization in most circumstances just because is tightens the usage contract of the returned row in ways most would not normally consider.

func (ReaderOptions) Comment

func (ReaderOptions) Comment(r rune) ReaderOption

func (ReaderOptions) CommentsAllowedAfterStartOfRecords

func (ReaderOptions) CommentsAllowedAfterStartOfRecords(b bool) ReaderOption

func (ReaderOptions) DiscoverRecordSeparator

func (ReaderOptions) DiscoverRecordSeparator(b bool) ReaderOption

func (ReaderOptions) ErrorOnNewlineInUnquotedField

func (ReaderOptions) ErrorOnNewlineInUnquotedField(b bool) ReaderOption

func (ReaderOptions) ErrorOnNoByteOrderMarker

func (ReaderOptions) ErrorOnNoByteOrderMarker(b bool) ReaderOption

func (ReaderOptions) ErrorOnNoRows

func (ReaderOptions) ErrorOnNoRows(b bool) ReaderOption

func (ReaderOptions) ErrorOnQuotesInUnquotedField

func (ReaderOptions) ErrorOnQuotesInUnquotedField(b bool) ReaderOption

func (ReaderOptions) Escape

func (ReaderOptions) Escape(r rune) ReaderOption

Escape is useful for specifying what character is used to escape a quote in a field and the literal escape character itself.

Without specifying this option a quote character is expected to be escaped by it just being doubled while the overall field is wrapped in quote characters.

This is mainly useful when processing a spark csv file as it does not follow strict rfc4180.

So set to '\\' if you have this need.

It is not valid to use this option without specifically setting a quote. Doing so will result in an error being returned on Reader creation.

func (ReaderOptions) ExpectHeaders

func (ReaderOptions) ExpectHeaders(h []string) ReaderOption

ExpectHeaders causes the first row to be recognized as a header row.

If the slice of header values does not match then the reader will error.

func (ReaderOptions) FieldSeparator

func (ReaderOptions) FieldSeparator(r rune) ReaderOption

func (ReaderOptions) NumFields

func (ReaderOptions) NumFields(n int) ReaderOption

func (ReaderOptions) Quote

func (ReaderOptions) Quote(r rune) ReaderOption

func (ReaderOptions) Reader

func (ReaderOptions) Reader(r io.Reader) ReaderOption

func (ReaderOptions) RecordSeparator

func (ReaderOptions) RecordSeparator(s string) ReaderOption

func (ReaderOptions) RemoveByteOrderMarker

func (ReaderOptions) RemoveByteOrderMarker(b bool) ReaderOption

func (ReaderOptions) RemoveHeaderRow

func (ReaderOptions) RemoveHeaderRow(b bool) ReaderOption

RemoveHeaderRow causes the first row to be recognized as a header row.

The row will be skipped over by Scan() and will not be returned by Row().

func (ReaderOptions) TerminalRecordSeparatorEmitsRecord

func (ReaderOptions) TerminalRecordSeparatorEmitsRecord(b bool) ReaderOption

TerminalRecordSeparatorEmitsRecord only exists to acknowledge an edge case when processing csv documents that contain one column. If the file contents end in a record separator it's impossible to determine if that should indicate that a new record with an empty field should be emitted unless that record is enclosed in quotes or a config option like this exists.

In most cases this should not be an issue, unless the dataset is a single column list that allows empty strings for some use case and the writer used to create the file chooses to not always write the last record followed by a record separator. (treating the record separator like a record terminator)

func (ReaderOptions) TrimHeaders

func (ReaderOptions) TrimHeaders(b bool) ReaderOption

TrimHeaders causes the first row to be recognized as a header row and all values are returned with whitespace trimmed.