csv

Documentation

Overview

Package csv reads and writes comma-separated values (CSV) files.

A csv file contains zero or more records of one or more fields per record. Each record is separated by the newline character. The final record may optionally be followed by a newline character.

field1,field2,field3

White space is considered part of a field.

Carriage returns before newline characters are silently removed.

Blank lines are ignored. A line with only whitespace characters (excluding the ending newline character) is not considered a blank line.

Fields which start and stop with the quote character " are called quoted-fields. The beginning and ending quote are not part of the field.

The source:

normal string,"quoted-field"

results in the fields

{`normal string`, `quoted-field`}

Within a quoted-field a quote character followed by a second quote character is considered a single quote.

"the ""word"" is true","a ""quoted-field"""

results in

{`the "word" is true`, `a "quoted-field"`}

Newlines and commas may be included in a quoted-field

"Multi-line
field","comma is ,"

results in

{`Multi-line
field`, `comma is ,`}

Index

• Constants
• Variables
• type CsvAdapter
  • func NewCsvAdapter() *CsvAdapter
  • func (c *CsvAdapter) GetSplits(connectionId string, aq adapter.AdapterQuery) (splits []adapter.Split, err error)
  • func (c *CsvAdapter) LoadConfiguration(config map[string]string)
  • func (c *CsvAdapter) ReadSplit(split adapter.Split, writer io.Writer) error
• type CsvDataSplit
  • func (cs CsvDataSplit) GetConfiguration() map[string]string
• type ParseError
  • func (e *ParseError) Error() string
• type Reader
  • func NewReader(r io.Reader) *Reader
  • func (r *Reader) Read() (record []string, err error)
  • func (r *Reader) ReadAll() (records [][]string, err error)
• type Source
  • func New(fileOrPattern string) *Source
  • func (q *Source) AdapterName() string
  • func (q *Source) GetParallelLimit() int
  • func (ci *Source) Match(fullPath string) bool
  • func (q *Source) SetHasHeader(hasHeader bool) *Source
  • func (q *Source) SetParallelLimit(paraLimit int) *Source

Constants

const (
	SINGLE_QUOTE = '\''
	DOUBLE_QUOTE = '"'
)

Variables

var (
	ErrTrailingComma = errors.New("extra delimiter at end of line") // no longer used
	ErrBareQuote     = errors.New("bare \" in non-quoted-field")
	ErrQuote         = errors.New("extraneous \" in field")
	ErrFieldCount    = errors.New("wrong number of fields in line")
)

These are the errors that can be returned in ParseError.Error

Types

type CsvAdapter

type CsvAdapter struct {
}

func NewCsvAdapter

func NewCsvAdapter() *CsvAdapter

func (*CsvAdapter) GetSplits

func (c *CsvAdapter) GetSplits(connectionId string, aq adapter.AdapterQuery) (splits []adapter.Split, err error)

func (*CsvAdapter) LoadConfiguration

func (c *CsvAdapter) LoadConfiguration(config map[string]string)

func (*CsvAdapter) ReadSplit

func (c *CsvAdapter) ReadSplit(split adapter.Split, writer io.Writer) error

type CsvDataSplit

type CsvDataSplit struct {
	Config    map[string]string
	FileName  string
	HasHeader bool
}

func (CsvDataSplit) GetConfiguration

func (cs CsvDataSplit) GetConfiguration() map[string]string

type ParseError

type ParseError struct {
	Line   int   // Line where the error occurred
	Column int   // Column (rune index) where the error occurred
	Err    error // The actual error
}

A ParseError is returned for parsing errors. The first line is 1. The first column is 0.

func (*ParseError) Error

func (e *ParseError) Error() string

type Reader

type Reader struct {
	Comma            rune // field delimiter (set to ',' by NewReader)
	Comment          rune // comment character for start of line
	FieldsPerRecord  int  // number of expected fields per record
	LazyQuotes       bool // allow lazy quotes
	TrailingComma    bool // ignored; here for backwards compatibility
	TrimLeadingSpace bool // trim leading space
	// contains filtered or unexported fields
}

A Reader reads records from a CSV-encoded file.

As returned by NewReader, a Reader expects input conforming to RFC 4180. The exported fields can be changed to customize the details before the first call to Read or ReadAll.

Comma is the field delimiter. It defaults to ','.

Comment, if not 0, is the comment character. Lines beginning with the Comment character are ignored.

If FieldsPerRecord is positive, Read requires each record to have the given number of fields. If FieldsPerRecord is 0, Read sets it to the number of fields in the first record, so that future records must have the same field count. If FieldsPerRecord is negative, no check is made and records may have a variable number of fields.

If LazyQuotes is true, a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field.

If TrimLeadingSpace is true, leading white space in a field is ignored.

func NewReader

func NewReader(r io.Reader) *Reader

NewReader returns a new Reader that reads from r.

func (*Reader) Read

func (r *Reader) Read() (record []string, err error)

Read reads one record from r. The record is a slice of strings with each string representing one field.

func (*Reader) ReadAll

func (r *Reader) ReadAll() (records [][]string, err error)

ReadAll reads all the remaining records from r. Each record is a slice of fields. A successful call returns err == nil, not err == EOF. Because ReadAll is defined to read until EOF, it does not treat end of file as an error to be reported.

type Source

type Source struct {
	Path      string
	HasHeader bool
	Parallel  int
	// contains filtered or unexported fields
}

func New

func New(fileOrPattern string) *Source

func (*Source) AdapterName

func (q *Source) AdapterName() string

func (*Source) GetParallelLimit

func (q *Source) GetParallelLimit() int

func (*Source) Match

func (ci *Source) Match(fullPath string) bool

func (*Source) SetHasHeader

func (q *Source) SetHasHeader(hasHeader bool) *Source

func (*Source) SetParallelLimit

func (q *Source) SetParallelLimit(paraLimit int) *Source