Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewCsvReader ¶ added in v0.16.0
func NewCsvReader(r io.ReadCloser) (*csvReader, error)
NewCsvReader creates a csvReader from a given ReadCloser.
The interpretation of the bytes of the supplied reader is a little murky. If there is a UTF8, UTF16LE or UTF16BE BOM as the first bytes read, then the BOM is stripped and the remaining contents of the reader are treated as that encoding. If we are not in any of those marked encodings, then some of the bytes go uninterpreted until we get to the SQL layer. It is currently the case that newlines must be encoded as a '0xa' byte.
func NewStringPrefixReader ¶ added in v0.16.0
NewStringPrefixReader creates a new stringPrefixReader that first returns the data in |prefix| and then returns data from |reader|.
Types ¶
type CsvDataLoader ¶
type CsvDataLoader struct {
// contains filtered or unexported fields
}
CsvDataLoader is an implementation of DataLoader that reads data from chunks of CSV files and inserts them into a table.
func NewCsvDataLoader ¶
func NewCsvDataLoader(ctx *sql.Context, table sql.InsertableTable, delimiter string, header bool) (*CsvDataLoader, error)
NewCsvDataLoader creates a new DataLoader instance that will insert records from chunks of CSV data into |table|. If |header| is true, the first line of the data will be treated as a header and ignored. If |delimiter| is not the empty string, it will be used as the delimiter separating value.
func (*CsvDataLoader) Abort ¶
func (cdl *CsvDataLoader) Abort(ctx *sql.Context) error
Abort implements the DataLoader interface
func (*CsvDataLoader) Finish ¶
func (cdl *CsvDataLoader) Finish(ctx *sql.Context) (*LoadDataResults, error)
Finish implements the DataLoader interface
type DataLoader ¶
type DataLoader interface {
// LoadChunk reads the records from |data| and inserts them into the previously configured table. Data records
// are not guaranteed to start and end cleanly on chunk boundaries, so implementations must recognize incomplete
// records and save them to prepend on the next processed chunk.
LoadChunk(ctx *sql.Context, data *bufio.Reader) error
// Abort aborts the current load operation and releases all used resources.
Abort(ctx *sql.Context) error
// Finish finalizes the current load operation and commits the inserted rows so that the data becomes visibile
// to clients. Implementations should check that the last call to LoadChunk did not end with an incomplete
// record and return an error to the caller if so. The returned LoadDataResults describe the load operation,
// including how many rows were inserted.
Finish(ctx *sql.Context) (*LoadDataResults, error)
}
DataLoader allows callers to insert rows from multiple chunks into a table. Rows encoded in each chunk will not necessarily end cleanly on a chunk boundary, so DataLoader implementations must handle recognizing partial, or incomplete records, and saving that partial record until the next call to LoadChunk, so that it may be prefixed with the incomplete record.
type LoadDataResults ¶
type LoadDataResults struct {
// RowsLoaded contains the total number of rows inserted during a load data operation.
RowsLoaded int32
}
LoadDataResults contains the results of a load data operation, including the number of rows loaded.
type TabularDataLoader ¶
type TabularDataLoader struct {
// contains filtered or unexported fields
}
TabularDataLoader tracks the state of a load data operation from a tabular data source.
func NewTabularDataLoader ¶
func NewTabularDataLoader(ctx *sql.Context, table sql.InsertableTable, delimiterChar, nullChar string, header bool) (*TabularDataLoader, error)
NewTabularDataLoader creates a new TabularDataLoader to insert into the specifeid |table| using the specified |delimiterChar| and |nullChar|. If |header| is true, the first line of the data will be treated as a header and ignored.
func (*TabularDataLoader) Abort ¶
func (tdl *TabularDataLoader) Abort(ctx *sql.Context) error
Abort ends the current load data operation and discards any changes that have been made.
func (*TabularDataLoader) Finish ¶
func (tdl *TabularDataLoader) Finish(ctx *sql.Context) (*LoadDataResults, error)
Finish completes the current load data operation and finalizes the data that has been inserted.
Source Files