csv

Documentation

Overview

Package csv implements the sq driver for CSV/TSV et al.

Index

• Constants
• Variables
• func DetectCSV(ctx context.Context, openFn source.FileOpenFunc) (detected source.DriverType, score float32, err error)
• func DetectTSV(ctx context.Context, openFn source.FileOpenFunc) (detected source.DriverType, score float32, err error)
• func Hash(detectors []*kind.Detector) (h string, err error)
• func NamedDelims() []string
• type Provider
  • func (d *Provider) DriverFor(typ source.DriverType) (driver.Driver, error)

Constants

const (
	// TypeCSV is the CSV driver type.
	TypeCSV = source.DriverType("csv")

	// TypeTSV is the TSV driver type.
	TypeTSV = source.DriverType("tsv")
)

Variables

var OptDelim = options.NewString(
	"driver.csv.delim",
	0,
	delimCommaKey,
	"Delimiter for ingest CSV data",
	`Delimiter to use for CSV files. Default is "comma".
Possible values are: comma, space, pipe, tab, colon, semi, period.`,
	"source",
	"csv",
)

OptDelim specifies the CSV delimiter to use.

var OptEmptyAsNull = options.NewBool(
	"driver.csv.empty-as-null",
	0,
	true,
	"Treat ingest empty CSV fields as NULL",
	`When true, empty CSV fields are treated as NULL. When false,
the zero value for that type is used, e.g. empty string or 0.`,
	"source",
	"csv",
)

OptEmptyAsNull determines if an empty CSV field is treated as NULL or as the zero value for the kind of that field.

Functions

func DetectCSV

func DetectCSV(ctx context.Context, openFn source.FileOpenFunc) (detected source.DriverType, score float32,
	err error,
)

DetectCSV implements source.DriverDetectFunc.

func DetectTSV

func DetectTSV(ctx context.Context, openFn source.FileOpenFunc) (detected source.DriverType,
	score float32, err error,
)

DetectTSV implements source.DriverDetectFunc.

func Hash

func Hash(detectors []*kind.Detector) (h string, err error)

Hash generates a hash from the kinds returned by the detectors. The detectors should already have sampled data.

TODO: move Hash to pkg libsq/core/kind?

func NamedDelims

func NamedDelims() []string

NamedDelims returns the named delimiters, such as [comma, tab, pipe...].

Types

type Provider

type Provider struct {
	Log       *slog.Logger
	Scratcher driver.ScratchDatabaseOpener
	Files     *source.Files
}

Provider implements driver.Provider.

func (*Provider) DriverFor

func (d *Provider) DriverFor(typ source.DriverType) (driver.Driver, error)

DriverFor implements driver.Provider.