config

Documentation

Index

• type Config
  • func New(opts ...Option) Config
• type Option
  • func OptBadRow(br gnfmt.BadRow) Option
  • func OptCacheDir(s string) Option
  • func OptCacheSfgaDir(s string) Option
  • func OptJobsNum(n int) Option
  • func OptWithQuotes(b bool) Option
  • func OptWithZipOutput(b bool) Option

Types

type Config

type Config struct {
	// GitRepo contains data for sfga schema Git repository.
	sfga.GitRepo

	// TempRepoDir is a temporary location to schema files downloaded from GitHub.
	TempRepoDir string

	// CacheDir is the root path for all cached files.
	CacheDir string

	// CacheSfgaDir is the path SFGA database.
	CacheSfgaDir string

	// JobsNum is the number of concurrent jobs to run.
	JobsNum int

	// BatchSize is the number of records to insert in one transaction.
	BatchSize int

	// BadRow dets decision what to do if a row has more/less fields
	// than it should.
	BadRow gnfmt.BadRow

	// WithZipOutput is a flag to return zipped SFGAarchive outpu.
	WithZipOutput bool

	// WithQuotes tells that coldp file has `"` to escape new lines and
	// delimiters inside fields. If true, RFC-based CSV algorithm is used.
	WithQuotes bool
}

Config is a configuration object for the Catalogue of Life Data Package Archive (CoLDP) data processing.

func New

func New(opts ...Option) Config

New creates a new Config object with default values, and allows to override them with options.

type Option

type Option func(*Config)

Option is a function type that allows to standardize how options to the configuration are organized.

func OptBadRow

func OptBadRow(br gnfmt.BadRow) Option

func OptCacheDir

func OptCacheDir(s string) Option

OptCacheDir sets the root path for all temporary files.

func OptCacheSfgaDir

func OptCacheSfgaDir(s string) Option

OptCacheSfgaDir sets the path to store resulting sqlite file with data imported from CoLDP file.

func OptJobsNum

func OptJobsNum(n int) Option

OptJobsNum sets the number of concurrent jobs to run.

func OptWithQuotes

func OptWithQuotes(b bool) Option

OptWithQuotes tells reader that CoLDP file uses quotes in CSV to help data integrity when a field contains either new lines, or designated field delimiters.

func OptWithZipOutput

func OptWithZipOutput(b bool) Option

OptWithZipOutput sets output as binary SQLite file.