cnf

Documentation

Index

• type FilterConf
• type NgramConf
  • func (nc *NgramConf) IsZero() bool
  • func (nc *NgramConf) MaxRequiredColumn() int
  • func (nc *NgramConf) UpgradeLegacy() error
• type VTEConf
  • func LoadConf(confPath string) (*VTEConf, error)
  • func (c *VTEConf) GetDefinedVerticals() []string
  • func (c *VTEConf) HasConfiguredFilter() bool
  • func (c *VTEConf) HasConfiguredVertical() bool
  • func (c *VTEConf) WithoutPasswords() VTEConf

Types

type FilterConf

type FilterConf struct {
	Lib string `json:"lib"`
	Fn  string `json:"fn"`
}

FilterConf specifies a plug-in containing a compatible filter (see LineFilter interface).

type NgramConf

type NgramConf struct {
	NgramSize   int            `json:"ngramSize"`
	CalcARF     bool           `json:"calcARF"`
	VertColumns db.VertColumns `json:"vertColumns"`

	// AttrColumns
	//
	// Deprecated: please use VertColumns instead which groups idx and mod function
	AttrColumns []int `json:"attrColumns,omitempty"`

	// ColumnMods
	//
	// Deprecated: please use VertColumns instead which groups idx and mod function
	ColumnMods []string `json:"columnMods,omitempty"`
}

NgramConf configures positional attributes (referred by their column position) we want to store and count as n-grams. This can be used to extract all the unique PoS tags or frequency information about words/lemmas.

func (*NgramConf) IsZero

func (nc *NgramConf) IsZero() bool

IsZero returns true if the object contains all the attributes set to their respective zero values (CalcARF == 0, len(VertColumns) == 0 etc.) This is used e.g. to reset n-gram configuration in CNC-MASM

func (*NgramConf) MaxRequiredColumn

func (nc *NgramConf) MaxRequiredColumn() int

func (*NgramConf) UpgradeLegacy

func (nc *NgramConf) UpgradeLegacy() error

type VTEConf

type VTEConf struct {
	Corpus              string `json:"corpus"`
	ParallelCorpus      string `json:"parallelCorpus,omitempty"`
	AtomStructure       string `json:"atomStructure"`
	AtomParentStructure string `json:"atomParentStructure"`
	StackStructEval     bool   `json:"stackStructEval"`

	// MaxNumErrors if reached then the process stops
	MaxNumErrors int                 `json:"maxNumErrors"`
	Structures   map[string][]string `json:"structures"`

	// Ngrams - see NgramConf
	// If omitted then the function is disabled.
	Ngrams NgramConf `json:"ngrams"`

	// VerticalFile can be either a path to a single file
	// or a path to a directory containing multiple vertical
	// files (then we assume all the vertical files are of the
	// same structure)
	VerticalFile string `json:"verticalFile,omitempty"`

	// VerticalFiles is an alternative to VerticalFile allowing
	// explicit selection of one or more files to be processed
	// as one.
	VerticalFiles []string `json:"verticalFiles,omitempty"`

	DB db.Conf `json:"db"`

	Encoding    string          `json:"encoding"`
	SelfJoin    db.SelfJoinConf `json:"selfJoin"`
	IndexedCols []string        `json:"indexedCols"`
	BibView     db.BibViewConf  `json:"bibView"`

	Filter FilterConf `json:"filter"`

	Verbosity int `json:"verbosity"`
}

VTEConf holds configuration for a concrete data extraction task.

func LoadConf

func LoadConf(confPath string) (*VTEConf, error)

func (*VTEConf) GetDefinedVerticals

func (c *VTEConf) GetDefinedVerticals() []string

func (*VTEConf) HasConfiguredFilter

func (c *VTEConf) HasConfiguredFilter() bool

func (*VTEConf) HasConfiguredVertical

func (c *VTEConf) HasConfiguredVertical() bool

func (*VTEConf) WithoutPasswords

func (c *VTEConf) WithoutPasswords() VTEConf

WithoutPassword returns a new semi-shallow copy of the called config with sensitive information replaced by `*`. By the "semi-shallownes" we mean that in case a sensitive information overwriting would affect the original object, such part will be provided as a deep copy.