Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FilterConf ¶
FilterConf specifies a plug-in containing a compatible filter (see LineFilter interface).
type NgramConf ¶
type NgramConf struct { NgramSize int `json:"ngramSize"` CalcARF bool `json:"calcARF"` VertColumns db.VertColumns `json:"vertColumns"` // AttrColumns // // Deprecated: please use VertColumns instead which groups idx and mod function AttrColumns []int `json:"attrColumns,omitempty"` // ColumnMods // // Deprecated: please use VertColumns instead which groups idx and mod function ColumnMods []string `json:"columnMods,omitempty"` }
NgramConf configures positional attributes (referred by their column position) we want to store and count as n-grams. This can be used to extract all the unique PoS tags or frequency information about words/lemmas.
func (*NgramConf) IsZero ¶
IsZero returns true if the object contains all the attributes set to their respective zero values (CalcARF == 0, len(VertColumns) == 0 etc.) This is used e.g. to reset n-gram configuration in CNC-MASM
func (*NgramConf) MaxRequiredColumn ¶
func (*NgramConf) UpgradeLegacy ¶
type VTEConf ¶
type VTEConf struct { Corpus string `json:"corpus"` ParallelCorpus string `json:"parallelCorpus,omitempty"` AtomStructure string `json:"atomStructure"` AtomParentStructure string `json:"atomParentStructure"` StackStructEval bool `json:"stackStructEval"` // MaxNumErrors if reached then the process stops MaxNumErrors int `json:"maxNumErrors"` Structures map[string][]string `json:"structures"` // Ngrams - see NgramConf // If omitted then the function is disabled. Ngrams NgramConf `json:"ngrams"` // VerticalFile can be either a path to a single file // or a path to a directory containing multiple vertical // files (then we assume all the vertical files are of the // same structure) VerticalFile string `json:"verticalFile,omitempty"` // VerticalFiles is an alternative to VerticalFile allowing // explicit selection of one or more files to be processed // as one. VerticalFiles []string `json:"verticalFiles,omitempty"` DB db.Conf `json:"db"` Encoding string `json:"encoding"` SelfJoin db.SelfJoinConf `json:"selfJoin"` IndexedCols []string `json:"indexedCols"` BibView db.BibViewConf `json:"bibView"` Filter FilterConf `json:"filter"` Verbosity int `json:"verbosity"` }
VTEConf holds configuration for a concrete data extraction task.
func (*VTEConf) GetDefinedVerticals ¶
func (*VTEConf) HasConfiguredFilter ¶
func (*VTEConf) HasConfiguredVertical ¶
func (*VTEConf) WithoutPasswords ¶
WithoutPassword returns a new semi-shallow copy of the called config with sensitive information replaced by `*`. By the "semi-shallownes" we mean that in case a sensitive information overwriting would affect the original object, such part will be provided as a deep copy.