plan

package
v0.0.0-...-5871508 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 6, 2024 License: Apache-2.0 Imports: 5 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Field

type Field struct {
	Name string `yaml:"name" validate:"required,alpha"`
	Type string `yaml:"type" validate:"required,oneof=raw text number decimal datetime"`
	// CSS Selector.
	Selector string `yaml:"selector" validate:"required"`
	// Regex to extract data from the selector.
	Regex string `yaml:"regex"`
	// See: https://github.com/golang-module/carbon#format-sign-table
	Format string `yaml:"format"`
	// TZ Database name e.g "Indian/Mauritius"
	Timezone string `yaml:"timezone"`
}

Field is a single piece of data.

func (*Field) SetDefaults

func (d *Field) SetDefaults()

SetDefaults sets the default values for a field.

type Plan

type Plan struct {
	Source     string   `yaml:"source" validate:"required,url"`
	Type       string   `yaml:"type" validate:"required,oneof=website"`
	UserAgents []string `yaml:"user_agents"`
	Fields     []Field  `yaml:",flow" validate:"required,dive"`
	// Location of the transformer file.
	Transformer string `yaml:"transformer"`
}

Plan defines the parameters for harvesting.

func Load

func Load(path string) (*Plan, error)

Load loads a plan from a file.

func (*Plan) SetDefaults

func (p *Plan) SetDefaults()

SetDefaults sets the default values for the plan.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL