config

Documentation

Index

• type Config
  • func NewConfig(configPath string) (*Config, error)
• type WriterConfig

Types

type Config

type Config struct {
	Writer   WriterConfig      `yaml:"writer"`
	Scrapers []scraper.Scraper `yaml:"scrapers"`
}

Config defines the overall structure of the scraper configuration. Values will be taken from a config yml file or environment variables or both.

func NewConfig

func NewConfig(configPath string) (*Config, error)

type WriterConfig

type WriterConfig struct {
	Type     string `yaml:"type" env:"WRITER_TYPE" env-default:"stdout"`
	Uri      string `yaml:"uri" env:"WRITER_URI"`
	User     string `yaml:"user" env:"WRITER_USER"`
	Password string `yaml:"password" env:"WRITER_PASSWORD"`
}

.WriterConfig defines the necessary paramters to make a new writer which is responsible for writing the scraped data to a specific output eg. stdout.