coles

Documentation

Index

• Constants
• Variables
• type Coles
  • func (c *Coles) GetSharedProductsUpdatedAfter(t time.Time, count int) ([]shared.ProductInfo, error)
  • func (c *Coles) GetTotalProductCount() (int, error)
  • func (c *Coles) Init(baseURL string, dbPath string, productMaxAge time.Duration) error
  • func (c *Coles) Run(cancel chan struct{})

Constants

const BROWSE_HOMEPAGE_URL_FORMAT = "%s/browse"

const BROWSE_JSON_URL_FORMAT = "%s/_next/data/%s/en/browse.json"

const CATEGORY_URL_FORMAT = "%s/_next/data/%s/en/browse/%s.json"

const COLES_ID_PREFIX = "coles_id_"

const DB_SCHEMA_VERSION = 1

const DEFAULT_API_VERSION = "20240827.02_v4.7.7"

const DEFAULT_LISTING_PAGE_CHECK_INTERVAL = 1 * time.Minute

const PRODUCTS_PER_PAGE = 48

const SCRAPE_TRAP_STRING = "Pardon Our Interruption"

Variables

var ErrHitScrapeTrap = errors.New("caught in a scrape trap")

Types

type Coles

type Coles struct {
	// contains filtered or unexported fields
}

Coles satisfies the ProductInfoGetter interface.

func (*Coles) GetSharedProductsUpdatedAfter

func (c *Coles) GetSharedProductsUpdatedAfter(t time.Time, count int) ([]shared.ProductInfo, error)

GetSharedProductsUpdatedAfter provides a list of product IDs that have been updated since the given time

func (*Coles) GetTotalProductCount

func (c *Coles) GetTotalProductCount() (int, error)

GetTotalProductCount returns the total number of products in the database.

func (*Coles) Init

func (c *Coles) Init(baseURL string, dbPath string, productMaxAge time.Duration) error

Init initialises the Coles struct.

func (*Coles) Run

func (c *Coles) Run(cancel chan struct{})

Runs up all the workers and mediates data flowing between them. Currently all sqlite writes happen via this function. This may move off to a separate goroutine in the future.