db

Documentation

Index

• Constants
• func DeleteFile(fs afero.Fs)
• type DB
  • func Open() *DB
  • func OpenDB(dir string, fs afero.Fs) *DB
  • func (store *DB) Close() error
  • func (store *DB) Lookup(u *urlpkg.URL) Item
  • func (store *DB) Store(u *urlpkg.URL, item Item)
• type Item
  • func (i Item) Empty() bool
  • func (i Item) String() string

Constants

const FileName = "goscrape-etags.txt"

Functions

func DeleteFile

func DeleteFile(fs afero.Fs)

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB provides a persistent store for HTTP ETags to reduce network traffic when repeating a download session. If the store is unavailable for some reason, its methods are no-ops.

func Open

func Open() *DB

func OpenDB

func OpenDB(dir string, fs afero.Fs) *DB

func (*DB) Close

func (store *DB) Close() error

func (*DB) Lookup

func (store *DB) Lookup(u *urlpkg.URL) Item

Lookup finds the ETags for a given URL.

func (*DB) Store

func (store *DB) Store(u *urlpkg.URL, item Item)

Store stores the ETags for a given URL.

type Item

type Item struct {
	ETags   string
	Expires time.Time
}

func (Item) Empty

func (i Item) Empty() bool

func (Item) String

func (i Item) String() string