store

Documentation

Overview

Key generation and related methods relevant to any storage backend.

Index

• Constants
• Variables
• func Key(url URLWithHostname) uint64
• type DatabaseOptions
• type Factory
• type Maintainable
• type StoredUrlData
  • func (u *StoredUrlData) AssertTimes()
• type URLDataStore
• type URLString
  • func (u URLString) Hostname() string
  • func (u URLString) String() string
• type URLWithHostname

Constants

const (
	MASK_56 uint64 = 0xffffffffffffff

	CHECKSUM_MASK uint64 = 127 << 56
	TIME_MASK     uint64 = 0xFFFFFF << 32
)

const (
	DEFAULT_TTL = 24 * time.Hour * 30
)

Variables

var (
	ErrorDatabaseNotFound = errors.New("database not found")
	ErrorResourceNotFound = errors.New("resource not found in data store")
)

Functions

func Key

func Key(url URLWithHostname) uint64

Produces a 63 bit uint contained in a uint64 (SQLite cannot accept uint64 with high bit set as a primary key) [Bit 63] Always 0 [Bits 62-56] A 7 bit checksum based on the domain name [Bits 31-0] A 56 bit hash of the URL (reduced from a 64 bit fnv1a hash)

Types

type DatabaseOptions

type DatabaseOptions fmt.Stringer

type Factory

type Factory func() (URLDataStore, error)

type Maintainable

type Maintainable interface {
	Create() error
	Clear() error
	Maintain() error
}

type StoredUrlData

type StoredUrlData struct {
	Data resource.WebPage //todo promote this to default embedding
	TTL  *time.Duration
}

func (*StoredUrlData) AssertTimes

func (u *StoredUrlData) AssertTimes()

type URLDataStore

type URLDataStore interface {
	Open(context.Context) error
	Store(*StoredUrlData) (uint64, error)
	Fetch(*nurl.URL) (*StoredUrlData, error)
	Close() error
}

type URLString

type URLString string

Type that provides the

func (URLString) Hostname

func (u URLString) Hostname() string

func (URLString) String

func (u URLString) String() string

type URLWithHostname

type URLWithHostname interface {
	fmt.Stringer
	Hostname() string
}

net.URL and URLString both implement this interface, which is needed to generate a key for the URL.