fetch

Documentation

Index

• Constants
• func CheckIcon(ctx context.Context, url string) (bool, error)
• func SetClient(c *http.Client)
• type Document
  • func Fetch(ctx context.Context, url string) (*Document, error)
• type Fetcher
• type HTMLFetcher
  • func NewHTMLFetcher(url string) *HTMLFetcher
  • func (f *HTMLFetcher) Fetch(ctx context.Context) (doc *Document, err error)
• type HTTPError
  • func (e HTTPError) Error() string
  • func (e HTTPError) Forbidden() bool
  • func (e HTTPError) NotFound() bool
  • func (e HTTPError) NotModified() bool
• type IconFetcher
  • func NewIconFetcher(url string) *IconFetcher
  • func (f *IconFetcher) Check(ctx context.Context) (exists bool, err error)

Constants

const (
	HeaderUserAgent       = "User-Agent"
	HeaderAccept          = "Accept"
	HeaderAcceptLang      = "Accept-Language"
	HeaderAcceptEncode    = "Accept-Encoding"
	HeaderCacheControl    = "Cache-Control"
	HeaderReferer         = "Referer"
	HeaderIfNoneMatch     = "If-None-Match"
	HeaderIfModifiedSince = "If-Modified-Since"
	HeaderRFC3229         = "A-IM"
	HeaderETag            = "ETag"
	HeaderLastModified    = "Last-Modified"
	HeaderContentEncoding = "Content-Encoding"
	HeaderContentType     = "Content-Type"
)

Canonical names of headers used by the fetch package

Functions

func CheckIcon

func CheckIcon(ctx context.Context, url string) (bool, error)

CheckIcon returns true if the icon at the specified URL can be retrieved and has an image mimetype, otherwise it returns false.

func SetClient

func SetClient(c *http.Client)

SetClient allows you to specify an alternative http.Client to the default one used by all http based Fetchers in this package. Use this function to change the timeouts of the client or to set a test client.

Types

type Document

type Document struct {
	Link         string `json:"link"`
	Title        string `json:"title"`
	Description  string `json:"description"`
	Favicon      string `json:"favicon"`
	FaviconCheck bool   `json:"favicon_exists,omitempty"`
}

func Fetch

func Fetch(ctx context.Context, url string) (*Document, error)

type Fetcher

type Fetcher interface {
	Fetch(context.Context) (any, error)
}

Fetcher is an interface for statefully making periodic requests to a resource.

type HTMLFetcher

type HTMLFetcher struct {
	// contains filtered or unexported fields
}

HTMLFetcher is an interface for fetching the full HTML associated with a feed item

func NewHTMLFetcher

func NewHTMLFetcher(url string) *HTMLFetcher

NewHTMLFetcher creates a new HTML fetcher that can fetch the full HTML from the specified URL.

func (*HTMLFetcher) Fetch

func (f *HTMLFetcher) Fetch(ctx context.Context) (doc *Document, err error)

The HTMLFetcher uses GET requests to retrieve details about the HTML page.

type HTTPError

type HTTPError struct {
	Code   int
	Status string
}

HTTPError contains status information from the request and can be returned as error. This type of error is returned from the Fetcher when the server replies successfully but without a 200 status. The suggested use of this error is in a switch statement, e.g. something like: switch he := err.(type) {case fetch.HTTPError: ... default: ...}

func (HTTPError) Error

func (e HTTPError) Error() string

Error implements the error interface and returns a string representation of the err.

func (HTTPError) Forbidden

func (e HTTPError) Forbidden() bool

Forbidden returns true if the error is an HTTP 403

func (HTTPError) NotFound

func (e HTTPError) NotFound() bool

NotFound returns true if the error is an HTTP 404

func (HTTPError) NotModified

func (e HTTPError) NotModified() bool

NotModified returns true if the error is an HTTP 304

type IconFetcher

type IconFetcher struct {
	HTMLFetcher
}

func NewIconFetcher

func NewIconFetcher(url string) *IconFetcher

func (*IconFetcher) Check

func (f *IconFetcher) Check(ctx context.Context) (exists bool, err error)