infrastructure

package
v0.9.39 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 17, 2025 License: MPL-2.0 Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source 
const CustomHTTPErrorCode = 528

CustomHTTPErrorCode is a custom error code to be able to recognize it externally see also: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml 

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Variables

View Source 
var Version string

Version is a global variable written by the linker during CI builds

Functions

func BinaryVersion 

func BinaryVersion() string

BinaryVersion returns the best guess at the server's version

func DomainOf added in v0.9.31 

func DomainOf(input string) string

DomainOf returns either the domain name or a placeholder in case of a parse error

func GetInstanceId added in v0.9.31 

func GetInstanceId() string

func GetRunningSince added in v0.9.31 

func GetRunningSince() string

func ResetGlobalStats added in v0.9.27 

func ResetGlobalStats()

ResetGlobalStats the global stats

func SetUpConsoleLogging added in v0.9.38 

func SetUpConsoleLogging()

func SetUpGlobalLogger added in v0.9.38 

func SetUpGlobalLogger()

Types

type BodyPatternConfig 

type BodyPatternConfig struct {
	Name  string
	Regex string
}

BodyPatternConfig is unmarshalled from the configuration file

type CCLimitedURLChecker 

type CCLimitedURLChecker struct {
	// contains filtered or unexported fields
}

CCLimitedURLChecker is a concurrency-limited wrapper around a URLCheckerClient

func NewCCLimitedURLChecker 

func NewCCLimitedURLChecker() *CCLimitedURLChecker

NewCCLimitedURLChecker instantiates a new concurrency-limited URL checker

func (*CCLimitedURLChecker) CheckURL 

func (r *CCLimitedURLChecker) CheckURL(ctx context.Context, url string) *URLCheckResult

CheckURL checks the desired URL

type CachedURLChecker 

type CachedURLChecker struct {
	// contains filtered or unexported fields
}

CachedURLChecker wraps a concurrency-limited URL checker

func NewCachedURLChecker 

func NewCachedURLChecker() *CachedURLChecker

NewCachedURLChecker creates a new cached URL checker instance

func (*CachedURLChecker) CheckURL 

func (c *CachedURLChecker) CheckURL(ctx context.Context, url string) *URLCheckResult

CheckURL checks the desired URL

type DomainRateLimitedChecker 

type DomainRateLimitedChecker struct {
	// contains filtered or unexported fields
}

DomainRateLimitedChecker is a domain-rate-limited URLCheckerClient wrapper

func NewDomainRateLimitedChecker 

func NewDomainRateLimitedChecker(ratePerSecond rate.Limit) *DomainRateLimitedChecker

NewDomainRateLimitedChecker Creates a new domain-rate-limited URLCheckerClient instance

func (*DomainRateLimitedChecker) CheckURL 

func (c *DomainRateLimitedChecker) CheckURL(ctx context.Context, url string) *URLCheckResult

CheckURL checks the desired URL applying rate limits per domain

type DomainStats added in v0.9.31 

type DomainStats struct {
	BrokenBecause map[string]int64
	Ok            int64
}

DomainStats for one domain

type DomainStatsResponse added in v0.9.31 

type DomainStatsResponse struct {
	DomainStats map[string]DomainStats
}

DomainStatsResponse for all domains

type Stats added in v0.9.27 

type Stats struct {
	IncomingRequests       int64
	OutgoingRequests       int64
	IncomingStreamRequests int64
	DNSResolutionsFailed   int64
	LinkChecksErrored      int64
	LinkChecksOk           int64
	LinkChecksBroken       int64
	LinkChecksDropped      int64
	LinkChecksSkipped      int64
	CacheHits              int64
	CacheMisses            int64
}

Stats of the link checker service

type StatsState added in v0.9.30 

type StatsState struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

StatsState is the protected instance of the Stats object

func GlobalStats added in v0.9.27 

func GlobalStats() *StatsState

GlobalStats returns the global handler to the stats collector

func (*StatsState) GetDomainStats added in v0.9.31 

func (stats *StatsState) GetDomainStats() DomainStatsResponse

GetDomainStats returns a copy of the detailed domain stats

func (*StatsState) GetStats added in v0.9.30 

func (stats *StatsState) GetStats() Stats

GetStats returns a copy of the stats

func (*StatsState) OnCacheHit added in v0.9.30 

func (stats *StatsState) OnCacheHit()

OnCacheHit called when the result is taken from the cache

func (*StatsState) OnCacheMiss added in v0.9.30 

func (stats *StatsState) OnCacheMiss()

OnCacheMiss called when the requested URL wasn't found in the cache

func (*StatsState) OnDNSResolutionFailed added in v0.9.30 

func (stats *StatsState) OnDNSResolutionFailed(domain string)

OnDNSResolutionFailed called on dns resolution failure

func (*StatsState) OnIncomingRequest added in v0.9.30 

func (stats *StatsState) OnIncomingRequest()

OnIncomingRequest call on incoming request

func (*StatsState) OnIncomingStreamRequest added in v0.9.30 

func (stats *StatsState) OnIncomingStreamRequest()

OnIncomingStreamRequest called on an incoming stream request

func (*StatsState) OnLinkBroken added in v0.9.30 

func (stats *StatsState) OnLinkBroken(domain string, status string)

OnLinkBroken called on link check broken

func (*StatsState) OnLinkDropped added in v0.9.30 

func (stats *StatsState) OnLinkDropped(domain string)

OnLinkDropped called on link check dropped

func (*StatsState) OnLinkErrored added in v0.9.30 

func (stats *StatsState) OnLinkErrored(domain string)

OnLinkErrored called on link check error

func (*StatsState) OnLinkOk added in v0.9.30 

func (stats *StatsState) OnLinkOk(domain string)

OnLinkOk called on link check ok

func (*StatsState) OnLinkSkipped added in v0.9.30 

func (stats *StatsState) OnLinkSkipped(domain string)

OnLinkSkipped called on link check skipped

func (*StatsState) OnOutgoingRequest added in v0.9.30 

func (stats *StatsState) OnOutgoingRequest()

OnOutgoingRequest called on outgoing request

type URLCheckResult 

type URLCheckResult struct {
	Status                URLCheckStatus
	Code                  int
	Error                 error
	FetchedAtEpochSeconds int64
	BodyPatternsFound     []string
	RemoteAddr            string
	CheckerTrace          []URLCheckerPluginTrace
	ElapsedMs             int64
}

URLCheckResult is the internal struct to hold URL check results

type URLCheckStatus 

type URLCheckStatus int

URLCheckStatus indicates the URL check outcome 

const (
	// Skipped indicates that the URL check wasn't performed
	Skipped URLCheckStatus = iota
	// Ok indicates the URL is accessible
	Ok
	// Broken indicates the URL cannot be accessed for some reason
	Broken
	// Dropped indicates an internal reason for not proceeding with the URL check
	Dropped
)

func URLCheckStatusString 

func URLCheckStatusString(s string) (URLCheckStatus, error)

URLCheckStatusString retrieves an enum value from the enum constants string name. Throws an error if the param is not part of the enum.

func URLCheckStatusValues 

func URLCheckStatusValues() []URLCheckStatus

URLCheckStatusValues returns all values of the enum

func (URLCheckStatus) IsAURLCheckStatus 

func (i URLCheckStatus) IsAURLCheckStatus() bool

IsAURLCheckStatus returns "true" if the value is listed in the enum definition. "false" otherwise

func (URLCheckStatus) MarshalJSON 

func (i URLCheckStatus) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for URLCheckStatus

func (URLCheckStatus) MarshalText 

func (i URLCheckStatus) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface for URLCheckStatus

func (URLCheckStatus) String 

func (i URLCheckStatus) String() string

func (*URLCheckStatus) UnmarshalJSON 

func (i *URLCheckStatus) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for URLCheckStatus

func (*URLCheckStatus) UnmarshalText 

func (i *URLCheckStatus) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface for URLCheckStatus

type URLChecker 

type URLChecker interface {
	CheckURL(ctx context.Context, url string) *URLCheckResult
}

URLChecker interface that all layers should conform to

type URLCheckerClient 

type URLCheckerClient struct {
	// contains filtered or unexported fields
}

URLCheckerClient contains the HTTP/URL checking logic

func NewURLCheckerClient 

func NewURLCheckerClient() *URLCheckerClient

NewURLCheckerClient instantiates a new basic URL checking client

func (*URLCheckerClient) CheckURL 

func (c *URLCheckerClient) CheckURL(ctx context.Context, url string) *URLCheckResult

CheckURL checks a single URL

type URLCheckerPlugin added in v0.9.18 

type URLCheckerPlugin interface {
	// Name returns the name of the plugin to use in logging and result reporting
	Name() string

	// CheckURL gets the urlToCheck and lastResult, which it can process, and return the next result
	// and a boolean flag, whether the chain should be interrupted, and the last result - simply returned
	// ctx can be used to cancel the request prematurely
	CheckURL(ctx context.Context, urlToCheck string, lastResult *URLCheckResult) (*URLCheckResult, bool)
}

URLCheckerPlugin represents one low-level URL checker in a chain of checkers

type URLCheckerPluginTrace added in v0.9.20 

type URLCheckerPluginTrace struct {
	Name      string
	Code      int
	ElapsedMs int64
	Error     string
}

URLCheckerPluginTrace is the internal struct to gather individual checker plugin stats

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.