README
¶
Lambda
Lamda helper functions and types for State Machine
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Error ¶
type Error struct {
Err *errors.Error `json:"error"`
Service string `json:"service"` // Currently running function, for example "scrape"
Meta Meta `json:"meta"`
}
Error represents an error that occurred during the processing of a Krang Lambda function.
type Meta ¶
type Meta struct {
GroupSlug string `json:"group_slug"`
TaskID int64 `json:"task_id"`
ScrapeID string `json:"scrape_id"`
URL string `json:"url"`
SearchTerm string `json:"search_term"`
Data map[string]any `json:"data"`
}
Meta represents the attributes of a failed task.
type Page ¶ added in v0.0.5
type Page struct {
ID primitive.ObjectID `bson:"_id,omitempty" json:"id"`
ScrapeID *primitive.ObjectID `bson:"scrape_id" json:"scrape_id"`
URL string `bson:"url" json:"url"`
GroupSlug string `bson:"group_slug,omitempty" json:"group_slug"`
TaskID int64 `bson:"task_id,omitempty" json:"task_id"`
SearchTerm string `json:"search_term" bson:"search_term"`
RelevancyScore uint `json:"relevancy_score" bson:"relevancy_score"`
SiteScore uint `json:"site_score" bson:"site_score"`
Scrape Scrape `bson:"scrape" json:"scrape"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
}
Page represents an individual task scrape including metadata from the Task.
type Scrape ¶
type Scrape struct {
ID primitive.ObjectID `json:"id" bson:"_id,omitempty"`
URL string `json:"url" bson:"url"`
HTTPStatus int `json:"http_status" bson:"http_status,omitempty"`
Content ScrapeContent `json:"content" bson:"content,omitempty"`
Metrics ScrapeMetrics `json:"metrics" bson:"metrics,omitempty"`
}
Scrape represents an individual scrape of a page and its various metrics.
type ScrapeContent ¶
type ScrapeContent struct {
H1 string `json:"h1" bson:"h1"`
H2 string `json:"h2" bson:"h2"`
Title string `json:"title" bson:"title"`
ExternalLinks int `json:"external_links" bson:"external_links"`
Body string `json:"body" bson:"body"`
}
ScrapeContent represents the HTML markup of a page including any <body> content that's relevant for scoring.
type ScrapeMetrics ¶
type ScrapeMetrics struct {
Backlinks int `json:"backlinks" bson:"backlinks"`
LoadingTime time.Duration `json:"loading_time" bson:"loading_time"`
AhrefsDA int `json:"ahrefs_da" bson:"ahrefs_da"` // Domain Authority
MozPA int `json:"moz_pa" bson:"moz_pa"` // Page Authority
MozDA int `json:"moz_da" bson:"moz_da"` // Domain Authority
MozSpamScore int `json:"moz_spam_score" bson:"moz_spam_score"`
MajesticCF int `json:"majestic_cf" bson:"majestic_cf"` // Citation Flow
MajesticTF int `json:"majestic_tf" bson:"majestic_tf"` // Trust Flow
}
ScrapeMetrics represents the scores and metrics retrieved from Ahrefs, Moz and Majestic.
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.