engine

package
v0.0.0-...-12d29e9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 7, 2024 License: Apache-2.0, MIT Imports: 27 Imported by: 0

Documentation

Overview

Core automod rules engine implementation and related types

Index

Constants

This section is empty.

Variables

View Source 
var (
	ReviewStateEscalated = "escalated"
	ReviewStateOpen      = "open"
	ReviewStateClosed    = "closed"
	ReviewStateNone      = "none"
)
View Source 
var (
	CreateOp = "create"
	UpdateOp = "update"
	DeleteOp = "delete"
)
View Source 
var (
	// time period within which automod will not re-report an account for the same reasonType
	ReportDupePeriod = 1 * 24 * time.Hour
	// number of reports automod can file per day, for all subjects and types combined (circuit breaker)
	QuotaModReportDay = 2000
	// number of takedowns automod can action per day, for all subjects combined (circuit breaker)
	QuotaModTakedownDay = 200
	// number of misc actions automod can do per day, for all subjects combined (circuit breaker)
	QuotaModActionDay = 1000
)
View Source 
var (
	ReportReasonSpam       = "com.atproto.moderation.defs#reasonSpam"
	ReportReasonViolation  = "com.atproto.moderation.defs#reasonViolation"
	ReportReasonMisleading = "com.atproto.moderation.defs#reasonMisleading"
	ReportReasonSexual     = "com.atproto.moderation.defs#reasonSexual"
	ReportReasonRude       = "com.atproto.moderation.defs#reasonRude"
	ReportReasonOther      = "com.atproto.moderation.defs#reasonOther"
)

Functions

func ReasonShortName 

func ReasonShortName(reason string) string

Types

type AbuseSignature 

type AbuseSignature struct {
	Property string
	Value    string
}

opaque fingerprints for correlating abusive accounts

type AccountContext 

type AccountContext struct {
	BaseContext

	Account AccountMeta
}

Both a useful context on it's own (eg, for identity events), and extended by other context types.

func NewAccountContext 

func NewAccountContext(ctx context.Context, eng *Engine, meta AccountMeta) AccountContext

func (*AccountContext) AcknowledgeAccount 

func (c *AccountContext) AcknowledgeAccount()

func (*AccountContext) AddAccountFlag 

func (c *AccountContext) AddAccountFlag(val string)

func (*AccountContext) AddAccountLabel 

func (c *AccountContext) AddAccountLabel(val string)

func (*AccountContext) AddAccountTag 

func (c *AccountContext) AddAccountTag(val string)

func (*AccountContext) EscalateAccount 

func (c *AccountContext) EscalateAccount()

func (*AccountContext) GetAccountRelationship 

func (c *AccountContext) GetAccountRelationship(other syntax.DID) AccountRelationship

fetch relationship metadata between this account and another account

func (*AccountContext) ReportAccount 

func (c *AccountContext) ReportAccount(reason, comment string)

func (*AccountContext) TakedownAccount 

func (c *AccountContext) TakedownAccount()

type AccountMeta 

type AccountMeta struct {
	Identity             *identity.Identity
	Profile              ProfileSummary
	Private              *AccountPrivate
	AccountLabels        []string
	AccountNegatedLabels []string
	AccountFlags         []string
	FollowersCount       int64
	FollowsCount         int64
	PostsCount           int64
	Takendown            bool
	Deactivated          bool
	// best effort public interpretation of account creation timestamp. not always available, and may be inaccurate/inconsistent for now.
	CreatedAt *time.Time
}

information about a repo/account/identity, always pre-populated and relevant to many rules

type AccountPrivate 

type AccountPrivate struct {
	Email          string
	EmailConfirmed bool
	IndexedAt      *time.Time
	AccountTags    []string
	// ReviewState will be one of ReviewStateEscalated, ReviewStateOpen, ReviewStateClosed, ReviewStateNone, or "" (unknown)
	ReviewState     string
	Appealed        bool
	AbuseSignatures []AbuseSignature
}

type AccountRelationship 

type AccountRelationship struct {
	// the DID of the "other" account
	DID syntax.DID
	// if the primary account is followed by this "other" DID
	FollowedBy bool
	// if the primary account follows this "other" DID
	Following bool
}

Represents app.bsky social graph relationship between a primary account, and an "other" account

type AccountRuleFunc 

type AccountRuleFunc = func(c *AccountContext) error

type BaseContext 

type BaseContext struct {
	// Actual golang "context.Context", if needed for timeouts etc
	Ctx context.Context
	// Any errors encountered while processing methods on this struct (or sub-types) get rolled up in this nullable field
	Err error
	// slog logger handle, with event-specific structured fields pre-populated. Pointer, but expected to never be nil.
	Logger *slog.Logger
	// contains filtered or unexported fields
}

The primary interface exposed to rules. All other contexts derive from this "base" struct.

func (*BaseContext) Directory 

func (c *BaseContext) Directory() identity.Directory

Access to engine's identity directory (without access to other engine fields)

func (*BaseContext) GetAccountMeta 

func (c *BaseContext) GetAccountMeta(did syntax.DID) *AccountMeta

fetch account metadata for the given DID. if there is any problem with lookup, returns nil.

TODO: should this take an AtIdentifier instead?

func (*BaseContext) GetCount 

func (c *BaseContext) GetCount(name, val, period string) int

request external state via engine (indirect)

func (*BaseContext) GetCountDistinct 

func (c *BaseContext) GetCountDistinct(name, bucket, period string) int

func (*BaseContext) InSet 

func (c *BaseContext) InSet(name, val string) bool

func (*BaseContext) Increment 

func (c *BaseContext) Increment(name, val string)

func (*BaseContext) IncrementDistinct 

func (c *BaseContext) IncrementDistinct(name, bucket, val string)

func (*BaseContext) IncrementPeriod 

func (c *BaseContext) IncrementPeriod(name, val string, period string)

func (*BaseContext) InternalEngine 

func (c *BaseContext) InternalEngine() *Engine

Returns a pointer to the underlying automod engine. This usually should NOT be used in rules.

This is an escape hatch for hacking on the system before features get fully integerated in to the content API surface. The Engine API is not stable.

func (*BaseContext) Notify 

func (c *BaseContext) Notify(srv string)

type BlobRuleFunc 

type BlobRuleFunc = func(c *RecordContext, blob lexutil.LexBlob, data []byte) error

type CounterDistinctRef 

type CounterDistinctRef struct {
	Name   string
	Bucket string
	Val    string
}

type CounterRef 

type CounterRef struct {
	Name   string
	Val    string
	Period *string
}

type Effects 

type Effects struct {

	// List of counters which should be incremented as part of processing this event. These are collected during rule execution and persisted in bulk at the end.
	CounterIncrements []CounterRef
	// Similar to "CounterIncrements", but for "distinct" style counters
	CounterDistinctIncrements []CounterDistinctRef // TODO: better variable names
	// Label values which should be applied to the overall account, as a result of rule execution.
	AccountLabels []string
	// Moderation tags (similar to labels, but private) which should be applied to the overall account, as a result of rule execution.
	AccountTags []string
	// automod flags (metadata) which should be applied to the account as a result of rule execution.
	AccountFlags []string
	// Reports which should be filed against this account, as a result of rule execution.
	AccountReports []ModReport
	// If "true", a rule decided that the entire account should have a takedown.
	AccountTakedown bool
	// If "true", a rule decided that the reported account should be escalated.
	AccountEscalate bool
	// If "true", a rule decided that the reports on account should be resolved as acknowledged.
	AccountAcknowledge bool
	// Same as "AccountLabels", but at record-level
	RecordLabels []string
	// Same as "AccountTags", but at record-level
	RecordTags []string
	// Same as "AccountFlags", but at record-level
	RecordFlags []string
	// Same as "AccountReports", but at record-level
	RecordReports []ModReport
	// Same as "AccountTakedown", but at record-level
	RecordTakedown bool
	// Set of Blob CIDs to takedown (eg, purge from CDN) when doing a record takedown
	BlobTakedowns []string
	// If "true", indicates that a rule indicates that the action causing the event should be blocked or prevented
	RejectEvent bool
	// Services, if any, which should blast out a notification about this even (eg, Slack)
	NotifyServices []string
	// contains filtered or unexported fields
}

Mutable container for all the possible side-effects from rule execution.

This single type tracks generic effects (eg, counter increments), account-level actions, and record-level actions (even for processing of account-level events which have no possible record-level effects).

func ExtractEffects 

func ExtractEffects(c *BaseContext) *Effects

Helper to access the private effects field from a context. Intended for use in test code, *not* from rules.

func (*Effects) AcknowledgeAccount 

func (e *Effects) AcknowledgeAccount()

Enqueues reports on account to be "acknowledged" (closed) at the end of rule processing.

func (*Effects) AddAccountFlag 

func (e *Effects) AddAccountFlag(val string)

Enqueues the provided flag (string value) to be recorded (in the Engine's flagstore) at the end of rule processing.

func (*Effects) AddAccountLabel 

func (e *Effects) AddAccountLabel(val string)

Enqueues the provided label (string value) to be added to the account at the end of rule processing.

func (*Effects) AddAccountTag 

func (e *Effects) AddAccountTag(val string)

Enqueues the provided label (string value) to be added to the account at the end of rule processing.

func (*Effects) AddRecordFlag 

func (e *Effects) AddRecordFlag(val string)

Enqueues the provided flag (string value) to be recorded (in the Engine's flagstore) at the end of rule processing.

func (*Effects) AddRecordLabel 

func (e *Effects) AddRecordLabel(val string)

Enqueues the provided label (string value) to be added to the record at the end of rule processing.

func (*Effects) AddRecordTag 

func (e *Effects) AddRecordTag(val string)

Enqueues the provided tag (string value) to be added to the record at the end of rule processing.

func (*Effects) EscalateAccount 

func (e *Effects) EscalateAccount()

Enqueues the account to be "escalated" for mod review at the end of rule processing.

func (*Effects) Increment 

func (e *Effects) Increment(name, val string)

Enqueues the named counter to be incremented at the end of all rule processing. Will automatically increment for all time periods.

"name" is the counter namespace. "val" is the specific counter with that namespace.

func (*Effects) IncrementDistinct 

func (e *Effects) IncrementDistinct(name, bucket, val string)

Enqueues the named "distinct value" counter based on the supplied string value ("val") to be incremented at the end of all rule processing. Will automatically increment for all time periods.

func (*Effects) IncrementPeriod 

func (e *Effects) IncrementPeriod(name, val string, period string)

Enqueues the named counter to be incremented at the end of all rule processing. Will only increment the indicated time period bucket.

func (*Effects) Notify 

func (e *Effects) Notify(srv string)

Records that the given service should be notified about this event

func (*Effects) Reject 

func (e *Effects) Reject()

func (*Effects) ReportAccount 

func (e *Effects) ReportAccount(reason, comment string)

Enqueues a moderation report to be filed against the account at the end of rule processing.

func (*Effects) ReportRecord 

func (e *Effects) ReportRecord(reason, comment string)

Enqueues a moderation report to be filed against the record at the end of rule processing.

func (*Effects) TakedownAccount 

func (e *Effects) TakedownAccount()

Enqueues the entire account to be taken down at the end of rule processing.

func (*Effects) TakedownBlob 

func (e *Effects) TakedownBlob(cid string)

Enqueues the blob CID to be taken down (aka, CDN purge) as part of any record takedown

func (*Effects) TakedownRecord 

func (e *Effects) TakedownRecord()

Enqueues the record to be taken down at the end of rule processing.

type Engine 

type Engine struct {
	Logger    *slog.Logger
	Directory identity.Directory
	Rules     RuleSet
	Counters  countstore.CountStore
	Sets      setstore.SetStore
	Cache     cachestore.CacheStore
	Flags     flagstore.FlagStore
	// unlike the other sub-modules, this field (Notifier) may be nil
	Notifier Notifier
	// use to fetch public account metadata from AppView; no auth
	BskyClient *xrpc.Client
	// used to persist moderation actions in ozone moderation service; optional, admin auth
	OzoneClient *xrpc.Client
	// used to fetch private account metadata from PDS or entryway; optional, admin auth
	AdminClient *xrpc.Client
	// used to fetch blobs from upstream PDS instances
	BlobClient *http.Client

	// internal configuration
	Config EngineConfig
}

runtime for executing rules, managing state, and recording moderation actions.

NOTE: careful when initializing: several fields must not be nil or zero, even though they are pointer type.

func EngineTestFixture 

func EngineTestFixture() Engine

func (*Engine) CanonicalLogLineAccount 

func (e *Engine) CanonicalLogLineAccount(c *AccountContext)

func (*Engine) CanonicalLogLineNotification 

func (e *Engine) CanonicalLogLineNotification(c *NotificationContext)

func (*Engine) CanonicalLogLineOzoneEvent 

func (e *Engine) CanonicalLogLineOzoneEvent(c *OzoneEventContext)

func (*Engine) CanonicalLogLineRecord 

func (e *Engine) CanonicalLogLineRecord(c *RecordContext)

func (*Engine) GetAccountMeta 

func (e *Engine) GetAccountMeta(ctx context.Context, ident *identity.Identity) (*AccountMeta, error)

Helper to hydrate metadata about an account from several sources: PDS (if access), mod service (if access), public identity resolution

func (*Engine) GetAccountRelationship 

func (eng *Engine) GetAccountRelationship(ctx context.Context, primary, other syntax.DID) (*AccountRelationship, error)

Helper to fetch social graph relationship metadata

func (*Engine) ProcessAccountEvent 

func (eng *Engine) ProcessAccountEvent(ctx context.Context, evt comatproto.SyncSubscribeRepos_Account) error

Entrypoint for external code pushing #account events in to the engine.

This method can be called concurrently, though cached state may end up inconsistent if multiple events for the same account (DID) are processed in parallel.

func (*Engine) ProcessIdentityEvent 

func (eng *Engine) ProcessIdentityEvent(ctx context.Context, evt comatproto.SyncSubscribeRepos_Identity) error

Entrypoint for external code pushing #identity events in to the engine.

This method can be called concurrently, though cached state may end up inconsistent if multiple events for the same account (DID) are processed in parallel.

func (*Engine) ProcessNotificationEvent 

func (eng *Engine) ProcessNotificationEvent(ctx context.Context, senderDID, recipientDID syntax.DID, reason string, subject syntax.ATURI) (bool, error)

returns a boolean indicating "block the event"

func (*Engine) ProcessOzoneEvent 

func (eng *Engine) ProcessOzoneEvent(ctx context.Context, eventView *toolsozone.ModerationDefs_ModEventView) error

Entrypoint for external code pushing ozone events.

This method can be called concurrently, though cached state may end up inconsistent if multiple events for the same account (DID) are processed in parallel.

func (*Engine) ProcessRecordOp 

func (eng *Engine) ProcessRecordOp(ctx context.Context, op RecordOp) error

Entrypoint for external code pushing repository updates. A simple repo commit results in multiple calls.

This method can be called concurrently, though cached state may end up inconsistent if multiple events for the same account (DID) are processed in parallel.

func (*Engine) PurgeAccountCaches 

func (e *Engine) PurgeAccountCaches(ctx context.Context, did syntax.DID) error

Purge metadata caches for a specific account.

type EngineConfig 

type EngineConfig struct {
	// if enabled, account metadata is not hydrated for every event by default
	SkipAccountMeta bool
}

type IdentityRuleFunc 

type IdentityRuleFunc = func(c *AccountContext) error

type ModReport 

type ModReport struct {
	ReasonType string
	Comment    string
}

Simplified variant of input parameters for com.atproto.moderation.createReport, for internal tracking

type NotificationContext 

type NotificationContext struct {
	AccountContext

	Recipient    AccountMeta
	Notification NotificationMeta
}

Originally intended for push notifications, but can also work for any inter-account notification.

func NewNotificationContext 

func NewNotificationContext(ctx context.Context, eng *Engine, sender, recipient AccountMeta, reason string, subject syntax.ATURI) NotificationContext

func (*NotificationContext) Reject 

func (c *NotificationContext) Reject()

type NotificationMeta 

type NotificationMeta struct {
	// Expected values are 'like', 'repost', 'follow', 'mention', 'reply', and 'quote'; arbitrary values may be added in the future.
	Reason string
	// The content (atproto record) which was the cause of this notification. Could be a post with a mention, or a like, follow, or repost record.
	Subject syntax.ATURI
}

Additional notification metadata, with fields aligning with the `app.bsky.notification.listNotifications` Lexicon schemas

type NotificationRuleFunc 

type NotificationRuleFunc = func(c *NotificationContext) error

type Notifier 

type Notifier interface {
	SendAccount(ctx context.Context, service string, c *AccountContext) error
	SendRecord(ctx context.Context, service string, c *RecordContext) error
}

Interface for a type that can handle sending notifications

type OzoneEvent 

type OzoneEvent struct {
	EventType  string
	EventID    int64
	CreatedAt  syntax.Datetime
	CreatedBy  syntax.DID
	SubjectDID syntax.DID
	SubjectURI *syntax.ATURI
	// TODO: SubjectBlobs []syntax.CID
	Event toolsozone.ModerationDefs_ModEventView_Event
}

type OzoneEventContext 

type OzoneEventContext struct {
	AccountContext

	Event OzoneEvent

	// Moderator team member (for ozone internal events) or account that created a report or appeal
	CreatorAccount AccountMeta

	// If the subject of the event is a record, this is the record metadata
	SubjectRecord *RecordMeta
}

Represents an ozone event on a subject account.

TODO: for ozone events with a record subject (not account subject), should we extend RecordContext instead?

func NewOzoneEventContext 

func NewOzoneEventContext(ctx context.Context, eng *Engine, eventView *toolsozone.ModerationDefs_ModEventView) (*OzoneEventContext, error)

type OzoneEventRuleFunc 

type OzoneEventRuleFunc = func(c *OzoneEventContext) error

type PostRuleFunc 

type PostRuleFunc = func(c *RecordContext, post *appbsky.FeedPost) error

type ProfileRuleFunc 

type ProfileRuleFunc = func(c *RecordContext, profile *appbsky.ActorProfile) error

type ProfileSummary 

type ProfileSummary struct {
	HasAvatar   bool
	AvatarCid   *string
	BannerCid   *string
	Description *string
	DisplayName *string
}

type RecordContext 

type RecordContext struct {
	AccountContext

	RecordOp RecordOp
}

Represents a repository operation on a single record: create, update, delete, etc.

func NewRecordContext 

func NewRecordContext(ctx context.Context, eng *Engine, meta AccountMeta, op RecordOp) RecordContext

func (*RecordContext) AddRecordFlag 

func (c *RecordContext) AddRecordFlag(val string)

func (*RecordContext) AddRecordLabel 

func (c *RecordContext) AddRecordLabel(val string)

func (*RecordContext) AddRecordTag 

func (c *RecordContext) AddRecordTag(val string)

func (*RecordContext) Blobs 

func (c *RecordContext) Blobs() ([]lexutil.LexBlob, error)

Parses out any blobs from the enclosed record.

NOTE: for consistency with other RecordContext methods, which don't usually return errors, maybe the error-returning version of this function should be a helper function, or defined on RecordOp, and the RecordContext version should return an empty array on error?

func (*RecordContext) ReportRecord 

func (c *RecordContext) ReportRecord(reason, comment string)

func (*RecordContext) TakedownBlob 

func (c *RecordContext) TakedownBlob(cid string)

func (*RecordContext) TakedownRecord 

func (c *RecordContext) TakedownRecord()

type RecordMeta 

type RecordMeta struct {
	DID        syntax.DID
	Collection syntax.NSID
	RecordKey  syntax.RecordKey
	CID        *syntax.CID
}

Immutable

type RecordOp 

type RecordOp struct {
	// Indicates type of record mutation: create, update, or delete.
	// The term "action" is copied from com.atproto.sync.subscribeRepos#repoOp
	Action     string
	DID        syntax.DID
	Collection syntax.NSID
	RecordKey  syntax.RecordKey
	CID        *syntax.CID
	RecordCBOR []byte
}

Immutable

func (*RecordOp) ATURI 

func (op *RecordOp) ATURI() syntax.ATURI

func (*RecordOp) Validate 

func (op *RecordOp) Validate() error

Checks that op has expected fields, based on the action type

type RecordRuleFunc 

type RecordRuleFunc = func(c *RecordContext) error

type RuleSet 

type RuleSet struct {
	PostRules         []PostRuleFunc
	ProfileRules      []ProfileRuleFunc
	RecordRules       []RecordRuleFunc
	RecordDeleteRules []RecordRuleFunc
	IdentityRules     []IdentityRuleFunc
	AccountRules      []AccountRuleFunc
	BlobRules         []BlobRuleFunc
	NotificationRules []NotificationRuleFunc
	OzoneEventRules   []OzoneEventRuleFunc
}

Holds configuration of which rules of various types should be run, and helps dispatch events to those rules.

func (*RuleSet) CallAccountRules 

func (r *RuleSet) CallAccountRules(c *AccountContext) error

Executes rules for account update events.

func (*RuleSet) CallIdentityRules 

func (r *RuleSet) CallIdentityRules(c *AccountContext) error

Executes rules for identity update events.

func (*RuleSet) CallNotificationRules 

func (r *RuleSet) CallNotificationRules(c *NotificationContext) error

func (*RuleSet) CallOzoneEventRules 

func (r *RuleSet) CallOzoneEventRules(c *OzoneEventContext) error

func (*RuleSet) CallRecordDeleteRules 

func (r *RuleSet) CallRecordDeleteRules(c *RecordContext) error

NOTE: this will probably be removed and merged in to `CallRecordRules`

func (*RuleSet) CallRecordRules 

func (r *RuleSet) CallRecordRules(c *RecordContext) error

Executes all the various record-related rules. Only dispatches execution, does no other de-dupe or pre/post processing.

type SlackNotifier 

type SlackNotifier struct {
	SlackWebhookURL string
}

func (*SlackNotifier) SendAccount 

func (n *SlackNotifier) SendAccount(ctx context.Context, service string, c *AccountContext) error

func (*SlackNotifier) SendRecord 

func (n *SlackNotifier) SendRecord(ctx context.Context, service string, c *RecordContext) error

type SlackWebhookBody 

type SlackWebhookBody struct {
	Text string `json:"text"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.