filters

package module
v0.0.0-...-0aad921 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 23, 2024 License: MIT Imports: 15 Imported by: 9

Documentation

Overview

Package filters contains helper functions to be used by the storage implementations for filtering out elements at load time.

Index

Examples

Constants

View Source 
const (
	// ActorsType is a constant that represents the URL path for the local actors collection.
	// It is used as the parent for all To IDs
	ActorsType = vocab.CollectionPath("actors")
	// ActivitiesType is a constant that represents the URL path for the local activities collection
	// It is used as the parent for all Activity IDs
	ActivitiesType = vocab.CollectionPath("activities")
	// ObjectsType is a constant that represents the URL path for the local objects collection
	// It is used as the parent for all non To, non Activity Object IDs
	ObjectsType = vocab.CollectionPath("objects")

	// BlockedType is an internally used collection, to store a list of actors the actor has blocked
	BlockedType = vocab.CollectionPath("blocked")

	// IgnoredType is an internally used collection, to store a list of actors the actor has ignored
	IgnoredType = vocab.CollectionPath("ignored")
)
View Source 
const (
	ByID                = index.ByID
	ByType              = index.ByType
	ByName              = index.ByName
	ByPreferredUsername = index.ByPreferredUsername
	BySummary           = index.BySummary
	ByContent           = index.ByContent
	ByActor             = index.ByActor
	ByObject            = index.ByObject
	ByRecipients        = index.ByRecipients
	ByAttributedTo      = index.ByAttributedTo
	ByInReplyTo         = index.ByInReplyTo
)
View Source 
const MaxItems int = 100

Variables

View Source 
var (
	HiddenCollections = vocab.CollectionPaths{
		BlockedType,
		IgnoredType,
	}

	FedBOXCollections = vocab.CollectionPaths{
		ActivitiesType,
		ActorsType,
		ObjectsType,
		BlockedType,
		IgnoredType,
	}
)

TODO(marius): here we need a better separation between the collections which are exposed in the HTTP API 

(activities,actors,objects) and the ones that are internal (blocked,ignored)
View Source 
var AbsentHash = []Hash{Hash(absentValue)}
View Source 
var AbsentIRI = CompStr{Str: absentValue, Operator: "="}
View Source 
var AbsentIRIs = CompStrs{AbsentIRI}
View Source 
var ContentEmpty = contentCheck("", naturalLanguageEmpty)

ContentEmpty checks an vocab.Object's Content, *and*, in the case of an vocab.Actor also the PreferredUsername to be empty. If *all* of the values are empty, the function returns true.

Please note that the logic of this check is different from ContentIs and ContentLike.

View Source 
var ErrNotFound = func(s string) error {
	return errors.Errorf("%s not found", s)
}
View Source 
var NameEmpty = nameCheck("", naturalLanguageEmpty)

NameEmpty checks an vocab.Object's Name, *and*, in the case of an vocab.Actor also its PreferredUsername to be empty. If *all* of the values are empty, the function returns true.

Please note that the logic of this check is different from NameIs and NameLike.

View Source 
var NilAttributedTo = attributedToNil{}
View Source 
var NilContext = contextNil{}
View Source 
var NilID = idNil{}

NilID checks if the vocab.Object's ID property matches any of the two magic values that denote an empty value: vocab.NilID = "-", or vocab.EmptyID = ""

View Source 
var NilIRI = iriNil{}

NilIRI checks if the activitypub.Item's IRI that matches any of the two magic values that denote an empty value: activitypub.NilID = "-", or activitypub.EmptyID = ""

View Source 
var NilInReplyTo = inReplyToNil{}
View Source 
var NotNilID = Not(NilID)

NotNilID checks if the vocab.Object's ID property is not nil

View Source 
var NotNilIRI = Not(iriNil{})

NotNilIRI checks if the activitypub.Object's URL property matches any of the two magic values that denote an empty value: activitypub.NilID = "-", or activitypub.EmptyID = ""

View Source 
var SummaryEmpty = summaryCheck("", naturalLanguageEmpty)

SummaryEmpty checks an vocab.Object's Summary, *and*, in the case of an vocab.Actor also the PreferredUsername to be empty. If *all* of the values are empty, the function returns true.

Please note that the logic of this check is different from SummaryIs and SummaryLike.

Functions

func CacheKey 

func CacheKey(f *Filters) vocab.IRI

CacheKey generates a unique vocab.IRI hash based on its authenticated user and other parameters

func Counted 

func Counted(fns ...Check) int

func CursorFromItem 

func CursorFromItem(it vocab.Item, filters ...Check) (vocab.Item, vocab.Item, vocab.Item)

func FilterIt 

func FilterIt(it vocab.Item, f Filterable) (vocab.Item, error)

FilterIt

func FirstPage 

func FirstPage() pagValues

FirstPage returns the default url.Values for getting to the first page of a collection.

func GetLimit 

func GetLimit(f ...Check) int

func GetWhereClauses 

func GetWhereClauses(f ...Check) ([]string, []any)

func IRIf 

func IRIf(f Filters, iri string) string

func IsItemIRI 

func IsItemIRI(f vocab.LinkOrIRI) bool

func LoadCollectionFilters 

func LoadCollectionFilters(f *Filters, auth vocab.Actor) error

LoadCollectionFilters uses specific logic for adding elements to the filters when loading collections from the database.

func LoadItemFilters 

func LoadItemFilters(f *Filters, auth vocab.Actor) error

LoadItemFilters uses specific logic for adding elements to the filters when loading single items from the database.

func MaxCount 

func MaxCount(fns ...Check) int

func NextPage 

func NextPage(it vocab.Item) pagValues

func PaginateCollection 

func PaginateCollection(it vocab.Item, filters ...Check) vocab.Item

PaginateCollection is a function that populates the received collection

func PaginatorValues 

func PaginatorValues(q url.Values) pagValues

func PrevPage 

func PrevPage(it vocab.Item) pagValues

func SearchIndex 

func SearchIndex(i *index.Index, ff ...Check) ([]vocab.IRI, error)

SearchIndex does a fast index search for the received filters.

Example 
activities := []vocab.LinkOrIRI{
	// NOTE(marius): if the object for the Create activity is not indexed independently, like we have here,
	// it will not be findable in the index, and the composite search by activity type: Create,
	// and object ID:https://federated.local/objects/1 will fail.
	// In a previous version of this example this object was embedded in the activity directly,
	// but we decided that the logic to add embedded objects to the index, would be too complicated,
	// so this is the compromise.
	&vocab.Object{
		ID:   "https://federated.local/objects/1",
		Type: vocab.PageType,
		Name: vocab.NaturalLanguageValues{{Ref: "-", Value: vocab.Content("Link to example.com")}},
		URL:  vocab.IRI("https://example.com"),
	},
	&vocab.Activity{
		ID:     "https://federated.local/1",
		Type:   vocab.CreateType,
		To:     vocab.ItemCollection{vocab.IRI("https://federated.local/~jdoe")},
		Actor:  vocab.IRI("https://federated.local/~jdoe"),
		Object: vocab.IRI("https://federated.local/objects/1"),
	},
	&vocab.Activity{
		ID:     "https://federated.local/2",
		Type:   vocab.LikeType,
		To:     vocab.ItemCollection{vocab.IRI("https://federated.local/~jdoe")},
		Actor:  vocab.IRI("https://federated.local/~jdoe"),
		Object: vocab.IRI("https://federated.local/objects/1"),
	},
	&vocab.Activity{
		ID:     "https://federated.local/3",
		Type:   vocab.DislikeType,
		To:     vocab.ItemCollection{vocab.IRI("https://federated.local/~jdoe")},
		Actor:  vocab.IRI("https://federated.local/~jdoe"),
		Object: vocab.IRI("https://federated.local/objects/1"),
	},
	&vocab.Activity{
		ID:     "https://federated.local/4",
		Type:   vocab.FlagType,
		To:     vocab.ItemCollection{vocab.IRI("https://federated.local/~jdoe")},
		Actor:  vocab.IRI("https://federated.local/~jdoe"),
		Object: vocab.IRI("https://federated.local/objects/1"),
	},
}

in := index.Full()
// Add the activities to the index
in.Add(activities...)

findCreate := Checks{
	HasType(vocab.CreateType),
	Object(SameID("https://federated.local/objects/1")),
}
iris, err := SearchIndex(in, findCreate...)
fmt.Printf("Find Create:\n")
fmt.Printf("Error: %v\n", err)
fmt.Printf("IRIs: %#v\n", iris)

findBlock := Checks{
	HasType(vocab.FlagType),
	AttributedToLike("https://federated.local/~jdoe"),
}
iris, err = SearchIndex(in, findBlock...)
fmt.Printf("Find Flag:\n")
fmt.Printf("Error: %v\n", err)
fmt.Printf("IRIs: %#v\n", iris)

Output:

Find Create:
Error: <nil>
IRIs: []activitypub.IRI{https://federated.local/1}
Find Flag:
Error: <nil>
IRIs: []activitypub.IRI{https://federated.local/4}

func ToValues 

func ToValues(ff ...Check) url.Values

func ValidActivityCollection 

func ValidActivityCollection(typ vocab.CollectionPath) bool

ValidActivityCollection shows if the current ActivityPub end-point type is a valid collection for handling Activities

func ValidCollection 

func ValidCollection(typ vocab.CollectionPath) bool

ValidCollection shows if the current ActivityPub end-point type is a valid collection

func ValidCollectionCount 

func ValidCollectionCount(col vocab.ItemCollection, f Filterable) bool

func ValidObjectCollection 

func ValidObjectCollection(typ vocab.CollectionPath) bool

ValidObjectCollection shows if the current ActivityPub end-point type is a valid collection for handling Objects

func VocabularyTypesFilter 

func VocabularyTypesFilter(types ...string) vocab.ActivityVocabularyTypes

VocabularyTypesFilter converts the received list of strings to a list of ActivityVocabularyTypes that can be used with the HasType filter function. The individual strings are not validated against the known vocabulary types.

Types

type Check 

type Check interface {
	Match(vocab.Item) bool
}

Check represents an interface for a filter that can be applied on a vocab.Item and it returns true if it matches and false if it does not.

func Actor 

func Actor(fns ...Check) Check

func After 

func After(fns ...Check) Check

After checks the activitypub.Item against a specified "fn" filter function. This should be used when iterating over a collection, and it resolves to true after fn returns true and to false check.

Due to relying on the static check function return value the After is not reentrant.

func All 

func All(fns ...Check) Check

All aggregates a list of individual Check functions into a single Check which resolves true if all individual members resolve as true, and false otherwise. It is equivalent to a sequence of AND operators.

func Any 

func Any(fns ...Check) Check

Any aggregates a list of individual Check functions into a single Check which resolves to false if all the individual members resolve as false, and true if any of them resolves as true. It is equivalent to a sequence of OR operators.

func AttributedToLike 

func AttributedToLike(frag string) Check

AttributedToLike creates a filter that checks the vocab.IRI against the attributedTo property of the item it gets applied on using a similarity match.

func Authorized 

func Authorized(iri vocab.IRI) Check

Authorized creates a filter that checks the vocab.IRI against the recipients list of the item it gets applied on. The ActivityStreams Public Namespace IRI gets special treatment, because servers use it to signify that the audience of an object is public.

func Before 

func Before(fn ...Check) Check

Before checks the activitypub.Item against a specified "fn" filter function. This should be used when iterating over a collection, and it resolves to true check the fn has returned true and to false after.

Due to relying on the static check function return value the function is not reentrant.

func ContentIs 

func ContentIs(cont string) Check

ContentIs checks an vocab.Object's Content against the "cont" value. If any of the Language Ref map values match the value, the function returns true.

func ContentLike 

func ContentLike(cont string) Check

ContentLike checks an vocab.Object's Content property against the "cont" value. If any of the Language Ref map values contains the value as a substring, the function returns true.

func ContextLike 

func ContextLike(frag string) Check

func HasType 

func HasType(ty ...vocab.ActivityVocabularyType) Check

HasType checks an activitypub.Object's Type property against a series of values. If any of the ty values matches, the function returns true.

func IDLike 

func IDLike(frag string) Check

IDLike

func IRILike 

func IRILike(frag string) Check

IRILike

func InReplyToLike 

func InReplyToLike(frag string) Check

func MaxCountCheck 

func MaxCountCheck(fns ...Check) Check

func NameIs 

func NameIs(name string) Check

NameIs checks an vocab.Object's Name, or, in the case of an vocab.Actor also the PreferredUsername against the "name" value. If any of the Language Ref map values match the value, the function returns true.

func NameLike 

func NameLike(name string) Check

NameLike checks an vocab.Object's Name, or, in the case of an vocab.Actor // also the PreferredUsername against the "name" value. If any of the Language Ref map values contains the value as a substring, the function returns true.

func Not 

func Not(fn Check) Check

Not negates the result of a Check function. It is equivalent to a unary NOT operator.

func Object 

func Object(fns ...Check) Check

func Recipients 

func Recipients(iri vocab.IRI) Check

Recipients creates a filter that checks the vocab.IRI against the recipients list of the item it gets applied on.

func SameAttributedTo 

func SameAttributedTo(iri vocab.IRI) Check

SameAttributedTo creates a filter that checks the vocab.IRI against the attributedTo property of the item it gets applied on.

func SameContext 

func SameContext(iri vocab.IRI) Check

func SameID 

func SameID(i vocab.IRI) Check

SameID checks a vocab.Object's ID property against the received iri.

func SameIRI 

func SameIRI(iri vocab.IRI) Check

SameIRI checks an activitypub.Object's IRI

func SameInReplyTo 

func SameInReplyTo(iri vocab.IRI) Check

SameInReplyTo checks an activitypub.Object's InReplyTo

func SameURL 

func SameURL(iri vocab.IRI) Check

SameURL checks an activitypub.Object's IRI

func SummaryIs 

func SummaryIs(sum string) Check

SummaryIs checks an vocab.Object's Summary against the "sum" value. If any of the Language Ref map values match the value, the function returns true.

func SummaryLike 

func SummaryLike(sum string) Check

SummaryLike checks an vocab.Object's Summary property against the "sum" value. If any of the Language Ref map values contains the value as a substring, the function returns true.

func Tag 

func Tag(fns ...Check) Check

func Target 

func Target(fns ...Check) Check

func URLLike 

func URLLike(frag string) Check

func WithMaxCount 

func WithMaxCount(max int) Check

WithMaxCount is used to limit a collection's items count to the 'max' value. It can be used from slicing from the first element of the collection to max. Due to relying on the static max value the function is not reentrant.

type Checks 

type Checks []Check

Checks aggregates a list of Check functions to be tested on a vocab.Item.

func ActivityChecks 

func ActivityChecks(fns ...Check) Checks

func ActorChecks 

func ActorChecks(fns ...Check) Checks

func AfterChecks 

func AfterChecks(fns ...Check) Checks

func AuthorizedChecks 

func AuthorizedChecks(fns ...Check) Checks

func BeforeChecks 

func BeforeChecks(fns ...Check) Checks

func CursorChecks 

func CursorChecks(fns ...Check) Checks

func FilterChecks 

func FilterChecks(fns ...Check) Checks

func FromIRI 

func FromIRI(i vocab.IRI) (Checks, error)

func FromURL 

func FromURL(u url.URL) Checks

func FromValues 

func FromValues(q url.Values) Checks

func IntransitiveActivityChecks 

func IntransitiveActivityChecks(fns ...Check) Checks

func ItemChecks 

func ItemChecks(fns ...Check) Checks

func ObjectChecks 

func ObjectChecks(fns ...Check) Checks

func RecipientsChecks 

func RecipientsChecks(fns ...Check) Checks

func TagChecks 

func TagChecks(fns ...Check) Checks

func TargetChecks 

func TargetChecks(fns ...Check) Checks

func TypeChecks 

func TypeChecks(fns ...Check) Checks

func (Checks) Filter 

func (ff Checks) Filter(item vocab.Item) vocab.Item

func (Checks) IndexMatch 

func (ff Checks) IndexMatch(indexes map[index.Type]index.Indexable) *roaring64.Bitmap

func (Checks) Paginate 

func (ff Checks) Paginate(item vocab.Item) vocab.Item

func (Checks) Run 

func (ff Checks) Run(item vocab.Item) vocab.Item

type CollectionFilterer 

type CollectionFilterer interface {
	FilterCollection(col vocab.ItemCollection) (vocab.ItemCollection, int)
}

type CompStr 

type CompStr = qstring.ComparativeString

func StringDifferent 

func StringDifferent(s string) CompStr

func StringEquals 

func StringEquals(s string) CompStr

func StringLike 

func StringLike(s string) CompStr

type CompStrs 

type CompStrs []CompStr

func (CompStrs) Contains 

func (cs CompStrs) Contains(f CompStr) bool

type FilterFn 

type FilterFn func(f *Filters) error

func BaseIRI 

func BaseIRI(iri vocab.IRI, col vocab.CollectionPath) FilterFn

func IRI 

func IRI(i vocab.IRI) FilterFn

func ItemKey 

func ItemKey(keys ...string) FilterFn

func Name 

func Name(names ...string) FilterFn

func Type 

func Type(types ...vocab.ActivityVocabularyType) FilterFn

type Filterable 

type Filterable interface {
	GetLink() vocab.IRI
}

func FiltersOnActivityActor 

func FiltersOnActivityActor(f Filterable) (bool, Filterable)

func FiltersOnActivityObject 

func FiltersOnActivityObject(f Filterable) (bool, Filterable)

func FiltersOnActivityTarget 

func FiltersOnActivityTarget(f Filterable) (bool, Filterable)

type FilterableActivity 

type FilterableActivity interface {
	FilterableObject
	Actors() vocab.IRIs
	Objects() vocab.IRIs
	Targets() vocab.IRIs
}

FilterableActivity can filter activities

type FilterableCollection 

type FilterableCollection interface {
	FilterableObject
	TotalItemsGt() uint
	TotalItemsLt() uint
	TotalItemsEq() uint
	TotalItemsGtE() uint
	TotalItemsLtE() uint
	Contains() vocab.IRIs
}

FilterableCollection can filter collections

type FilterableItems 

type FilterableItems interface {
	Filterable
	Types() vocab.ActivityVocabularyTypes
	IRIs() vocab.IRIs
}

type FilterableObject 

type FilterableObject interface {
	FilterableItems
	AttributedTo() vocab.IRIs
	InReplyTo() vocab.IRIs
	MediaTypes() []vocab.MimeType
	Names() []string
	Content() []string
	//PublishedBefore() time.Time
	//PublishedAfter() time.Time
	URLs() vocab.IRIs
	// Audience returns the list of IRIs to check against full Audience targeting for the object
	// It should include all relevant fields: To, CC, BTo, BCC, and Audience
	// ---
	// An element of the Audience is used to get its Inbox end-point and then disseminate the current Activity
	// to it.
	Audience() vocab.IRIs
	// Context returns the list of IRIs to check against an Object's Context property.
	Context() vocab.IRIs
	// Generator returns the list of IRIs to check against an Object's Generator property.
	Generator() vocab.IRIs
}

FilterableObject can filter objects

type Filters deprecated 

type Filters struct {
	BaseURL       vocab.IRI            `qstring:"-"`
	Name          CompStrs             `qstring:"name,omitempty"`
	Cont          CompStrs             `qstring:"content,omitempty"`
	Authenticated *vocab.Actor         `qstring:"-"`
	To            *vocab.Actor         `qstring:"-"`
	Author        *vocab.Actor         `qstring:"-"`
	Parent        *vocab.Actor         `qstring:"-"`
	IRI           vocab.IRI            `qstring:"-"`
	Collection    vocab.CollectionPath `qstring:"-"`
	URL           CompStrs             `qstring:"url,omitempty"`
	MedTypes      []vocab.MimeType     `qstring:"mediaType,omitempty"`
	Aud           CompStrs             `qstring:"recipients,omitempty"`
	Gen           CompStrs             `qstring:"generator,omitempty"`
	Key           []Hash               `qstring:"-"`
	ItemKey       CompStrs             `qstring:"iri,omitempty"`
	Type          CompStrs             `qstring:"type,omitempty"`
	AttrTo        CompStrs             `qstring:"attributedTo,omitempty"`
	InReplTo      CompStrs             `qstring:"inReplyTo,omitempty"`
	OP            CompStrs             `qstring:"context,omitempty"`
	FollowedBy    []Hash               `qstring:"followedBy,omitempty"` // todo(marius): not really used
	OlderThan     time.Time            `qstring:"olderThan,omitempty"`
	NewerThan     time.Time            `qstring:"newerThan,omitempty"`
	Prev          Hash                 `qstring:"before,omitempty"`
	Next          Hash                 `qstring:"after,omitempty"`
	Object        *Filters             `qstring:"object,omitempty"`
	Actor         *Filters             `qstring:"actor,omitempty"`
	Target        *Filters             `qstring:"target,omitempty"`
	Tag           *Filters             `qstring:"tag,omitempty"`
	CurPage       int                  `qstring:"page,omitempty"`
	MaxItems      int                  `qstring:"maxItems,omitempty"`
	Req           *http.Request        `qstring:"-"`
}

Filters

Deprecated: This is functionality cribbed from the original FedBOX filtering mechanism which we want to move away from in favour of using []Check TODO(marius) we can make some small changes so it's not necessary to export this struct

func FiltersFromIRI 

func FiltersFromIRI(i vocab.IRI) (*Filters, error)

func FiltersNew 

func FiltersNew(filters ...FilterFn) *Filters

func FromRequest 

func FromRequest(r *http.Request, baseUrl string) *Filters

FromRequest loads the filters we use for generating storage queries from the HTTP request

func (Filters) Actors 

func (f Filters) Actors() vocab.IRIs

func (Filters) After 

func (f Filters) After() Hash

After

func (Filters) AttributedTo 

func (f Filters) AttributedTo() CompStrs

func (Filters) Audience 

func (f Filters) Audience() CompStrs

Audience returns a filter for audience members. This is important for filtering out objects that don't have a public audience.

func (Filters) Before 

func (f Filters) Before() Hash

Before

func (Filters) Content 

func (f Filters) Content() CompStrs

func (Filters) Context 

func (f Filters) Context() CompStrs

Context returns a list of ActivityVocabularyTypes to filter against

func (Filters) Count 

func (f Filters) Count() int

Count

func (Filters) FilterCollection 

func (f Filters) FilterCollection(col vocab.ItemCollection) (vocab.ItemCollection, int)

func (Filters) Generator 

func (f Filters) Generator() CompStrs
func (f *Filters) GetLink() vocab.IRI

GetLink returns a list of IRIs to filter against

func (Filters) IRIs 

func (f Filters) IRIs() CompStrs

IRIs returns a list of IRIs to filter against

func (Filters) InReplyTo 

func (f Filters) InReplyTo() CompStrs

func (*Filters) IsItemIRI 

func (f *Filters) IsItemIRI() bool

func (*Filters) ItemsMatch 

func (f *Filters) ItemsMatch(col ...vocab.Item) bool

ItemsMatch

func (Filters) MediaTypes 

func (f Filters) MediaTypes() []vocab.MimeType

func (Filters) Names 

func (f Filters) Names() CompStrs

func (Filters) Objects 

func (f Filters) Objects() vocab.IRIs

func (Filters) Page 

func (f Filters) Page() int

Page

func (Filters) Tags 

func (f Filters) Tags() *Filters

Tags returns a list of filters to filter an object's tag collections against

func (Filters) Targets 

func (f Filters) Targets() vocab.IRIs

func (Filters) Types 

func (f Filters) Types() CompStrs

Types returns a list of ActivityVocabularyTypes to filter against

func (Filters) URLs 

func (f Filters) URLs() CompStrs

type Hash 

type Hash string

Hash

func (Hash) Matches 

func (h Hash) Matches(i vocab.IRI) bool

Matches

func (Hash) String 

func (h Hash) String() string

String returns the hash as a string

type ItemMatcher 

type ItemMatcher interface {
	ItemsMatch(it ...vocab.Item) bool
}

type KeysetPaginator 

type KeysetPaginator interface {
	Before() string
	After() string
	Count() int
}

KeysetPaginator

type OffsetPaginator 

type OffsetPaginator interface {
	Count() int
	Page() int
}

OffsetPaginator

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.