timeline

Documentation

Index

• type FilterFunction
• type GrabFunction
• type Manager
  • func NewManager(grabFunction GrabFunction, filterFunction FilterFunction, ...) Manager
• type Preparable
• type PrepareFunction
• type SkipInsertFunction
• type Timeline
  • func NewTimeline(ctx context.Context, timelineID string, grabFunction GrabFunction, ...) Timeline
• type Timelineable
• type Timelines

Types

type FilterFunction

type FilterFunction func(ctx context.Context, timelineID string, item Timelineable) (shouldIndex bool, err error)

FilterFunction is used by a Timeline to filter whether or not a grabbed item should be indexed.

type GrabFunction

type GrabFunction func(ctx context.Context, timelineID string, maxID string, sinceID string, minID string, limit int) (items []Timelineable, stop bool, err error)

GrabFunction is used by a Timeline to grab more items to index.

It should be provided to NewTimeline when the caller is creating a timeline (of statuses, notifications, etc).

• timelineID: ID of the timeline.
• maxID: the maximum item ID desired.
• sinceID: the minimum item ID desired.
• minID: see sinceID
• limit: the maximum amount of items to be returned

If an error is returned, the timeline will stop processing whatever request called GrabFunction, and return the error. If no error is returned, but stop = true, this indicates to the caller of GrabFunction that there are no more items to return, and processing should continue with the items already grabbed.

type Manager

type Manager interface {
	// IngestOne takes one timelineable and indexes it into the given timeline, and then immediately prepares it for serving.
	// This is useful in cases where we know the item will need to be shown at the top of a user's timeline immediately (eg., a new status is created).
	//
	// It should already be established before calling this function that the item actually belongs in the timeline!
	//
	// The returned bool indicates whether the item was actually put in the timeline. This could be false in cases where
	// a status is a boost, but a boost of the original status or the status itself already exists recently in the timeline.
	IngestOne(ctx context.Context, timelineID string, item Timelineable) (bool, error)

	// GetTimeline returns limit n amount of prepared entries from the given timeline, in descending chronological order.
	GetTimeline(ctx context.Context, timelineID string, maxID string, sinceID string, minID string, limit int, local bool) ([]Preparable, error)

	// GetIndexedLength returns the amount of items that have been indexed for the given account ID.
	GetIndexedLength(ctx context.Context, timelineID string) int

	// GetOldestIndexedID returns the id ID for the oldest item that we have indexed for the given timeline.
	// Will be an empty string if nothing is (yet) indexed.
	GetOldestIndexedID(ctx context.Context, timelineID string) string

	// Remove removes one item from the given timeline.
	Remove(ctx context.Context, timelineID string, itemID string) (int, error)

	// RemoveTimeline completely removes one timeline.
	RemoveTimeline(ctx context.Context, timelineID string) error

	// WipeItemFromAllTimelines removes one item from the index and prepared items of all timelines
	WipeItemFromAllTimelines(ctx context.Context, itemID string) error

	// WipeStatusesFromAccountID removes all items by the given accountID from the given timeline.
	WipeItemsFromAccountID(ctx context.Context, timelineID string, accountID string) error

	// UnprepareItem unprepares/uncaches the prepared version fo the given itemID from the given timelineID.
	// Use this for cache invalidation when the prepared representation of an item has changed.
	UnprepareItem(ctx context.Context, timelineID string, itemID string) error

	// UnprepareItemFromAllTimelines unprepares/uncaches the prepared version of the given itemID from all timelines.
	// Use this for cache invalidation when the prepared representation of an item has changed.
	UnprepareItemFromAllTimelines(ctx context.Context, itemID string) error

	// Prune manually triggers a prune operation for the given timelineID.
	Prune(ctx context.Context, timelineID string, desiredPreparedItemsLength int, desiredIndexedItemsLength int) (int, error)

	// Start starts hourly cleanup jobs for this timeline manager.
	Start() error

	// Stop stops the timeline manager (currently a stub, doesn't do anything).
	Stop() error
}

Manager abstracts functions for creating multiple timelines, and adding, removing, and fetching entries from those timelines.

By the time a timelineable hits the manager interface, it should already have been filtered and it should be established that the item indeed belongs in the given timeline.

The manager makes a distinction between *indexed* items and *prepared* items.

Indexed items consist of just that item's ID (in the database) and the time it was created. An indexed item takes up very little memory, so it's not a huge priority to keep trimming the indexed items list.

Prepared items consist of the item's database ID, the time it was created, AND the apimodel representation of that item, for quick serialization. Prepared items of course take up more memory than indexed items, so they should be regularly pruned if they're not being actively served.

func NewManager

func NewManager(grabFunction GrabFunction, filterFunction FilterFunction, prepareFunction PrepareFunction, skipInsertFunction SkipInsertFunction) Manager

NewManager returns a new timeline manager.

type Preparable

type Preparable interface {
	GetID() string
	GetAccountID() string
	GetBoostOfID() string
	GetBoostOfAccountID() string
}

Preparable represents any item that can be prepared in a timeline.

type PrepareFunction

type PrepareFunction func(ctx context.Context, timelineID string, itemID string) (Preparable, error)

PrepareFunction converts a Timelineable into a Preparable.

For example, this might result in the converstion of a *gtsmodel.Status with the given itemID into a serializable *apimodel.Status.

type SkipInsertFunction

type SkipInsertFunction func(ctx context.Context,
	newItemID string,
	newItemAccountID string,
	newItemBoostOfID string,
	newItemBoostOfAccountID string,
	nextItemID string,
	nextItemAccountID string,
	nextItemBoostOfID string,
	nextItemBoostOfAccountID string,
	depth int) (bool, error)

SkipInsertFunction indicates whether a new item about to be inserted in the prepared list should be skipped, based on the item itself, the next item in the timeline, and the depth at which nextItem has been found in the list.

This will be called for every item found while iterating through a timeline, so callers should be very careful not to do anything expensive here.

type Timeline

type Timeline interface {

	// Get returns an amount of prepared items with the given parameters.
	// If prepareNext is true, then the next predicted query will be prepared already in a goroutine,
	// to make the next call to Get faster.
	Get(ctx context.Context, amount int, maxID string, sinceID string, minID string, prepareNext bool) ([]Preparable, error)

	// IndexAndPrepareOne puts a item into the timeline at the appropriate place
	// according to its id, and then immediately prepares it.
	//
	// The returned bool indicates whether or not the item was actually inserted
	// into the timeline. This will be false if the item is a boost and the original
	// item, or a boost of it, already exists recently in the timeline.
	IndexAndPrepareOne(ctx context.Context, itemID string, boostOfID string, accountID string, boostOfAccountID string) (bool, error)

	// Unprepare clears the prepared version of the given item (and any boosts
	// thereof) from the timeline, but leaves the indexed version in place.
	//
	// This is useful for cache invalidation when the prepared version of the
	// item has changed for some reason (edits, updates, etc), but the item does
	// not need to be removed: it will be prepared again next time Get is called.
	Unprepare(ctx context.Context, itemID string) error

	// TimelineID returns the id of this timeline.
	TimelineID() string

	// Len returns the length of the item index at this point in time.
	Len() int

	// OldestIndexedItemID returns the id of the rearmost (ie., the oldest) indexed item.
	// If there's no oldest item, an empty string will be returned so make sure to check for this.
	OldestIndexedItemID() string

	// LastGot returns the time that Get was last called.
	LastGot() time.Time

	// Prune prunes prepared and indexed items in this timeline to the desired lengths.
	// This will be a no-op if the lengths are already < the desired values.
	//
	// The returned int indicates the amount of entries that were removed or unprepared.
	Prune(desiredPreparedItemsLength int, desiredIndexedItemsLength int) int

	// Remove removes an item with the given ID.
	//
	// If a item has multiple entries in a timeline, they will all be removed.
	//
	// The returned int indicates the amount of entries that were removed.
	Remove(ctx context.Context, itemID string) (int, error)

	// RemoveAllByOrBoosting removes all items created by or boosting the given accountID.
	//
	// The returned int indicates the amount of entries that were removed.
	RemoveAllByOrBoosting(ctx context.Context, accountID string) (int, error)
}

Timeline represents a timeline for one account, and contains indexed and prepared items.

func NewTimeline

func NewTimeline(
	ctx context.Context,
	timelineID string,
	grabFunction GrabFunction,
	filterFunction FilterFunction,
	prepareFunction PrepareFunction,
	skipInsertFunction SkipInsertFunction,
) Timeline

NewTimeline returns a new Timeline with the given ID, using the given functions.

type Timelineable

type Timelineable interface {
	GetID() string
	GetAccountID() string
	GetBoostOfID() string
	GetBoostOfAccountID() string
}

Timelineable represents any item that can be indexed in a timeline.

type Timelines

type Timelines struct {
	// Home provides access to account home timelines.
	Home Manager

	// List provides access to list timelines.
	List Manager
	// contains filtered or unexported fields
}