Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type GraphAPI ¶
type GraphAPI interface {
Links(fromID, toID uuid.UUID, accessedBefore time.Time) (graph.LinkIterator, error)
Edges(fromID, toID uuid.UUID, updatedBefore time.Time) (graph.EdgeIterator, error)
}
GraphAPI defines as set of API methods for fetching the links and edges from the link graph.
type IndexAPI ¶
IndexAPI defines a set of API methods for updating PageRank scores for indexed documents.
type MasterConfig ¶
type MasterConfig struct {
// The address to listen for incoming worker connections.
ListenAddress string
// The minimum required number of connected workers for starting a new
// PageRank pass. If not specified, a new pass will start when at least
// one worker has connected.
MinWorkers int
// The timeout for the required number of workers to connect before
// aborting a new pass attempt. If not specified, the master will wait
// indefinitely.
WorkerAcquireTimeout time.Duration
// A clock instance for generating time-related events. If not specified,
// the default wall-clock will be used instead.
Clock clock.Clock
// The time between subsequent pagerank updates.
UpdateInterval time.Duration
// The logger to use. If not defined an output-discarding logger will
// be used instead.
Logger *logrus.Entry
}
MasterConfig encapsulates the settings for configuring the master node for the PageRank calculator service.
type MasterNode ¶
type MasterNode struct {
// contains filtered or unexported fields
}
MasterNode implements a master node for calculating PageRank scores in a distributed fashion.
func NewMasterNode ¶
func NewMasterNode(cfg MasterConfig) (*MasterNode, error)
NewMasterNode creates a new master node for the PageRank calculator service.
func (*MasterNode) AbortJob ¶
func (n *MasterNode) AbortJob(_ job.Details)
AbortJob implements job.Runner.
func (*MasterNode) CompleteJob ¶
func (n *MasterNode) CompleteJob(_ job.Details) error
CompleteJob implements job.Runner.
func (*MasterNode) Run ¶
func (n *MasterNode) Run(ctx context.Context) error
Run implements the main loop of the master node for the distributed PageRank calculator. It periodically wakes up and orchestrates the execution of a new PageRank update pass across all connected workers.
Run blocks until the provided context expires.
func (*MasterNode) StartJob ¶
func (n *MasterNode) StartJob(_ job.Details, execFactory bspgraph.ExecutorFactory) (*bspgraph.Executor, error)
StartJob implements job.Runner. It initializes the underlying bspgraph.Graph instance and invokes the provided ExecutorFactory to create an executor for the graph supersteps.
type WorkerConfig ¶
type WorkerConfig struct {
// The master node endpoint.
MasterEndpoint string
// The timeout for establishing a connection to the master node.
MasterDialTimeout time.Duration
// An API for interating links and edges from the link graph.
GraphAPI GraphAPI
// An API for updating the PageRank score for indexed documents.
IndexAPI IndexAPI
// The number of workers to spin up for computing PageRank scores. If
// not specified, a default value of 1 will be used instead.
ComputeWorkers int
// The logger to use. If not defined an output-discarding logger will
// be used instead.
Logger *logrus.Entry
}
WorkerConfig encapsulates the settings for configuring a worker node for the PageRank calculator service.
type WorkerNode ¶
type WorkerNode struct {
// contains filtered or unexported fields
}
WorkerNode implements a master node for calculating PageRank scores in a distributed fashion.
func NewWorkerNode ¶
func NewWorkerNode(cfg WorkerConfig) (*WorkerNode, error)
NewWorkerNode creates a new worker node for the PageRank calculator service.
func (*WorkerNode) AbortJob ¶
func (n *WorkerNode) AbortJob(_ job.Details)
AbortJob implements job.Runner.
func (*WorkerNode) CompleteJob ¶
func (n *WorkerNode) CompleteJob(_ job.Details) error
CompleteJob implements job.Runner. It persists the locally computed PageRank scores after a successful execution of a distributed PageRank run.
func (*WorkerNode) Run ¶
func (n *WorkerNode) Run(ctx context.Context) error
Run implements the main loop of a worker that executes the PageRank algorithm on a subset of the link graph. The worker waits for the master node to publish a new PageRank job and then begins the algorithm execution constrained to the assigned partition range.
Run blocks until the provided context expires.
func (*WorkerNode) StartJob ¶
func (n *WorkerNode) StartJob(jobDetails job.Details, execFactory bspgraph.ExecutorFactory) (*bspgraph.Executor, error)
StartJob implements job.Runner. It initializes the underlying bspgraph.Graph instance and invokes the provided ExecutorFactory to create an executor for the graph supersteps.
Source Files