Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MonitorWorkers ¶
func MonitorWorkers(ctx context.Context, node *masa.OracleNode)
MonitorWorkers monitors worker data by subscribing to the completed work topic, computing a CID for each received data, and writing the data to the database.
Parameters:
- ctx: The context for the monitoring operation.
- node: A pointer to the OracleNode instance.
The function uses a ticker to periodically log a debug message every 60 seconds. It subscribes to the completed work topic using the PubSubManager and handles the received data. For each received data, it computes a CID using the computeCid function, logs the CID, marshals the data to JSON, and writes it to the database using the WriteData function. The monitoring continues until the context is done.
func NewWorker ¶
NewWorker creates a new instance of the Worker actor. It implements the actor.Receiver interface, allowing it to receive and handle messages.
Returns:
- An instance of the Worker struct that implements the actor.Receiver interface.
func SendWork ¶
func SendWork(node *masa.OracleNode, data []byte)
SendWork is responsible for handling work messages and processing them based on the request type. It supports the following request types: - "web": Scrapes web data from the specified URL with the given depth. - "twitter": Scrapes tweets based on the provided query and count.
The Worker actor receives messages through its Receive method, which is called by the actor system when a message is sent to the actor. It handles the following message types:
- *messages.Connect: Indicates that a client has connected to the worker. The client's sender information is added to the clients set.
- *actor.Started: Indicates that the actor has started. It prints a debug message.
- *messages.Work: Contains the work data to be processed. The work data is parsed based on the request type, and the corresponding scraping function is called. The scraped data is then sent to the workerStatusCh channel for further processing.
The Worker actor is responsible for the NewWorker function, which returns an actor.Producer that can be used to spawn new instances of the Worker actor.
Source Files
Directories