Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Connection ¶
type Connection interface {
// Publish pushes new messages to the queue.
Publish(string, string, uint) error
// Subscribe pulls messages from the queue and processes them using the processor function.
Subscribe(Processor)
}
Connection is an interface that defines how messages are published are received from a queue.
func NewNatsConn ¶
func NewNatsConn(d db.Connection, conn *nats.EncodedConn) Connection
NewNatsConn initializes the connection to Gnatsd. It assumes that the client has been initialized by the application context.
func NewPoolConn ¶
func NewPoolConn(d db.Connection) Connection
NewPoolConn initializes the channel connection
type Message ¶
type Message struct {
Depth uint // depth level where the url was found
JobUUID string // unique identifiler for the job that trigerred this message
URL string // url to crawl
}
Message is the structure that the crawler sends and receives in the queue.
func NewMessage ¶
NewMessage creates new messages to crawl an url.
type NatsConn ¶
type NatsConn struct {
// contains filtered or unexported fields
}
NatsConn implements the queue.Connection interface using Gnatsd as a queue.
type PoolConn ¶
type PoolConn struct {
// contains filtered or unexported fields
}
PoolConn implements queue.Connection using a channel as a backend. This interface is only suitable for testing. It offers no guarantees about the elements pushed and pulled from the queue.
type Processor ¶
type Processor func(Connection, db.Connection, *Message)
Processor defines a function interface to process messages.
Source Files