jobq

Documentation

Index

• Variables
• func ConvertJobToProtobuf(job RepairJob) *pb.RepairJob
• type Client
  • func Dial(addr net.Addr) (*Client, error)
  • func WrapConn(rawConn net.Conn) *Client
  • func (c *Client) AddPlacementQueue(ctx context.Context, placement storj.PlacementConstraint) error
  • func (c *Client) Clean(ctx context.Context, placement storj.PlacementConstraint, ...) (removedSegments int32, err error)
  • func (c *Client) Close() error
  • func (c *Client) DestroyPlacementQueue(ctx context.Context, placement storj.PlacementConstraint) error
  • func (c *Client) Inspect(ctx context.Context, placement storj.PlacementConstraint, streamID uuid.UUID, ...) (job RepairJob, err error)
  • func (c *Client) Len(ctx context.Context, placement storj.PlacementConstraint) (repairLen, retryLen int64, err error)
  • func (c *Client) Peek(ctx context.Context, placement storj.PlacementConstraint) (job RepairJob, err error)
  • func (c *Client) Pop(ctx context.Context, ...) (job RepairJob, err error)
  • func (c *Client) Push(ctx context.Context, job RepairJob) (wasNew bool, err error)
  • func (c *Client) PushBatch(ctx context.Context, jobs []RepairJob) (wasNew []bool, err error)
  • func (c *Client) Trim(ctx context.Context, placement storj.PlacementConstraint, ...) (removedSegments int32, err error)
  • func (c *Client) Truncate(ctx context.Context, placement storj.PlacementConstraint) error
• type RepairJob
  • func ConvertJobFromProtobuf(protoJob *pb.RepairJob) (RepairJob, error)
  • func (rj RepairJob) LastAttemptedAtTime() time.Time
• type SegmentIdentifier

Variables

var ErrJobNotFound = errors.New("job not found")

ErrJobNotFound is returned by the client when a particular job is not found.

var ErrQueueEmpty = errors.New("queue is empty")

ErrQueueEmpty is returned by the client when the queue is empty.

var RecordSize = unsafe.Sizeof(RepairJob{})

RecordSize is the size of a RepairJob record in bytes. It includes any padding that may be added by the compiler to align the record to a multiple of the word size for the target arch.

Functions

func ConvertJobToProtobuf

func ConvertJobToProtobuf(job RepairJob) *pb.RepairJob

ConvertJobToProtobuf converts a RepairJob record to a protobuf representation.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps a DRPCJobQueueClient.

func Dial

func Dial(addr net.Addr) (*Client, error)

Dial dials an address and creates a new client.

func WrapConn

func WrapConn(rawConn net.Conn) *Client

WrapConn wraps an existing connection in a client.

func (*Client) AddPlacementQueue

func (c *Client) AddPlacementQueue(ctx context.Context, placement storj.PlacementConstraint) error

AddPlacementQueue adds a new queue for the given placement.

func (*Client) Clean

func (c *Client) Clean(ctx context.Context, placement storj.PlacementConstraint, updatedBefore time.Time) (removedSegments int32, err error)

Clean removes all jobs with UpdatedAt time before the given cutoff.

func (*Client) Close

func (c *Client) Close() error

Close closes the underlying connection.

func (*Client) DestroyPlacementQueue

func (c *Client) DestroyPlacementQueue(ctx context.Context, placement storj.PlacementConstraint) error

DestroyPlacementQueue truncates and removes the queue for the given placement.

func (*Client) Inspect

func (c *Client) Inspect(ctx context.Context, placement storj.PlacementConstraint, streamID uuid.UUID, position uint64) (job RepairJob, err error)

Inspect finds a job in the queue by streamID and position and returns all of the job information. If the job is not found, it returns ErrJobNotFound.

func (*Client) Len

func (c *Client) Len(ctx context.Context, placement storj.PlacementConstraint) (repairLen, retryLen int64, err error)

Len returns the number of items in the job queue.

func (*Client) Peek

func (c *Client) Peek(ctx context.Context, placement storj.PlacementConstraint) (job RepairJob, err error)

Peek returns the lowest-health item from the job queue without removing it. If there are no items in the queue, it returns ErrQueueEmpty.

func (*Client) Pop

func (c *Client) Pop(ctx context.Context, includedPlacements, excludedPlacements []storj.PlacementConstraint) (job RepairJob, err error)

Pop removes and returns the lowest-health item from the job queue. If there are no items in the queue, it returns ErrQueueEmpty.

func (*Client) Push

func (c *Client) Push(ctx context.Context, job RepairJob) (wasNew bool, err error)

Push adds a new item to the job queue with the given health.

It returns an indication of whether the given segment was newly inserted or if it already existed in the target queue.

func (*Client) PushBatch

func (c *Client) PushBatch(ctx context.Context, jobs []RepairJob) (wasNew []bool, err error)

PushBatch adds multiple items to the appropriate job queues with the given health values.

It returns a slice of booleans indicating whether each segment was newly inserted or if it already existed in the target queue.

func (*Client) Trim

func (c *Client) Trim(ctx context.Context, placement storj.PlacementConstraint, healthGreaterThan float64) (removedSegments int32, err error)

Trim removes all jobs with Health greater than the given threshold.

func (*Client) Truncate

func (c *Client) Truncate(ctx context.Context, placement storj.PlacementConstraint) error

Truncate removes all items from a job queue.

type RepairJob

type RepairJob struct {
	ID                SegmentIdentifier
	Health            float64
	InsertedAt        uint64
	LastAttemptedAt   uint64
	UpdatedAt         uint64
	NumAttempts       uint16
	Placement         uint16
	NumMissing        uint16
	NumOutOfPlacement uint16
}

RepairJob represents the in-memory structure of a job in the queue. This structure does not _need_ to be a multiple of 64 bits in size, but it's probably going to be aligned to a multiple of 64 bits in memory either way, so it's more efficient if we use that whole space.

func ConvertJobFromProtobuf

func ConvertJobFromProtobuf(protoJob *pb.RepairJob) (RepairJob, error)

ConvertJobFromProtobuf converts a protobuf representation of a RepairJob to a RepairJob record.

func (RepairJob) LastAttemptedAtTime

func (rj RepairJob) LastAttemptedAtTime() time.Time

LastAttemptedAtTime returns the LastAttemptedAt field as a time.Time.

type SegmentIdentifier

type SegmentIdentifier struct {
	// StreamID is the stream ID of the segment.
	StreamID uuid.UUID
	// Position is the position of the segment.
	Position uint64
}

SegmentIdentifier identifies individual segments in the repair queue.