partition

Documentation

Index

• type Committer
• type Consumer
• type ConsumerFactory
• type Reader
  • func NewReader(kafkaCfg kafka.Config, partitionID int32, instanceID string, ...) (*Reader, error)
• type Record

Types

type Committer

type Committer interface {
	Commit(ctx context.Context, offset int64) error
	EnqueueOffset(offset int64)
}

Committer defines an interface for committing offsets

type Consumer

type Consumer interface {
	Start(ctx context.Context, recordsChan <-chan []Record) func()
}

type ConsumerFactory

type ConsumerFactory func(committer Committer) (Consumer, error)

type Reader

type Reader struct {
	services.Service
	// contains filtered or unexported fields
}

Reader is responsible for reading data from a specific Kafka partition and passing it to the consumer for processing. It is a core component of the Loki ingester's Kafka-based ingestion pipeline.

func NewReader

func NewReader(
	kafkaCfg kafka.Config,
	partitionID int32,
	instanceID string,
	consumerFactory ConsumerFactory,
	logger log.Logger,
	reg prometheus.Registerer,
) (*Reader, error)

NewReader creates and initializes a new PartitionReader. It sets up the basic service and initializes the reader with the provided configuration.

type Record

type Record struct {
	// Context holds the tracing (and potentially other) info, that the record was enriched with on fetch from Kafka.
	Ctx      context.Context
	TenantID string
	Content  []byte
	Offset   int64
}