monitoring

Documentation

Index

• type MetricsMonitor
  • func NewMetricsMonitor(p *infprocessor.P, logger logr.Logger) *MetricsMonitor
  • func (m *MetricsMonitor) ObserveCompletionLatency(modelID string, latency time.Duration)
  • func (m *MetricsMonitor) ObserveEmbeddingLatency(modelID string, latency time.Duration)
  • func (m *MetricsMonitor) Run(ctx context.Context, interval time.Duration) error
  • func (m *MetricsMonitor) UnregisterAllCollectors()
  • func (m *MetricsMonitor) UpdateCompletionRequest(modelID string, c int)
  • func (m *MetricsMonitor) UpdateEmbeddingRequest(modelID string, c int)

Types

type MetricsMonitor

type MetricsMonitor struct {
	// contains filtered or unexported fields
}

MetricsMonitor holds and updates Prometheus metrics.

func NewMetricsMonitor

func NewMetricsMonitor(p *infprocessor.P, logger logr.Logger) *MetricsMonitor

NewMetricsMonitor returns a new MetricsMonitor.

func (*MetricsMonitor) ObserveCompletionLatency

func (m *MetricsMonitor) ObserveCompletionLatency(modelID string, latency time.Duration)

ObserveCompletionLatency observes a new latency data for a completion request.

func (*MetricsMonitor) ObserveEmbeddingLatency

func (m *MetricsMonitor) ObserveEmbeddingLatency(modelID string, latency time.Duration)

ObserveEmbeddingLatency observes a new latency data for an embedding request.

func (*MetricsMonitor) Run

func (m *MetricsMonitor) Run(ctx context.Context, interval time.Duration) error

Run updates the metrics periodically.

func (*MetricsMonitor) UnregisterAllCollectors

func (m *MetricsMonitor) UnregisterAllCollectors()

UnregisterAllCollectors unregisters all connectors.

func (*MetricsMonitor) UpdateCompletionRequest

func (m *MetricsMonitor) UpdateCompletionRequest(modelID string, c int)

UpdateCompletionRequest updates the number of completion requests.

func (*MetricsMonitor) UpdateEmbeddingRequest

func (m *MetricsMonitor) UpdateEmbeddingRequest(modelID string, c int)

UpdateEmbeddingRequest updates the number of embedding requests.