metrics

Documentation

Overview

Package metrics contains definitions for all JMX collected Metrics, and core collection methods for Brokers, Consumers, and Producers.

Index

• Variables
• func CollectBrokerRequestMetrics(sample *metric.Set, metricSets []*JMXMetricSet)
• func CollectMetricDefinitions(sample *metric.Set, metricSets []*JMXMetricSet, beanModifier BeanModifier)
• func CollectTopicSubMetrics(entity *integration.Entity, entityType string, metricSets []*JMXMetricSet, ...)
• func GetBrokerMetrics(sample *metric.Set)
• func GetConsumerMetrics(consumerName string, sample *metric.Set)
• func GetProducerMetrics(producerName string, sample *metric.Set)
• type BeanModifier
  • func ApplyConsumerTopicName(consumerName, topicName string) BeanModifier
  • func ApplyProducerTopicName(producerName, topicName string) BeanModifier
  • func ApplyTopicName(topicName string) BeanModifier
• type JMXMetricSet
• type MetricDefinition

Variables

var BrokerTopicMetricDefs = []*JMXMetricSet{
	{
		MBean:        "kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=" + topicHolder,
		MetricPrefix: "kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=" + topicHolder + ",",
		MetricDefs: []*MetricDefinition{
			{
				Name:       "broker.bytesWrittenToTopicPerSecond",
				SourceType: metric.RATE,
				JMXAttr:    "attr=Count",
			},
		},
	},
}

BrokerTopicMetricDefs metric definitions for topic metrics that are specific to a Broker

var ConsumerTopicMetricDefs = []*JMXMetricSet{
	{
		MBean:        "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=" + consumerHolder + ",topic=*",
		MetricPrefix: "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=" + consumerHolder + ",topic=" + topicHolder + ",",
		MetricDefs: []*MetricDefinition{
			{
				Name:       "consumer.avgFetchSizeInBytes",
				SourceType: metric.GAUGE,
				JMXAttr:    "attr=fetch-size-avg",
			},
			{
				Name:       "consumer.maxFetchSizeInBytes",
				SourceType: metric.GAUGE,
				JMXAttr:    "attr=fetch-size-max",
			},
			{
				Name:       "consumer.avgRecordConsumedPerTopicPerSecond",
				SourceType: metric.GAUGE,
				JMXAttr:    "attr=records-consumed-rate",
			},
			{
				Name:       "consumer.avgRecordConsumedPerTopic",
				SourceType: metric.GAUGE,
				JMXAttr:    "attr=records-per-request-avg",
			},
		},
	},
}

ConsumerTopicMetricDefs metric definitions for topic metrics that are specific to a Consumer

var ProducerTopicMetricDefs = []*JMXMetricSet{
	{
		MBean:        "kafka.producer:type=producer-topic-metrics,client-id=" + producerHolder + ",topic=*",
		MetricPrefix: "kafka.producer:type=producer-topic-metrics,client-id=" + producerHolder + ",topic=" + topicHolder + ",",
		MetricDefs: []*MetricDefinition{
			{
				Name:       "producer.avgRecordsSentPerTopicPerSecond",
				SourceType: metric.GAUGE,
				JMXAttr:    "attr=record-send-rate",
			},
			{
				Name:       "producer.avgCompressionRateRecordBatches",
				SourceType: metric.GAUGE,
				JMXAttr:    "attr=compression-rate",
			},
		},
	},
}

ProducerTopicMetricDefs metric definitions for topic metrics that are specific to a Producer

var TopicOffsetMetricDef = &JMXMetricSet{
	MBean: "kafka.log:type=Log,name=LogEndOffset,topic=" + topicHolder + ",partition=*",
}

TopicOffsetMetricDef metric definition for calculating the roll up for a Topic's offset for a given Broker

var TopicSizeMetricDef = &JMXMetricSet{
	MBean: "kafka.log:type=Log,name=Size,topic=" + topicHolder + ",partition=*",
}

TopicSizeMetricDef metric definition for calculating the roll up for a Topic's on disk size for a given Broker

Functions

func CollectBrokerRequestMetrics

func CollectBrokerRequestMetrics(sample *metric.Set, metricSets []*JMXMetricSet)

CollectBrokerRequestMetrics collects request metrics from brokers

func CollectMetricDefinitions

func CollectMetricDefinitions(sample *metric.Set, metricSets []*JMXMetricSet, beanModifier BeanModifier)

CollectMetricDefinitions collects the set of metrics from the current open JMX connection and add them to the sample

func CollectTopicSubMetrics

func CollectTopicSubMetrics(
	entity *integration.Entity,
	entityType string,
	metricSets []*JMXMetricSet,
	beanModifier func(string, string) BeanModifier,
)

CollectTopicSubMetrics collects Topic metrics that are related to either a Producer or Consumer

beanModifier is a function that is used to replace place holder with actual Consumer/Producer and Topic names for a given MBean

func GetBrokerMetrics

func GetBrokerMetrics(sample *metric.Set)

GetBrokerMetrics collects all Broker JMX metrics and stores them in sample

func GetConsumerMetrics

func GetConsumerMetrics(consumerName string, sample *metric.Set)

GetConsumerMetrics collects all Consumer metrics for the given consumerName and stores them in sample.

func GetProducerMetrics

func GetProducerMetrics(producerName string, sample *metric.Set)

GetProducerMetrics collects all Producer and Producer metrics for the given producerName and stores them in sample.

Types

type BeanModifier

type BeanModifier func(string) string

BeanModifier is a function that takes in an MBean and modifies any place holders

func ApplyConsumerTopicName

func ApplyConsumerTopicName(consumerName, topicName string) BeanModifier

ApplyConsumerTopicName to be used when passed to CollectMetricDefinitions to modified bean name for Consumer and Topic

func ApplyProducerTopicName

func ApplyProducerTopicName(producerName, topicName string) BeanModifier

ApplyProducerTopicName to be used when passed to collectMetricDefinitions to modified bean name for Producer and Topic

func ApplyTopicName

func ApplyTopicName(topicName string) BeanModifier

ApplyTopicName to modified bean name for Topic

type JMXMetricSet

type JMXMetricSet struct {
	MBean        string
	MetricPrefix string
	MetricDefs   []*MetricDefinition
}

JMXMetricSet defines a set of MetricDefinitions that can be collected from a specific MBean

type MetricDefinition

type MetricDefinition struct {
	Name       string
	SourceType metric.SourceType
	JMXAttr    string
}

MetricDefinition defines a single Infrastructure metric