metadata

package
v0.116.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 17, 2024 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	MetricsStability = component.StabilityLevelAlpha
)

Variables

View Source 
var (
	Type      = component.MustNewType("apachespark")
	ScopeName = "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/apachesparkreceiver"
)
View Source 
var MapAttributeDirection = map[string]AttributeDirection{
	"in":  AttributeDirectionIn,
	"out": AttributeDirectionOut,
}

MapAttributeDirection is a helper map of string to AttributeDirection attribute value.

View Source 
var MapAttributeExecutorTaskResult = map[string]AttributeExecutorTaskResult{
	"completed": AttributeExecutorTaskResultCompleted,
	"failed":    AttributeExecutorTaskResultFailed,
}

MapAttributeExecutorTaskResult is a helper map of string to AttributeExecutorTaskResult attribute value.

View Source 
var MapAttributeGcType = map[string]AttributeGcType{
	"major": AttributeGcTypeMajor,
	"minor": AttributeGcTypeMinor,
}

MapAttributeGcType is a helper map of string to AttributeGcType attribute value.

View Source 
var MapAttributeJobResult = map[string]AttributeJobResult{
	"completed": AttributeJobResultCompleted,
	"failed":    AttributeJobResultFailed,
	"skipped":   AttributeJobResultSkipped,
}

MapAttributeJobResult is a helper map of string to AttributeJobResult attribute value.

View Source 
var MapAttributeLocation = map[string]AttributeLocation{
	"on_heap":  AttributeLocationOnHeap,
	"off_heap": AttributeLocationOffHeap,
}

MapAttributeLocation is a helper map of string to AttributeLocation attribute value.

View Source 
var MapAttributePoolMemoryType = map[string]AttributePoolMemoryType{
	"direct": AttributePoolMemoryTypeDirect,
	"mapped": AttributePoolMemoryTypeMapped,
}

MapAttributePoolMemoryType is a helper map of string to AttributePoolMemoryType attribute value.

View Source 
var MapAttributeSchedulerStatus = map[string]AttributeSchedulerStatus{
	"waiting": AttributeSchedulerStatusWaiting,
	"running": AttributeSchedulerStatusRunning,
}

MapAttributeSchedulerStatus is a helper map of string to AttributeSchedulerStatus attribute value.

View Source 
var MapAttributeSource = map[string]AttributeSource{
	"local":  AttributeSourceLocal,
	"remote": AttributeSourceRemote,
}

MapAttributeSource is a helper map of string to AttributeSource attribute value.

View Source 
var MapAttributeStageTaskResult = map[string]AttributeStageTaskResult{
	"completed": AttributeStageTaskResultCompleted,
	"failed":    AttributeStageTaskResultFailed,
	"killed":    AttributeStageTaskResultKilled,
}

MapAttributeStageTaskResult is a helper map of string to AttributeStageTaskResult attribute value.

View Source 
var MapAttributeState = map[string]AttributeState{
	"used": AttributeStateUsed,
	"free": AttributeStateFree,
}

MapAttributeState is a helper map of string to AttributeState attribute value.

Functions

This section is empty.

Types

type AttributeDirection 

type AttributeDirection int

AttributeDirection specifies the a value direction attribute. 

const (
	AttributeDirectionIn AttributeDirection
	AttributeDirectionOut
)

func (AttributeDirection) String 

func (av AttributeDirection) String() string

String returns the string representation of the AttributeDirection.

type AttributeExecutorTaskResult 

type AttributeExecutorTaskResult int

AttributeExecutorTaskResult specifies the a value executor_task_result attribute. 

const (
	AttributeExecutorTaskResultCompleted AttributeExecutorTaskResult
	AttributeExecutorTaskResultFailed
)

func (AttributeExecutorTaskResult) String 

func (av AttributeExecutorTaskResult) String() string

String returns the string representation of the AttributeExecutorTaskResult.

type AttributeGcType 

type AttributeGcType int

AttributeGcType specifies the a value gc_type attribute. 

const (
	AttributeGcTypeMajor AttributeGcType
	AttributeGcTypeMinor
)

func (AttributeGcType) String 

func (av AttributeGcType) String() string

String returns the string representation of the AttributeGcType.

type AttributeJobResult 

type AttributeJobResult int

AttributeJobResult specifies the a value job_result attribute. 

const (
	AttributeJobResultCompleted AttributeJobResult
	AttributeJobResultFailed
	AttributeJobResultSkipped
)

func (AttributeJobResult) String 

func (av AttributeJobResult) String() string

String returns the string representation of the AttributeJobResult.

type AttributeLocation 

type AttributeLocation int

AttributeLocation specifies the a value location attribute. 

const (
	AttributeLocationOnHeap AttributeLocation
	AttributeLocationOffHeap
)

func (AttributeLocation) String 

func (av AttributeLocation) String() string

String returns the string representation of the AttributeLocation.

type AttributePoolMemoryType 

type AttributePoolMemoryType int

AttributePoolMemoryType specifies the a value pool_memory_type attribute. 

const (
	AttributePoolMemoryTypeDirect AttributePoolMemoryType
	AttributePoolMemoryTypeMapped
)

func (AttributePoolMemoryType) String 

func (av AttributePoolMemoryType) String() string

String returns the string representation of the AttributePoolMemoryType.

type AttributeSchedulerStatus added in v0.79.0 

type AttributeSchedulerStatus int

AttributeSchedulerStatus specifies the a value scheduler_status attribute. 

const (
	AttributeSchedulerStatusWaiting AttributeSchedulerStatus
	AttributeSchedulerStatusRunning
)

func (AttributeSchedulerStatus) String added in v0.79.0 

func (av AttributeSchedulerStatus) String() string

String returns the string representation of the AttributeSchedulerStatus.

type AttributeSource 

type AttributeSource int

AttributeSource specifies the a value source attribute. 

const (
	AttributeSourceLocal AttributeSource
	AttributeSourceRemote
)

func (AttributeSource) String 

func (av AttributeSource) String() string

String returns the string representation of the AttributeSource.

type AttributeStageTaskResult 

type AttributeStageTaskResult int

AttributeStageTaskResult specifies the a value stage_task_result attribute. 

const (
	AttributeStageTaskResultCompleted AttributeStageTaskResult
	AttributeStageTaskResultFailed
	AttributeStageTaskResultKilled
)

func (AttributeStageTaskResult) String 

func (av AttributeStageTaskResult) String() string

String returns the string representation of the AttributeStageTaskResult.

type AttributeState 

type AttributeState int

AttributeState specifies the a value state attribute. 

const (
	AttributeStateUsed AttributeState
	AttributeStateFree
)

func (AttributeState) String 

func (av AttributeState) String() string

String returns the string representation of the AttributeState.

type MetricBuilderOption added in v0.110.0 

type MetricBuilderOption interface {
	// contains filtered or unexported methods
}

MetricBuilderOption applies changes to default metrics builder.

func WithStartTime 

func WithStartTime(startTime pcommon.Timestamp) MetricBuilderOption

WithStartTime sets startTime on the metrics builder.

type MetricConfig 

type MetricConfig struct {
	Enabled bool `mapstructure:"enabled"`
	// contains filtered or unexported fields
}

MetricConfig provides common config for a particular metric.

func (*MetricConfig) Unmarshal 

func (ms *MetricConfig) Unmarshal(parser *confmap.Conf) error

type MetricsBuilder 

type MetricsBuilder struct {
	// contains filtered or unexported fields
}

MetricsBuilder provides an interface for scrapers to report metrics while taking care of all the transformations required to produce metric representation defined in metadata and user config.

func NewMetricsBuilder 

func NewMetricsBuilder(mbc MetricsBuilderConfig, settings receiver.Settings, options ...MetricBuilderOption) *MetricsBuilder

func (*MetricsBuilder) Emit 

func (mb *MetricsBuilder) Emit(options ...ResourceMetricsOption) pmetric.Metrics

Emit returns all the metrics accumulated by the metrics builder and updates the internal state to be ready for recording another set of metrics. This function will be responsible for applying all the transformations required to produce metric representation defined in metadata and user config, e.g. delta or cumulative.

func (*MetricsBuilder) EmitForResource 

func (mb *MetricsBuilder) EmitForResource(options ...ResourceMetricsOption)

EmitForResource saves all the generated metrics under a new resource and updates the internal state to be ready for recording another set of data points as part of another resource. This function can be helpful when one scraper needs to emit metrics from several resources. Otherwise calling this function is not required, just `Emit` function can be called instead. Resource attributes should be provided as ResourceMetricsOption arguments.

func (*MetricsBuilder) NewResourceBuilder added in v0.83.0 

func (mb *MetricsBuilder) NewResourceBuilder() *ResourceBuilder

NewResourceBuilder returns a new resource builder that should be used to build a resource associated with for the emitted metrics.

func (*MetricsBuilder) RecordSparkDriverBlockManagerDiskUsageDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverBlockManagerDiskUsageDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverBlockManagerDiskUsageDataPoint adds a data point to spark.driver.block_manager.disk.usage metric.

func (*MetricsBuilder) RecordSparkDriverBlockManagerMemoryUsageDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverBlockManagerMemoryUsageDataPoint(ts pcommon.Timestamp, val int64, locationAttributeValue AttributeLocation, stateAttributeValue AttributeState)

RecordSparkDriverBlockManagerMemoryUsageDataPoint adds a data point to spark.driver.block_manager.memory.usage metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorCompilationAverageTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorCompilationAverageTimeDataPoint(ts pcommon.Timestamp, val float64)

RecordSparkDriverCodeGeneratorCompilationAverageTimeDataPoint adds a data point to spark.driver.code_generator.compilation.average_time metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorCompilationCountDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorCompilationCountDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverCodeGeneratorCompilationCountDataPoint adds a data point to spark.driver.code_generator.compilation.count metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedClassAverageSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedClassAverageSizeDataPoint(ts pcommon.Timestamp, val float64)

RecordSparkDriverCodeGeneratorGeneratedClassAverageSizeDataPoint adds a data point to spark.driver.code_generator.generated_class.average_size metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedClassCountDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedClassCountDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverCodeGeneratorGeneratedClassCountDataPoint adds a data point to spark.driver.code_generator.generated_class.count metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedMethodAverageSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedMethodAverageSizeDataPoint(ts pcommon.Timestamp, val float64)

RecordSparkDriverCodeGeneratorGeneratedMethodAverageSizeDataPoint adds a data point to spark.driver.code_generator.generated_method.average_size metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedMethodCountDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorGeneratedMethodCountDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverCodeGeneratorGeneratedMethodCountDataPoint adds a data point to spark.driver.code_generator.generated_method.count metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorSourceCodeAverageSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorSourceCodeAverageSizeDataPoint(ts pcommon.Timestamp, val float64)

RecordSparkDriverCodeGeneratorSourceCodeAverageSizeDataPoint adds a data point to spark.driver.code_generator.source_code.average_size metric.

func (*MetricsBuilder) RecordSparkDriverCodeGeneratorSourceCodeOperationsDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverCodeGeneratorSourceCodeOperationsDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverCodeGeneratorSourceCodeOperationsDataPoint adds a data point to spark.driver.code_generator.source_code.operations metric.

func (*MetricsBuilder) RecordSparkDriverDagSchedulerJobActiveDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverDagSchedulerJobActiveDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverDagSchedulerJobActiveDataPoint adds a data point to spark.driver.dag_scheduler.job.active metric.

func (*MetricsBuilder) RecordSparkDriverDagSchedulerJobCountDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverDagSchedulerJobCountDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverDagSchedulerJobCountDataPoint adds a data point to spark.driver.dag_scheduler.job.count metric.

func (*MetricsBuilder) RecordSparkDriverDagSchedulerStageCountDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverDagSchedulerStageCountDataPoint(ts pcommon.Timestamp, val int64, schedulerStatusAttributeValue AttributeSchedulerStatus)

RecordSparkDriverDagSchedulerStageCountDataPoint adds a data point to spark.driver.dag_scheduler.stage.count metric.

func (*MetricsBuilder) RecordSparkDriverDagSchedulerStageFailedDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverDagSchedulerStageFailedDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverDagSchedulerStageFailedDataPoint adds a data point to spark.driver.dag_scheduler.stage.failed metric.

func (*MetricsBuilder) RecordSparkDriverExecutorGcOperationsDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverExecutorGcOperationsDataPoint(ts pcommon.Timestamp, val int64, gcTypeAttributeValue AttributeGcType)

RecordSparkDriverExecutorGcOperationsDataPoint adds a data point to spark.driver.executor.gc.operations metric.

func (*MetricsBuilder) RecordSparkDriverExecutorGcTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverExecutorGcTimeDataPoint(ts pcommon.Timestamp, val int64, gcTypeAttributeValue AttributeGcType)

RecordSparkDriverExecutorGcTimeDataPoint adds a data point to spark.driver.executor.gc.time metric.

func (*MetricsBuilder) RecordSparkDriverExecutorMemoryExecutionDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverExecutorMemoryExecutionDataPoint(ts pcommon.Timestamp, val int64, locationAttributeValue AttributeLocation)

RecordSparkDriverExecutorMemoryExecutionDataPoint adds a data point to spark.driver.executor.memory.execution metric.

func (*MetricsBuilder) RecordSparkDriverExecutorMemoryJvmDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverExecutorMemoryJvmDataPoint(ts pcommon.Timestamp, val int64, locationAttributeValue AttributeLocation)

RecordSparkDriverExecutorMemoryJvmDataPoint adds a data point to spark.driver.executor.memory.jvm metric.

func (*MetricsBuilder) RecordSparkDriverExecutorMemoryPoolDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverExecutorMemoryPoolDataPoint(ts pcommon.Timestamp, val int64, poolMemoryTypeAttributeValue AttributePoolMemoryType)

RecordSparkDriverExecutorMemoryPoolDataPoint adds a data point to spark.driver.executor.memory.pool metric.

func (*MetricsBuilder) RecordSparkDriverExecutorMemoryStorageDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverExecutorMemoryStorageDataPoint(ts pcommon.Timestamp, val int64, locationAttributeValue AttributeLocation)

RecordSparkDriverExecutorMemoryStorageDataPoint adds a data point to spark.driver.executor.memory.storage metric.

func (*MetricsBuilder) RecordSparkDriverHiveExternalCatalogFileCacheHitsDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverHiveExternalCatalogFileCacheHitsDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverHiveExternalCatalogFileCacheHitsDataPoint adds a data point to spark.driver.hive_external_catalog.file_cache_hits metric.

func (*MetricsBuilder) RecordSparkDriverHiveExternalCatalogFilesDiscoveredDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverHiveExternalCatalogFilesDiscoveredDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverHiveExternalCatalogFilesDiscoveredDataPoint adds a data point to spark.driver.hive_external_catalog.files_discovered metric.

func (*MetricsBuilder) RecordSparkDriverHiveExternalCatalogHiveClientCallsDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverHiveExternalCatalogHiveClientCallsDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverHiveExternalCatalogHiveClientCallsDataPoint adds a data point to spark.driver.hive_external_catalog.hive_client_calls metric.

func (*MetricsBuilder) RecordSparkDriverHiveExternalCatalogParallelListingJobsDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverHiveExternalCatalogParallelListingJobsDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverHiveExternalCatalogParallelListingJobsDataPoint adds a data point to spark.driver.hive_external_catalog.parallel_listing_jobs metric.

func (*MetricsBuilder) RecordSparkDriverHiveExternalCatalogPartitionsFetchedDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverHiveExternalCatalogPartitionsFetchedDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverHiveExternalCatalogPartitionsFetchedDataPoint adds a data point to spark.driver.hive_external_catalog.partitions_fetched metric.

func (*MetricsBuilder) RecordSparkDriverJvmCPUTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverJvmCPUTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverJvmCPUTimeDataPoint adds a data point to spark.driver.jvm_cpu_time metric.

func (*MetricsBuilder) RecordSparkDriverLiveListenerBusDroppedDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverLiveListenerBusDroppedDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverLiveListenerBusDroppedDataPoint adds a data point to spark.driver.live_listener_bus.dropped metric.

func (*MetricsBuilder) RecordSparkDriverLiveListenerBusPostedDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverLiveListenerBusPostedDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverLiveListenerBusPostedDataPoint adds a data point to spark.driver.live_listener_bus.posted metric.

func (*MetricsBuilder) RecordSparkDriverLiveListenerBusProcessingTimeAverageDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverLiveListenerBusProcessingTimeAverageDataPoint(ts pcommon.Timestamp, val float64)

RecordSparkDriverLiveListenerBusProcessingTimeAverageDataPoint adds a data point to spark.driver.live_listener_bus.processing_time.average metric.

func (*MetricsBuilder) RecordSparkDriverLiveListenerBusQueueSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkDriverLiveListenerBusQueueSizeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkDriverLiveListenerBusQueueSizeDataPoint adds a data point to spark.driver.live_listener_bus.queue_size metric.

func (*MetricsBuilder) RecordSparkExecutorDiskUsageDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorDiskUsageDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorDiskUsageDataPoint adds a data point to spark.executor.disk.usage metric.

func (*MetricsBuilder) RecordSparkExecutorGcTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorGcTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorGcTimeDataPoint adds a data point to spark.executor.gc_time metric.

func (*MetricsBuilder) RecordSparkExecutorInputSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorInputSizeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorInputSizeDataPoint adds a data point to spark.executor.input_size metric.

func (*MetricsBuilder) RecordSparkExecutorMemoryUsageDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorMemoryUsageDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorMemoryUsageDataPoint adds a data point to spark.executor.memory.usage metric.

func (*MetricsBuilder) RecordSparkExecutorShuffleIoSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorShuffleIoSizeDataPoint(ts pcommon.Timestamp, val int64, directionAttributeValue AttributeDirection)

RecordSparkExecutorShuffleIoSizeDataPoint adds a data point to spark.executor.shuffle.io.size metric.

func (*MetricsBuilder) RecordSparkExecutorStorageMemoryUsageDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorStorageMemoryUsageDataPoint(ts pcommon.Timestamp, val int64, locationAttributeValue AttributeLocation, stateAttributeValue AttributeState)

RecordSparkExecutorStorageMemoryUsageDataPoint adds a data point to spark.executor.storage_memory.usage metric.

func (*MetricsBuilder) RecordSparkExecutorTaskActiveDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorTaskActiveDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorTaskActiveDataPoint adds a data point to spark.executor.task.active metric.

func (*MetricsBuilder) RecordSparkExecutorTaskLimitDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorTaskLimitDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorTaskLimitDataPoint adds a data point to spark.executor.task.limit metric.

func (*MetricsBuilder) RecordSparkExecutorTaskResultDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorTaskResultDataPoint(ts pcommon.Timestamp, val int64, executorTaskResultAttributeValue AttributeExecutorTaskResult)

RecordSparkExecutorTaskResultDataPoint adds a data point to spark.executor.task.result metric.

func (*MetricsBuilder) RecordSparkExecutorTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkExecutorTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkExecutorTimeDataPoint adds a data point to spark.executor.time metric.

func (*MetricsBuilder) RecordSparkJobStageActiveDataPoint 

func (mb *MetricsBuilder) RecordSparkJobStageActiveDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkJobStageActiveDataPoint adds a data point to spark.job.stage.active metric.

func (*MetricsBuilder) RecordSparkJobStageResultDataPoint 

func (mb *MetricsBuilder) RecordSparkJobStageResultDataPoint(ts pcommon.Timestamp, val int64, jobResultAttributeValue AttributeJobResult)

RecordSparkJobStageResultDataPoint adds a data point to spark.job.stage.result metric.

func (*MetricsBuilder) RecordSparkJobTaskActiveDataPoint 

func (mb *MetricsBuilder) RecordSparkJobTaskActiveDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkJobTaskActiveDataPoint adds a data point to spark.job.task.active metric.

func (*MetricsBuilder) RecordSparkJobTaskResultDataPoint 

func (mb *MetricsBuilder) RecordSparkJobTaskResultDataPoint(ts pcommon.Timestamp, val int64, jobResultAttributeValue AttributeJobResult)

RecordSparkJobTaskResultDataPoint adds a data point to spark.job.task.result metric.

func (*MetricsBuilder) RecordSparkStageDiskSpilledDataPoint 

func (mb *MetricsBuilder) RecordSparkStageDiskSpilledDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageDiskSpilledDataPoint adds a data point to spark.stage.disk.spilled metric.

func (*MetricsBuilder) RecordSparkStageExecutorCPUTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageExecutorCPUTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageExecutorCPUTimeDataPoint adds a data point to spark.stage.executor.cpu_time metric.

func (*MetricsBuilder) RecordSparkStageExecutorRunTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageExecutorRunTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageExecutorRunTimeDataPoint adds a data point to spark.stage.executor.run_time metric.

func (*MetricsBuilder) RecordSparkStageIoRecordsDataPoint 

func (mb *MetricsBuilder) RecordSparkStageIoRecordsDataPoint(ts pcommon.Timestamp, val int64, directionAttributeValue AttributeDirection)

RecordSparkStageIoRecordsDataPoint adds a data point to spark.stage.io.records metric.

func (*MetricsBuilder) RecordSparkStageIoSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageIoSizeDataPoint(ts pcommon.Timestamp, val int64, directionAttributeValue AttributeDirection)

RecordSparkStageIoSizeDataPoint adds a data point to spark.stage.io.size metric.

func (*MetricsBuilder) RecordSparkStageJvmGcTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageJvmGcTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageJvmGcTimeDataPoint adds a data point to spark.stage.jvm_gc_time metric.

func (*MetricsBuilder) RecordSparkStageMemoryPeakDataPoint 

func (mb *MetricsBuilder) RecordSparkStageMemoryPeakDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageMemoryPeakDataPoint adds a data point to spark.stage.memory.peak metric.

func (*MetricsBuilder) RecordSparkStageMemorySpilledDataPoint 

func (mb *MetricsBuilder) RecordSparkStageMemorySpilledDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageMemorySpilledDataPoint adds a data point to spark.stage.memory.spilled metric.

func (*MetricsBuilder) RecordSparkStageShuffleBlocksFetchedDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleBlocksFetchedDataPoint(ts pcommon.Timestamp, val int64, sourceAttributeValue AttributeSource)

RecordSparkStageShuffleBlocksFetchedDataPoint adds a data point to spark.stage.shuffle.blocks_fetched metric.

func (*MetricsBuilder) RecordSparkStageShuffleFetchWaitTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleFetchWaitTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageShuffleFetchWaitTimeDataPoint adds a data point to spark.stage.shuffle.fetch_wait_time metric.

func (*MetricsBuilder) RecordSparkStageShuffleIoDiskDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleIoDiskDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageShuffleIoDiskDataPoint adds a data point to spark.stage.shuffle.io.disk metric.

func (*MetricsBuilder) RecordSparkStageShuffleIoReadSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleIoReadSizeDataPoint(ts pcommon.Timestamp, val int64, sourceAttributeValue AttributeSource)

RecordSparkStageShuffleIoReadSizeDataPoint adds a data point to spark.stage.shuffle.io.read.size metric.

func (*MetricsBuilder) RecordSparkStageShuffleIoRecordsDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleIoRecordsDataPoint(ts pcommon.Timestamp, val int64, directionAttributeValue AttributeDirection)

RecordSparkStageShuffleIoRecordsDataPoint adds a data point to spark.stage.shuffle.io.records metric.

func (*MetricsBuilder) RecordSparkStageShuffleIoWriteSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleIoWriteSizeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageShuffleIoWriteSizeDataPoint adds a data point to spark.stage.shuffle.io.write.size metric.

func (*MetricsBuilder) RecordSparkStageShuffleWriteTimeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageShuffleWriteTimeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageShuffleWriteTimeDataPoint adds a data point to spark.stage.shuffle.write_time metric.

func (*MetricsBuilder) RecordSparkStageStatusDataPoint 

func (mb *MetricsBuilder) RecordSparkStageStatusDataPoint(ts pcommon.Timestamp, val int64, stageActiveAttributeValue bool, stageCompleteAttributeValue bool, stagePendingAttributeValue bool, stageFailedAttributeValue bool)

RecordSparkStageStatusDataPoint adds a data point to spark.stage.status metric.

func (*MetricsBuilder) RecordSparkStageTaskActiveDataPoint 

func (mb *MetricsBuilder) RecordSparkStageTaskActiveDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageTaskActiveDataPoint adds a data point to spark.stage.task.active metric.

func (*MetricsBuilder) RecordSparkStageTaskResultDataPoint 

func (mb *MetricsBuilder) RecordSparkStageTaskResultDataPoint(ts pcommon.Timestamp, val int64, stageTaskResultAttributeValue AttributeStageTaskResult)

RecordSparkStageTaskResultDataPoint adds a data point to spark.stage.task.result metric.

func (*MetricsBuilder) RecordSparkStageTaskResultSizeDataPoint 

func (mb *MetricsBuilder) RecordSparkStageTaskResultSizeDataPoint(ts pcommon.Timestamp, val int64)

RecordSparkStageTaskResultSizeDataPoint adds a data point to spark.stage.task.result_size metric.

func (*MetricsBuilder) Reset 

func (mb *MetricsBuilder) Reset(options ...MetricBuilderOption)

Reset resets metrics builder to its initial state. It should be used when external metrics source is restarted, and metrics builder should update its startTime and reset it's internal state accordingly.

type MetricsBuilderConfig 

type MetricsBuilderConfig struct {
	Metrics            MetricsConfig            `mapstructure:"metrics"`
	ResourceAttributes ResourceAttributesConfig `mapstructure:"resource_attributes"`
}

MetricsBuilderConfig is a configuration for apachespark metrics builder.

func DefaultMetricsBuilderConfig 

func DefaultMetricsBuilderConfig() MetricsBuilderConfig

type MetricsConfig 

type MetricsConfig struct {
	SparkDriverBlockManagerDiskUsage                   MetricConfig `mapstructure:"spark.driver.block_manager.disk.usage"`
	SparkDriverBlockManagerMemoryUsage                 MetricConfig `mapstructure:"spark.driver.block_manager.memory.usage"`
	SparkDriverCodeGeneratorCompilationAverageTime     MetricConfig `mapstructure:"spark.driver.code_generator.compilation.average_time"`
	SparkDriverCodeGeneratorCompilationCount           MetricConfig `mapstructure:"spark.driver.code_generator.compilation.count"`
	SparkDriverCodeGeneratorGeneratedClassAverageSize  MetricConfig `mapstructure:"spark.driver.code_generator.generated_class.average_size"`
	SparkDriverCodeGeneratorGeneratedClassCount        MetricConfig `mapstructure:"spark.driver.code_generator.generated_class.count"`
	SparkDriverCodeGeneratorGeneratedMethodAverageSize MetricConfig `mapstructure:"spark.driver.code_generator.generated_method.average_size"`
	SparkDriverCodeGeneratorGeneratedMethodCount       MetricConfig `mapstructure:"spark.driver.code_generator.generated_method.count"`
	SparkDriverCodeGeneratorSourceCodeAverageSize      MetricConfig `mapstructure:"spark.driver.code_generator.source_code.average_size"`
	SparkDriverCodeGeneratorSourceCodeOperations       MetricConfig `mapstructure:"spark.driver.code_generator.source_code.operations"`
	SparkDriverDagSchedulerJobActive                   MetricConfig `mapstructure:"spark.driver.dag_scheduler.job.active"`
	SparkDriverDagSchedulerJobCount                    MetricConfig `mapstructure:"spark.driver.dag_scheduler.job.count"`
	SparkDriverDagSchedulerStageCount                  MetricConfig `mapstructure:"spark.driver.dag_scheduler.stage.count"`
	SparkDriverDagSchedulerStageFailed                 MetricConfig `mapstructure:"spark.driver.dag_scheduler.stage.failed"`
	SparkDriverExecutorGcOperations                    MetricConfig `mapstructure:"spark.driver.executor.gc.operations"`
	SparkDriverExecutorGcTime                          MetricConfig `mapstructure:"spark.driver.executor.gc.time"`
	SparkDriverExecutorMemoryExecution                 MetricConfig `mapstructure:"spark.driver.executor.memory.execution"`
	SparkDriverExecutorMemoryJvm                       MetricConfig `mapstructure:"spark.driver.executor.memory.jvm"`
	SparkDriverExecutorMemoryPool                      MetricConfig `mapstructure:"spark.driver.executor.memory.pool"`
	SparkDriverExecutorMemoryStorage                   MetricConfig `mapstructure:"spark.driver.executor.memory.storage"`
	SparkDriverHiveExternalCatalogFileCacheHits        MetricConfig `mapstructure:"spark.driver.hive_external_catalog.file_cache_hits"`
	SparkDriverHiveExternalCatalogFilesDiscovered      MetricConfig `mapstructure:"spark.driver.hive_external_catalog.files_discovered"`
	SparkDriverHiveExternalCatalogHiveClientCalls      MetricConfig `mapstructure:"spark.driver.hive_external_catalog.hive_client_calls"`
	SparkDriverHiveExternalCatalogParallelListingJobs  MetricConfig `mapstructure:"spark.driver.hive_external_catalog.parallel_listing_jobs"`
	SparkDriverHiveExternalCatalogPartitionsFetched    MetricConfig `mapstructure:"spark.driver.hive_external_catalog.partitions_fetched"`
	SparkDriverJvmCPUTime                              MetricConfig `mapstructure:"spark.driver.jvm_cpu_time"`
	SparkDriverLiveListenerBusDropped                  MetricConfig `mapstructure:"spark.driver.live_listener_bus.dropped"`
	SparkDriverLiveListenerBusPosted                   MetricConfig `mapstructure:"spark.driver.live_listener_bus.posted"`
	SparkDriverLiveListenerBusProcessingTimeAverage    MetricConfig `mapstructure:"spark.driver.live_listener_bus.processing_time.average"`
	SparkDriverLiveListenerBusQueueSize                MetricConfig `mapstructure:"spark.driver.live_listener_bus.queue_size"`
	SparkExecutorDiskUsage                             MetricConfig `mapstructure:"spark.executor.disk.usage"`
	SparkExecutorGcTime                                MetricConfig `mapstructure:"spark.executor.gc_time"`
	SparkExecutorInputSize                             MetricConfig `mapstructure:"spark.executor.input_size"`
	SparkExecutorMemoryUsage                           MetricConfig `mapstructure:"spark.executor.memory.usage"`
	SparkExecutorShuffleIoSize                         MetricConfig `mapstructure:"spark.executor.shuffle.io.size"`
	SparkExecutorStorageMemoryUsage                    MetricConfig `mapstructure:"spark.executor.storage_memory.usage"`
	SparkExecutorTaskActive                            MetricConfig `mapstructure:"spark.executor.task.active"`
	SparkExecutorTaskLimit                             MetricConfig `mapstructure:"spark.executor.task.limit"`
	SparkExecutorTaskResult                            MetricConfig `mapstructure:"spark.executor.task.result"`
	SparkExecutorTime                                  MetricConfig `mapstructure:"spark.executor.time"`
	SparkJobStageActive                                MetricConfig `mapstructure:"spark.job.stage.active"`
	SparkJobStageResult                                MetricConfig `mapstructure:"spark.job.stage.result"`
	SparkJobTaskActive                                 MetricConfig `mapstructure:"spark.job.task.active"`
	SparkJobTaskResult                                 MetricConfig `mapstructure:"spark.job.task.result"`
	SparkStageDiskSpilled                              MetricConfig `mapstructure:"spark.stage.disk.spilled"`
	SparkStageExecutorCPUTime                          MetricConfig `mapstructure:"spark.stage.executor.cpu_time"`
	SparkStageExecutorRunTime                          MetricConfig `mapstructure:"spark.stage.executor.run_time"`
	SparkStageIoRecords                                MetricConfig `mapstructure:"spark.stage.io.records"`
	SparkStageIoSize                                   MetricConfig `mapstructure:"spark.stage.io.size"`
	SparkStageJvmGcTime                                MetricConfig `mapstructure:"spark.stage.jvm_gc_time"`
	SparkStageMemoryPeak                               MetricConfig `mapstructure:"spark.stage.memory.peak"`
	SparkStageMemorySpilled                            MetricConfig `mapstructure:"spark.stage.memory.spilled"`
	SparkStageShuffleBlocksFetched                     MetricConfig `mapstructure:"spark.stage.shuffle.blocks_fetched"`
	SparkStageShuffleFetchWaitTime                     MetricConfig `mapstructure:"spark.stage.shuffle.fetch_wait_time"`
	SparkStageShuffleIoDisk                            MetricConfig `mapstructure:"spark.stage.shuffle.io.disk"`
	SparkStageShuffleIoReadSize                        MetricConfig `mapstructure:"spark.stage.shuffle.io.read.size"`
	SparkStageShuffleIoRecords                         MetricConfig `mapstructure:"spark.stage.shuffle.io.records"`
	SparkStageShuffleIoWriteSize                       MetricConfig `mapstructure:"spark.stage.shuffle.io.write.size"`
	SparkStageShuffleWriteTime                         MetricConfig `mapstructure:"spark.stage.shuffle.write_time"`
	SparkStageStatus                                   MetricConfig `mapstructure:"spark.stage.status"`
	SparkStageTaskActive                               MetricConfig `mapstructure:"spark.stage.task.active"`
	SparkStageTaskResult                               MetricConfig `mapstructure:"spark.stage.task.result"`
	SparkStageTaskResultSize                           MetricConfig `mapstructure:"spark.stage.task.result_size"`
}

MetricsConfig provides config for apachespark metrics.

func DefaultMetricsConfig 

func DefaultMetricsConfig() MetricsConfig

type ResourceAttributeConfig 

type ResourceAttributeConfig struct {
	Enabled bool `mapstructure:"enabled"`
	// Experimental: MetricsInclude defines a list of filters for attribute values.
	// If the list is not empty, only metrics with matching resource attribute values will be emitted.
	MetricsInclude []filter.Config `mapstructure:"metrics_include"`
	// Experimental: MetricsExclude defines a list of filters for attribute values.
	// If the list is not empty, metrics with matching resource attribute values will not be emitted.
	// MetricsInclude has higher priority than MetricsExclude.
	MetricsExclude []filter.Config `mapstructure:"metrics_exclude"`
	// contains filtered or unexported fields
}

ResourceAttributeConfig provides common config for a particular resource attribute.

func (*ResourceAttributeConfig) Unmarshal added in v0.87.0 

func (rac *ResourceAttributeConfig) Unmarshal(parser *confmap.Conf) error

type ResourceAttributesConfig 

type ResourceAttributesConfig struct {
	SparkApplicationID   ResourceAttributeConfig `mapstructure:"spark.application.id"`
	SparkApplicationName ResourceAttributeConfig `mapstructure:"spark.application.name"`
	SparkExecutorID      ResourceAttributeConfig `mapstructure:"spark.executor.id"`
	SparkJobID           ResourceAttributeConfig `mapstructure:"spark.job.id"`
	SparkStageAttemptID  ResourceAttributeConfig `mapstructure:"spark.stage.attempt.id"`
	SparkStageID         ResourceAttributeConfig `mapstructure:"spark.stage.id"`
}

ResourceAttributesConfig provides config for apachespark resource attributes.

func DefaultResourceAttributesConfig 

func DefaultResourceAttributesConfig() ResourceAttributesConfig

type ResourceBuilder added in v0.82.0 

type ResourceBuilder struct {
	// contains filtered or unexported fields
}

ResourceBuilder is a helper struct to build resources predefined in metadata.yaml. The ResourceBuilder is not thread-safe and must not to be used in multiple goroutines.

func NewResourceBuilder added in v0.82.0 

func NewResourceBuilder(rac ResourceAttributesConfig) *ResourceBuilder

NewResourceBuilder creates a new ResourceBuilder. This method should be called on the start of the application.

func (*ResourceBuilder) Emit added in v0.82.0 

func (rb *ResourceBuilder) Emit() pcommon.Resource

Emit returns the built resource and resets the internal builder state.

func (*ResourceBuilder) SetSparkApplicationID added in v0.82.0 

func (rb *ResourceBuilder) SetSparkApplicationID(val string)

SetSparkApplicationID sets provided value as "spark.application.id" attribute.

func (*ResourceBuilder) SetSparkApplicationName added in v0.82.0 

func (rb *ResourceBuilder) SetSparkApplicationName(val string)

SetSparkApplicationName sets provided value as "spark.application.name" attribute.

func (*ResourceBuilder) SetSparkExecutorID added in v0.82.0 

func (rb *ResourceBuilder) SetSparkExecutorID(val string)

SetSparkExecutorID sets provided value as "spark.executor.id" attribute.

func (*ResourceBuilder) SetSparkJobID added in v0.82.0 

func (rb *ResourceBuilder) SetSparkJobID(val int64)

SetSparkJobID sets provided value as "spark.job.id" attribute.

func (*ResourceBuilder) SetSparkStageAttemptID added in v0.82.0 

func (rb *ResourceBuilder) SetSparkStageAttemptID(val int64)

SetSparkStageAttemptID sets provided value as "spark.stage.attempt.id" attribute.

func (*ResourceBuilder) SetSparkStageID added in v0.82.0 

func (rb *ResourceBuilder) SetSparkStageID(val int64)

SetSparkStageID sets provided value as "spark.stage.id" attribute.

type ResourceMetricsOption 

type ResourceMetricsOption interface {
	// contains filtered or unexported methods
}

ResourceMetricsOption applies changes to provided resource metrics.

func WithResource added in v0.82.0 

func WithResource(res pcommon.Resource) ResourceMetricsOption

WithResource sets the provided resource on the emitted ResourceMetrics. It's recommended to use ResourceBuilder to create the resource.

func WithStartTimeOverride 

func WithStartTimeOverride(start pcommon.Timestamp) ResourceMetricsOption

WithStartTimeOverride overrides start time for all the resource metrics data points. This option should be only used if different start time has to be set on metrics coming from different resources.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.