stats

Documentation

Overview

Package stats collects statistics about running Sif pipelines

Index

• type RunStatistics
  • func (rs *RunStatistics) EndPartition(sidx int, numRows int)
  • func (rs *RunStatistics) EndShuffle(sidx int)
  • func (rs *RunStatistics) EndStage(sidx int)
  • func (rs *RunStatistics) EndTransform(sidx int)
  • func (rs *RunStatistics) Finish()
  • func (rs *RunStatistics) GetCurrentPartitionProcessingTime() int64
  • func (rs *RunStatistics) GetNumPartitionsProcessed() []int64
  • func (rs *RunStatistics) GetNumRowsProcessed() []int64
  • func (rs *RunStatistics) GetRuntime() int64
  • func (rs *RunStatistics) GetStageRuntimes() []int64
  • func (rs *RunStatistics) GetStageShuffleRuntimes() []int64
  • func (rs *RunStatistics) GetStageTransformRuntimes() []int64
  • func (rs *RunStatistics) GetStartTime() time.Time
  • func (rs *RunStatistics) Start(numStages int)
  • func (rs *RunStatistics) StartPartition()
  • func (rs *RunStatistics) StartShuffle()
  • func (rs *RunStatistics) StartStage()
  • func (rs *RunStatistics) StartTransform()
  • func (rs *RunStatistics) ToMessage() *stats.MStatisticsResponse

Types

type RunStatistics

type RunStatistics struct {
	// contains filtered or unexported fields
}

RunStatistics contains statistics about a running Sif pipeline

func (*RunStatistics) EndPartition

func (rs *RunStatistics) EndPartition(sidx int, numRows int)

EndPartition tracks the end of the processing of a partition

func (*RunStatistics) EndShuffle

func (rs *RunStatistics) EndShuffle(sidx int)

EndShuffle tracks the end of the shuffle portion of a Stage

func (*RunStatistics) EndStage

func (rs *RunStatistics) EndStage(sidx int)

EndStage tracks the end of a Stage

func (*RunStatistics) EndTransform

func (rs *RunStatistics) EndTransform(sidx int)

EndTransform tracks the end of the transformation portion of a Stage

func (*RunStatistics) Finish

func (rs *RunStatistics) Finish()

Finish completes statistics tracking

func (*RunStatistics) GetCurrentPartitionProcessingTime

func (rs *RunStatistics) GetCurrentPartitionProcessingTime() int64

GetCurrentPartitionProcessingTime returns a rolling average of partition processing time

func (*RunStatistics) GetNumPartitionsProcessed

func (rs *RunStatistics) GetNumPartitionsProcessed() []int64

GetNumPartitionsProcessed returns the number of Partitions which have been processed so far, counted by stage

func (*RunStatistics) GetNumRowsProcessed

func (rs *RunStatistics) GetNumRowsProcessed() []int64

GetNumRowsProcessed returns the number of Rows which have been processed so far, counted by stage

func (*RunStatistics) GetRuntime

func (rs *RunStatistics) GetRuntime() int64

GetRuntime returns the running time of the Sif pipeline

func (*RunStatistics) GetStageRuntimes

func (rs *RunStatistics) GetStageRuntimes() []int64

GetStageRuntimes returns all recorded stage runtimes, from the most recent run of each Stage

func (*RunStatistics) GetStageShuffleRuntimes

func (rs *RunStatistics) GetStageShuffleRuntimes() []int64

GetStageShuffleRuntimes returns all recorded stage shuffle-phase runtimes, from the most recent run of each Stage

func (*RunStatistics) GetStageTransformRuntimes

func (rs *RunStatistics) GetStageTransformRuntimes() []int64

GetStageTransformRuntimes returns all recorded stage transform-phase runtimes, from the most recent run of each Stage

func (*RunStatistics) GetStartTime

func (rs *RunStatistics) GetStartTime() time.Time

GetStartTime returns the start time of the Sif pipeline

func (*RunStatistics) Start

func (rs *RunStatistics) Start(numStages int)

Start triggers statistics tracking, if it hasn't been started already

func (*RunStatistics) StartPartition

func (rs *RunStatistics) StartPartition()

StartPartition tracks the beginning of the processing of a partition

func (*RunStatistics) StartShuffle

func (rs *RunStatistics) StartShuffle()

StartShuffle tracks the beginning of the shuffle portion of a Stage

func (*RunStatistics) StartStage

func (rs *RunStatistics) StartStage()

StartStage tracks the beginning of a new Stage

func (*RunStatistics) StartTransform

func (rs *RunStatistics) StartTransform()

StartTransform tracks the beginning of the transformation portion of a Stage

func (*RunStatistics) ToMessage

func (rs *RunStatistics) ToMessage() *stats.MStatisticsResponse

ToMessage converts this struct into a protobuf message