bench

package
v0.3.30-0...-7ef3155 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 1, 2021 License: AGPL-3.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Benchmark 

type Benchmark interface {
	// Prepare for the benchmark run
	Prepare(ctx context.Context) error

	// Start will execute the main benchmark.
	// Operations should begin executing when the start channel is closed.
	Start(ctx context.Context, wait chan struct{}) (Operations, error)

	// Clean up after the benchmark run.
	Cleanup(ctx context.Context)

	// Common returns the common parameters.
	GetCommon() *Common
}

type CmpSegment 

type CmpSegment struct {
	Before, After    *Segment
	ThroughputPerSec float64
	ObjPerSec        float64
	OpsEndedPerSec   float64
}

CmpSegment is s comparisons between two segments.

func (*CmpSegment) Compare 

func (c *CmpSegment) Compare(before, after Segment)

Compare sets c to a comparison between before and after.

func (CmpSegment) String 

func (c CmpSegment) String() string

String returns a string representation of the segment comparison.

type Collector 

type Collector struct {
	// contains filtered or unexported fields
}

func NewCollector 

func NewCollector() *Collector

func (*Collector) AutoTerm 

func (c *Collector) AutoTerm(ctx context.Context, op string, threshold float64, wantSamples, splitInto int, minDur time.Duration) context.Context

AutoTerm will check if throughput is within 'threshold' (0 -> ) for wantSamples, when the current operations are split into 'splitInto' segments. The minimum duration for the calculation can be set as well. Segment splitting may cause less than this duration to be used.

func (*Collector) Close 

func (c *Collector) Close() Operations

func (*Collector) Receiver 

func (c *Collector) Receiver() chan<- Operation

type Common 

type Common struct {
	Client func() (cl *minio.Client, done func())

	Concurrency int
	Source      func() generator.Source
	Bucket      string
	Location    string

	// Running in client mode.
	ClientMode bool
	// Clear bucket before benchmark
	Clear           bool
	PrepareProgress chan float64
	// Does destination support versioning?
	Versioned bool

	// Auto termination is set when this is > 0.
	AutoTermDur   time.Duration
	AutoTermScale float64

	// Default Put options.
	PutOpts minio.PutObjectOptions

	// Error should log an error similar to fmt.Print(data...)
	Error func(data ...interface{})
}

Common contains common benchmark parameters.

func (*Common) ErrorF 

func (c *Common) ErrorF(format string, data ...interface{})

func (*Common) GetCommon 

func (c *Common) GetCommon() *Common

type Comparison 

type Comparison struct {
	Op string

	TTFB *TTFBCmp

	Average CmpSegment
	Fastest CmpSegment
	Median  CmpSegment
	Slowest CmpSegment
}

Comparison is a comparison between two benchmarks.

func Compare 

func Compare(before, after Operations, analysis time.Duration, allThreads bool) (*Comparison, error)

Compare compares operations of a single operation type.

type Delete 

type Delete struct {
	CreateObjects int
	BatchSize     int
	Collector     *Collector

	Common
	// contains filtered or unexported fields
}

Delete benchmarks delete speed.

func (*Delete) Cleanup 

func (d *Delete) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Delete) Prepare 

func (d *Delete) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*Delete) Start 

func (d *Delete) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type Get 

type Get struct {
	CreateObjects int
	RandomRanges  bool
	Collector     *Collector

	// Default Get options.
	GetOpts minio.GetObjectOptions
	Common
	// contains filtered or unexported fields
}

Get benchmarks download speed.

func (*Get) Cleanup 

func (g *Get) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Get) Prepare 

func (g *Get) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*Get) Start 

func (g *Get) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type List 

type List struct {
	CreateObjects int
	NoPrefix      bool
	Collector     *Collector

	Common
	// contains filtered or unexported fields
}

List benchmarks listing speed.

func (*List) Cleanup 

func (d *List) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*List) Prepare 

func (d *List) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*List) Start 

func (d *List) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type Mixed 

type Mixed struct {
	CreateObjects int
	Collector     *Collector
	Dist          *MixedDistribution

	GetOpts  minio.GetObjectOptions
	StatOpts minio.StatObjectOptions
	Common
}

Mixed benchmarks mixed operations all inclusive.

func (*Mixed) Cleanup 

func (g *Mixed) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Mixed) Prepare 

func (g *Mixed) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*Mixed) Start 

func (g *Mixed) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type MixedDistribution 

type MixedDistribution struct {
	// Operation -> distribution.
	Distribution map[string]float64
	// contains filtered or unexported fields
}

MixedDistribution keeps track of operation distribution and currently available objects.

func (*MixedDistribution) Generate 

func (m *MixedDistribution) Generate(allocObjs int) error

func (*MixedDistribution) Objects 

func (m *MixedDistribution) Objects() generator.Objects

type Operation 

type Operation struct {
	OpType    string     `json:"type"`
	ObjPerOp  int        `json:"ops"`
	Start     time.Time  `json:"start"`
	FirstByte *time.Time `json:"first_byte"`
	End       time.Time  `json:"end"`
	Err       string     `json:"err"`
	Size      int64      `json:"size"`
	File      string     `json:"file"`
	Thread    uint16     `json:"thread"`
	ClientID  string     `json:"client_id"`
	Endpoint  string     `json:"endpoint"`
}

func (Operation) Aggregate 

func (o Operation) Aggregate(s *Segment) (done bool)

Aggregate the operation into segment if it belongs there. Done returns true if operation is starting after segment ended.

func (Operation) BytesPerSec 

func (o Operation) BytesPerSec() Throughput

func (Operation) Duration 

func (o Operation) Duration() time.Duration

Duration returns the duration o.End-o.Start

func (Operation) String 

func (o Operation) String() string

func (Operation) TTFB 

func (o Operation) TTFB() time.Duration

TTFB returns the time to first byte or 0 if nothing was recorded.

type Operations 

type Operations []Operation

func OperationsFromCSV 

func OperationsFromCSV(r io.Reader, analyzeOnly bool, offset, limit int, log func(msg string, v ...interface{})) (Operations, error)

OperationsFromCSV will load operations from CSV.

func (Operations) ActiveTimeRange 

func (o Operations) ActiveTimeRange(allThreads bool) (start, end time.Time)

ActiveTimeRange returns the "active" time range. All threads must have completed at least one request and the last start time of any thread. If there is no active time range both values will be the same.

func (Operations) AvgDuration 

func (o Operations) AvgDuration() time.Duration

AvgDuration returns the average operation duration.

func (Operations) AvgSize 

func (o Operations) AvgSize() int64

AvgSize returns the average operation size.

func (Operations) ByEndpoint 

func (o Operations) ByEndpoint() map[string]Operations

ByOp separates the operations by endpoint.

func (Operations) ByOp 

func (o Operations) ByOp() map[string]Operations

ByOp separates the operations by op.

func (Operations) CSV 

func (o Operations) CSV(w io.Writer, comment string) error

CSV will write the operations to w as CSV. The comment, if any, is written at the end of the file, each line prefixed with '# '.

func (Operations) Clients 

func (o Operations) Clients() int

Clients returns the number of clients.

func (Operations) Clone 

func (o Operations) Clone() Operations

Clone the operations.

func (Operations) Duration 

func (o Operations) Duration() time.Duration

Duration returns the full duration from start of first operation to end of the last.

func (Operations) Endpoints 

func (o Operations) Endpoints() []string

Endpoints returns the endpoints as a sorted slice.

func (Operations) Errors 

func (o Operations) Errors() []string

Errors returns the errors found.

func (Operations) FilterByEndpoint 

func (o Operations) FilterByEndpoint(endpoint string) Operations

FilterByEndpoint returns operations run against a specific endpoint. Always returns a copy.

func (Operations) FilterByHasTTFB 

func (o Operations) FilterByHasTTFB(hasTTFB bool) Operations

FilterByHasTTFB returns operations that has or has not time to first byte.

func (Operations) FilterByOp 

func (o Operations) FilterByOp(opType string) Operations

FilterByOp returns operations of a specific type.

func (Operations) FilterErrors 

func (o Operations) FilterErrors() Operations

Errors returns the errors found.

func (Operations) FilterFirst 

func (o Operations) FilterFirst() Operations

FilterFirst returns the first operation on any file.

func (Operations) FilterInsideRange 

func (o Operations) FilterInsideRange(start, end time.Time) Operations

FilterInsideRange returns operations that are inside the specified time range. Operations starting before start or ending after end are discarded.

func (Operations) FilterSuccessful 

func (o Operations) FilterSuccessful() Operations

FilterSuccessful returns the successful requests.

func (Operations) FirstObjPerOp 

func (o Operations) FirstObjPerOp() int

FirstObjPerOp returns the number of objects per operation of the first entry, or 0 if there are no ops.

func (Operations) FirstObjSize 

func (o Operations) FirstObjSize() int64

FirstObjSizeType returns the size of the first entry, 0 if there are no ops.

func (Operations) FirstOpType 

func (o Operations) FirstOpType() string

FirstOpType returns the type of the first entry empty string if there are no ops.

func (Operations) HasError 

func (o Operations) HasError() bool

HasError returns whether one or more operations failed.

func (Operations) Hosts 

func (o Operations) Hosts() int

Hosts returns the number of servers.

func (Operations) IsMixed 

func (o Operations) IsMixed() bool

IsMixed returns true if different operation types are overlapping.

func (Operations) IsMultiTouch 

func (o Operations) IsMultiTouch() bool

IsMultiTouch returns true if the same files are touched multiple times.

func (Operations) Median 

func (o Operations) Median(m float64) Operation

Median returns the m part median of the assumed sorted list of operations. m is clamped to the range 0 -> 1.

func (Operations) MinMaxSize 

func (o Operations) MinMaxSize() (min, max int64)

MinMaxSize returns the minimum and maximum operation sizes.

func (Operations) MultipleSizes 

func (o Operations) MultipleSizes() bool

MultipleSizes returns whether there are multiple operation sizes.

func (Operations) OffsetThreads 

func (o Operations) OffsetThreads(n uint16) uint16

OffsetThreads adds an offset to all thread ids and returns the next thread number.

func (Operations) OpThroughput 

func (o Operations) OpThroughput() Throughput

OpThroughput returns the average throughput in B/s.

func (Operations) OpTypes 

func (o Operations) OpTypes() []string

OpTypes returns a list of the operation types in the order they appear if not overlapping or in alphabetical order if mixed.

func (Operations) Segment 

func (o Operations) Segment(so SegmentOptions) Segments

Segment will segment the operations o. Operations should be of the same type. Operations will be sorted by start time.

func (Operations) SetClientID 

func (o Operations) SetClientID(id string)

SetClientID will set the client ID for all operations.

func (Operations) SingleSizeSegment 

func (o Operations) SingleSizeSegment() SizeSegment

func (Operations) SortByDuration 

func (o Operations) SortByDuration()

SortByDuration will sort the operations by duration taken to complete. Fastest operations first.

func (Operations) SortByEndTime 

func (o Operations) SortByEndTime()

SortByEndTime will sort the operations by end time. Earliest operations first.

func (Operations) SortByStartTime 

func (o Operations) SortByStartTime()

SortByStartTime will sort the operations by start time. Earliest operations first.

func (Operations) SortByTTFB 

func (o Operations) SortByTTFB()

SortByTTFB sorts by time to first byte. Smallest first.

func (Operations) SortByThroughput 

func (o Operations) SortByThroughput()

SortByThroughput will sort the operations by throughput. Fastest operations first.

func (Operations) SplitSizes 

func (o Operations) SplitSizes(minShare float64) []SizeSegment

SplitSizes will return log10 separated data. Specify the share of requests that must be in a segment to return it.

func (Operations) TTFB 

func (o Operations) TTFB(start, end time.Time) TTFB

TTFB returns time to first byte stats for all operations completely within the time segment.

func (Operations) Threads 

func (o Operations) Threads() int

Threads returns the number of threads found.

func (Operations) TimeRange 

func (o Operations) TimeRange() (start, end time.Time)

TimeRange returns the full time range from start of first operation to end of the last.

func (Operations) Total 

func (o Operations) Total(allThreads bool) Segment

Total will return the total of active operations. See ActiveTimeRange how this is determined. Specify whether one operation for all threads should be skipped or just a single.

type Put 

type Put struct {
	Common
	// contains filtered or unexported fields
}

Put benchmarks upload speed.

func (*Put) Cleanup 

func (u *Put) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Put) Prepare 

func (u *Put) Prepare(ctx context.Context) error

Prepare will create an empty bucket ot delete any content already there.

func (*Put) Start 

func (u *Put) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type Segment 

type Segment struct {
	OpType     string    `json:"op"`
	Host       string    `json:"host"`
	ObjsPerOp  int       `json:"objects_per_op"`
	TotalBytes int64     `json:"total_bytes"`
	FullOps    int       `json:"full_ops"`
	PartialOps int       `json:"partial_ops"`
	OpsStarted int       `json:"ops_started"`
	OpsEnded   int       `json:"ops_ended"`
	Objects    float64   `json:"objects"`
	Errors     int       `json:"errors"`
	Start      time.Time `json:"start"`
	EndsBefore time.Time `json:"ends_before"`
}

A Segment represents totals of operations in a specific time segment starting at Start and ending before EndsBefore.

func (Segment) CSV 

func (s Segment) CSV(w *csv.Writer, idx int) error

CSV writes a CSV representation of the segment to the supplied writer.

func (Segment) Duration 

func (s Segment) Duration() time.Duration

String returns a string representation of the segment

func (Segment) ShortString 

func (s Segment) ShortString() string

ShortString returns a string representation of the segment without ops ended/s.

func (Segment) SpeedPerSec 

func (s Segment) SpeedPerSec() (mib, ops, objs float64)

SpeedPerSec returns mb/s for the segment and the ops ended per second.

func (Segment) String 

func (s Segment) String() string

String returns a string representation of the segment

type SegmentOptions 

type SegmentOptions struct {
	From           time.Time
	PerSegDuration time.Duration
	AllThreads     bool
	MultiOp        bool
}

SegmentOptions describe options used to segment operations.

type Segments 

type Segments []Segment

Segments is a slice of segment elements.

func (Segments) CSV 

func (s Segments) CSV(w io.Writer) error

CSV writes segments to a supplied writer as CSV data.

func (Segments) Clone 

func (s Segments) Clone() Segments

Clone returns a copy of the segment elements.

func (Segments) Median 

func (s Segments) Median(m float64) Segment

Median returns the m part median. m is clamped to the range 0 -> 1.

func (Segments) Print 

func (s Segments) Print(w io.Writer) error

Print segments to a supplied writer.

func (Segments) SortByObjsPerSec 

func (s Segments) SortByObjsPerSec()

SortByOpsEnded sorts the segments by the number of distributed objects processed. Lowest first.

func (Segments) SortByOpsEnded 

func (s Segments) SortByOpsEnded()

SortByOpsEnded sorts the segments by the number of ops ended in segment. Lowest first.

func (Segments) SortByThroughput 

func (s Segments) SortByThroughput()

SortByThroughput sorts the segments by throughput. Slowest first.

func (Segments) SortByTime 

func (s Segments) SortByTime()

SortByTime sorts the segments by start time. Earliest first.

type Select 

type Select struct {
	CreateObjects int
	Collector     *Collector

	// Default Select options.
	SelectOpts minio.SelectObjectOptions
	Common
	// contains filtered or unexported fields
}

Select benchmarks download speed.

func (*Select) Cleanup 

func (g *Select) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Select) Prepare 

func (g *Select) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*Select) Start 

func (g *Select) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type SizeSegment 

type SizeSegment struct {
	Smallest      int64
	SmallestLog10 int
	Biggest       int64
	BiggestLog10  int
	Ops           Operations
}

SizeSegment is a size segment.

func (SizeSegment) SizeString 

func (s SizeSegment) SizeString() string

SizeString returns the size as a string.

func (SizeSegment) SizesString 

func (s SizeSegment) SizesString() (lo, hi string)

SizesString returns the lower and upper limit as strings.

type Stat 

type Stat struct {
	CreateObjects int
	Collector     *Collector

	// Default Stat options.
	StatOpts minio.StatObjectOptions
	Common
	// contains filtered or unexported fields
}

Stat benchmarks HEAD speed.

func (*Stat) Cleanup 

func (g *Stat) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Stat) Prepare 

func (g *Stat) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*Stat) Start 

func (g *Stat) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type TTFB 

type TTFB struct {
	Average time.Duration
	Worst   time.Duration
	Best    time.Duration
	Median  time.Duration
}

TTFB contains time to first byte stats.

func (TTFB) Compare 

func (t TTFB) Compare(after TTFB) *TTFBCmp

Compare will set t to the difference between before and after.

func (TTFB) String 

func (t TTFB) String() string

String returns a human printable version of the time to first byte.

type TTFBCmp 

type TTFBCmp struct {
	TTFB
	Before, After TTFB
}

TTFBCmp is a comparison between two TTFB runs.

func (*TTFBCmp) String 

func (t *TTFBCmp) String() string

String returns a human readable representation of the TTFB comparison.

type Throughput 

type Throughput float64

Throughput is the throughput as bytes/second.

func (Throughput) Float 

func (t Throughput) Float() float64

Float returns a rounded (to 0.1) float value of the throughput.

func (Throughput) String 

func (t Throughput) String() string

type Versioned 

type Versioned struct {
	CreateObjects int
	Collector     *Collector
	Dist          *VersionedDistribution

	GetOpts  minio.GetObjectOptions
	StatOpts minio.StatObjectOptions
	Common
}

Versioned benchmarks mixed operations all inclusive.

func (*Versioned) Cleanup 

func (g *Versioned) Cleanup(ctx context.Context)

Cleanup deletes everything uploaded to the bucket.

func (*Versioned) Prepare 

func (g *Versioned) Prepare(ctx context.Context) error

Prepare will create an empty bucket or delete any content already there and upload a number of objects.

func (*Versioned) Start 

func (g *Versioned) Start(ctx context.Context, wait chan struct{}) (Operations, error)

Start will execute the main benchmark. Operations should begin executing when the start channel is closed.

type VersionedDistribution 

type VersionedDistribution struct {
	// Operation -> distribution.
	Distribution map[string]float64
	// contains filtered or unexported fields
}

VersionedDistribution keeps track of operation distribution and currently available objects.

func (*VersionedDistribution) Generate 

func (m *VersionedDistribution) Generate(allocObjs int) error

Generate versioned objects.

func (*VersionedDistribution) Objects 

func (m *VersionedDistribution) Objects() generator.Objects

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.