generator

package
v0.0.0-...-c74e22e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 22, 2023 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// The size of the window of pending id ack.
	AcknowledgedWindowSize = int64(1 << 20)
	// The mask to use to turn an id into a slot in window.
	AcknowledgedWindowMask = AcknowledgedWindowSize - 1
)
View Source 
const (
	FNVOffsetBasis32 = uint32(0x811c9dc5)
	FNVPrime32       = uint32(16777619)
)
View Source 
const (
	FNVOffsetBasis64 = uint64(0xCBF29CE484222325)
	FNVPrime64       = uint64(1099511628211)
)
View Source 
const (
	ZipfianConstant = float64(0.99)
)

Variables

View Source 
var (
	Zetan               = float64(26.46902820178302)
	UsedZipfianConstant = float64(0.99)
	ItemCount           = float64(10000000000)
)

Functions

func FNVHash32 

func FNVHash32(value uint32) uint32

32 bit FNV hash. Refer to http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash

func FNVHash64 

func FNVHash64(value uint64) uint64

64 bit FNV hash. Refer to http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash

func Hash 

func Hash(value int64) uint64

Hash a integer value.

func NewErrorf 

func NewErrorf(format string, args ...interface{}) error

func NextFloat64 

func NextFloat64() float64

Return a random float64 value.

func NextInt64 

func NextInt64(n int64) int64

Return a random int64 value.

Types

type AcknowledgedCounterGenerator 

type AcknowledgedCounterGenerator struct {
	*CounterGenerator
	// contains filtered or unexported fields
}

A CounterGenerator that reports generated integers via LastInt() only after they have been acknowledged.

func NewAcknowledgedCounterGenerator 

func NewAcknowledgedCounterGenerator(startCount int64) *AcknowledgedCounterGenerator

Create a counter that starts at startCount.

func (*AcknowledgedCounterGenerator) Acknowledge 

func (self *AcknowledgedCounterGenerator) Acknowledge(value int64)

Make a generated counter value available via LastInt().

func (*AcknowledgedCounterGenerator) LastInt 

func (self *AcknowledgedCounterGenerator) LastInt() int64

In this generator, the highest acknowledged counter value (as opposed to the highest generated counter value).

func (*AcknowledgedCounterGenerator) LastString 

func (self *AcknowledgedCounterGenerator) LastString() string

type ConstantIntegerGenerator 

type ConstantIntegerGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

ConstantIntegerGenerator is a trivial integer generator that always returns the same value.

func NewConstantIntegerGenerator 

func NewConstantIntegerGenerator(i int64) *ConstantIntegerGenerator

func (*ConstantIntegerGenerator) Mean 

func (self *ConstantIntegerGenerator) Mean() float64

func (*ConstantIntegerGenerator) NextInt 

func (self *ConstantIntegerGenerator) NextInt() int64

func (*ConstantIntegerGenerator) NextString 

func (self *ConstantIntegerGenerator) NextString() string

type CounterGenerator 

type CounterGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

Generates a sequence of integers 0, 1, ...

func NewCounterGenerator 

func NewCounterGenerator(startCount int64) *CounterGenerator

Create a counter that starts at startCount.

func (*CounterGenerator) Mean 

func (self *CounterGenerator) Mean() float64

func (*CounterGenerator) NextInt 

func (self *CounterGenerator) NextInt() int64

If the generator returns numeric(integer) values, return the next value as an int. Default is to return -1, which is appropriate for generators that do not return numeric values.

func (*CounterGenerator) NextString 

func (self *CounterGenerator) NextString() string

type DiscreteGenerator 

type DiscreteGenerator struct {
	// contains filtered or unexported fields
}

Generates a distribution by choosing from a discrete set of values.

func NewDiscreteGenerator 

func NewDiscreteGenerator() *DiscreteGenerator

func (*DiscreteGenerator) AddValue 

func (self *DiscreteGenerator) AddValue(weight float64, value string)

func (*DiscreteGenerator) LastString 

func (self *DiscreteGenerator) LastString() string

Return the previous string generated by the distribution; e.g. returned from the last NextString() call. Calling LastString() should not advance the distribution or have any side effects. If NextString() has not yet been called, LastString() should return something reasonable.

func (*DiscreteGenerator) NextString 

func (self *DiscreteGenerator) NextString() string

Generate the next string in the distribution.

type ExponentialGenerator 

type ExponentialGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

A generator of an Exponential distribution. It produces a sequence of time intervals(integers) according to an exponential distribution. Smaller intervals are more frequent than larger ones, and there is no bound on the length of an interval. When you construct an instance of this class, you specify a parameter gamma, which corresponds to the rate at which events occur. Alternatively, 1/gamma is the average length of an interval.

func NewExponentialGenerator 

func NewExponentialGenerator(percentile, theRange float64) *ExponentialGenerator

func NewExponentialGeneratorByMean 

func NewExponentialGeneratorByMean(mean float64) *ExponentialGenerator

Create an exponential generator with a mean arrival rate of gamma. (And half life of 1/gamma).

func (*ExponentialGenerator) Mean 

func (self *ExponentialGenerator) Mean() float64

func (*ExponentialGenerator) NextInt 

func (self *ExponentialGenerator) NextInt() int64

Generate the next item. This distribution will be skewed toward lower integers; e.g. 0 will be the most popular, 1 the next most popular, etc.

func (*ExponentialGenerator) NextString 

func (self *ExponentialGenerator) NextString() string

type FileGenerator 

type FileGenerator struct {
	// contains filtered or unexported fields
}

A generator, whose sequence is the lines of a file.

func NewFileGenerator 

func NewFileGenerator(filename string) (*FileGenerator, error)

Create a FileGenerator with the given file.

func (*FileGenerator) Close 

func (self *FileGenerator) Close() error

Close the file.

func (*FileGenerator) LastString 

func (self *FileGenerator) LastString() string

Return the previous read line.

func (*FileGenerator) NextString 

func (self *FileGenerator) NextString() string

Return the next string of the sequence, which is the next line of the file.

func (*FileGenerator) ReloadFile 

func (self *FileGenerator) ReloadFile() error

Reopen the file to reuse values.

type Generator 

type Generator interface {

	// NextString generates the next string in the distribution.
	NextString() string

	// LastString returns the previous string generated by the distribution;
	// e.g., returned from the last NextString() call.
	// Calling LastString() should not advance the distribution or
	// have any side effects. If NextString() has not yet been called,
	// LastString() should return something reasonable.
	LastString() string
}

Generator defines the interface that generates a sequence of string values, following some distribution (Uniform, Zipfian, Sequential, etc.)

type HistogramGenerator 

type HistogramGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

Generate integers according to a histogram distribution. The histogram buckets are of width one, but the values are multiplied by a block size. Therefore, instead of drawing sizes uniformly at random within each bucket, we always draw the largest value in the current bucket, so the value drawn is always a multiple of blockSize. The minimum value this distribution returns is blockSize(not zero).

func NewHistogramGenerator 

func NewHistogramGenerator(buckets []int64, blockSize int64) *HistogramGenerator

func NewHistogramGeneratorFromFile 

func NewHistogramGeneratorFromFile(file string) (*HistogramGenerator, error)

func (*HistogramGenerator) Mean 

func (self *HistogramGenerator) Mean() float64

func (*HistogramGenerator) NextInt 

func (self *HistogramGenerator) NextInt() int64

func (*HistogramGenerator) NextString 

func (self *HistogramGenerator) NextString() string

type HotspotIntegerGenerator 

type HotspotIntegerGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

Generate integers resembling a hotspot distribution where x% of operations access y% of data items. The parameters specify the bounds for the numbers, the percentage of the interval which comprises the hot set and the percentage of operations that access the hot set. Numbers of the host set are always smaller than any number in the cold set. Elements from the hot set and the cold set are chosen using a uniform distribution.

func NewHotspotIntegerGenerator 

func NewHotspotIntegerGenerator(
	lowerBound, upperBound int64,
	hotsetFraction, hotOpnFraction float64) *HotspotIntegerGenerator

Create a generator for hotspot distribution.

func (*HotspotIntegerGenerator) GetHotOpnFraction 

func (self *HotspotIntegerGenerator) GetHotOpnFraction() float64

func (*HotspotIntegerGenerator) GetHotsetFraction 

func (self *HotspotIntegerGenerator) GetHotsetFraction() float64

func (*HotspotIntegerGenerator) GetLowerBound 

func (self *HotspotIntegerGenerator) GetLowerBound() int64

func (*HotspotIntegerGenerator) GetUpperBound 

func (self *HotspotIntegerGenerator) GetUpperBound() int64

func (*HotspotIntegerGenerator) Mean 

func (self *HotspotIntegerGenerator) Mean() float64

func (*HotspotIntegerGenerator) NextInt 

func (self *HotspotIntegerGenerator) NextInt() int64

func (*HotspotIntegerGenerator) NextString 

func (self *HotspotIntegerGenerator) NextString() string

type IntegerGenerator 

type IntegerGenerator interface {
	Generator
	// NextInt returns the next value as an int. When overriding this method,
	// be sure to call setLastString() properly, or the LastString() call
	// won't work.
	NextInt() int64

	// LastInt returns the previous int generated by the distribution.
	// This call is unique to IntegerGenerator implementation struct, and
	// assumes all implementation of this interface always return ints for
	// NextInt (e.g. not arbitrary values).
	LastInt() int64

	// Mean returns the expected value(mean) of the values this generator will
	// return.
	Mean() float64
}

IntegerGenerator is a generator capable of generating integers and strings.

type IntegerGeneratorBase 

type IntegerGeneratorBase struct {
	// contains filtered or unexported fields
}

IntegerGeneratorBase is a parent class for all IntegerGenerator subclasses.

func NewIntegerGeneratorBase 

func NewIntegerGeneratorBase(last int64) *IntegerGeneratorBase

func (*IntegerGeneratorBase) LastInt 

func (self *IntegerGeneratorBase) LastInt() int64

func (*IntegerGeneratorBase) LastString 

func (self *IntegerGeneratorBase) LastString() string

func (*IntegerGeneratorBase) NextString 

func (self *IntegerGeneratorBase) NextString(g IntegerGenerator) string

NextString generates the next string in the distribution.

func (*IntegerGeneratorBase) SetLastInt 

func (self *IntegerGeneratorBase) SetLastInt(value int64)

SetLastInt sets the last value to be generated. IntegerGenerator subclasses must use this call to properly set the last int value, or the LastString() and LastInt() calls won't work.

type Pair 

type Pair struct {
	Weight float64
	Value  string
}

type ScrambledZipfianGenerator 

type ScrambledZipfianGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

A generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either by specifying an itemCount(so that the sequence is of items from 0 to itemCount-1) or by specifying a min and a max (so that the sequence is of items from min to max inclusive). After you construct the instance, you can change the number of items by calling NextInt(itemCount) or Next(itemCount). Unlike ZipfianGenerator, this class scatters the "popular" items across the item space. Use this, instead of ZipfianGenerator, if you don't want the head of the distribution(the popular items) clustered together.

func NewScrambledZipfianGenerator 

func NewScrambledZipfianGenerator(min, max int64) *ScrambledZipfianGenerator

Create a zipfian generator for items between min and max(inclusive).

func NewScrambledZipfianGeneratorByItems 

func NewScrambledZipfianGeneratorByItems(items int64) *ScrambledZipfianGenerator

Create a zipfian generator for the specified number of items.

func NewScrambledZipfianGeneratorConstant 

func NewScrambledZipfianGeneratorConstant(min, max int64, constant float64) *ScrambledZipfianGenerator

Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant. If you use a zipfian constant other than 0.99, this will take a long time complete because we need to recompute zeta.

func (*ScrambledZipfianGenerator) Mean 

func (self *ScrambledZipfianGenerator) Mean() float64

Since the values are scrambed (hopefully uniformly), the mean is simply the middle of the range.

func (*ScrambledZipfianGenerator) Next 

func (self *ScrambledZipfianGenerator) Next() int64

return the next item in the sequence.

func (*ScrambledZipfianGenerator) NextInt 

func (self *ScrambledZipfianGenerator) NextInt() int64

Return the next int in the sequence.

func (*ScrambledZipfianGenerator) NextString 

func (self *ScrambledZipfianGenerator) NextString() string

type SkewedLatestGenerator 

type SkewedLatestGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

Generate a popularity distribution of items, skewed to favor recent items significantly more than older items.

func NewSkewedLatestGenerator 

func NewSkewedLatestGenerator(basis *CounterGenerator) *SkewedLatestGenerator

func (*SkewedLatestGenerator) Mean 

func (self *SkewedLatestGenerator) Mean() float64

func (*SkewedLatestGenerator) NextInt 

func (self *SkewedLatestGenerator) NextInt() int64

Generate the next value in the distribution, skewed Zipfian favoring the items most recently returned by the basis generator.

func (*SkewedLatestGenerator) NextString 

func (self *SkewedLatestGenerator) NextString() string

type UniformGenerator 

type UniformGenerator struct {
	// contains filtered or unexported fields
}

An expression that generates a random integer in the specified range.

func NewUniformGenerator 

func NewUniformGenerator(values []string) *UniformGenerator

Create a generator that will return strings from the specified set uniformly randomly.

func (*UniformGenerator) LastString 

func (self *UniformGenerator) LastString() string

Return the previous string generated by the distribution; e.g., returned from the last NextString() call. Calling LastString() should not advance the distribution or have any side effects. If NextString() has not yet been called, LastString() should return something reasonable.

func (*UniformGenerator) NextString 

func (self *UniformGenerator) NextString() string

Generate the next string in the distribution.

type UniformIntegerGenerator 

type UniformIntegerGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

Generate integers randomly uniform from an interval.

func NewUniformIntegerGenerator 

func NewUniformIntegerGenerator(lowerBound, upperBound int64) *UniformIntegerGenerator

Create a generator that will return integers uniformly randomly from the interval[lowerBound, upperBound] inclusive(that is, lowerBound and upperBound are possible values)

func (*UniformIntegerGenerator) Mean 

func (self *UniformIntegerGenerator) Mean() float64

func (*UniformIntegerGenerator) NextInt 

func (self *UniformIntegerGenerator) NextInt() int64

func (*UniformIntegerGenerator) NextString 

func (self *UniformIntegerGenerator) NextString() string

type ZipfianGenerator 

type ZipfianGenerator struct {
	*IntegerGeneratorBase
	// contains filtered or unexported fields
}

A generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either by specifying an itemcount (so that the sequence is of items from 0 to itemcount-1) or by specifying a min and a max (so that the sequence is of items from min to max inclusive). After you construct the instance, you can change the number of items by calling NextInt() or nextLong().

Note that the popular items will be clustered together, e.g. item 0 is the most popular, item 1 the second most popular, and so on (or min is the most popular, min+1 the next most popular, etc.) If you don't want this clustering, and instead want the popular items scattered throughout the item space, then use ScrambledZipfianGenerator instead.

Be aware: initializing this generator may take a long time if there are lots of items to choose from (e.g. over a minute for 100 million objects). This is because certain mathematical values need to be computed to properly generate a zipfian skew, and one of those values (zeta) is a sum sequence from 1 to n, where n is the itemCount. Note that if you increase the number of items in the set, we can compute a new zeta incrementally, so it should be fast unless you have added millions of items. However, if you decrease the number of items, we recompute zeta from scratch, so this can take a long time.

The algorithm used here is from "Quickly Generating Billion-Record Synthetic Databases", Jim Gray et al, SIGMOD 1994.

func NewZipfianGenerator 

func NewZipfianGenerator(
	min, max int64, zipfianConstant, zetan float64) *ZipfianGenerator

Create a zipfian generator for items between min and max(inclusive) for the specified zipfian constant, using the precomputed value of zeta.

func NewZipfianGeneratorByInterval 

func NewZipfianGeneratorByInterval(min, max int64) *ZipfianGenerator

Create a zipfian generator for items between min and max(inclusive).

func (*ZipfianGenerator) Mean 

func (self *ZipfianGenerator) Mean() float64

func (*ZipfianGenerator) Next 

func (self *ZipfianGenerator) Next(itemCount int64) int64

Generate the next item. this distribution will be skewed toward lower itegers; e.g. 0 will be the most popular, 1 the next most popular, etc.

func (*ZipfianGenerator) NextInt 

func (self *ZipfianGenerator) NextInt() int64

Return the next value, skewed by the zipfian distribution. The 0th item will be the most popular, followed by the 1st, followed by the 2nd, etc. (or, if min != 0, the min-th item is the most popular, the min+1th item the next most popular, etc.) If you want the popular items scattered throughout the item space, use ScrambledZipfianGenerator instead.

func (*ZipfianGenerator) NextLong 

func (self *ZipfianGenerator) NextLong() int64

Return the next value, skewed by the zipfian distribution. The 0th item will be the most popular, followed by the 1st, followed by the 2nd, etc. (same as NextInt())

func (*ZipfianGenerator) NextString 

func (self *ZipfianGenerator) NextString() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.