y

package
v1.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 10, 2020 License: Apache-2.0 Imports: 16 Imported by: 433

Documentation

Index

Constants

View Source 
const (
	// Sync indicates that O_DSYNC should be set on the underlying file,
	// ensuring that data writes do not return until the data is flushed
	// to disk.
	Sync = 1 << iota
	// ReadOnly opens the underlying file on a read-only basis.
	ReadOnly
)

Variables

View Source 
var (
	// LSMSize has size of the LSM in bytes
	LSMSize *expvar.Map
	// VlogSize has size of the value log in bytes
	VlogSize *expvar.Map
	// PendingWrites tracks the number of pending writes.
	PendingWrites *expvar.Map

	// NumReads has cumulative number of reads
	NumReads *expvar.Int
	// NumWrites has cumulative number of writes
	NumWrites *expvar.Int
	// NumBytesRead has cumulative number of bytes read
	NumBytesRead *expvar.Int
	// NumBytesWritten has cumulative number of bytes written
	NumBytesWritten *expvar.Int
	// NumLSMGets is number of LMS gets
	NumLSMGets *expvar.Map
	// NumLSMBloomHits is number of LMS bloom hits
	NumLSMBloomHits *expvar.Map
	// NumGets is number of gets
	NumGets *expvar.Int
	// NumPuts is number of puts
	NumPuts *expvar.Int
	// NumBlockedPuts is number of blocked puts
	NumBlockedPuts *expvar.Int
	// NumMemtableGets is number of memtable gets
	NumMemtableGets *expvar.Int
)
View Source 
var (

	// CastagnoliCrcTable is a CRC32 polynomial table
	CastagnoliCrcTable = crc32.MakeTable(crc32.Castagnoli)
)
View Source 
var ErrEOF = errors.New("End of mapped region")

ErrEOF indicates an end of file when trying to read from a memory mapped file and encountering the end of slice.

View Source 
var (
	NoEventLog trace.EventLog = nilEventLog{}
)

Functions

func AssertTrue 

func AssertTrue(b bool)

AssertTrue asserts that b is true. Otherwise, it would log fatal.

func AssertTruef 

func AssertTruef(b bool, format string, args ...interface{})

AssertTruef is AssertTrue with extra info.

func Check 

func Check(err error)

Check logs fatal if err != nil.

func Check2 

func Check2(_ interface{}, err error)

Check2 acts as convenience wrapper around Check, using the 2nd argument as error.

func CompareKeys added in v0.9.0 

func CompareKeys(key1, key2 []byte) int

CompareKeys checks the key without timestamp and checks the timestamp if keyNoTs is same. a<timestamp> would be sorted higher than aa<timestamp> if we use bytes.compare All keys should have timestamp.

func Copy added in v0.9.0 

func Copy(a []byte) []byte

Copy copies a byte slice and returns the copied slice.

func CreateSyncedFile 

func CreateSyncedFile(filename string, sync bool) (*os.File, error)

CreateSyncedFile creates a new file (using O_EXCL), errors if it already existed.

func FileSync added in v1.6.0 

func FileSync(f *os.File) error

FileSync calls os.File.Sync with the right parameters. This function can be removed once we stop supporting Go 1.11 on MacOS.

More info: https://golang.org/issue/26650.

func FixedDuration added in v1.6.0 

func FixedDuration(d time.Duration) string

FixedDuration returns a string representation of the given duration with the hours, minutes, and seconds.

func KeyWithTs added in v0.9.0 

func KeyWithTs(key []byte, ts uint64) []byte

KeyWithTs generates a new key by appending ts to key.

func Madvise 

func Madvise(b []byte, readahead bool) error

Madvise uses the madvise system call to give advise about the use of memory when using a slice that is memory-mapped to a file. Set the readahead flag to false if page references are expected in random order.

func Mmap 

func Mmap(fd *os.File, writable bool, size int64) ([]byte, error)

Mmap uses the mmap system call to memory-map a file. If writable is true, memory protection of the pages is set so that they may be written to as well.

func Munmap 

func Munmap(b []byte) error

Munmap unmaps a previously mapped slice.

func OpenExistingFile added in v1.4.0 

func OpenExistingFile(filename string, flags uint32) (*os.File, error)

OpenExistingFile opens an existing file, errors if it doesn't exist.

func OpenSyncedFile 

func OpenSyncedFile(filename string, sync bool) (*os.File, error)

OpenSyncedFile creates the file if one doesn't exist.

func OpenTruncFile 

func OpenTruncFile(filename string, sync bool) (*os.File, error)

OpenTruncFile opens the file with O_RDWR | O_CREATE | O_TRUNC

func ParseKey added in v0.9.0 

func ParseKey(key []byte) []byte

ParseKey parses the actual key from the key bytes.

func ParseTs added in v0.9.0 

func ParseTs(key []byte) uint64

ParseTs parses the timestamp from the key bytes.

func SafeCopy added in v1.1.0 

func SafeCopy(a, src []byte) []byte

SafeCopy does append(a[:0], src...).

func SameKey added in v0.9.0 

func SameKey(src, dst []byte) bool

SameKey checks for key equality ignoring the version timestamp suffix.

func Wrap 

func Wrap(err error) error

Wrap wraps errors from external lib.

func Wrapf 

func Wrapf(err error, format string, args ...interface{}) error

Wrapf is Wrap with extra info.

Types

type Closer 

type Closer struct {
	// contains filtered or unexported fields
}

Closer holds the two things we need to close a goroutine and wait for it to finish: a chan to tell the goroutine to shut down, and a WaitGroup with which to wait for it to finish shutting down.

func NewCloser 

func NewCloser(initial int) *Closer

NewCloser constructs a new Closer, with an initial count on the WaitGroup.

func (*Closer) AddRunning 

func (lc *Closer) AddRunning(delta int)

AddRunning Add()'s delta to the WaitGroup.

func (*Closer) Done 

func (lc *Closer) Done()

Done calls Done() on the WaitGroup.

func (*Closer) HasBeenClosed 

func (lc *Closer) HasBeenClosed() <-chan struct{}

HasBeenClosed gets signaled when Signal() is called.

func (*Closer) Signal 

func (lc *Closer) Signal()

Signal signals the HasBeenClosed signal.

func (*Closer) SignalAndWait 

func (lc *Closer) SignalAndWait()

SignalAndWait calls Signal(), then Wait().

func (*Closer) Wait 

func (lc *Closer) Wait()

Wait waits on the WaitGroup. (It waits for NewCloser's initial value, AddRunning, and Done calls to balance out.)

type Iterator 

type Iterator interface {
	Next()
	Rewind()
	Seek(key []byte)
	Key() []byte
	Value() ValueStruct
	Valid() bool

	// All iterators should be closed so that file garbage collection works.
	Close() error
}

Iterator is an interface for a basic iterator.

type Slice 

type Slice struct {
	// contains filtered or unexported fields
}

Slice holds a reusable buf, will reallocate if you request a larger size than ever before. One problem is with n distinct sizes in random order it'll reallocate log(n) times.

func (*Slice) Resize 

func (s *Slice) Resize(sz int) []byte

Resize reuses the Slice's buffer (or makes a new one) and returns a slice in that buffer of length sz.

type Throttle added in v1.6.0 

type Throttle struct {
	// contains filtered or unexported fields
}

Throttle allows a limited number of workers to run at a time. It also provides a mechanism to check for errors encountered by workers and wait for them to finish.

func NewThrottle added in v1.6.0 

func NewThrottle(max int) *Throttle

NewThrottle creates a new throttle with a max number of workers.

func (*Throttle) Do added in v1.6.0 

func (t *Throttle) Do() error

Do should be called by workers before they start working. It blocks if there are already maximum number of workers working. If it detects an error from previously Done workers, it would return it.

func (*Throttle) Done added in v1.6.0 

func (t *Throttle) Done(err error)

Done should be called by workers when they finish working. They can also pass the error status of work done.

func (*Throttle) Finish added in v1.6.0 

func (t *Throttle) Finish() error

Finish waits until all workers have finished working. It would return any error passed by Done. If Finish is called multiple time, it will wait for workers to finish only once(first time). From next calls, it will return same error as found on first call.

type ValueStruct 

type ValueStruct struct {
	Meta      byte
	UserMeta  byte
	ExpiresAt uint64
	Value     []byte

	Version uint64 // This field is not serialized. Only for internal usage.
}

ValueStruct represents the value info that can be associated with a key, but also the internal Meta field.

func (*ValueStruct) Decode added in v0.9.0 

func (v *ValueStruct) Decode(b []byte)

Decode uses the length of the slice to infer the length of the Value field.

func (*ValueStruct) Encode 

func (v *ValueStruct) Encode(b []byte)

Encode expects a slice of length at least v.EncodedSize().

func (*ValueStruct) EncodeTo added in v1.0.0 

func (v *ValueStruct) EncodeTo(buf *bytes.Buffer)

EncodeTo should be kept in sync with the Encode function above. The reason this function exists is to avoid creating byte arrays per key-value pair in table/builder.go.

func (*ValueStruct) EncodedSize 

func (v *ValueStruct) EncodedSize() uint16

EncodedSize is the size of the ValueStruct when encoded

type WaterMark added in v1.5.0 

type WaterMark struct {
	Name string
	// contains filtered or unexported fields
}

WaterMark is used to keep track of the minimum un-finished index. Typically, an index k becomes finished or "done" according to a WaterMark once Done(k) has been called

  1. as many times as Begin(k) has, AND
  2. a positive number of times.

An index may also become "done" by calling SetDoneUntil at a time such that it is not inter-mingled with Begin/Done calls.

Since doneUntil and lastIndex addresses are passed to sync/atomic packages, we ensure that they are 64-bit aligned by putting them at the beginning of the structure.

func (*WaterMark) Begin added in v1.5.0 

func (w *WaterMark) Begin(index uint64)

Begin sets the last index to the given value.

func (*WaterMark) BeginMany added in v1.6.0 

func (w *WaterMark) BeginMany(indices []uint64)

BeginMany works like Begin but accepts multiple indices.

func (*WaterMark) Done added in v1.5.0 

func (w *WaterMark) Done(index uint64)

Done sets a single index as done.

func (*WaterMark) DoneMany added in v1.6.0 

func (w *WaterMark) DoneMany(indices []uint64)

DoneMany works like Done but accepts multiple indices.

func (*WaterMark) DoneUntil added in v1.6.0 

func (w *WaterMark) DoneUntil() uint64

DoneUntil returns the maximum index that has the property that all indices less than or equal to it are done.

func (*WaterMark) Init added in v1.5.0 

func (w *WaterMark) Init(closer *Closer, eventLogging bool)

Init initializes a WaterMark struct. MUST be called before using it.

func (*WaterMark) LastIndex added in v1.6.0 

func (w *WaterMark) LastIndex() uint64

LastIndex returns the last index for which Begin has been called.

func (*WaterMark) SetDoneUntil added in v1.6.0 

func (w *WaterMark) SetDoneUntil(val uint64)

SetDoneUntil sets the maximum index that has the property that all indices less than or equal to it are done.

func (*WaterMark) WaitForMark added in v1.6.0 

func (w *WaterMark) WaitForMark(ctx context.Context, index uint64) error

WaitForMark waits until the given index is marked as done.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.