util

package
v0.0.0-...-9750751 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package util implements some utility functions.

Index

Constants

View Source 
const (
	EXTRA_PERCENT = 20
)

Yaping shi Dec 20, 2016

lock-free RingBuffer for single producer - single consumer - This RingBuffer can be used for communication between maximum two go routines without locking

- producer can only update tail - conusmer can only update head

View Source 
const MinRead = 512

MinRead is the minimum slice size passed to a Read call by Buffer.ReadFrom. As long as the Buffer has at least MinRead bytes beyond what is required to hold the contents of r, ReadFrom will not grow the underlying buffer.

Variables

View Source 
var ErrTooLarge = errors.New("bytes.Buffer: too large")

ErrTooLarge is passed to panic if memory cannot be allocated to store data in a buffer.

Functions

func GetExpirationTime 

func GetExpirationTime(ttl uint32) uint32

func GetExpirationTimeFrom 

func GetExpirationTimeFrom(now time.Time, ttl uint32) (expirationTime uint32)

func GetGID 

func GetGID() uint64

http://blog.sgmansfield.com/2015/12/goroutine-ids/ Goroutine Id, used for debugging purpose

func GetMicroShardId 

func GetMicroShardId(key []byte, numMicroShards uint32) (microShardId uint8)

func GetNumOpenFDs 

func GetNumOpenFDs() (n int)

func GetPartitionId 

func GetPartitionId(key []byte, numShards uint32) uint16

func GetShardIds 

func GetShardIds(key []byte, numShards uint32, numMicroShards uint32) (shardId uint16, microShardId uint8)

lower two bytes are used for Shard Id, and higher two bytes are used for Micro Shard Id

func GetShardInfoByKey 

func GetShardInfoByKey(key []byte, numShards uint32, numZones uint32, AlgVersion uint32) (shardId uint16, start_zoneid uint32)

func GetTimeFromUUIDv1 

func GetTimeFromUUIDv1(id uuid.UUID) (tm time.Time, err error)

func GetTimeToLive 

func GetTimeToLive(expirationTime uint32) uint32

func GetTimeToLiveFrom 

func GetTimeToLiveFrom(expirationTime uint32, now time.Time) uint32

func GetTotalMemMB 

func GetTotalMemMB() int

in MB

func HexDump 

func HexDump(data []byte)

func IsSocket 

func IsSocket(f *os.File) bool

func IsSocketFD 

func IsSocketFD(fd int) bool

func Lsof 

func Lsof(w io.Writer)

func Murmur3Hash 

func Murmur3Hash(data []byte) uint32

func NewBufioReader 

func NewBufioReader(r io.Reader, bufSize int) *bufio.Reader

func NewBufioWriter 

func NewBufioWriter(w io.Writer, bufSize int) *bufio.Writer

func Now 

func Now() uint32

func Now64 

func Now64() uint64

func PrintBytesForTest 

func PrintBytesForTest(data []byte)

func PutBufioReader 

func PutBufioReader(br *bufio.Reader)

func PutBufioWriter 

func PutBufioWriter(bw *bufio.Writer)

func ToHexString 

func ToHexString(data []byte) string

func ToPrintableAndHexString 

func ToPrintableAndHexString(data []byte) string

func ToPrintableString 

func ToPrintableString(b []byte) string

func WriteFileInfo 

func WriteFileInfo(files []*os.File, w io.Writer)

Types

type AtomicCounter 

type AtomicCounter struct {
	// contains filtered or unexported fields
}

func (*AtomicCounter) Add 

func (c *AtomicCounter) Add(delta int32)

func (*AtomicCounter) Get 

func (c *AtomicCounter) Get() int32

func (*AtomicCounter) Reset 

func (c *AtomicCounter) Reset()

func (*AtomicCounter) Set 

func (c *AtomicCounter) Set(cnt int32)

type AtomicShareCounter 

type AtomicShareCounter struct {
	// contains filtered or unexported fields
}

func NewAtomicShareCounter 

func NewAtomicShareCounter(cnt *uint64) *AtomicShareCounter

func (*AtomicShareCounter) Add 

func (c *AtomicShareCounter) Add(delta uint64)

func (*AtomicShareCounter) Get 

func (c *AtomicShareCounter) Get() uint64

func (*AtomicShareCounter) Reset 

func (c *AtomicShareCounter) Reset()

type AtomicUint64Counter 

type AtomicUint64Counter struct {
	// contains filtered or unexported fields
}

func (*AtomicUint64Counter) Add 

func (c *AtomicUint64Counter) Add(delta uint64)

func (*AtomicUint64Counter) Get 

func (c *AtomicUint64Counter) Get() uint64

func (*AtomicUint64Counter) Reset 

func (c *AtomicUint64Counter) Reset()

func (*AtomicUint64Counter) Set 

func (c *AtomicUint64Counter) Set(cnt uint64)

type Buffer 

type Buffer struct {
	// contains filtered or unexported fields
}

A Buffer is a variable-sized buffer of bytes with Read and Write methods. The zero value for Buffer is an empty buffer ready to use.

func NewBuffer 

func NewBuffer(buf []byte) *Buffer

NewBuffer creates and initializes a new Buffer using buf as its initial contents. It is intended to prepare a Buffer to read existing data. It can also be used to size the internal buffer for writing. To do that, buf should have the desired capacity but a length of zero.

In most cases, new(Buffer) (or just declaring a Buffer variable) is sufficient to initialize a Buffer.

func NewBufferString 

func NewBufferString(s string) *Buffer

NewBufferString creates and initializes a new Buffer using string s as its initial contents. It is intended to prepare a buffer to read an existing string.

In most cases, new(Buffer) (or just declaring a Buffer variable) is sufficient to initialize a Buffer.

func (*Buffer) Bytes 

func (b *Buffer) Bytes() []byte

Bytes returns a slice of length b.Len() holding the unread portion of the buffer. The slice is valid for use only until the next buffer modification (that is, only until the next call to a method like Read, Write, Reset, or Truncate). The slice aliases the buffer content at least until the next buffer modification, so immediate changes to the slice will affect the result of future reads.

func (*Buffer) Cap 

func (b *Buffer) Cap() int

Cap returns the capacity of the buffer's underlying byte slice, that is, the total space allocated for the buffer's data.

func (*Buffer) Grow 

func (b *Buffer) Grow(n int)

Grow grows the buffer's capacity, if necessary, to guarantee space for another n bytes. After Grow(n), at least n bytes can be written to the buffer without another allocation. If n is negative, Grow will panic. If the buffer can't grow it will panic with ErrTooLarge.

func (*Buffer) Len 

func (b *Buffer) Len() int

Len returns the number of bytes of the unread portion of the buffer; b.Len() == len(b.Bytes()).

func (*Buffer) Next 

func (b *Buffer) Next(n int) []byte

Next returns a slice containing the next n bytes from the buffer, advancing the buffer as if the bytes had been returned by Read. If there are fewer than n bytes in the buffer, Next returns the entire buffer. The slice is only valid until the next call to a read or write method.

func (*Buffer) Read 

func (b *Buffer) Read(p []byte) (n int, err error)

Read reads the next len(p) bytes from the buffer or until the buffer is drained. The return value n is the number of bytes read. If the buffer has no data to return, err is io.EOF (unless len(p) is zero); otherwise it is nil.

func (*Buffer) ReadByte 

func (b *Buffer) ReadByte() (byte, error)

ReadByte reads and returns the next byte from the buffer. If no byte is available, it returns error io.EOF.

func (*Buffer) ReadBytes 

func (b *Buffer) ReadBytes(delim byte) (line []byte, err error)

ReadBytes reads until the first occurrence of delim in the input, returning a slice containing the data up to and including the delimiter. If ReadBytes encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadBytes returns err != nil if and only if the returned data does not end in delim.

func (*Buffer) ReadFrom 

func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error)

ReadFrom reads data from r until EOF and appends it to the buffer, growing the buffer as needed. The return value n is the number of bytes read. Any error except io.EOF encountered during the read is also returned. If the buffer becomes too large, ReadFrom will panic with ErrTooLarge.

func (*Buffer) ReadRune 

func (b *Buffer) ReadRune() (r rune, size int, err error)

ReadRune reads and returns the next UTF-8-encoded Unicode code point from the buffer. If no bytes are available, the error returned is io.EOF. If the bytes are an erroneous UTF-8 encoding, it consumes one byte and returns U+FFFD, 1.

func (*Buffer) ReadString 

func (b *Buffer) ReadString(delim byte) (line string, err error)

ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadString returns err != nil if and only if the returned data does not end in delim.

func (*Buffer) Reset 

func (b *Buffer) Reset()

Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).

func (*Buffer) String 

func (b *Buffer) String() string

String returns the contents of the unread portion of the buffer as a string. If the Buffer is a nil pointer, it returns "<nil>".

func (*Buffer) Truncate 

func (b *Buffer) Truncate(n int)

Truncate discards all but the first n unread bytes from the buffer but continues to use the same allocated storage. It panics if n is negative or greater than the length of the buffer.

func (*Buffer) UnreadByte 

func (b *Buffer) UnreadByte() error

UnreadByte unreads the last byte returned by the most recent read operation. If write has happened since the last read, UnreadByte returns an error.

func (*Buffer) UnreadRune 

func (b *Buffer) UnreadRune() error

UnreadRune unreads the last rune returned by ReadRune. If the most recent read or write operation on the buffer was not a ReadRune, UnreadRune returns an error. (In this regard it is stricter than UnreadByte, which will unread the last byte from any read operation.)

func (*Buffer) Write 

func (b *Buffer) Write(p []byte) (n int, err error)

Write appends the contents of p to the buffer, growing the buffer as needed. The return value n is the length of p; err is always nil. If the buffer becomes too large, Write will panic with ErrTooLarge.

func (*Buffer) WriteByte 

func (b *Buffer) WriteByte(c byte) error

WriteByte appends the byte c to the buffer, growing the buffer as needed. The returned error is always nil, but is included to match bufio.Writer's WriteByte. If the buffer becomes too large, WriteByte will panic with ErrTooLarge.

func (*Buffer) WriteRune 

func (b *Buffer) WriteRune(r rune) (n int, err error)

WriteRune appends the UTF-8 encoding of Unicode code point r to the buffer, returning its length and an error, which is always nil but is included to match bufio.Writer's WriteRune. The buffer is grown as needed; if it becomes too large, WriteRune will panic with ErrTooLarge.

func (*Buffer) WriteString 

func (b *Buffer) WriteString(s string) (n int, err error)

WriteString appends the contents of s to the buffer, growing the buffer as needed. The return value n is the length of s; err is always nil. If the buffer becomes too large, WriteString will panic with ErrTooLarge.

func (*Buffer) WriteTo 

func (b *Buffer) WriteTo(w io.Writer) (n int64, err error)

WriteTo writes data to w until the buffer is drained or an error occurs. The return value n is the number of bytes written; it always fits into an int, but it is int64 to match the io.WriterTo interface. Any error encountered during the write is also returned.

type BufferPool 

type BufferPool interface {
	Get() *PPBuffer
	Put(buf *PPBuffer)
}

func GetBufferPool 

func GetBufferPool(size int) BufferPool

func NewChanBufferPool 

func NewChanBufferPool(chansize int, bufsize int) BufferPool

func NewSyncBufferPool 

func NewSyncBufferPool(size int) BufferPool

type BytePool 

type BytePool interface {
	Get() []byte
	Put([]byte)
}

func NewChanBytePool 

func NewChanBytePool(chansize int, bytesize int) BytePool

func NewSyncBytePool 

func NewSyncBytePool(size int) BytePool

type CMap 

type CMap struct {
	// contains filtered or unexported fields
}

func NewCMap 

func NewCMap(paritionsCount uint32) *CMap

func (*CMap) Delete 

func (m *CMap) Delete(key []byte)

func (*CMap) Get 

func (m *CMap) Get(key []byte) (interface{}, bool)

func (*CMap) PrintAll 

func (m *CMap) PrintAll(bucketId uint32, dataType string)

Testing and logging purpose only. Don't dump in production as it has overhead and file IO will take more time.

func (*CMap) Put 

func (m *CMap) Put(key []byte, value interface{})

func (*CMap) PutIfAbsent 

func (m *CMap) PutIfAbsent(key []byte, value interface{}) (interface{}, bool)

type ChanBufferPool 

type ChanBufferPool struct {
	// contains filtered or unexported fields
}

channel based buffer pool

func (*ChanBufferPool) Get 

func (p *ChanBufferPool) Get() (buf *PPBuffer)

func (*ChanBufferPool) Put 

func (p *ChanBufferPool) Put(buf *PPBuffer)

type ChanBytePool 

type ChanBytePool struct {
	// contains filtered or unexported fields
}

channel based buffer pool

func (*ChanBytePool) Get 

func (p *ChanBytePool) Get() (b []byte)

func (*ChanBytePool) Put 

func (p *ChanBytePool) Put(b []byte)

type ChanPool 

type ChanPool struct {
	New func() interface{}
	// contains filtered or unexported fields
}

func NewChanPool 

func NewChanPool(chansize int, f func() interface{}) *ChanPool

func (*ChanPool) Get 

func (p *ChanPool) Get() (item interface{})

func (*ChanPool) Put 

func (p *ChanPool) Put(item interface{})

type Duration 

type Duration struct {
	time.Duration
}

func (Duration) MarshalText 

func (d Duration) MarshalText() (text []byte, err error)

func (*Duration) UnmarshalText 

func (d *Duration) UnmarshalText(text []byte) error

type MapPartition 

type MapPartition struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

type PPBuffer 

type PPBuffer struct {
	Buffer
}

func NewPPBuffer 

func NewPPBuffer(buf []byte) *PPBuffer

func (*PPBuffer) Resize 

func (b *PPBuffer) Resize(n int)

type QueItem 

type QueItem interface {
	OnCleanup()
	OnExpiration()
	Deadline() (deadline time.Time)
	ResetDeadline()
	SetId(id uint32)
	GetId() uint32
	SetInUse(flag bool)
	SetQueTimeout(t time.Duration)
	GetQueTimeout() (t time.Duration)
	IsInUse() bool
}

type QueItemBase 

type QueItemBase struct {
	// contains filtered or unexported fields
}

func (*QueItemBase) Deadline 

func (q *QueItemBase) Deadline() (deadline time.Time)

func (*QueItemBase) GetId 

func (q *QueItemBase) GetId() uint32

func (*QueItemBase) GetQueTimeout 

func (q *QueItemBase) GetQueTimeout() time.Duration

func (*QueItemBase) IsInUse 

func (q *QueItemBase) IsInUse() bool

func (*QueItemBase) ResetDeadline 

func (q *QueItemBase) ResetDeadline()

func (*QueItemBase) SetDeadline 

func (q *QueItemBase) SetDeadline(d time.Time)

func (*QueItemBase) SetId 

func (q *QueItemBase) SetId(id uint32)

func (*QueItemBase) SetInUse 

func (q *QueItemBase) SetInUse(flag bool)

func (*QueItemBase) SetQueTimeout 

func (q *QueItemBase) SetQueTimeout(t time.Duration)

type RingBuffer 

type RingBuffer struct {
	// contains filtered or unexported fields
}

func NewRingBuffer 

func NewRingBuffer(size uint32) *RingBuffer

func NewRingBufferWithExtra 

func NewRingBufferWithExtra(size uint32, extra_pct uint32) *RingBuffer

func (*RingBuffer) CleanAll 

func (rb *RingBuffer) CleanAll()

drain everything in the ringbuffer

func (*RingBuffer) CleanUp 

func (rb *RingBuffer) CleanUp() bool

only called by reader (head) two cases: if full, clean up size/10, dequeue if not nil if head is free (nil), move head till not nil

func (*RingBuffer) DeQueue 

func (rb *RingBuffer) DeQueue() (item QueItem, err error)

func (*RingBuffer) EnQueue 

func (rb *RingBuffer) EnQueue(item QueItem) (id uint32, err error)

func (*RingBuffer) GetSize 

func (rb *RingBuffer) GetSize() uint32

func (*RingBuffer) IsEmpty 

func (rb *RingBuffer) IsEmpty() bool

func (*RingBuffer) IsFull 

func (rb *RingBuffer) IsFull() bool

func (*RingBuffer) Remove 

func (rb *RingBuffer) Remove(id uint32) (item QueItem, err error)

func (*RingBuffer) WriteStats 

func (rb *RingBuffer) WriteStats(w io.Writer)

type StringListFlags 

type StringListFlags []string

func (*StringListFlags) Set 

func (l *StringListFlags) Set(value string) error

func (*StringListFlags) String 

func (l *StringListFlags) String() string

type SyncBufferPool 

type SyncBufferPool struct {
	// contains filtered or unexported fields
}

sync.Pool based buffer pool

func (*SyncBufferPool) Get 

func (p *SyncBufferPool) Get() *PPBuffer

func (*SyncBufferPool) Put 

func (p *SyncBufferPool) Put(buf *PPBuffer)

type SyncBytePool 

type SyncBytePool struct {
	// contains filtered or unexported fields
}

sync.Pool based buffer pool

func (*SyncBytePool) Get 

func (p *SyncBytePool) Get() []byte

func (*SyncBytePool) Put 

func (p *SyncBytePool) Put(buf []byte)

type TimerWrapper 

type TimerWrapper struct {
	// contains filtered or unexported fields
}

This Wrapper class it to work around the issue with time.Timer.Reset(), mentioned below: https://github.com/golang/go/issues/11513

timer.C is buffered, so if the timer has just expired, the newly reset timer can actually trigger immediately.

func NewTimerWrapper 

func NewTimerWrapper(d time.Duration) *TimerWrapper

func (*TimerWrapper) GetTimeoutCh 

func (t *TimerWrapper) GetTimeoutCh() <-chan time.Time

func (*TimerWrapper) IsStopped 

func (t *TimerWrapper) IsStopped() bool

func (*TimerWrapper) Reset 

func (t *TimerWrapper) Reset(d time.Duration)

func (*TimerWrapper) Stop 

func (t *TimerWrapper) Stop()

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.