pool

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2024 License: Unlicense Imports: 5 Imported by: 6

README

pool

import git.tcp.direct/kayos/common/pool

Overview

pool contains two components, both of which are forms of buffer pools.

BufferFactory

BufferFactory is a potentially safer sync.Pool of bytes.Buffer types that will not allow you to accidentally re-use buffers after you return them to the pool.

  • func NewBufferFactory() BufferFactory
  • func NewSizedBufferFactory(size int) BufferFactory
  • func (cf BufferFactory) Get() *Buffer
  • func (cf BufferFactory) MustPut(buf *Buffer)
  • func (cf BufferFactory) Put(buf *Buffer) error
StringFactory

StringFactory is very much like BufferFactory, except for it's a pool of strings.Builder types instead.

  • func NewStringFactory() StringFactory
  • func (sf StringFactory) Get() *String
  • func (sf StringFactory) MustPut(buf *String)
  • func (sf StringFactory) Put(buf *String) error

Benchmarks

foreword

In some usecases, this package will actually allocate slightly more than if one were not using it. That is because each Buffer or String type has the additional header of a sync.Once. The overhead of this is minimal, and in the benchmarks I've put together that attempt to emulate a more realistic use case, the benefits are easy to see. This is comparing it to use without buffer pools at all. Using sync.Pool alone without this package will always be ever-so-slightly more efficient, but the consequences of mis-using it could be catastrophic. That is the trade-off we make.

Note: "bytes" here refers to the size of buffer pre-allocation. Either with NewSizedBufferFactory or bytes.NewBuffer

Using this package
BenchmarkBufferFactory/SingleProc-64-bytes-24         	 410792	     3467 ns/op	  24685 B/op	      5 allocs/op
BenchmarkBufferFactory/SingleProc-1024-bytes-24       	 376539	     3668 ns/op	  24687 B/op	      5 allocs/op
BenchmarkBufferFactory/SingleProc-4096-bytes-24       	 281402	     3599 ns/op	  24688 B/op	      5 allocs/op
BenchmarkBufferFactory/SingleProc-65536-bytes-24      	 340872	     3591 ns/op	  24830 B/op	      5 allocs/op

BenchmarkBufferFactory/Concurrent-x2-64-bytes-24      	 498634	     2076 ns/op	  24678 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x2-1024-bytes-24    	 569001	     2128 ns/op	  24684 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x2-4096-bytes-24    	 602946	     2131 ns/op	  24688 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x2-65536-bytes-24   	 779770	     1522 ns/op	  24747 B/op	      5 allocs/op

BenchmarkBufferFactory/Concurrent-x4-64-bytes-24      	 319677	     3271 ns/op	  24695 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x4-1024-bytes-24    	 597859	     2005 ns/op	  24677 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x4-4096-bytes-24    	 586940	     2092 ns/op	  24685 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x4-65536-bytes-24   	 861326	     1420 ns/op	  24719 B/op	      5 allocs/op

BenchmarkBufferFactory/Concurrent-x8-64-bytes-24      	 621253	     1893 ns/op	  24672 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x8-1024-bytes-24    	 626067	     2155 ns/op	  24678 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x8-4096-bytes-24    	 644078	     2030 ns/op	  24682 B/op	      5 allocs/op
BenchmarkBufferFactory/Concurrent-x8-65536-bytes-24   	 878794	     1382 ns/op	  24718 B/op	      5 allocs/op
Allocating new bytes.Buffers
BenchmarkNotUsingPackage/SingleProc-64-bytes-24       	 121978	     9401 ns/op	  52000 B/op	      5 allocs/op
BenchmarkNotUsingPackage/SingleProc-1024-bytes-24     	 129132	     9004 ns/op	  52000 B/op	      5 allocs/op
BenchmarkNotUsingPackage/SingleProc-4096-bytes-24     	 138590	     8894 ns/op	  52000 B/op	      5 allocs/op
BenchmarkNotUsingPackage/SingleProc-65536-bytes-24    	 134730	     8891 ns/op	  52000 B/op	      5 allocs/op

BenchmarkNotUsingPackage/Concurrent-x2-64-bytes-24    	 150890	     8488 ns/op	  52225 B/op	      8 allocs/op
BenchmarkNotUsingPackage/Concurrent-x2-1024-bytes-24  	 174734	     8924 ns/op	  55937 B/op	      7 allocs/op
BenchmarkNotUsingPackage/Concurrent-x2-4096-bytes-24  	 101695	    11032 ns/op	  65537 B/op	      6 allocs/op
BenchmarkNotUsingPackage/Concurrent-x2-65536-bytes-24 	  43708	    28977 ns/op	 286724 B/op	      5 allocs/op

BenchmarkNotUsingPackage/Concurrent-x4-64-bytes-24    	 189124	     6314 ns/op	  52224 B/op	      8 allocs/op
BenchmarkNotUsingPackage/Concurrent-x4-1024-bytes-24  	 144324	     7010 ns/op	  55937 B/op	      7 allocs/op
BenchmarkNotUsingPackage/Concurrent-x4-4096-bytes-24  	 129194	     9325 ns/op	  65537 B/op	      6 allocs/op
BenchmarkNotUsingPackage/Concurrent-x4-65536-bytes-24 	  43641	    28730 ns/op	 286723 B/op	      5 allocs/op

BenchmarkNotUsingPackage/Concurrent-x8-64-bytes-24    	 184123	     5911 ns/op	  52224 B/op	      8 allocs/op
BenchmarkNotUsingPackage/Concurrent-x8-1024-bytes-24  	 172016	     6602 ns/op	  55937 B/op	      7 allocs/op
BenchmarkNotUsingPackage/Concurrent-x8-4096-bytes-24  	 129916	     9030 ns/op	  65537 B/op	      6 allocs/op
BenchmarkNotUsingPackage/Concurrent-x8-65536-bytes-24 	  45729	    26344 ns/op	 286723 B/op	      5 allocs/op

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrBufferReturned = errors.New("buffer already returned")

Functions

This section is empty.

Types

type Buffer added in v0.7.5 

type Buffer struct {
	*bytes.Buffer
	// contains filtered or unexported fields
}

Buffer is a wrapper around bytes.Buffer that can only be returned to a pool once.

func (Buffer) Bytes added in v0.7.5 

func (c Buffer) Bytes() []byte

Bytes returns a slice of length b.Len() holding the unread portion of the buffer. The slice is valid for use only until the next buffer modification (that is, only until the next call to a method like Read, Write, Reset, or Truncate). The slice aliases the buffer content at least until the next buffer modification, so immediate changes to the slice will affect the result of future reads.

*This is from the bytes.Buffer docs.*

func (Buffer) Cap added in v0.7.5 

func (c Buffer) Cap() int

Cap returns the capacity of the buffer's underlying byte slice, that is, the total space allocated for the buffer's data.

*This is from the bytes.Buffer docs.* If the buffer has already been returned to the pool, Cap will return 0.

func (Buffer) Close added in v0.8.5 

func (c Buffer) Close() error

Close implements io.Closer. It returns the buffer to the pool. This

func (Buffer) Grow added in v0.7.5 

func (c Buffer) Grow(n int) error

Grow grows the buffer's capacity, if necessary, to guarantee space for another n bytes. After Grow(n), at least n bytes can be written to the buffer without another allocation. If n is negative, Grow will panic. If the buffer can't grow it will panic with ErrTooLarge.

If the buffer has already been returned to the pool, Grow will return ErrBufferReturned.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) IsClosed added in v0.8.6 

func (c Buffer) IsClosed() bool

IsClosed returns true if the buffer has been returned to the pool.

func (Buffer) Len added in v0.7.5 

func (c Buffer) Len() int

Len returns the number of bytes of the unread portion of the buffer; b.Len() == len(b.Bytes()).

*This is from the bytes.Buffer docs.* This wrapper returns 0 if the buffer has already been returned to the pool.

func (Buffer) MustBytes added in v0.7.5 

func (c Buffer) MustBytes() []byte

MustBytes is the same as Bytes but panics if the buffer has already been returned to the pool.

func (Buffer) MustLen added in v0.7.5 

func (c Buffer) MustLen() int

MustLen is the same as Len but panics if the buffer has already been returned to the pool.

func (Buffer) MustReadFrom added in v0.7.5 

func (c Buffer) MustReadFrom(r io.Reader)

MustReadFrom is the same as ReadFrom but panics if the buffer has already been returned to the pool.

func (Buffer) MustReset added in v0.7.5 

func (c Buffer) MustReset()

MustReset is the same as Reset but panics if the buffer has already been returned to the pool.

func (Buffer) MustString added in v0.7.5 

func (c Buffer) MustString() string

MustString is the same as String but panics if the buffer has already been returned to the pool.

func (Buffer) MustTruncate added in v0.7.5 

func (c Buffer) MustTruncate(n int)

MustTruncate is the same as Truncate but panics if the buffer has already been returned to the pool.

func (Buffer) MustWrite added in v0.7.5 

func (c Buffer) MustWrite(p []byte)

MustWrite is the same as Write but panics if the buffer has already been returned to the pool.

func (Buffer) MustWriteByte added in v0.7.5 

func (c Buffer) MustWriteByte(cyte byte)

MustWriteByte is the same as WriteByte but panics if the buffer has already been returned to the pool.

func (Buffer) MustWriteRune added in v0.7.5 

func (c Buffer) MustWriteRune(r rune)

MustWriteRune is the same as WriteRune but panics if the buffer has already been returned to the pool.

func (Buffer) MustWriteTo added in v0.7.5 

func (c Buffer) MustWriteTo(w io.Writer)

MustWriteTo is the same as WriteTo but panics if the buffer has already been returned to the pool.

func (Buffer) Next added in v0.7.5 

func (c Buffer) Next(n int) []byte

Next returns a slice containing the next n bytes from the buffer, advancing the buffer as if the bytes had been returned by Read. If there are fewer than n bytes in the buffer, Next returns the entire buffer. The slice is only valid until the next call to a read or write method.

*This is from the bytes.Buffer docs.* This wrapper returns nil if the buffer has already been returned to the pool.

func (Buffer) Read added in v0.7.5 

func (c Buffer) Read(p []byte) (int, error)

Read reads the next len(p) bytes from the buffer or until the buffer is drained. The return value n is the number of bytes read. If the buffer has no data to return, err is io.EOF (unless len(p) is zero); otherwise it is nil.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) ReadByte added in v0.7.5 

func (c Buffer) ReadByte() (byte, error)

ReadByte reads and returns the next byte from the buffer. If no byte is available, it returns error io.EOF.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) ReadBytes added in v0.7.5 

func (c Buffer) ReadBytes(delim byte) ([]byte, error)

ReadBytes reads until the first occurrence of delim in the input, returning a slice containing the data up to and including the delimiter. If ReadBytes encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadBytes returns err != nil if and only if the returned data does not end in delim.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) ReadFrom added in v0.7.5 

func (c Buffer) ReadFrom(r io.Reader) (int64, error)

ReadFrom reads data from r until EOF and appends it to the buffer, growing the buffer as needed. The return value n is the number of bytes read. Any error except io.EOF encountered during the read is also returned. If the buffer becomes too large, ReadFrom will panic with ErrTooLarge.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) ReadRune added in v0.7.5 

func (c Buffer) ReadRune() (rune, int, error)

ReadRune reads and returns the next UTF-8-encoded Unicode code point from the buffer. If no bytes are available, the error returned is io.EOF. If the bytes are an erroneous UTF-8 encoding, it consumes one byte and returns U+FFFD, 1.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) Reset added in v0.7.5 

func (c Buffer) Reset() error

Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) String added in v0.7.5 

func (c Buffer) String() string

String returns the contents of the unread portion of the buffer as a string. If the Buffer is a nil pointer, it returns "<nil>".

To build strings more efficiently, see the strings.Builder type.

*This is from the bytes.Buffer docs.*

func (Buffer) Truncate added in v0.7.5 

func (c Buffer) Truncate(n int) error

Truncate discards all but the first n unread bytes from the buffer but continues to use the same allocated storage. It panics if n is negative or greater than the length of the buffer.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) UnreadByte added in v0.7.5 

func (c Buffer) UnreadByte() error

UnreadByte unreads the last byte returned by the most recent successful read operation that read at least one byte. If a write has happened since the last read, if the last read returned an error, or if the read read zero bytes, UnreadByte returns an error.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) UnreadRune added in v0.7.5 

func (c Buffer) UnreadRune() error

UnreadRune unreads the last rune returned by ReadRune. If the most recent read or write operation on the buffer was not a successful ReadRune, UnreadRune returns an error. (In this regard it is stricter than UnreadByte, which will unread the last byte from any read operation.)

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) WithParent added in v0.8.5 

func (c Buffer) WithParent(p *BufferFactory) *Buffer

WithParent sets the parent of the buffer. This is useful for chaining factories, and for facilitating in-line buffer return with functions like Buffer.Close(). Be mindful, however, that this adds a bit of overhead.

func (Buffer) Write added in v0.7.5 

func (c Buffer) Write(p []byte) (int, error)

Write appends the contents of p to the buffer, growing the buffer as needed. The return value n is the length of p; err is always nil. If the buffer becomes too large, Write will panic with ErrTooLarge.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) WriteByte added in v0.7.5 

func (c Buffer) WriteByte(cyte byte) error

WriteByte appends the byte c to the buffer, growing the buffer as needed. The returned error is always nil, but is included to match bufio.Writer's WriteByte. If the buffer becomes too large, WriteByte will panic with ErrTooLarge.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) WriteRune added in v0.7.5 

func (c Buffer) WriteRune(r rune) (int, error)

WriteRune appends the UTF-8 encoding of Unicode code point r to the buffer, returning its length and an error, which is always nil but is included to match bufio.Writer's WriteRune. The buffer is grown as needed; if it becomes too large, WriteRune will panic with ErrTooLarge.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) WriteString added in v0.7.5 

func (c Buffer) WriteString(str string) (int, error)

WriteString appends the contents of s to the buffer, growing the buffer as needed. The return value n is the length of s; err is always nil. If the buffer becomes too large, WriteString will panic with ErrTooLarge.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

func (Buffer) WriteTo added in v0.7.5 

func (c Buffer) WriteTo(w io.Writer) (int64, error)

WriteTo writes data to w until the buffer is drained or an error occurs. The return value n is the number of bytes written; it always fits into an int, but it is int64 to match the io.WriterTo interface. Any error encountered during the write is also returned.

*This is from the bytes.Buffer docs.* This wrapper returns an error if the buffer has already been returned to the pool.

type BufferFactory added in v0.7.5 

type BufferFactory struct {
	// contains filtered or unexported fields
}

BufferFactory is a factory for creating and reusing bytes.Buffers. BufferFactory tries to be safer than using a sync.Pool directly by ensuring that the buffer is not returned twice.

func NewBufferFactory added in v0.7.5 

func NewBufferFactory() BufferFactory

NewBufferFactory creates a new BufferFactory that creates new buffers on demand.

func NewSizedBufferFactory added in v0.8.1 

func NewSizedBufferFactory(size int) BufferFactory

NewSizedBufferFactory creates a new BufferFactory that creates new buffers of the given size on demand.

func (BufferFactory) Get added in v0.7.5 

func (cf BufferFactory) Get() *Buffer

Get returns a buffer from the pool.

func (BufferFactory) MustPut added in v0.7.5 

func (cf BufferFactory) MustPut(buf *Buffer)

MustPut is the same as Put but panics if the buffer has already been returned to the pool.

func (BufferFactory) Put added in v0.7.5 

func (cf BufferFactory) Put(buf *Buffer) error

Put returns the buffer to the pool. It returns an error if the buffer has already been returned to the pool.

type BufferFactoryByteBufferCompat added in v0.9.8 

type BufferFactoryByteBufferCompat struct {
	BufferFactory
}

func (BufferFactoryByteBufferCompat) Get added in v0.9.8 

func (bf BufferFactoryByteBufferCompat) Get() ByteBuffer

func (BufferFactoryByteBufferCompat) Put added in v0.9.8 

func (bf BufferFactoryByteBufferCompat) Put(buf ByteBuffer)

type BufferFactoryInterfaceCompat added in v0.9.4 

type BufferFactoryInterfaceCompat struct {
	BufferFactory
}

func (BufferFactoryInterfaceCompat) Put added in v0.9.4 

func (b BufferFactoryInterfaceCompat) Put(buf *Buffer)

type ByteBuffer added in v0.9.8 

type ByteBuffer interface {
	Len() int
	Cap() int
	Bytes() []byte
	WriteRune(rune) (int, error)
	WriteString(string) (int, error)

	io.WriterTo
	io.ByteWriter
	io.ReadWriter

	io.ReaderFrom
	io.ByteReader
	io.RuneReader
}

ByteBuffer is satisfied by *pool.Buffer and *bytes.Buffer.

Note, we can't include Reset and Grow as our implementations return an error while a *bytes.Buffer does not.

type Pool added in v0.9.4 

type Pool[T any] interface {
	Get() T
	Put(T)
}

type String 

type String struct {
	*strings.Builder
	// contains filtered or unexported fields
}

func (String) Cap 

func (s String) Cap() int

func (String) Grow 

func (s String) Grow(n int) error

func (String) Len 

func (s String) Len() int

func (String) MustGrow added in v0.7.5 

func (s String) MustGrow(n int)

func (String) MustLen added in v0.7.5 

func (s String) MustLen() int

func (String) MustReset added in v0.7.5 

func (s String) MustReset()

func (String) MustString added in v0.7.5 

func (s String) MustString() string

func (String) MustWriteString added in v0.7.5 

func (s String) MustWriteString(str string)

MustWriteString means Must Write String, like WriteString but will panic on error.

func (String) Reset 

func (s String) Reset() error

func (String) String 

func (s String) String() string

func (String) Write 

func (s String) Write(p []byte) (int, error)

func (String) WriteByte 

func (s String) WriteByte(c byte) error

func (String) WriteRune 

func (s String) WriteRune(r rune) (int, error)

func (String) WriteString 

func (s String) WriteString(str string) (int, error)

type StringFactory 

type StringFactory struct {
	// contains filtered or unexported fields
}

func NewStringFactory 

func NewStringFactory() StringFactory

NewStringFactory creates a new strings.Builder pool.

func (StringFactory) Get 

func (sf StringFactory) Get() *String

Get returns a strings.Builder from the pool.

func (StringFactory) MustPut added in v0.7.5 

func (sf StringFactory) MustPut(buf *String)

func (StringFactory) Put 

func (sf StringFactory) Put(buf *String) error

Put returns a strings.Builder back into to the pool after resetting it.

type WithPutError added in v0.9.4 

type WithPutError[T any] interface {
	Get() T
	Put(T) error
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.