memory

Documentation

Index

• Variables
• func WithAllocator(ctx context.Context, mem Allocator) context.Context
• type Allocator
  • func GetAllocator(ctx context.Context) Allocator
• type GoAllocator
  • func (*GoAllocator) Account(size int) error
• type LimitExceededError
  • func (a LimitExceededError) Error() string
• type Manager
• type ResourceAllocator
  • func NewResourceAllocator(allocator memory.Allocator) *ResourceAllocator
  • func (a *ResourceAllocator) Account(size int) error
  • func (a *ResourceAllocator) Allocate(size int) []byte
  • func (a *ResourceAllocator) Allocated() int64
  • func (a *ResourceAllocator) Free(b []byte)
  • func (a *ResourceAllocator) MaxAllocated() int64
  • func (a *ResourceAllocator) Reallocate(size int, b []byte) []byte
  • func (a *ResourceAllocator) TotalAllocated() int64

Variables

var DefaultAllocator = &GoAllocator{}

DefaultAllocator is the default memory allocator for Flux.

This implements the memory.Allocator interface from arrow.

Functions

func WithAllocator

func WithAllocator(ctx context.Context, mem Allocator) context.Context

Types

type Allocator

type Allocator interface {
	memory.Allocator

	// Account provides direct access to record to the allocator
	// that the given memory was allocated outside the Allocator.
	// In order to release the memory, a negative size can be used.
	Account(size int) error
}

Allocator defines how memory is allocated and released within the flux engine.

func GetAllocator

func GetAllocator(ctx context.Context) Allocator

type GoAllocator

type GoAllocator struct {
	memory.GoAllocator
}

GoAllocator implements a version of the allocator that uses native Go slices. It does not track or limit the amount of memory that is used.

func (*GoAllocator) Account

func (*GoAllocator) Account(size int) error

type LimitExceededError

type LimitExceededError struct {
	Limit     int64
	Allocated int64
	Wanted    int64
}

LimitExceededError is an error when the allocation limit is exceeded.

func (LimitExceededError) Error

func (a LimitExceededError) Error() string

type Manager

type Manager interface {
	// RequestMemory will request that the given amount of memory
	// be reserved for the caller. The manager will return the number
	// of bytes that were successfully reserved. The n value will be
	// either equal to or greater than the requested number of bytes.
	// If the manager cannot reserve at least bytes in memory, then
	// it will return an error with the details.
	RequestMemory(want int64) (got int64, err error)

	// FreeMemory will inform the memory manager that this memory
	// is not being used anymore.
	// It is not required for this to be called similar to how
	// it is not necessary for a program to free the memory.
	// It is the responsibility of the manager itself to identify
	// when this allocator is not used anymore and to reclaim any
	// unfreed memory when the resource is dead.
	FreeMemory(bytes int64)
}

Manager will manage the memory allowed for the Allocator. The Allocator may use the Manager to request additional memory or to give back memory that is currently in use by the Allocator when/if it is no longer needed.

type ResourceAllocator

type ResourceAllocator struct {

	// Limit is the limit on the amount of memory that this allocator
	// can assign. If this is null, there is no limit.
	Limit *int64

	// Manager holds the Manager for this Allocator.
	// If this Allocator has a limit set and the limit is to be exceeded,
	// it will attempt to use the Manager to request more memory.
	// If this fails, then the Allocator will panic.
	Manager Manager

	// Allocator is the underlying memory allocator used to
	// allocate and free memory.
	// If this is unset, the DefaultAllocator is used.
	Allocator memory.Allocator
	// contains filtered or unexported fields
}

ResourceAllocator tracks the amount of memory being consumed by a query.

func NewResourceAllocator

func NewResourceAllocator(allocator memory.Allocator) *ResourceAllocator

func (*ResourceAllocator) Account

func (a *ResourceAllocator) Account(size int) error

Account will manually account for the amount of memory being used. This is typically used for memory that is allocated outside of the Allocator that must be recorded in some way.

func (*ResourceAllocator) Allocate

func (a *ResourceAllocator) Allocate(size int) []byte

Allocate will ensure that the requested memory is available and record that it is in use.

func (*ResourceAllocator) Allocated

func (a *ResourceAllocator) Allocated() int64

Allocated returns the amount of currently allocated memory.

func (*ResourceAllocator) Free

func (a *ResourceAllocator) Free(b []byte)

Free will reduce the amount of memory used by this Allocator. In general, memory should be freed using the Reference returned by Allocate. Not all code is capable of using this though so this method provides a low-level way of releasing the memory without using a Reference. Free will release the memory associated with the byte slice.

func (*ResourceAllocator) MaxAllocated

func (a *ResourceAllocator) MaxAllocated() int64

MaxAllocated reports the maximum amount of allocated memory at any point in the query.

func (*ResourceAllocator) Reallocate

func (a *ResourceAllocator) Reallocate(size int, b []byte) []byte

func (*ResourceAllocator) TotalAllocated

func (a *ResourceAllocator) TotalAllocated() int64

TotalAllocated reports the total amount of memory allocated. It counts all memory that was allocated at any time even if it was released.