pgalloc

Documentation

Overview

Package pgalloc contains the page allocator subsystem, which manages memory that may be mapped into application address spaces.

Lock order:

 pgalloc.MemoryFile.mu
	pgalloc.MemoryFile.mappingsMu

Index

• Constants
• type AllocOpts
• type DelayedEvictionType
• type Direction
  • func (d Direction) String() string
• type EvictableMemoryUser
• type MemoryFile
  • func MemoryFileFromContext(ctx context.Context) *MemoryFile
  • func NewMemoryFile(file *os.File, opts MemoryFileOpts) (*MemoryFile, error)
  • func (f *MemoryFile) Allocate(length uint64, opts AllocOpts) (memmap.FileRange, error)
  • func (f *MemoryFile) AllocateAndFill(length uint64, kind usage.MemoryKind, populate bool, r safemem.Reader) (memmap.FileRange, error)
  • func (f *MemoryFile) DecRef(fr memmap.FileRange)
  • func (f *MemoryFile) Decommit(fr memmap.FileRange) error
  • func (f *MemoryFile) Destroy()
  • func (f *MemoryFile) FD() int
  • func (f *MemoryFile) File() *os.File
  • func (f *MemoryFile) IncRef(fr memmap.FileRange)
  • func (f *MemoryFile) LoadFrom(ctx context.Context, r wire.Reader) error
  • func (f *MemoryFile) MapInternal(fr memmap.FileRange, at hostarch.AccessType) (safemem.BlockSeq, error)
  • func (f *MemoryFile) MarkAllUnevictable(user EvictableMemoryUser)
  • func (f *MemoryFile) MarkEvictable(user EvictableMemoryUser, er EvictableRange)
  • func (f *MemoryFile) MarkUnevictable(user EvictableMemoryUser, er EvictableRange)
  • func (f *MemoryFile) MemoryFile() *MemoryFile
  • func (f *MemoryFile) SaveTo(ctx context.Context, w wire.Writer) error
  • func (f *MemoryFile) ShouldCacheEvictable() bool
  • func (f *MemoryFile) StartEvictions()
  • func (f *MemoryFile) String() string
  • func (f *MemoryFile) TotalSize() uint64
  • func (f *MemoryFile) TotalUsage() (uint64, error)
  • func (f *MemoryFile) UpdateUsage() error
  • func (f *MemoryFile) WaitForEvictions()
• type MemoryFileOpts
• type MemoryFileProvider
  • func MemoryFileProviderFromContext(ctx context.Context) MemoryFileProvider

Constants

const (
	// CtxMemoryFile is a Context.Value key for a MemoryFile.
	CtxMemoryFile contextID = iota

	// CtxMemoryFileProvider is a Context.Value key for a MemoryFileProvider.
	CtxMemoryFileProvider
)

Types

type AllocOpts

type AllocOpts struct {
	Kind usage.MemoryKind
	Dir  Direction
}

AllocOpts are options used in MemoryFile.Allocate.

type DelayedEvictionType

type DelayedEvictionType int

DelayedEvictionType is the type of MemoryFileOpts.DelayedEviction.

const (
	// DelayedEvictionDefault has unspecified behavior.
	DelayedEvictionDefault DelayedEvictionType = iota

	// DelayedEvictionDisabled requires that evictable allocations are evicted
	// as soon as possible.
	DelayedEvictionDisabled

	// DelayedEvictionEnabled requests that the MemoryFile delay eviction of
	// evictable allocations until doing so is considered necessary to avoid
	// performance degradation due to host memory pressure, or OOM kills.
	//
	// As of this writing, the behavior of DelayedEvictionEnabled depends on
	// whether or not MemoryFileOpts.UseHostMemcgPressure is enabled:
	//
	//	- If UseHostMemcgPressure is true, evictions are delayed until memory
	//		pressure is indicated.
	//
	//	- Otherwise, evictions are only delayed until the reclaimer goroutine
	//		is out of work (pages to reclaim).
	DelayedEvictionEnabled

	// DelayedEvictionManual requires that evictable allocations are only
	// evicted when MemoryFile.StartEvictions() is called. This is extremely
	// dangerous outside of tests.
	DelayedEvictionManual
)

type Direction

type Direction int

Direction describes how to allocate offsets from MemoryFile.

const (
	// BottomUp allocates offsets in increasing offsets.
	BottomUp Direction = iota
	// TopDown allocates offsets in decreasing offsets.
	TopDown
)

func (Direction) String

func (d Direction) String() string

String implements fmt.Stringer.

type EvictableMemoryUser

type EvictableMemoryUser interface {
	// Evict requests that the EvictableMemoryUser deallocate memory used by
	// er, which was registered as evictable by a previous call to
	// MemoryFile.MarkEvictable.
	//
	// Evict is not required to deallocate memory. In particular, since pgalloc
	// must call Evict without holding locks to avoid circular lock ordering,
	// it is possible that the passed range has already been marked as
	// unevictable by a racing call to MemoryFile.MarkUnevictable.
	// Implementations of EvictableMemoryUser must detect such races and handle
	// them by making Evict have no effect on unevictable ranges.
	//
	// After a call to Evict, the MemoryFile will consider the evicted range
	// unevictable (i.e. it will not call Evict on the same range again) until
	// informed otherwise by a subsequent call to MarkEvictable.
	Evict(ctx context.Context, er EvictableRange)
}

An EvictableMemoryUser represents a user of MemoryFile-allocated memory that may be asked to deallocate that memory in the presence of memory pressure.

type MemoryFile

type MemoryFile struct {
	// contains filtered or unexported fields
}

MemoryFile is a memmap.File whose pages may be allocated to arbitrary users.

func MemoryFileFromContext

func MemoryFileFromContext(ctx context.Context) *MemoryFile

MemoryFileFromContext returns the MemoryFile used by ctx, or nil if no such MemoryFile exists.

func NewMemoryFile

func NewMemoryFile(file *os.File, opts MemoryFileOpts) (*MemoryFile, error)

NewMemoryFile creates a MemoryFile backed by the given file. If NewMemoryFile succeeds, ownership of file is transferred to the returned MemoryFile.

func (*MemoryFile) Allocate

func (f *MemoryFile) Allocate(length uint64, opts AllocOpts) (memmap.FileRange, error)

Allocate returns a range of initially-zeroed pages of the given length with the given accounting kind and a single reference held by the caller. When the last reference on an allocated page is released, ownership of the page is returned to the MemoryFile, allowing it to be returned by a future call to Allocate.

Preconditions: length must be page-aligned and non-zero.

func (*MemoryFile) AllocateAndFill

func (f *MemoryFile) AllocateAndFill(length uint64, kind usage.MemoryKind, populate bool, r safemem.Reader) (memmap.FileRange, error)

AllocateAndFill allocates memory of the given kind and fills it by calling r.ReadToBlocks() repeatedly until either length bytes are read or a non-nil error is returned. It returns the memory filled by r, truncated down to the nearest page. If this is shorter than length bytes due to an error returned by r.ReadToBlocks(), it returns that error.

If populate is true, AllocateAndFill will attempt to pre-fault pages in bulk in the safemem.BlockSeq passed to r. Callers that will fill the allocated memory by writing to it in the sentry should pass populate = true to avoid faulting page-by-page. Callers that will fill the allocated memory by invoking host system calls should pass populate = false.

Preconditions:

• length > 0.
• length must be page-aligned.

func (*MemoryFile) DecRef

func (f *MemoryFile) DecRef(fr memmap.FileRange)

DecRef implements memmap.File.DecRef.

func (*MemoryFile) Decommit

func (f *MemoryFile) Decommit(fr memmap.FileRange) error

Decommit releases resources associated with maintaining the contents of the given pages. If Decommit succeeds, future accesses of the decommitted pages will read zeroes.

Preconditions: fr.Length() > 0.

func (*MemoryFile) Destroy

func (f *MemoryFile) Destroy()

Destroy releases all resources used by f.

Preconditions: All pages allocated by f have been freed.

Postconditions: None of f's methods may be called after Destroy.

func (*MemoryFile) FD

func (f *MemoryFile) FD() int

FD implements memmap.File.FD.

func (*MemoryFile) File

func (f *MemoryFile) File() *os.File

File returns the backing file.

func (*MemoryFile) IncRef

func (f *MemoryFile) IncRef(fr memmap.FileRange)

IncRef implements memmap.File.IncRef.

func (*MemoryFile) LoadFrom

func (f *MemoryFile) LoadFrom(ctx context.Context, r wire.Reader) error

LoadFrom loads MemoryFile state from the given stream.

func (*MemoryFile) MapInternal

func (f *MemoryFile) MapInternal(fr memmap.FileRange, at hostarch.AccessType) (safemem.BlockSeq, error)

MapInternal implements memmap.File.MapInternal.

func (*MemoryFile) MarkAllUnevictable

func (f *MemoryFile) MarkAllUnevictable(user EvictableMemoryUser)

MarkAllUnevictable informs f that user no longer considers any offsets to be evictable. It otherwise has the same semantics as MarkUnevictable.

func (*MemoryFile) MarkEvictable

func (f *MemoryFile) MarkEvictable(user EvictableMemoryUser, er EvictableRange)

MarkEvictable allows f to request memory deallocation by calling user.Evict(er) in the future.

Redundantly marking an already-evictable range as evictable has no effect.

func (*MemoryFile) MarkUnevictable

func (f *MemoryFile) MarkUnevictable(user EvictableMemoryUser, er EvictableRange)

MarkUnevictable informs f that user no longer considers er to be evictable, so the MemoryFile should no longer call user.Evict(er). Note that, per EvictableMemoryUser.Evict's documentation, user.Evict(er) may still be called even after MarkUnevictable returns due to race conditions, and implementations of EvictableMemoryUser must handle this possibility.

Redundantly marking an already-unevictable range as unevictable has no effect.

func (*MemoryFile) MemoryFile

func (f *MemoryFile) MemoryFile() *MemoryFile

MemoryFile implements MemoryFileProvider.MemoryFile.

func (*MemoryFile) SaveTo

func (f *MemoryFile) SaveTo(ctx context.Context, w wire.Writer) error

SaveTo writes f's state to the given stream.

func (*MemoryFile) ShouldCacheEvictable

func (f *MemoryFile) ShouldCacheEvictable() bool

ShouldCacheEvictable returns true if f is meaningfully delaying evictions of evictable memory, such that it may be advantageous to cache data in evictable memory. The value returned by ShouldCacheEvictable may change between calls.

func (*MemoryFile) StartEvictions

func (f *MemoryFile) StartEvictions()

StartEvictions requests that f evict all evictable allocations. It does not wait for eviction to complete; for this, see MemoryFile.WaitForEvictions.

func (*MemoryFile) String

func (f *MemoryFile) String() string

String implements fmt.Stringer.String.

Note that because f.String locks f.mu, calling f.String internally (including indirectly through the fmt package) risks recursive locking. Within the pgalloc package, use f.usage directly instead.

func (*MemoryFile) TotalSize

func (f *MemoryFile) TotalSize() uint64

TotalSize returns the current size of the backing file in bytes, which is an upper bound on the amount of memory that can currently be allocated from the MemoryFile. The value returned by TotalSize is permitted to change.

func (*MemoryFile) TotalUsage

func (f *MemoryFile) TotalUsage() (uint64, error)

TotalUsage returns an aggregate usage for all memory statistics except Mapped (which is external to MemoryFile). This is generally much cheaper than UpdateUsage, but will not provide a fine-grained breakdown.

func (*MemoryFile) UpdateUsage

func (f *MemoryFile) UpdateUsage() error

UpdateUsage ensures that the memory usage statistics in usage.MemoryAccounting are up to date.

func (*MemoryFile) WaitForEvictions

func (f *MemoryFile) WaitForEvictions()

WaitForEvictions blocks until f is no longer evicting any evictable allocations.

type MemoryFileOpts

type MemoryFileOpts struct {
	// DelayedEviction controls the extent to which the MemoryFile may delay
	// eviction of evictable allocations.
	DelayedEviction DelayedEvictionType

	// If UseHostMemcgPressure is true, use host memory cgroup pressure level
	// notifications to determine when eviction is necessary. This option has
	// no effect unless DelayedEviction is DelayedEvictionEnabled.
	UseHostMemcgPressure bool

	// If ManualZeroing is true, MemoryFile must not assume that new pages
	// obtained from the host are zero-filled, such that MemoryFile must manually
	// zero newly-allocated pages.
	ManualZeroing bool
}

MemoryFileOpts provides options to NewMemoryFile.

type MemoryFileProvider

type MemoryFileProvider interface {
	// MemoryFile returns the Kernel MemoryFile.
	MemoryFile() *MemoryFile
}

MemoryFileProvider provides the MemoryFile method.

This type exists to work around a save/restore defect. The only object in a saved object graph that S/R allows to be replaced at time of restore is the starting point of the restore, kernel.Kernel. However, the MemoryFile changes between save and restore as well, so objects that need persistent access to the MemoryFile must instead store a pointer to the Kernel and call Kernel.MemoryFile() as required. In most cases, depending on the kernel package directly would create a package dependency loop, so the stored pointer must instead be a MemoryProvider interface object. Correspondingly, kernel.Kernel is the only implementation of this interface.

func MemoryFileProviderFromContext

func MemoryFileProviderFromContext(ctx context.Context) MemoryFileProvider

MemoryFileProviderFromContext returns the MemoryFileProvider used by ctx, or nil if no such MemoryFileProvider exists.