memsys

package
v1.3.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 10, 2023 License: MIT Imports: 17 Imported by: 2

README

Overview

MMSA is, simultaneously, a) Slab and SGL allocator, and b) memory manager that is responsible to optimize memory usage between different (more vs less) utilized Slabs.

Multiple MMSA instances may coexist in the system, each having its own constraints and managing its own Slabs and SGLs.

MMSA includes a "house-keeping" part to monitor system resources, adjust Slab sizes based on their respective usages, and incrementally deallocate idle Slabs. To that end, MMSA utilizes housekeep (project and runner).

Construction

A typical initialization sequence includes steps, e.g.:

  1. Construct:

    mm := &memsys.MMSA{Name: ..., TimeIval: ..., MinPctFree: ..., Name: ...}
    

    Note that with the only exception of Name all the rest member variables (above) have their system defaults and can be omitted.

  2. Initialize:

    err := mm.Init(false /* don't panic on error */)
    if err != nil {
        ...
    }
    

The example above shows initialization that ignores errors - in particular, insifficient minimum required memory (see the previous section).

Alternatively, MMSA can be initialized not to panic on errors:

 mm.Init(true /* panic on error */)

In addition, there are several environment variables that can be used (to circumvent the need to change the code, for instance):

AIS_MINMEM_FREE
AIS_MINMEM_PCT_TOTAL
AIS_MINMEM_PCT_FREE
AIS_DEBUG

Minimum Available Memory

MMSA will try to make sure that there's a certain specified amount of memory that remains available at all times. Following are the rules to set this minimum:

  1. environment AIS_MINMEM_FREE takes precedence over everything else listed below;
  2. if AIS_MINMEM_FREE is not defined, variables AIS_MINMEM_PCT_TOTAL and/or AIS_MINMEM_PCT_FREE define percentages to compute the minimum based on the total or the currently available memory, respectively;
  3. with no environment, the minimum is computed based on the following MMSA member variables:
    • MinFree - memory that must be available at all times
    • MinPctTotal - same, via percentage of total
    • MinPctFree - ditto, as % of free at init time
    • Example:
      memsys.MMSA{MinPctTotal: 4, MinFree: cmn.GiB * 2}
      
  4. finally, if none of the above is specified, the constant minMemFree in the source

Termination

If the memory manager is no longer needed, terminating the MMSA instance is recommended. This will free up all the slabs allocated to the memory manager instance. Halt a running or initialized MMSA instance is done by:

mm.Terminate()

Operation

Once constructed and initialized, memory-manager-and-slab-allocator (MMSA) can be exercised via its public API that includes GetSlab, on the one hand and Alloc/AllocSize on the other.

Notice the difference:

  • GetSlab(fixed-bufsize) returns Slab that contains presizely fixed-bufsize sized reusable buffers
  • Alloc() and AllocSize() return both a Slab and an already allocated buffer from this Slab.

Note as well that Alloc() uses default buffer size for a given MMSA, while AllocSize() accepts the specified size (as the name implies).

Once selected, each Slab can be used via its own public API that includes Alloc and Free methods. In addition, each allocated SGL internally utilizes one of the existing enumerated slabs to "grow" (that is, allocate more buffers from the slab) on demand. For details, look for "grow" in the iosgl.go.

When running, the memory manager periodically evaluates the remaining free memory resource and adjusts its slabs accordingly. The entire logic is consolidated in one work() method that can, for instance, "cleanup" (see cleanup()) an existing "idle" slab, or forcefully "reduce" (see reduce()) one if and when the amount of free memory falls below watermark.

Testing

  • Run all tests while redirecting glog to STDERR:
$ go test -v -logtostderr=true
  • Same as above, with debug enabled and glog level = 4 (verbose):
$ AIS_DEBUG=memsys=4 go test -v -logtostderr=true -duration 2m -tags=debug
  • Run one of the named tests for 100 seconds:
$ go test -v -logtostderr=true -run=Test_Sleep -duration=100s
  • Same as above with debug and deadbeef (build tags) enabled:
$ go test -v -tags=debug,deadbeef -logtostderr=true -run=Test_Sleep -duration=100s
  • Run each test for 10 minutes with the permission to use up to 90% of total RAM (and glog => STDERR)
$ AIS_MINMEM_PCT_TOTAL=10 AIS_DEBUG=memsys=1 go test -v -logtostderr=true -run=No -duration 10m -timeout=1h
  • Same as above, with debug enabled, glog level = 1 (non-verbose), and verbose output generated by the tests:
$ AIS_MINMEM_PCT_TOTAL=10 AIS_DEBUG=memsys=1 go test -v -run=No -duration 10m -verbose true -tags=debug -timeout=1h

Global Memory Manager

In the interest of reusing a single memory manager instance across multiple packages outside the ais core package, the memsys package declares a gMem2 variable that can be accessed through the matching exported Getter. The notable runtime parameters that are used for the global memory manager are MinFreePct and TimeIval which are set to 50% and 2 minutes, respectively. Note that more specialized use cases which warrant custom memory managers with finely tuned parameters are free to create their own separate MMSA instances.

Usage:

To access the global memory manager, a single call to memsys.Init() is all that is required. Separate Init() nor Run() calls should not be made on the returned MMSA instance.

Documentation

Overview

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2022, NVIDIA CORPORATION. All rights reserved.

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.

Package memsys provides memory management and slab/SGL allocation with io.Reader and io.Writer interfaces on top of scatter-gather lists of reusable buffers.

  • Copyright (c) 2018-2022, NVIDIA CORPORATION. All rights reserved.

Index

Constants

View Source
const (
	PageSize            = cos.KiB * 4
	DefaultBufSize      = PageSize * 8
	DefaultSmallBufSize = cos.KiB
)
View Source
const (
	MaxPageSlabSize = 128 * cos.KiB
	PageSlabIncStep = PageSize
	NumPageSlabs    = MaxPageSlabSize / PageSlabIncStep // = 32
)

page slabs: pagesize increments up to MaxPageSlabSize

View Source
const (
	MaxSmallSlabSize = PageSize
	SmallSlabIncStep = 128
	NumSmallSlabs    = MaxSmallSlabSize / SmallSlabIncStep // = 32
)

small slabs: 128 byte increments up to MaxSmallSlabSize

View Source
const (
	PressureLow = iota
	PressureModerate
	PressureHigh
	PressureExtreme
	OOM
)
View Source
const NumStats = NumPageSlabs // NOTE: must be >= NumSmallSlabs

Variables

This section is empty.

Functions

func Init

func Init(gmmName, smmName string, config *cmn.Config)

Types

type FreeSpec

type FreeSpec struct {
	IdleDuration time.Duration // reduce only the slabs that are idling for at least as much time
	MinSize      int64         // minimum freed size that'd warrant calling GC (default = sizetoGC)
	Totally      bool          // true: free all slabs regardless of their idle-ness and size
	ToOS         bool          // GC and then return the memory to the operating system
}

type MMSA

type MMSA struct {
	// public
	MinFree     uint64        // memory that must be available at all times
	TimeIval    time.Duration // interval of time to watch for low memory and make steps
	MinPctTotal int           // same, via percentage of total
	MinPctFree  int           // ditto, as % of free at init time
	Name        string
	// contains filtered or unexported fields
}

func ByteMM

func ByteMM() *MMSA

system small-size allocator (range 1 - 4K)

func NewMMSA

func NewMMSA(name string) (mem *MMSA, err error)

func PageMM

func PageMM() *MMSA

system page-based memory-manager-slab-allocator (MMSA)

func (*MMSA) Alloc

func (r *MMSA) Alloc() (buf []byte, slab *Slab)

func (*MMSA) AllocSize

func (r *MMSA) AllocSize(size int64) (buf []byte, slab *Slab)

uses SelectMemAndSlab to select both MMSA (page or small) and its Slab

func (*MMSA) Append

func (r *MMSA) Append(buf []byte, bytes string) (nbuf []byte)

func (*MMSA) Free

func (r *MMSA) Free(buf []byte)

func (*MMSA) FreeSpec

func (r *MMSA) FreeSpec(spec FreeSpec)

API: on-demand memory freeing to the user-provided specification

func (*MMSA) GetSlab

func (r *MMSA) GetSlab(bufSize int64) (s *Slab, err error)

gets Slab for a given fixed buffer size that must be within expected range of sizes - the range supported by _this_ MMSA (compare w/ SelectMemAndSlab())

func (*MMSA) GetStats

func (r *MMSA) GetStats() (stats *Stats)

func (*MMSA) Init

func (r *MMSA) Init(maxUse int64) (err error)

initialize new MMSA instance

func (*MMSA) NewSGL

func (r *MMSA) NewSGL(immediateSize int64, sbufSize ...int64) *SGL

allocate SGL

  • immediateSize: known size, OR minimum expected size, OR size to preallocate immediateSize == 0 translates as DefaultBufSize - for page MMSA, and DefaultSmallBufSize - for small-size MMSA
  • sbufSize: slab buffer size (optional)

func (*MMSA) Pressure

func (r *MMSA) Pressure(mems ...*sys.MemStat) (pressure int)

returns an estimate for the current memory pressure expressed as enumerated values also, tracks swapping stateful vars

func (*MMSA) RegWithHK

func (r *MMSA) RegWithHK()

func (*MMSA) SelectMemAndSlab

func (r *MMSA) SelectMemAndSlab(size int64) (mmsa *MMSA, slab *Slab)

Given a known, expected or minimum size to allocate, selects MMSA (page or small, if initialized) and its Slab

func (*MMSA) Str

func (r *MMSA) Str(mem *sys.MemStat) string

func (*MMSA) String

func (r *MMSA) String() string

func (*MMSA) Terminate

func (r *MMSA) Terminate(unregHK bool)

terminate this MMSA instance and, possibly, GC as well

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

uses the underlying SGL to implement io.ReadWriteCloser + io.Seeker

func NewReader

func NewReader(z *SGL) *Reader

func (*Reader) Close

func (*Reader) Close() error

func (*Reader) Open

func (r *Reader) Open() (cos.ReadOpenCloser, error)

func (*Reader) Read

func (r *Reader) Read(b []byte) (n int, err error)

func (*Reader) Seek

func (r *Reader) Seek(from int64, whence int) (offset int64, err error)

type SGL

type SGL struct {
	// contains filtered or unexported fields
}

implements io.ReadWriteCloser + Reset

func (*SGL) Bytes

func (z *SGL) Bytes() (b []byte)

NOTE assert and use with caution: heap allocation (via ReadAll) is intended for tests (and only tests)

func (*SGL) Cap

func (z *SGL) Cap() int64

func (*SGL) Close

func (*SGL) Close() error

func (*SGL) Free

func (z *SGL) Free()

func (*SGL) IsNil

func (z *SGL) IsNil() bool

func (*SGL) Len

func (z *SGL) Len() int64

func (*SGL) Open

func (z *SGL) Open() (cos.ReadOpenCloser, error)

func (*SGL) Read

func (z *SGL) Read(b []byte) (n int, err error)

func (*SGL) ReadAll

func (z *SGL) ReadAll() (b []byte)

ReadAll is a strictly _convenience_ method as it performs heap allocation. Still, it's an optimized alternative to the generic io.ReadAll which normally returns err == nil (and not io.EOF) upon successful reading until EOF. ReadAll always returns err == nil.

func (*SGL) ReadByte

func (z *SGL) ReadByte() (byte, error)

func (*SGL) ReadFrom

func (z *SGL) ReadFrom(r io.Reader) (n int64, err error)

func (*SGL) Reset

func (z *SGL) Reset()

reuse already allocated SGL (compare with Reader below)

func (*SGL) Size

func (z *SGL) Size() int64

func (*SGL) Slab

func (z *SGL) Slab() *Slab

func (*SGL) UnreadByte

func (z *SGL) UnreadByte() error

func (*SGL) Write

func (z *SGL) Write(p []byte) (n int, err error)

func (*SGL) WriteAt

func (z *SGL) WriteAt(p []byte, off int64) (n int, err error)

NOTE assert and use with caution.

func (*SGL) WriteByte

func (z *SGL) WriteByte(c byte) error

func (*SGL) WriteTo

func (z *SGL) WriteTo(dst io.Writer) (n int64, err error)

NOTE: not advancing roff here - see usage

type Slab

type Slab struct {
	// contains filtered or unexported fields
}

func (*Slab) Alloc

func (s *Slab) Alloc() (buf []byte)

func (*Slab) Free

func (s *Slab) Free(buf []byte)

func (*Slab) MMSA

func (s *Slab) MMSA() *MMSA

func (*Slab) Size

func (s *Slab) Size() int64

func (*Slab) Tag

func (s *Slab) Tag() string

type Stats

type Stats struct {
	Hits [NumStats]uint64
	Idle [NumStats]time.Duration
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL