memory

Documentation

Index

• Constants
• func MemTotal() (uint64, error)
• func MemUsed() (uint64, error)
• type ActionOnExceed
• type LogOnExceed
  • func (a *LogOnExceed) Action(t *Tracker)
  • func (a *LogOnExceed) SetLogHook(hook func(uint64))
• type PanicOnExceed
  • func (a *PanicOnExceed) Action(t *Tracker)
  • func (a *PanicOnExceed) SetLogHook(hook func(uint64))
• type Tracker
  • func NewTracker(label fmt.Stringer, bytesLimit int64) *Tracker
  • func (t *Tracker) AttachTo(parent *Tracker)
  • func (t *Tracker) BytesConsumed() int64
  • func (t *Tracker) BytesToString(numBytes int64) string
  • func (t *Tracker) Consume(bytes int64)
  • func (t *Tracker) Detach()
  • func (t *Tracker) MaxConsumed() int64
  • func (t *Tracker) ReplaceChild(oldChild, newChild *Tracker)
  • func (t *Tracker) SetActionOnExceed(a ActionOnExceed)
  • func (t *Tracker) SetLabel(label fmt.Stringer)
  • func (t *Tracker) String() string

Constants

const (

	// PanicMemoryExceed represents the panic message when out of memory quota.
	PanicMemoryExceed string = "Out Of Memory Quota!"
)

Functions

func MemTotal

func MemTotal() (uint64, error)

MemTotal returns the total amount of RAM on this system

func MemUsed

func MemUsed() (uint64, error)

MemUsed returns the total used amount of RAM on this system

Types

type ActionOnExceed

type ActionOnExceed interface {
	// Action will be called when memory usage exceeds memory quota by the
	// corresponding Tracker.
	Action(t *Tracker)
	// SetLogHook binds a log hook which will be triggered and log an detailed
	// message for the out-of-memory sql.
	SetLogHook(hook func(uint64))
}

ActionOnExceed is the action taken when memory usage exceeds memory quota. NOTE: All the implementors should be thread-safe.

type LogOnExceed

type LogOnExceed struct {
	ConnID uint64
	// contains filtered or unexported fields
}

LogOnExceed logs a warning only once when memory usage exceeds memory quota.

func (*LogOnExceed) Action

func (a *LogOnExceed) Action(t *Tracker)

Action logs a warning only once when memory usage exceeds memory quota.

func (*LogOnExceed) SetLogHook

func (a *LogOnExceed) SetLogHook(hook func(uint64))

SetLogHook sets a hook for LogOnExceed.

type PanicOnExceed

type PanicOnExceed struct {
	ConnID uint64
	// contains filtered or unexported fields
}

PanicOnExceed panics when memory usage exceeds memory quota.

func (*PanicOnExceed) Action

func (a *PanicOnExceed) Action(t *Tracker)

Action panics when memory usage exceeds memory quota.

func (*PanicOnExceed) SetLogHook

func (a *PanicOnExceed) SetLogHook(hook func(uint64))

SetLogHook sets a hook for PanicOnExceed.

type Tracker

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker is used to track the memory usage during query execution. It contains an optional limit and can be arranged into a tree structure such that the consumption tracked by a Tracker is also tracked by its ancestors. The main idea comes from Apache Impala:

https://github.com/cloudera/Impala/blob/cdh5-trunk/be/src/runtime/mem-tracker.h

By default, memory consumption is tracked via calls to "Consume()", either to the tracker itself or to one of its descendents. A typical sequence of calls for a single Tracker is: 1. tracker.SetLabel() / tracker.SetActionOnExceed() / tracker.AttachTo() 2. tracker.Consume() / tracker.ReplaceChild() / tracker.BytesConsumed()

NOTE: We only protect concurrent access to "bytesConsumed" and "children", that is to say: 1. Only "BytesConsumed()", "Consume()", "AttachTo()" and "Detach" are thread-safe. 2. Other operations of a Tracker tree is not thread-safe.

func NewTracker

func NewTracker(label fmt.Stringer, bytesLimit int64) *Tracker

NewTracker creates a memory tracker.

1. "label" is the label used in the usage string.
2. "bytesLimit < 0" means no limit.

func (*Tracker) AttachTo

func (t *Tracker) AttachTo(parent *Tracker)

AttachTo attaches this memory tracker as a child to another Tracker. If it already has a parent, this function will remove it from the old parent. Its consumed memory usage is used to update all its ancestors.

func (*Tracker) BytesConsumed

func (t *Tracker) BytesConsumed() int64

BytesConsumed returns the consumed memory usage value in bytes.

func (*Tracker) BytesToString

func (t *Tracker) BytesToString(numBytes int64) string

BytesToString converts the memory consumption to a readable string.

func (*Tracker) Consume

func (t *Tracker) Consume(bytes int64)

Consume is used to consume a memory usage. "bytes" can be a negative value, which means this is a memory release operation.

func (*Tracker) Detach

func (t *Tracker) Detach()

Detach detaches this Tracker from its parent.

func (*Tracker) MaxConsumed

func (t *Tracker) MaxConsumed() int64

MaxConsumed returns max number of bytes consumed during execution.

func (*Tracker) ReplaceChild

func (t *Tracker) ReplaceChild(oldChild, newChild *Tracker)

ReplaceChild removes the old child specified in "oldChild" and add a new child specified in "newChild". old child's memory consumption will be removed and new child's memory consumption will be added.

func (*Tracker) SetActionOnExceed

func (t *Tracker) SetActionOnExceed(a ActionOnExceed)

SetActionOnExceed sets the action when memory usage is out of memory quota.

func (*Tracker) SetLabel

func (t *Tracker) SetLabel(label fmt.Stringer)

SetLabel sets the label of a Tracker.

func (*Tracker) String

func (t *Tracker) String() string

String returns the string representation of this Tracker tree.