memory

Documentation

Index

• Constants
• type ActionOnExceed
• type LogOnExceed
  • func (a *LogOnExceed) Action(t *Tracker)
• type PanicOnExceed
  • func (a *PanicOnExceed) Action(t *Tracker)
• type Tracker
  • func NewTracker(label string, bytesLimit int64) *Tracker
  • func (t *Tracker) AttachTo(parent *Tracker)
  • func (t *Tracker) BytesConsumed() int64
  • func (t *Tracker) Consume(bytes int64)
  • func (t *Tracker) Detach()
  • func (t *Tracker) ReplaceChild(oldChild, newChild *Tracker)
  • func (t *Tracker) SetActionOnExceed(a ActionOnExceed)
  • func (t *Tracker) SetLabel(label string)
  • func (t *Tracker) String() string

Constants

const (

	// PanicMemoryExceed represents the panic message when out of memory quota.
	PanicMemoryExceed string = "Out Of Memory Quota!"
)

Types

type ActionOnExceed

type ActionOnExceed interface {
	// Action will be called when memory usage exceeds memory quota by the
	// corresponding Tracker.
	Action(t *Tracker)
}

ActionOnExceed is the action taken when memory usage exceeds memory quota. NOTE: All the implementors should be thread-safe.

type LogOnExceed

type LogOnExceed struct {
	// contains filtered or unexported fields
}

LogOnExceed logs a warning only once when memory usage exceeds memory quota.

func (*LogOnExceed) Action

func (a *LogOnExceed) Action(t *Tracker)

Action logs a warning only once when memory usage exceeds memory quota.

type PanicOnExceed

type PanicOnExceed struct {
	// contains filtered or unexported fields
}

PanicOnExceed panics when when memory usage exceeds memory quota.

func (*PanicOnExceed) Action

func (a *PanicOnExceed) Action(t *Tracker)

Action panics when when memory usage exceeds memory quota.

type Tracker

type Tracker struct {
	sync.Mutex // For synchronization.
	// contains filtered or unexported fields
}

Tracker is used to track the memory usage during query execution. It contains an optional limit and can be arranged into a tree structure such that the consumption tracked by a Tracker is also tracked by its ancestors. The main idea comes from Apache Impala:

https://github.com/cloudera/Impala/blob/cdh5-trunk/be/src/runtime/mem-tracker.h

By default, memory consumption is tracked via calls to "Consume()", either to the tracker itself or to one of its descendents. A typical sequence of calls for a single Tracker is: 1. tracker.SetLabel() / tracker.SetActionOnExceed() / tracker.AttachTo() 2. tracker.Consume() / tracker.ReplaceChild() / tracker.BytesConsumed()

NOTE: We only protect concurrent access to "bytesConsumed" and "children", that is to say: 1. Only "BytesConsumed()", "Consume()", "AttachTo()" and "Detach" are thread-safe. 2. Other operations of a Tracker tree is not thread-safe.

func NewTracker

func NewTracker(label string, bytesLimit int64) *Tracker

NewTracker creates a memory tracker.

1. "label" is the label used in the usage string.
2. "bytesLimit < 0" means no limit.

func (*Tracker) AttachTo

func (t *Tracker) AttachTo(parent *Tracker)

AttachTo attaches this memory tracker as a child to another Tracker. If it already has a parent, this function will remove it from the old parent. Its consumed memory usage is used to update all its ancestors.

func (*Tracker) BytesConsumed

func (t *Tracker) BytesConsumed() int64

BytesConsumed returns the consumed memory usage value in bytes.

func (*Tracker) Consume

func (t *Tracker) Consume(bytes int64)

Consume is used to consume a memory usage. "bytes" can be a negative value, which means this is a memory release operation.

func (*Tracker) Detach

func (t *Tracker) Detach()

Detach detaches this Tracker from its parent.

func (*Tracker) ReplaceChild

func (t *Tracker) ReplaceChild(oldChild, newChild *Tracker)

ReplaceChild removes the old child specified in "oldChild" and add a new child specified in "newChild". old child's memory consumption will be removed and new child's memory consumption will be added.

func (*Tracker) SetActionOnExceed

func (t *Tracker) SetActionOnExceed(a ActionOnExceed)

SetActionOnExceed sets the action when memory usage is out of memory quota.

func (*Tracker) SetLabel

func (t *Tracker) SetLabel(label string)

SetLabel sets the label of a Tracker.

func (*Tracker) String

func (t *Tracker) String() string

String returns the string representation of this Tracker tree.