ring

README

MMU-ring

MMU-ring is a copy-free hardware ring buffer implementation.

It never copies data around, yet it always provide single contiguous buffers for both content and unused space.

This is achieved by using the CPU's MMU to map the same physical memory twice head to tail. This means, in virtual memory we allocate a buffer twice as big, however the second half points to the same physical memory as the first half. Then if the ring would wrap around, rather than having to handle this into two steps or return two buffers we can overflow from the first mapping into the second mirror mapping.

The buffers are valid for any operation, can be passed to the OS like with os.File.Read.

Creating and destroying an MMU buffer require doing a couple of syscalls, altho no IO and is slightly costier than a heap allocation. There is no ongoing CPU cost once it has been created.

You need to call .Close when you are done using the buffers, the special mappings are not collected by GC.

This is incompatible with linux no MMU (altho go does not support linux no MMU anyway so not sure how you would manage to run anyway).

TODO:

• Windows support, they have APIs designed to do very exactly this.
• MacOS Support, idk.
• Generic posix support, shm ¿ altho that not anonymous.
• Copy-free Resizing (remap)

Documentation

Index

• type Ring
  • func New(size uintptr) (*Ring, error)
  • func (r *Ring) Advance(n uintptr) error
  • func (r *Ring) Close() error
  • func (r *Ring) Consume(n uintptr) error
  • func (r *Ring) Content() []byte
  • func (r *Ring) Init(size uintptr) (err error)
  • func (r *Ring) Read(f func(buffer []byte) (consumed uintptr, err error)) (consumed uintptr, err error)
  • func (r *Ring) Size() uintptr
  • func (r *Ring) Unused() []byte
  • func (r *Ring) Write(f func(buffer []byte) (newData uintptr, err error)) (newData uintptr, err error)

Types

type Ring

type Ring struct {
	// contains filtered or unexported fields
}

func New

func New(size uintptr) (*Ring, error)

func (*Ring) Advance

func (r *Ring) Advance(n uintptr) error

Advance bumps the head, in other words it tells the ring that you have written to the slice returned by .Unused.

func (*Ring) Close

func (r *Ring) Close() error

Close frees up the mmaped memory and must be manually called before the Ring is garbage collected. Very important, any buffer returned by .Context or .Unused, or leaked from .Write or .Read will become invalid and point to who knows what and cannot be used. If you *retain* theses buffers, you might need to set them to nil before calling Close.

func (*Ring) Consume

func (r *Ring) Consume(n uintptr) error

func (*Ring) Content

func (r *Ring) Content() []byte

Content returns a single contiguous slice of the content in the ring so you can pass it to os.File.Write or whatever is gonna read from it. It returns a byte slice that is only valid before calling Ring.Close, you should avoid retaining this slice and might need to manually nil it out before calling Close to avoid the GC attempting to scan unmapped memory.

func (*Ring) Init

func (r *Ring) Init(size uintptr) (err error)

Init initializes the ring buffer with the given size.

func (*Ring) Read

func (r *Ring) Read(f func(buffer []byte) (consumed uintptr, err error)) (consumed uintptr, err error)

Read is an alternative to Content and Consume, you get called back with a reference to the used buffer and return how many bytes you have consumed. The passed-in slice is not valid once f returns.

func (*Ring) Size

func (r *Ring) Size() uintptr

func (*Ring) Unused

func (r *Ring) Unused() []byte

Unused returns a single contiguous slice of unused memory in the ring so you can pass it to os.File.Read or whatever is gonna write to it. It returns a byte slice that is only valid before calling Ring.Close, you should avoid retaining this slice and might need to manually nil it out before calling Close to avoid the GC attempting to scan unmapped memory.

func (*Ring) Write

func (r *Ring) Write(f func(buffer []byte) (newData uintptr, err error)) (newData uintptr, err error)

Write is an alternative to Unused and Advance, you get called back with a reference to the unused buffer and return how many new bytes are there. The passed-in slice is not valid once f returns.