mmap

README

mmap

Interface for mmap syscall to provide safe and efficient access to memory. *mmap.File satisfies both io.ReaderAt and io.WriterAt interfaces.

Only works for darwin OS, Linux and Little Endian 64 bit architectures.

Safety & Efficiency

Golang mmap syscall function exposes the mapped memory as array of bytes. If the array is referenced even after the memory region is unmapped, this can lead to segmentation fault. mmap package provides safe access to the array of bytes by providing ReadAt and WriteAt functions.

WriteAt function copies a slice into the memory mapped region whereas ReadAt function copies data from memory mapped region to a given slice, therefore, avoiding exposing the array of bytes referring to mapped memory. This also avoids any extra data copy providing efficient access to the memory mapped region.

We have also added functions such as WriteUint64At, ReadUint64At that can directly typecast the mmaped memory to Uint64 and avoids an extra copy. We will add more functions in the library based on our use cases. If you need support for a particular function, let us know or better, raise a pull request.

Similar Packages

• golang.org/x/exp/mmap
• github.com/riobard/go-mmap
• launchpad.net/gommap
• github.com/edsrzf/mmap-go

Documentation

Overview

Package mmap provides higher level abstractions around a memory mapped file. For now, package only support creating mmap with backed file on disk (shared mappings). Anonymous mappings are currently not supported. Please see godoc for various function references.

Index

• Variables
• type File
  • func NewSharedFileMmap(f *os.File, offset int64, length int, prot int) (*File, error)
  • func (m *File) Advise(advice int) error
  • func (m *File) Flush(flags int) error
  • func (m *File) Lock() error
  • func (m *File) ReadAt(dest []byte, offset int64) (int, error)
  • func (m *File) ReadStringAt(dest *strings.Builder, offset int64) int
  • func (m *File) ReadUint64At(offset int64) uint64
  • func (m *File) Unlock() error
  • func (m *File) Unmap() error
  • func (m *File) WriteAt(src []byte, offset int64) (int, error)
  • func (m *File) WriteStringAt(src string, offset int64) int
  • func (m *File) WriteUint64At(num uint64, offset int64)

Variables

var (
	// ErrUnmappedMemory is returned when a function is called on unmapped memory.
	ErrUnmappedMemory = errors.New("unmapped memory")
	// ErrIndexOutOfBound is returned when given offset lies beyond the mapped region.
	ErrIndexOutOfBound = errors.New("offset out of mapped region")
)

Types

type File

type File struct {
	// contains filtered or unexported fields
}

File provides abstraction around a memory mapped file.

func NewSharedFileMmap

func NewSharedFileMmap(f *os.File, offset int64, length int, prot int) (*File, error)

NewSharedFileMmap maps a file into memory starting at a given offset, for given length. For documentation regarding prot, see documentation for syscall package. possible cases:

case 1 => if   file size > memory region (offset + length)
          then all the mapped memory is accessible
case 2 => if   file size <= memory region (offset + length)
          then from offset to file size memory region is accessible

func (*File) Advise

func (m *File) Advise(advice int) error

Advise provides hints to kernel regarding the use of memory mapped region.

func (*File) Flush

func (m *File) Flush(flags int) error

Flush flushes the memory mapped region to disk. Flush makes a syscall only if the memory region is modified since the last flush.

func (*File) Lock

func (m *File) Lock() error

Lock locks all the mapped memory to RAM, preventing the pages from swapping out.

func (*File) ReadAt

func (m *File) ReadAt(dest []byte, offset int64) (int, error)

ReadAt copies data to dest slice from mapped region starting at given offset and returns number of bytes copied to the dest slice. There are two possibilities -

Case 1: len(dest) >= (m.length - offset)
     => copies (m.length - offset) bytes to dest from mapped region
Case 2: len(dest) < (m.length - offset)
     => copies len(dest) bytes to dest from mapped region

err is always nil, hence, can be ignored.

func (*File) ReadStringAt

func (m *File) ReadStringAt(dest *strings.Builder, offset int64) int

ReadStringAt copies data to dest string builder from mapped region starting at given offset until the min value of (length - offset) or (dest.Cap() - dest.Len()) and returns number of bytes copied to the dest slice.

func (*File) ReadUint64At

func (m *File) ReadUint64At(offset int64) uint64

ReadUint64At reads uint64 from offset.

func (*File) Unlock

func (m *File) Unlock() error

Unlock unlocks the mapped memory from RAM, enabling swapping out of RAM if required.

func (*File) Unmap

func (m *File) Unmap() error

Unmap unmaps the memory mapped file. An error will be returned if any of the functions are called on Mmap after calling Unmap.

func (*File) WriteAt

func (m *File) WriteAt(src []byte, offset int64) (int, error)

WriteAt copies data to mapped region from the src slice starting at given offset and returns number of bytes copied to the mapped region. There are two possibilities -

Case 1: len(src) >= (m.length - offset)
    => copies (m.length - offset) bytes to the mapped region from src
Case 2: len(src) < (m.length - offset)
    => copies len(src) bytes to the mapped region from src

err is always nil, hence, can be ignored.

func (*File) WriteStringAt

func (m *File) WriteStringAt(src string, offset int64) int

WriteStringAt copies data to mapped region from the src string starting at given offset and returns number of bytes copied to the mapped region.

func (*File) WriteUint64At

func (m *File) WriteUint64At(num uint64, offset int64)

WriteUint64At writes num at offset.