dump

README

dump

Simple Go library for in-memory data storage with file persistence.

features

• No data races
• Everything is stored in memory
• Multiple ways to persist to disk
• Small and fast

what is this for?

I wanted a safe way to access Go slices from multiple goroutines and the ability to save those slices to the disk. This is useful for APIs (see example) that require fast data access.

persistence

Dumps save to the disk (usually with a ".db" file extension). There are currently three persistence settings available.

manually

Using the dump.PERSIST_MANUAL constant allows you to control when the dump is saved to disk. You can manually save the dump by calling the *Dump.Save() function.

... = dump.NewDump(..., dump.PERSIST_MANUAL, ...)

on writes

Using the dump.PERSIST_WRITES constant will cause the dump to save to disk when *Dump.Add() or *Dump.Update() is called.

... = dump.NewDump(..., dump.PERSIST_WRITES, ...)

on an interval

Using the dump.PERSIST_INTERVAL constant will cause the dump to save to disk on a timed interval (currently 60 seconds).

... = dump.NewDump(..., dump.PERSIST_INTERVAL, ...)

examples

creating a dump

users, err := dump.NewDump("users.db", dump.PERSIST_WRITES, dump.Type{"main.User", User{}})

adding an item

// id is assigned to the index location of the item after it is added
id, err := users.Add(&User{Name: "karl"})

getting an item

err := users.View(func(items []dump.Item) error {
    println(items[id].(*User).Name) // will output "karl"
    return nil
})

updating an item

err := users.Update(func(items []dump.Item) error {
    items[id].(*User).Name = "santa"
    return nil
})

Documentation

Overview

Package dump manages a list of items in memory with concurrency-safe functions for accessing and manipulating them.

Index

• Constants
• Variables
• type Dump
  • func NewDump(filename string, persist int, types ...Type) (*Dump, error)
  • func (d *Dump) Add(item Item) (int, error)
  • func (d *Dump) Load() error
  • func (d *Dump) Map(f func(item Item) error) error
  • func (d *Dump) MarshalJSON() ([]byte, error)
  • func (d *Dump) Save() error
  • func (d *Dump) Update(f func(items []Item) error) error
  • func (d *Dump) View(f func(items []Item) error) error
• type Item
• type Type

Constants

const (
	// PERSIST_MANUAL is the default disk-persistence setting. The user has to
	// manually save the dump to disk using the Save() function.
	PERSIST_MANUAL = iota

	// PERSIST_WRITES is a disk-persistence setting that will save the dump
	// to disk when either Add() or Update() is called.
	PERSIST_WRITES

	// PERSIST_INTERVAL is a disk-persistence setting that will save the dump
	// on a set interval (currently once every 60 seconds).
	PERSIST_INTERVAL
)

Variables

var (
	// ErrInvalidPersist is thrown when an invalid disk-persistence setting is
	// provided when calling NewDump().
	ErrInvalidPersist = errors.New("invalid persist type")

	// ErrInvalidTypes is thrown when no types are listed when calling
	// NewDump(). Without type definitions it is impossible to persist on disk
	// without possible data loss.
	ErrInvalidTypes = errors.New("no types were provided")

	// ErrInvalidFilename is thrown when NewDump() is called with an empty
	// filename - making persistence impossible.
	ErrInvalidFilename = errors.New("invalid filename")
)

Types

type Dump

type Dump struct {
	// contains filtered or unexported fields
}

Dump represents a collection of items that persist on disk.

func NewDump

func NewDump(filename string, persist int, types ...Type) (*Dump, error)

NewDump is the primary constructor function for creating dumps. The provided filename is where the dump will persist to disk (and read from disk). The persist int is one of the dump.PERSIST_ constants. The provided types register the types that will be held in the dump.

NewDump will return an error if the persist parameter is not a valid dump.PERSIST_ constant.

func (*Dump) Add

func (d *Dump) Add(item Item) (int, error)

Add appends an Item on the end of the dump. It returns the id of the item and an error if there was a problem persisting the dump on the disk (if PERSIST_WRITE is enabled).

func (*Dump) Load

func (d *Dump) Load() error

Load reads the dump from disk using the filename provided when NewDump() was called.

func (*Dump) Map

func (d *Dump) Map(f func(item Item) error) error

Map applies the function f to each item in the dump. It returns an error if f returns an error for one of the items. If PERSIST_WRITES is enabled Map might also return an error if there is an error saving the dump to disk.

func (*Dump) MarshalJSON

func (d *Dump) MarshalJSON() ([]byte, error)

MarshalJSON returns the dump as a JSON list. It returns an error if there was an error marshaling one of the items.

func (*Dump) Save

func (d *Dump) Save() error

Save persists the dump on disk using the filename provided when NewDump() was called.

func (*Dump) Update

func (d *Dump) Update(f func(items []Item) error) error

Update is used to manipulate an item (or items) in the dump. It returns an error if there is an error saving the dump (if PERSIST_WRITES is enabled) or if there is an error inside the f function.

func (*Dump) View

func (d *Dump) View(f func(items []Item) error) error

View is used to read an item (or items) in the dump. It returns an error if there is an error inside the f function.

type Item

type Item interface {
	MarshalJSON() ([]byte, error)
}

Item implements the json.Marshaler interface and is used so that the dump itself can implement the json.Marshaler function by aggregating all items.

type Type

type Type struct {
	// Name is the name of this type. Usually it takes the form of
	// "package.Name" -- so for a struct User{} in package main this field
	// would be "main.User".
	Name string

	// Value is an empty struct of this type. For a struct User{}, this would
	// be User{}.
	Value interface{}
}

Type is used to register types from outside packages so that they are recognized when loading or saving the dump.