jsobs

README

jsobs - Javascript Object Store

JSOBS (pronounced "jay-sobs") is simple object storage for JSON-encodable objects in a PostgreSQL database.

Future versions may also support SQLite, Amazon S3, and (maybe) the file system.

WARNING! ALPHA SOFTWARE!

This package is new (as of June 2023) and has not been tested much. Like all software, it probably contains bugs, and like all new software it probably contains a lot of them. 🪲🪲🪲

Use Cases

Debugging API Roundtrips

Generate a request/response pair and serialize it to JSON, then store it with a ULID that matches the transaction in your logs. If you find a problem, you can reconstitute the full converstaion from the JSOBS data.

(This is the original use case that led to the JSOBS package.)

Data Warehouse for Serialized Objects

Warehouse your objects in JSON and query them using the powerful features of PostgreSQL's JSONB type. Reconstitute the objects in code as needed. This is a great match for ORM-centric systems.

Short-Term Local, Long-Term Remote Storage

Want discoverability in the short term, and stable long-term storage for things outside the window of attention? Double-store your data with two simple calls, each with its own TTL.

Limitations

Besides the limitations of your database(s), please keep in mind:

Minimal Metadata

There is no support for complex metadata at this time. Maybe later? Maybe not.

Purge On Shutdown

Currently the only time data is purged is when Shutdown is called. This may be problematic for your use case!

One solution is to just call Backend.Purge() at your leisure. Another, which may be added here in the future, is to run the same on a timer so that as long as the process is running, a purge happens ever (configurable) so often.

The latter seems nicer, but it also has a big downside: in busy systems, you might end up with resource contention.

Documentation

Overview

Package jsobs provides simple storage for objects serializable to JSON.

Index

• Variables
• func IsNotFound(err error) bool
• type Client
  • func New(bc backend.BackendClient, err error) (*Client, error)
  • func NewPgClient() (*Client, error)
  • func (c *Client) Count(prefix string) (int, error)
  • func (c *Client) CountAll() (int, error)
  • func (c *Client) Delete(path string) error
  • func (c *Client) List(prefix string) ([]string, error)
  • func (c *Client) ListDetail(prefix string) ([]backend.Detailer, error)
  • func (c *Client) Load(path string, obj any) error
  • func (c *Client) LoadDetail(path string) (backend.Detailer, error)
  • func (c *Client) LoadRaw(path string) ([]byte, error)
  • func (c *Client) Save(path string, obj any) error
  • func (c *Client) SaveExpiry(path string, obj any, expiry time.Time) error
  • func (c *Client) SaveRaw(path string, raw_obj []byte) error
  • func (c *Client) SaveRawExpiry(path string, raw_obj []byte, expiry time.Time) error
  • func (c *Client) Shutdown(code int)

Variables

var ExitFunc = os.Exit

Functions

func IsNotFound

func IsNotFound(err error) bool

Types

type Client

type Client struct {
	Backend backend.BackendClient
}

Client handles save, load, list and delete operations for its Backend.

func New

func New(bc backend.BackendClient, err error) (*Client, error)

New returns a client with the provided backend. Any error returned from the backend creation function is returned here.

func NewPgClient

func NewPgClient() (*Client, error)

NewPgClient returns a client with a connection pool to the PostgreSQL database specified at DATABASE_URL in the environment.

Configuration for pgx may be included in the URL.

func (*Client) Count

func (c *Client) Count(prefix string) (int, error)

Count returns the number of non-expired objects beginning with prefix. If none are found, zero is returned.

func (*Client) CountAll

func (c *Client) CountAll() (int, error)

CountAll returns the total number of non-expired objects.

func (*Client) Delete

func (c *Client) Delete(path string) error

Delete deletes the object at path. If the object does not exist, the error returned will be ErrNotFound.

func (*Client) List

func (c *Client) List(prefix string) ([]string, error)

List returns an array of all objects beginning with prefix. An empty array is not considered an error.

For S3, this operation may be slow if paged results are returned!

func (*Client) ListDetail

func (c *Client) ListDetail(prefix string) ([]backend.Detailer, error)

ListDetail returns an array of all Detailers describing all objects beginning with prefix. An empty array is not considered an error.

This operation may be slow for any backend returning paged results!

func (*Client) Load

func (c *Client) Load(path string, obj any) error

Load retrieves the object at path from storage and unmarshals it to obj.

func (*Client) LoadDetail

func (c *Client) LoadDetail(path string) (backend.Detailer, error)

LoadDetail retrieves the object at path and returns its details, but not the actual data.

func (*Client) LoadRaw

func (c *Client) LoadRaw(path string) ([]byte, error)

LoadRaw retrieves the object at path and returns its raw value.

func (*Client) Save

func (c *Client) Save(path string, obj any) error

Save marshals obj to json and stores it at path with no expiry. Any existing object at path will be overwritten.

func (*Client) SaveExpiry

func (c *Client) SaveExpiry(path string, obj any, expiry time.Time) error

SaveExpiry marshals obj to json and stores it at path with expiry set. Any existing object at path will be overwritten.

func (*Client) SaveRaw

func (c *Client) SaveRaw(path string, raw_obj []byte) error

SaveRaw behaves like Save but sends raw_obj directly.

Use with caution!

func (*Client) SaveRawExpiry

func (c *Client) SaveRawExpiry(path string, raw_obj []byte, expiry time.Time) error

SaveRawExpiry behaves like SaveExpiry but sends raw_obj directly.

Use with caution!

func (*Client) Shutdown

func (c *Client) Shutdown(code int)

Shutdown calls Backend.Shutdown, which should perform any shutdown operations such as purging expired items from the pool; and then calls ExitFunc with the provided exit code.

If Backend.Shutdown returns an error, 99 is used instead of code.