libstorage

README

libStorage

libStorage provides a vendor agnostic storage orchestration model, API, and reference client and server implementations.

Overview

libStorage enables storage consumption by leveraging methods commonly available, locally and/or externally, to an operating system (OS).

The Past

The libStorage project and its architecture represents a culmination of experience gained from the project authors' building of several different storage orchestration tools. While created using different languages and targeting disparate storage platforms, all the tools were architecturally aligned and embedded functionality directly inside the tools and affected storage platforms.

This shared design goal enabled tools that natively consumed storage, sans external dependencies.

The Present

Today libStorage focuses on adding value to container runtimes and storage orchestration tools such as Docker and Mesos, however the libStorage framework is available abstractly for more general usage across:

• Operating systems
• Storage platforms
• Hardware platforms
• Virtualization platforms

The client side implementation, focused on operating system activities, has a minimal set of dependencies in order to avoid a large, runtime footprint.

Storage Orchestration Tools Today

Today there are many storage orchestration and abstraction tools relevant to to container runtimes. These tools often must be installed locally and run alongside the container runtime.

The solid green lines represent active communication paths. The dotted black lines represent passive paths. The orange volume represents a operating system device and volume path available to the container runtime.

libStorage Embedded Architecture

Embedding libStorage client and server components enable container runtimes to communicate directly with storage platforms, the ideal architecture. This design requires minimal operational dependencies and is still able to provide volume management for container runtimes.

libStorage Centralized Architecture

In a centralized architecture, libStorage is hosted as a service, acting as a go-between for container runtimes and backend storage platforms.

The libStorage endpoint is advertised by a tool like REX-Ray, run from anywhere, and is responsible for all control plane operations to the storage platform along with maintaining escalated credentials for these platforms. All client based processes within the operating system are still embedded in the container runtime.

libStorage Decentralized Architecture

Similar to the centralized architecture, this implementation design involves running a separate libStorage process alongside each container runtime, across one or several hosts.

API

Central to libStorage is the HTTP/JSON API. It defines the control plane calls that occur between the client and server. While the libStorage package includes reference implementations of the client and server written using Go, both the client and server could be written using any language as long as both adhere to the published libStorage API.

Client

The libStorage client is responsible for discovering a host's instance ID and the next, available device name. The client's reference implementation is written using Go and is compatible with C++.

The design goal of the client is to be lightweight, portable, and avoid obsolescence by minimizing dependencies and focusing on deferring as much of the logic as possible to the server.

Server

The libStorage server implements the libStorage API and is responsible for coordinating requests between clients and backend orchestration packages. The server's reference implementation is also written using Go.

Model

The libStorage model defines several data structures that are easily represented using Go structs or a portable format such as JSON.

Documentation

The libStorage documentation is available at libstorage.rtfd.org.

Documentation

Overview

Package libstorage provides a vendor agnostic storage orchestration model, API, and reference client and server implementations. libStorage enables storage consumption by leveraging methods commonly available, locally and/or externally, to an operating system (OS).

The Past

The libStorage project and its architecture represents a culmination of experience gained from the project authors' building of several (http://bit.ly/1HIAet6) different storage (http://bit.ly/1Ya9Uft) orchestration tools (https://github.com/emccode/rexray). While created using different languages and targeting disparate storage platforms, all the tools were architecturally aligned and embedded functionality directly inside the tools and affected storage platforms.

This shared design goal enabled tools that natively consumed storage, sans external dependencies.

The Present

Today libStorage focuses on adding value to container runtimes and storage orchestration tools such as Docker and Mesos, however the libStorage framework is available abstractly for more general usage across:

• Operating systems
• Storage platforms
• Hardware platforms
• Virtualization platforms

The client side implementation, focused on operating system activities, has a minimal set of dependencies in order to avoid a large, runtime footprint.

Index

• func Dial(config gofig.Config) (types.Client, error)
• func New(config gofig.Config) (types.Client, io.Closer, error, <-chan error)
• func RegisterIntegrationDriver(name string, ctor types.NewIntegrationDriver)
• func RegisterOSDriver(name string, ctor types.NewOSDriver)
• func RegisterStorageDriver(name string, ctor types.NewStorageDriver)
• func Serve(config gofig.Config) (io.Closer, error, <-chan error)

Functions

func Dial

func Dial(config gofig.Config) (types.Client, error)

Dial opens a connection to a remote libStorage serice and returns the client that can be used to communicate with said endpoint.

If the config parameter is nil a default instance is created. The function dials the libStorage service specified by the configuration property libstorage.host.

func New

func New(config gofig.Config) (types.Client, io.Closer, error, <-chan error)

New returns a new libStorage client like the `Dial` function, but with one difference. If the `libstorag.host` key is not present in the provided configuration instance, a new server will be automatically started and returned.

While a new server may be launched, it's still up to the caller to provide a config instance with the correct properties to specify service information for a libStorage server.

func RegisterIntegrationDriver

func RegisterIntegrationDriver(name string, ctor types.NewIntegrationDriver)

RegisterIntegrationDriver registers a new IntegrationDriver with the libStorage service.

func RegisterOSDriver

func RegisterOSDriver(name string, ctor types.NewOSDriver)

RegisterOSDriver registers a new StorageDriver with the libStorage service.

func RegisterStorageDriver

func RegisterStorageDriver(
	name string, ctor types.NewStorageDriver)

RegisterStorageDriver registers a new StorageDriver with the libStorage service.

func Serve

func Serve(config gofig.Config) (io.Closer, error, <-chan error)

Serve starts the reference implementation of a server hosting an HTTP/JSON service that implements the libStorage API endpoint.

If the config parameter is nil a default instance is created. The libStorage service is served at the address specified by the configuration property libstorage.host.