continuity

README

continuity

A transport-agnostic, filesystem metadata manifest system

This project is a staging area for experiments in providing transport agnostic metadata storage.

Please see https://github.com/opencontainers/specs/issues/11 for more details.

Manifest Format

A continuity manifest encodes filesystem metadata in Protocol Buffers. Please refer to proto/manifest.proto.

Usage

Build:

$ make

Create a manifest (of this repo itself):

$ ./bin/continuity build . > /tmp/a.pb

Dump a manifest:

$ ./bin/continuity ls /tmp/a.pb
...
-rw-rw-r--      270 B   /.gitignore
-rw-rw-r--      88 B    /.mailmap
-rw-rw-r--      187 B   /.travis.yml
-rw-rw-r--      359 B   /AUTHORS
-rw-rw-r--      11 kB   /LICENSE
-rw-rw-r--      1.5 kB  /Makefile
...
-rw-rw-r--      986 B   /testutil_test.go
drwxrwxr-x      0 B     /version
-rw-rw-r--      478 B   /version/version.go

Verify a manifest:

$ ./bin/continuity verify . /tmp/a.pb

Break the directory and restore using the manifest:

$ chmod 777 Makefile
$ ./bin/continuity verify . /tmp/a.pb
2017/06/23 08:00:34 error verifying manifest: resource "/Makefile" has incorrect mode: -rwxrwxrwx != -rw-rw-r--
$ ./bin/continuity apply . /tmp/a.pb
$ stat -c %a Makefile
664
$ ./bin/continuity verify . /tmp/a.pb

Platforms

continuity primarily targets Linux. continuity may compile for and work on other operating systems, but those platforms are not tested.

Contribution Guide

Building Proto Package

If you change the proto file you will need to rebuild the generated Go with go generate.

$ go generate ./proto

Project details

continuity is a containerd sub-project, licensed under the Apache 2.0 license. As a containerd sub-project, you will find the:

• Project governance,
• Maintainers,
• and Contributing guidelines

information in our containerd/project repository.

Documentation

Index

• Variables
• func ApplyManifest(ctx Context, manifest *Manifest) error
• func AtomicWriteFile(filename string, data []byte, perm os.FileMode) error
• func Marshal(m *Manifest) ([]byte, error)
• func MarshalText(w io.Writer, m *Manifest) error
• func VerifyManifest(ctx Context, manifest *Manifest) error
• type ByPath
  • func (bp ByPath) Len() int
  • func (bp ByPath) Less(i, j int) bool
  • func (bp ByPath) Swap(i, j int)
• type ContentProvider
• type Context
  • func NewContext(root string) (Context, error)
  • func NewContextWithOptions(root string, options ContextOptions) (Context, error)
• type ContextOptions
• type Device
• type Digester
• type Directory
• type Hardlinkable
• type Manifest
  • func BuildManifest(ctx Context) (*Manifest, error)
  • func Unmarshal(p []byte) (*Manifest, error)
• type NamedPipe
• type RegularFile
• type Resource
  • func Merge(fs ...Resource) (Resource, error)
• type SymLink
• type SymlinkPath
• type XAttrer

Variables

var (
	// ErrNotFound represents the resource not found
	ErrNotFound = fmt.Errorf("not found")
	// ErrNotSupported represents the resource not supported
	ErrNotSupported = fmt.Errorf("not supported")
)

Functions

func ApplyManifest

func ApplyManifest(ctx Context, manifest *Manifest) error

ApplyManifest applies on the resources in a manifest to the given context.

func AtomicWriteFile

func AtomicWriteFile(filename string, data []byte, perm os.FileMode) error

AtomicWriteFile atomically writes data to a file by first writing to a temp file and calling rename.

func Marshal

func Marshal(m *Manifest) ([]byte, error)

func MarshalText

func MarshalText(w io.Writer, m *Manifest) error

func VerifyManifest

func VerifyManifest(ctx Context, manifest *Manifest) error

VerifyManifest verifies all the resources in a manifest against files from the given context.

Types

type ByPath

type ByPath []Resource

ByPath provides the canonical sort order for a set of resources. Use with sort.Stable for deterministic sorting.

func (ByPath) Len

func (bp ByPath) Len() int

func (ByPath) Less

func (bp ByPath) Less(i, j int) bool

func (ByPath) Swap

func (bp ByPath) Swap(i, j int)

type ContentProvider

type ContentProvider interface {
	Reader(digest.Digest) (io.ReadCloser, error)
}

ContentProvider produces a read stream for a given digest

type Context

type Context interface {
	Apply(Resource) error
	Verify(Resource) error
	Resource(string, os.FileInfo) (Resource, error)
	Walk(filepath.WalkFunc) error
}

Context represents a file system context for accessing resources. The responsibility of the context is to convert system specific resources to generic Resource objects. Most of this is safe path manipulation, as well as extraction of resource details.

func NewContext

func NewContext(root string) (Context, error)

NewContext returns a Context associated with root. The default driver will be used, as returned by NewDriver.

func NewContextWithOptions

func NewContextWithOptions(root string, options ContextOptions) (Context, error)

NewContextWithOptions returns a Context associate with the root.

type ContextOptions

type ContextOptions struct {
	Digester   Digester
	Driver     driverpkg.Driver
	PathDriver pathdriver.PathDriver
	Provider   ContentProvider
}

ContextOptions represents options to create a new context.

type Device

type Device interface {
	Resource
	Hardlinkable
	XAttrer

	Major() uint64
	Minor() uint64
}

type Digester

type Digester interface {
	Digest(io.Reader) (digest.Digest, error)
}

Digester produces a digest for a given read stream

type Directory

type Directory interface {
	Resource
	XAttrer

	// Directory is a no-op method to identify directory objects by interface.
	Directory()
}

type Hardlinkable

type Hardlinkable interface {
	// Paths returns all paths of the resource, including the primary path
	// returned by Resource.Path. If len(Paths()) > 1, the resource is a hard
	// link.
	Paths() []string
}

Hardlinkable is an interface that a resource type satisfies if it can be a hardlink target.

type Manifest

type Manifest struct {
	// Resources specifies all the resources for a manifest in order by path.
	Resources []Resource
}

Manifest provides the contents of a manifest. Users of this struct should not typically modify any fields directly.

func BuildManifest

func BuildManifest(ctx Context) (*Manifest, error)

BuildManifest creates the manifest for the given context

func Unmarshal

func Unmarshal(p []byte) (*Manifest, error)

type NamedPipe

type NamedPipe interface {
	Resource
	Hardlinkable
	XAttrer

	// Pipe is a no-op method to allow consistent resolution of NamedPipe
	// interface.
	Pipe()
}

type RegularFile

type RegularFile interface {
	Resource
	XAttrer
	Hardlinkable

	Size() int64
	Digests() []digest.Digest
}

type Resource

type Resource interface {
	// Path provides the primary resource path relative to the bundle root. In
	// cases where resources have more than one path, such as with hard links,
	// this will return the primary path, which is often just the first entry.
	Path() string

	// Mode returns the
	Mode() os.FileMode

	UID() int64
	GID() int64
}

func Merge

func Merge(fs ...Resource) (Resource, error)

Merge two or more Resources into new file. Typically, this should be used to merge regular files as hardlinks. If the files are not identical, other than Paths and Digests, the merge will fail and an error will be returned.

type SymLink

type SymLink interface {
	Resource

	// Target returns the target of the symlink contained in the .
	Target() string
}

type SymlinkPath

type SymlinkPath func(root, linkname, target string) (string, error)

SymlinkPath is intended to give the symlink target value in a root context. Target and linkname are absolute paths not under the given root.

type XAttrer

type XAttrer interface {
	XAttrs() map[string][]byte
}