contentaddressable

README

Content Addressable

Package contentaddressable contains tools for writing content addressable files. Files are written to a temporary location, and only renamed to the final location after the file's OID (Object ID) has been verified.

filename := "path/to/01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b"
file, err := contentaddressable.NewFile(filename)
if err != nil {
  panic(err)
}
defer file.Close()

file.Oid // 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b

written, err := io.Copy(file, someReader)

if err == nil {
// Move file to final location if OID is verified.
  err = file.Accept()
}

if err != nil {
  panic(err)
}

See the godocs for details.

Installation

$ go get github.com/technoweenie/go-contentaddressable

Then import it:

import "github.com/technoweenie/go-contentaddressable"

Note on Patches/Pull Requests

1. Fork the project on GitHub.
2. Make your feature addition or bug fix.
3. Add tests for it. This is important so I don't break it in a future version unintentionally.
4. Commit, do not mess with version or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
5. Send me a pull request. Bonus points for topic branches.

Documentation

Overview

Package contentaddressable contains tools for reading and writing content addressable files. Files are written to a temporary location, and only renamed to the final location after the file's OID (Object ID) has been verified.

filename := "path/to/01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b"
file, err := contentaddressable.NewFile(filename)
if err != nil {
  panic(err)
}
defer file.Close()

file.Oid // 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b

written, err := io.Copy(file, someReader)

if err == nil {
// Move file to final location if OID is verified.
  err = file.Accept()
}

if err != nil {
  panic(err)
}

Currently SHA-256 is used for a file's OID.

You can also read files, while verifying that they are not corrupt.

filename := "path/to/01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b"

// get this from doing an os.Stat() or something
expectedSize := 123

// returns an io.ReadCloser
reader, err := contentaddressable.Open(filename, expectedSize)
if err != nil {
  panic(err)
}
defer file.Close()

// A contentaddressable.ReadCloser ensures that exactly the expectedSize
// number of bytes were read, and that the content matches the OID in the
// filename.
written, err := io.Copy(ioutil.Discard, reader)
if err != nil {
  panic(err)
}

Index

• Variables
• func Open(filename string) (io.ReadCloser, error)
• func Reader(reader io.ReadCloser, oid string) io.ReadCloser
• type File
  • func NewFile(filename string) (*File, error)
  • func NewWithSuffix(filename, suffix string) (*File, error)
  • func (w *File) Accept() error
  • func (w *File) Close() error
  • func (w *File) Closed() bool
  • func (w *File) Write(p []byte) (int, error)

Variables

var (
	AlreadyClosed = errors.New("Already closed.")
	HasData       = errors.New("Destination file already has data.")
	DefaultSuffix = "-temp"
)

Functions

func Open

func Open(filename string) (io.ReadCloser, error)

func Reader

func Reader(reader io.ReadCloser, oid string) io.ReadCloser

Types

type File

type File struct {
	Oid string
	// contains filtered or unexported fields
}

File handles the atomic writing of a content addressable file. It writes to a temp file, and then renames to the final location after Accept().

func NewFile

func NewFile(filename string) (*File, error)

NewFile initializes a content addressable file for writing. It is identical to NewWithSuffix, except it uses DefaultSuffix as the suffix.

func NewWithSuffix

func NewWithSuffix(filename, suffix string) (*File, error)

NewWithSuffix initializes a content addressable file for writing. It opens both the given filename, and a temp filename in exclusive mode. The *File OID is taken from the base name of the given filename.

func (*File) Accept

func (w *File) Accept() error

Accept verifies the written content SHA-256 signature matches the given OID. If it matches, the temp file is renamed to the original filename. If not, an error is returned.

func (*File) Close

func (w *File) Close() error

Close cleans up the internal file objects.

func (*File) Closed

func (w *File) Closed() bool

Closed reports whether this file object has been closed.

func (*File) Write

func (w *File) Write(p []byte) (int, error)

Write sends data to the temporary file.