Documentation
¶
Overview ¶
Package storage is for metadata of a tar archive.
Packing and unpacking the Entries of the stream. The types of streams are either segments of raw bytes (for the raw headers and various padding) and for an entry marking a file payload.
The raw bytes are stored precisely in the packed (marshalled) Entry, whereas the file payload marker include the name of the file, size, and crc64 checksum (for basic file integrity).
Index ¶
Constants ¶
This section is empty.
Variables ¶
var CRCTable = crc64.MakeTable(crc64.ISO)
CRCTable is the default table used for crc64 sum calculations
var ErrDuplicatePath = errors.New("duplicates of file paths not supported")
ErrDuplicatePath occurs when a tar archive has more than one entry for the same file path
Functions ¶
This section is empty.
Types ¶
type Entry ¶
type Entry struct {
Type Type `json:"type"`
Name string `json:"name,omitempty"`
NameRaw []byte `json:"name_raw,omitempty"`
Size int64 `json:"size,omitempty"`
Payload []byte `json:"payload"` // SegmentType stores payload here; FileType stores crc64 checksum here;
Position int `json:"position"`
}
Entry is the structure for packing and unpacking the information read from the Tar archive.
FileType Payload checksum is using `hash/crc64` for basic file integrity, _not_ for cryptography. From http://www.backplane.com/matt/crc64.html, CRC32 has almost 40,000 collisions in a sample of 18.2 million, CRC64 had none.
func (*Entry) GetName ¶ added in v0.9.8
GetName returns the string for the entry's name, regardless of the field stored in
func (*Entry) GetNameBytes ¶ added in v0.9.8
GetNameBytes returns the bytes for the entry's name, regardless of the field stored in
func (*Entry) SetName ¶ added in v0.9.8
SetName will check name for valid UTF-8 string, and set the appropriate field. See https://github.com/vbatts/tar-split/issues/17
func (*Entry) SetNameBytes ¶ added in v0.9.8
SetNameBytes will check name for valid UTF-8 string, and set the appropriate field
type FileGetPutter ¶
type FileGetPutter interface {
FileGetter
FilePutter
}
FileGetPutter is the interface that groups both Getting and Putting file payloads.
func NewBufferFileGetPutter ¶
func NewBufferFileGetPutter() FileGetPutter
NewBufferFileGetPutter is a simple in-memory FileGetPutter
Implication is this is memory intensive... Probably best for testing or light weight cases.
type FileGetter ¶
type FileGetter interface {
// Get returns a stream for the provided file path
Get(filename string) (output io.ReadCloser, err error)
}
FileGetter is the interface for getting a stream of a file payload, addressed by name/filename. Presumably, the names will be scoped to relative file paths.
func NewPathFileGetter ¶
func NewPathFileGetter(relpath string) FileGetter
NewPathFileGetter returns a FileGetter that is for files relative to path relpath.
type FilePutter ¶
type FilePutter interface {
// Put returns the size of the stream received, and the crc64 checksum for
// the provided stream
Put(filename string, input io.Reader) (size int64, checksum []byte, err error)
}
FilePutter is the interface for storing a stream of a file payload, addressed by name/filename.
func NewDiscardFilePutter ¶
func NewDiscardFilePutter() FilePutter
NewDiscardFilePutter is a bit bucket FilePutter
type Packer ¶
type Packer interface {
// AddEntry packs the Entry and returns its position
AddEntry(e Entry) (int, error)
}
Packer describes the methods to pack Entries to a storage destination
func NewJSONPacker ¶
NewJSONPacker provides a Packer that writes each Entry (SegmentType and FileType) as a json document.
The Entries are delimited by new line.
type Type ¶
type Type int
Type of Entry
const ( // FileType represents a file payload from the tar stream. // // This will be used to map to relative paths on disk. Only Size > 0 will get // read into a resulting output stream (due to hardlinks). FileType Type = 1 + iota // SegmentType represents a raw bytes segment from the archive stream. These raw // byte segments consist of the raw headers and various padding. // // Its payload is to be marshalled base64 encoded. SegmentType )
type Unpacker ¶
type Unpacker interface {
// Next returns the next Entry being unpacked, or error, until io.EOF
Next() (*Entry, error)
}
Unpacker describes the methods to read Entries from a source
func NewJSONUnpacker ¶
NewJSONUnpacker provides an Unpacker that reads Entries (SegmentType and FileType) as a json document.
Each Entry read are expected to be delimited by new line.
Source Files