vfsswift

Documentation

Overview

Package vfsswift is the implementation of the Virtual File System by using Swift from the OpenStack project. The file contents are saved in the object storage (Swift), and the metadata are indexed in CouchDB.

Index

• func DeleteContainer(ctx context.Context, c *swift.Connection, container string) error
• func MakeObjectName(docID string) string
• func MakeObjectNameV3(docID, internalID string) string
• func NewInternalID() string
• func NewThumbsFsV3(c *swift.Connection, db prefixer.Prefixer) vfs.Thumbser
• func NewV3(db vfs.Prefixer, index vfs.Indexer, disk vfs.DiskThresholder, ...) (vfs.VFS, error)

Functions

func DeleteContainer

func DeleteContainer(ctx context.Context, c *swift.Connection, container string) error

DeleteContainer removes all the files inside the given container, and then deletes it.

func MakeObjectName

func MakeObjectName(docID string) string

MakeObjectName build the swift object name for a given file document.It creates a virtual subfolder by splitting the document ID, which should be 32 bytes long, on the 27nth byte. This avoid having a flat hierarchy in swift with no bound

func MakeObjectNameV3

func MakeObjectNameV3(docID, internalID string) string

MakeObjectNameV3 builds the swift object name for a given file document. It creates a virtual subfolder by splitting the document ID, which should be 32 bytes long, on the 27nth byte. This avoid having a flat hierarchy in swift with no bound. And it appends the internalID at the end to regroup all the versions of a file in the same virtual subfolder.

func NewInternalID

func NewInternalID() string

NewInternalID returns a random string that can be used as an internal_vfs_id.

func NewThumbsFsV3

func NewThumbsFsV3(c *swift.Connection, db prefixer.Prefixer) vfs.Thumbser

NewThumbsFsV3 creates a new thumb filesystem base on swift.

This version stores the thumbnails in the same container as the main data container.

func NewV3

func NewV3(db vfs.Prefixer, index vfs.Indexer, disk vfs.DiskThresholder, mu lock.ErrorRWLocker) (vfs.VFS, error)

NewV3 returns a vfs.VFS instance associated with the specified indexer and the swift storage url.

This new V3 version uses only a single swift container per instance. We can easily put the thumbnails in the same container that the data. And, for the versioning, Swift Object Versioning is not what we want: it is not as robust as we expected (we had encoding issue with the V1 layout for file with `? ` in the name), and it is poor in features (for example, we want to swap an old version with the current version without having to download/upload contents, and it is not supported).