axis2

README

AXIS VFS, a simple virtual file system API.

AXIS is based on a few simple interfaces and a set of API functions that operate on these interfaces. Clients use the provided implementations of these interfaces (or provide their own custom implementations) to create "data sources" that may be mounted on a "file system" and used for OS-independent file IO.

AXIS was originally written to allow files inside of archives to be handled with exactly the same API as used for files inside of directories, but it has since grown to allow "logical" files and directories as well as "multiplexing" multiple items on the same location (to, for example, make two directories look and act like one). These properties make AXIS perfect for handling data and configuration files for any program where flexibility is important, the program does not need to know where its files are actually located, it simply needs them to be at a certain place in it's AXIS file system. Changing where a program loads it's files from is then as simple as changing the code that initializes the file system.

AXIS uses standard slash separated paths. File names may not contain any "special" characters, basically any of the following:

< > ? * | : " \ /

Additionally "." and ".." are not valid names.

Multiple slashes in an AXIS path are condensed into a single slash, and any trailing slashes will be stripped off. For example the following two paths are equivalent:

test/path/to/dir

/test///path//to/dir/

Obviously you should always use the first form, but the second is still legal (barely)

This copy of AXIS VFS was downloaded from my public repository. This repository is used strictly for releases and third party contributions, I do not use it for normal development.

AXIS VFS "officially" stands for Absurdly eXtremely Incredibly Simple Virtual File System (adjectives are good for making cool acronyms!). If you think the name is stupid (it is) you can just call it AXIS and forget what it is supposed to mean, after all the "official" name is more of a joke than anything...

Changes:

This documents changes listed by the date I added them to the repository.

2016Oct28

• Changed the error type returned for an invalid path so it makes more sense.
• Updated a lot of documentation comments with more info.
• Added an example.

Documentation

Overview

AXIS VFS, a simple virtual file system API.

AXIS is based on a few simple interfaces and a set of API functions that operate on these interfaces. Clients use the provided implementations of these interfaces (or provide their own custom implementations) to create "data sources" that may be mounted on a "file system" and used for OS-independent file IO.

AXIS was originally written to allow files inside of archives to be handled with exactly the same API as used for files inside of directories, but it has since grown to allow "logical" files and directories as well as "multiplexing" multiple items on the same location (to, for example, make two directories look and act like one). These properties make AXIS perfect for handling data and configuration files for any program where flexibility is important, the program does not need to know where its files are actually located, it simply needs them to be at a certain place in it's AXIS file system. Changing where a program loads it's files from is then as simple as changing the code that initializes the file system.

AXIS uses standard slash separated paths. File names may not contain any "special" characters, basically any of the following:

< > ? * | : " \ /

Additionally "." and ".." are not valid names.

Multiple slashes in an AXIS path are condensed into a single slash, and any trailing slashes will be stripped off. For example the following two paths are equivalent:

test/path/to/dir
/test///path//to/dir/

Obviously you should always use the first form, but the second is still legal (barely)

AXIS VFS "officially" stands for Absurdly eXtremely Incredibly Simple Virtual File System (adjectives are good for making cool acronyms!). If you think the name is stupid (it is) you can just call it AXIS and forget what it is supposed to mean, after all the "official" name is more of a joke than anything...

Example

package main

import (
	"encoding/base64"
	"fmt"
	"io/ioutil"
	"sort"

	"github.com/milochristiansen/axis2"
	"github.com/milochristiansen/axis2/sources/zip"
)

func main() {
	// Create the filesystem
	fs := new(axis2.FileSystem)

	// Create our base data source.
	// There are several kinds of DataSource provided, here we will use the one that reads a zip file from
	// a byte slice. Most of the time you will want to use one that reads from an OS file or directory.
	ds, err := zip.NewRawDir(data)
	if err != nil {
		fmt.Println(err)
		return
	}
	// Then mount the data source as "base" for reading only.
	fs.Mount("base", ds, false)

	// At this point you would generally store "fs" somewhere you can reach it easily and use it over and over.
	// Keep in mind that you can mount many items on a single FileSystem, they don't even need unique names!
	//
	// Items that do not have unique names will act like a single item with the caveat that each action will
	// be tried on them in the order they were mounted until it succeeds on one of them. For example you can
	// mount a user settings directory for reading and writing, then mount a directory with the default settings
	// for reading only. Any setting files the user has provided will be read from their directory and any
	// settings that the program writes will be written there, but anything that has not been provided will
	// be read from the default directory.
	//
	// Anyway, back to our regularly scheduled example:

	// Now let's print the file tree... (you should read the code for this function!)
	printTree(fs, "", "")

	// ...and finally let's print the contents of "base/a/y.txt".
	// (note there are also ReadAll and WriteAll convenience methods, but I won't use
	// them here so you can see the whole process)
	rdr, err := fs.Read("base/a/y.txt")
	if err != nil {
		fmt.Println(err)
		return
	}
	defer rdr.Close()
	contents, err := ioutil.ReadAll(rdr)
	fmt.Printf("\n%q\n", contents)

	// Errors from AXIS are wrapped in a special type, much like the "os" package warps most errors with the
	// os.PathError type.
	rdr, err = fs.Read("base/a/m.txt")
	if err == nil {
		rdr.Close()
		fmt.Println("Reading a non-existent file worked... Odd.")
		return
	}
	err2, ok := err.(*axis2.Error)
	if !ok {
		fmt.Println("AXIS returned an error of the incorrect type, this is supposed to be impossible.")
		return
	}

	// File not found errors from the "os" package are converted directly to the associated AXIS error type. Other
	// errors from os (that are wrapped in a os.PathError) are unwrapped and then rewrapped with the AXIS Error type.
	// This has the effect of stripping the OS path from the error message (the OS path is generally redundant and/or
	// undesirable, so this is intentional).
	if err2.Typ != axis2.ErrNotFound {
		fmt.Println("Error has unexpected type, this is supposed to be impossible.")
		return
	}
	fmt.Println("Errors OK!")

}

func printTree(fs *axis2.FileSystem, p, d string) {
	// We need to sort the lists we get from ListDirs and ListFiles because the zip
	// file data sources are not guaranteed to list in the same order each time!
	// Normally a stable order is of no importance, but the test randomly failing due
	// to different output orders is not desirable...

	// List all the directories OR mount points in the current path.
	// Since we pass in the empty string as the initial path this should return a
	// list of the root mount point subsets the first time printTree is called
	// (in this case just "base"). Mount point subsets are generally treated like
	// directories, but it is an error to read or write something to them. Use IsMP
	// to see if a "directory" is actually a mount point subset (generally you will
	// already know unless you are blindly walking a tree like this).
	l := fs.ListDirs(d)
	sort.Strings(l)
	for _, dir := range l {
		fmt.Println(p + dir + "/")
		printTree(fs, p+"  ", d+dir+"/")
	}

	// Rather than using ListDirs and ListFiles I could have just used List, but
	// then I would have had to use IsDir and IsMP to filter the results (and
	// directories wouldn't list first without some fancy code).
	l = fs.ListFiles(d)
	sort.Strings(l)
	for _, file := range l {
		fmt.Println(p + file)
	}
}

// After init runs data will contain a zip file with the following contents:
//
//	a/x.txt
//	a/y.txt
//	a/z.txt
//	b.txt
//	c.txt
//
// Each file contains its name as an ASCII string.
var data []byte

func init() {
	// Yes, I am throwing the error away. How evil... (Don't try this at home!)
	data, _ = base64.StdEncoding.DecodeString(`
UEsDBBQAAAAAAHJgW0kAAAAAAAAAAAAAAAACAAAAYS9QSwMECgAAAAAACmFbSUkCG6wFAAAABQAAAAcA
AABhL3gudHh0eC50eHRQSwMECgAAAAAADWFbSfkre5EFAAAABQAAAAcAAABhL3kudHh0eS50eHRQSwME
CgAAAAAAEWFbSSlR29YFAAAABQAAAAcAAABhL3oudHh0ei50eHRQSwMECgAAAAAAFmFbSWqNS4YFAAAA
BQAAAAUAAABiLnR4dGIudHh0UEsDBAoAAAAAABlhW0napCu7BQAAAAUAAAAFAAAAYy50eHRjLnR4dFBL
AQI/ABQAAAAAAHJgW0kAAAAAAAAAAAAAAAACACQAAAAAAAAAEAAAAAAAAABhLwoAIAAAAAAAAQAYAPli
b6trMNIB+WJvq2sw0gFgGMOaazDSAVBLAQI/AAoAAAAAAAphW0lJAhusBQAAAAUAAAAHACQAAAAAAAAA
IAAAACAAAABhL3gudHh0CgAgAAAAAAABABgAxGEfVWww0gFrH32kazDSAWsffaRrMNIBUEsBAj8ACgAA
AAAADWFbSfkre5EFAAAABQAAAAcAJAAAAAAAAAAgAAAASgAAAGEveS50eHQKACAAAAAAAAEAGADkA0NZ
bDDSAa5cvqdrMNIBax99pGsw0gFQSwECPwAKAAAAAAARYVtJKVHb1gUAAAAFAAAABwAkAAAAAAAAACAA
AAB0AAAAYS96LnR4dAoAIAAAAAAAAQAYAOWx0l1sMNIBF0F2qmsw0gFrH32kazDSAVBLAQI/AAoAAAAA
ABZhW0lqjUuGBQAAAAUAAAAFACQAAAAAAAAAIAAAAJ4AAABiLnR4dAoAIAAAAAAAAQAYAGe6SWNsMNIB
2qQzmGsw0gHQ9qOVazDSAVBLAQI/AAoAAAAAABlhW0napCu7BQAAAAUAAAAFACQAAAAAAAAAIAAAAMYA
AABjLnR4dAoAIAAAAAAAAQAYANmuWGdsMNIB0PajlWsw0gHQ9qOVazDSAVBLBQYAAAAABgAGAA0CAADu
AAAAAAA=`)
}

Output:

base/
  a/
    x.txt
    y.txt
    z.txt
  b.txt
  c.txt

"y.txt"
Errors OK!

Index

• Constants
• func NewError(typ ErrTyp) error
• type DataSource
• type Dir
• type ErrTyp
• type Error
  • func (err *Error) Error() string
• type File
• type FileSystem
  • func (fs *FileSystem) Append(path string) (io.WriteCloser, error)
  • func (fs *FileSystem) Delete(path string) error
  • func (fs *FileSystem) Exists(path string) bool
  • func (fs *FileSystem) GetDSAt(path string, create, r bool) (DataSource, error)
  • func (fs *FileSystem) GetDSsAt(path string, create, r bool) ([]DataSource, error)
  • func (fs *FileSystem) IsDir(path string) bool
  • func (fs *FileSystem) IsMP(path string) bool
  • func (fs *FileSystem) List(path string) []string
  • func (fs *FileSystem) ListDirs(path string) []string
  • func (fs *FileSystem) ListFiles(path string) []string
  • func (fs *FileSystem) Mount(path string, ds DataSource, rw bool) error
  • func (fs *FileSystem) Read(path string) (io.ReadCloser, error)
  • func (fs *FileSystem) ReadAll(path string) ([]byte, error)
  • func (fs *FileSystem) Size(path string) int64
  • func (fs *FileSystem) SwapMount(path string, ds DataSource, rw bool) DataSource
  • func (fs *FileSystem) Unmount(path string, r bool) error
  • func (fs *FileSystem) Write(path string) (io.WriteCloser, error)
  • func (fs *FileSystem) WriteAll(path string, content []byte) error

Examples

• Package

Constants

const (
	// Do not create the item if it does not exist.
	CreateNone int = iota

	// If the item does not exist try to create it as a child directory.
	CreateDir

	// If the item does not exist try to create it as a file.
	CreateFile
)

Flags for "Dir.Child".

Functions

func NewError

func NewError(typ ErrTyp) error

NewError creates a new AXIS Error with the given type. Error path information is automatically filled in by the API just before it is returned to the user.

Types

type DataSource

type DataSource interface{}

DataSource is any item that implements either File or Dir (or, more rarely, both).

This property is enforced by the API, any functions that takes a DataSource will return an error if it does not implement the required interface(s), likewise functions that return a DataSource will always return a value that implements the required interface(s).

type Dir

type Dir interface {
	// Child returns a reference to the requested child item, possibly creating it if needed and allowed.
	//
	// If the item exists then return a reference to it (ignoring the value of create), else create the item using the
	// value of create as a hint.
	//
	// Note that you do not need to actually create the item, simply creating the *possibility to have the item*
	// is enough. A request to create an item is always followed by a request to open it (or in the case of a
	// directory, one of its children) for writing, so you can delay the actual creation until then.
	Child(id string, create int) DataSource

	// Delete the given child item.
	Delete(id string) error

	// List all the children of this Dir.
	List() []string
}

Dir is the interface directories (or items that act like directories) must implement.

type ErrTyp

type ErrTyp int

const (
	// The path does not point to a valid item.
	ErrNotFound ErrTyp = iota

	// The requested action could not be carried out because the item is read-only.
	ErrReadOnly

	// The action cannot be done with the item (for example trying to write a directory).
	ErrBadAction

	// The given path is not absolute or it contains invalid characters.
	ErrBadPath

	// An error from an external library, with an attached AXIS path.
	ErrRaw
)

type Error

type Error struct {
	// Automatically set by the API functions before return, don't write this field!
	Path string

	// The error type. If you want to intelligently handle errors from this library this field will be invaluable.
	// See the ErrTyp constants for a list of valid values for this field.
	Typ ErrTyp

	// If Typ == ErrRaw this will contain an error value originating from a specific implementation of File or Dir.
	Err error
}

Error wraps a path and an error type, together with an error from an external library if applicable. Errors returned by API functions will be of this type.

Implementers of File or Dir do not need to use this type for their return values, but they are encouraged to do so (any returned error will be wrapped if it is not already of this type).

func (*Error) Error

func (err *Error) Error() string

Error prints the path associated with the error prefixed by a short string explanation of the error type.

Raw (wrapped) errors simply have "AXIS path: <path>" tacked onto the output of their own Error function.

type File

type File interface {
	// Read opens an AXIS file for reading and returns the result and any error that may have happened.
	Read() (io.ReadCloser, error)

	// Write opens an AXIS file for writing and returns the result and any error that may have happened.
	// Any existing contents the file may have are truncated.
	Write() (io.WriteCloser, error)

	// Append is exactly like Write, except the file is not truncated before writing.
	Append() (io.WriteCloser, error)

	// Size returns the size of the file or -1 if the value could not be retrieved.
	Size() int64
}

File is the interface files (or items that act like files) must implement.

type FileSystem

type FileSystem struct {
	// contains filtered or unexported fields
}

FileSystem is the center of an AXIS setup.

FileSystems have two halves: A "read" half and a "write" half. Any action that changes something (Write, Delete, etc) is carried out on the "write" half and any action that involves reading existing information is carried out on the "read" half. For this reason most DataSources are mounted on both halves or just the read half, never on just the write half.

If you mount more than one item on a location they will be tried in order, the first one to work is the one that is used.

The zero value of FileSystem is an empty FileSystem ready to use.

func (*FileSystem) Append

func (fs *FileSystem) Append(path string) (io.WriteCloser, error)

Append opens the file at the given path for writing. The write cursor is set beyond any existing file contents.

func (*FileSystem) Delete

func (fs *FileSystem) Delete(path string) error

Delete attempts to delete the item at the given path. This may or may not work. Deleting is always carried out on the write portion of the FileSystem, objects on the read portion will not be effected unless they are also mounted for writing. Only the first item found is deleted.

func (*FileSystem) Exists

func (fs *FileSystem) Exists(path string) bool

Exists returns true if the path points to a valid DataSource or a mount point subset. If the queried item is a mount point subset you won't be able to read or write it!

func (*FileSystem) GetDSAt

func (fs *FileSystem) GetDSAt(path string, create, r bool) (DataSource, error)

GetDSAt returns the first DataSource that matches the given path. This is mostly for internal use, but it is useful for certain advanced actions.

If the path is a mount point subset with no DataSources mounted at that level then this will return an error with type ErrBadAction (ErrNotFound isn't appropriate in that case, because something exists at the path, just not a data source).

func (*FileSystem) GetDSsAt

func (fs *FileSystem) GetDSsAt(path string, create, r bool) ([]DataSource, error)

GetDSsAt returns all of the DataSources that match the given path. This is mostly for internal use, but it is useful for certain advanced actions.

If the path is a mount point subset with no DataSources mounted at that level then this will return an error with type ErrBadAction (ErrNotFound isn't appropriate in that case, because something exists at the path, just not a data source).

func (*FileSystem) IsDir

func (fs *FileSystem) IsDir(path string) bool

IsDir returns true if the path points to a valid Dir or mount point subset.

func (*FileSystem) IsMP

func (fs *FileSystem) IsMP(path string) bool

IsMP returns true if the path is a mount point or mount point subset.

func (*FileSystem) List

func (fs *FileSystem) List(path string) []string

List returns a slice of the names of all the items available in the given Dir. If the item at the path is not a Dir and is not a subset of any mount points this returns nil.

The order of the returned list is undefined, or more correctly, is defined by the individual Dir implementations. Most of the time this means lexically by filename, but not always.

If the path is a mount point subset this may return more mount point subsets or a mix of mount point subsets and data sources!

func (*FileSystem) ListDirs

func (fs *FileSystem) ListDirs(path string) []string

ListDirs returns a slice of all the items that are Dirs in the given Dir. If the item at the path is not a Dir or mount point subset this returns nil.

The order of the returned list is undefined, or more correctly, is defined by the individual Dir implementations. Most of the time this means lexically by filename, but not always.

If the path is a mount point subset this may return more mount point subsets or a mix of mount point subsets and data sources!

func (*FileSystem) ListFiles

func (fs *FileSystem) ListFiles(path string) []string

ListFiles returns a slice of all the items that are Files in the given Dir. If the item at the path is not a Dir or if the Dir contains no Files this returns nil.

The order of the returned list is undefined, or more correctly, is defined by the individual Dir implementations. Most of the time this means lexically by filename, but not always.

func (*FileSystem) Mount

func (fs *FileSystem) Mount(path string, ds DataSource, rw bool) error

Mount a given DataSource onto the FileSystem at the given path. If rw is true the DataSource is mounted for writing as well as reading.

func (*FileSystem) Read

func (fs *FileSystem) Read(path string) (io.ReadCloser, error)

Read opens the File at the given path for reading.

func (*FileSystem) ReadAll

func (fs *FileSystem) ReadAll(path string) ([]byte, error)

ReadAll reads the File at the given path and returns it's contents.

func (*FileSystem) Size

func (fs *FileSystem) Size(path string) int64

Size returns the size of the File at the given path. If the object is not a File or the path is invalid -1 is returned.

func (*FileSystem) SwapMount

func (fs *FileSystem) SwapMount(path string, ds DataSource, rw bool) DataSource

SwapMount replaces the first data source with the given mount point and returns the old data source. Returns nil on error.

func (*FileSystem) Unmount

func (fs *FileSystem) Unmount(path string, r bool) error

Unmount deletes all mounted DataSources with the given mount point.

func (*FileSystem) Write

func (fs *FileSystem) Write(path string) (io.WriteCloser, error)

Write opens the the File at the given path for writing. Any existing file contents are truncated.

func (*FileSystem) WriteAll

func (fs *FileSystem) WriteAll(path string, content []byte) error

WriteAll replace the contents of the File at the given path with the contents given.