fs

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 16, 2024 License: LGPL-2.1 Imports: 10 Imported by: 0

README

Gonatus File System

Gonatus file system redefines the the architecture of a file system. The term "directory" does not exist, all records in the FS are called files. Each file may have content, topology (child records) or both (so files can also behave as directories).

File types

  • undetermined - The file does not exist or its state is unknown.
  • topology - The file can have descendants (directory in traditional FS).
  • content - The file holds a binary content (file in traditional FS).
  • topology|content - The file has both topology and content (file and directory at the same time, impossible in traditional FS).

Limitations

Design of the filesystem moves limitation to the edge of the technical possibilities of compute machines generally.

  • Path length - limited by RAM size.
  • Path forbidden characters - none (also / and \ may be used).
  • Records count - according to the used storage driver (usually near to mmap limits).
  • Record content size - limited by the storage capacity.
  • Record topology depth - unlimited (listing is done via the Lazy Streams).

Path

File path is internally defined as a type derived a slice of strings. Each slice element is meant to be a name of a path level. For example /home/foo/bar/c.bin is represented as ["home", "foo", "bar", "c.bin"]. The advantages are that all slice operations may be used and there are no forbidden characters.

Usage

Storage

Storage is a file system device abstraction. Every storage has its own storage driver, which is a set of methods for communication with the data storage device. The storage interface encapsulates the actual device used and allowes to use all of them the same way.

The public API of the storage:

type Storage interface {
	gonatus.Gobjecter

	Merge(source Storage) error
	Tree(depth Depth) (streams.ReadableOutputStreamer[File], error)

	Commit() error
	Clear() error

	Id() StorageId
}

There are two reference storage driver implementations in the drivers package. Native storage is a read-only access point to the file system on the local machine. Local counted storage stores data as binary files in a tree of counted folders. Physical location of the files is independent of the virtual file table. That avoids all naming and path restrictions. You can create your own driver by implementing the StorageDriver interface:

type StorageDriver interface {
	gonatus.Gobjecter

	Open(path Path, mode FileMode, givenFlags FileFlags, origTime time.Time) (FileDescriptor, error)
	Close(path Path) error

	MkDir(path Path, origTime time.Time) error

	Copy(srcPath Path, dstPath Path) error
	Move(srcPath Path, dstPath Path) error
	Delete(path Path) error

	Tree(path Path, depth Depth) (streams.ReadableOutputStreamer[File], error)
	Size(path Path) (uint64, error)
	Flags(path Path) (FileFlags, error)

	Commit() error
	Clear() error

	Features() StorageFeatures
	Id() StorageId
	SetId(id StorageId)
}

To use it, you have to register the storage to the storage manager. The storage is a global Gobject managing all storages used in the system and assigning each of them its unique ID.

File

File is a basic interface of the file system. It gives access to all operations needed for the FS to use. The file Gobject is created by a standard constructor NewFile(FileConf) File. When created, the file is not initially attached to the storage with the given ID (it even does not have to exist), the corresponding storage driver is accessed when some operation is performed with the file.

If file flags are specified in the configuration structure, they are used if Open() or MkDir() methods are invoked. For example, if you open a file with the topology flag set, it will be stored with both content and topology flags. User is also able to manually set an original creation time (if not specified, current time is used).

The configuration structure:

type FileConf struct {
  Path      Path
  StorageId StorageId
  Flags     FileFlags
  OrigTime  time.Time
}

The file interface:

type File interface {
	gonatus.Gobjecter
	io.Closer
	FileDescriptor

	Storage() Storage

	Path() Path
	Name() string

	Copy(dst File) error
	Move(dst File) error
	Delete() error

	Open(mode FileMode) error
	MkDir() error

	Tree(depth Depth) (streams.ReadableOutputStreamer[File], error)
	Stat() (FileStat, error)

	SetOrigTime(time time.Time)
}
File descriptor

The FileDescriptor interface mimics the standard Golang file descriptor, but abstracts a handler of an opened file in any storage. It is returned by by Open() method of the storage driver and is integrated in the File interface. However, the descriptor methods can only be used if the file is open.

Example

nativeStorage := drivers.NewNativeStorage(drivers.NativeStorageConf{
	Prefix: "/home/admin/Documents",
})
fs.GStorageManager.RegisterStorage(nativeStorage)

targetStorage := drivers.NewLocalCountedStorage(drivers.LocalCountedStorageConf{
	Prefix: "/tmp/storage",
})
fs.GStorageManager.RegisterStorage(targetStorage)

sourceFile := fs.NewFile(fs.FileConf{
	StorageId: nativeStorage.Id(),
	Path:      fs.Path{"trash", "fs.go"},
})

destinationFile := fs.NewFile(fs.FileConf{
	StorageId: targetStorage.Id(),
	Path:      fs.Path{"c", "copy.txt"},
})

if err := sourceFile.Copy(destinationFile); err != nil {
	panic(err)
}

Documentation

Index

Constants

View Source 
const (
	ModeRead = iota
	ModeWrite
	ModeAppend
	ModeRW
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Depth 

type Depth uint64

Depth of the tree command. 

const (
	DepthUnlimited Depth = math.MaxUint64
	DepthSelf      Depth = iota - 1
	DepthLs
)

type File 

type File interface {
	gonatus.Gobjecter
	FileDescriptor
	io.Closer

	/*
		Acquires a storage where the file is stored.

		Returns:
		  - the storage.
	*/
	Storage() Storage

	/*
		Acquires the fullpath to the file.

		Returns:
		  - path as a slice of strings.
	*/
	Path() Path

	/*
		Acquires the real location (native path) of the file.

		Returns:
		  - native path,
		  - error if any occurred.
	*/
	Location() (string, error)

	/*
		Acquires the name of the file (last element of the path).

		Returns:
		  - the name.
	*/
	Name() string

	/*
		Copies content and topology of the file to another one.
		If the files are located on the same storage, invokes same-storage copy method.

		Parameters:
		  - dst - destination file.

		Returns:
		  - error if any occurred.
	*/
	Copy(dst File) error

	/*
		Moves content and topology of the file to another one.
		If the files are located on the same storage, invokes same-storage move method, otherwise copies the file and then deletes it.

		Parameters:
		  - dst - destination file.

		Returns:
		  - error if any occurred.
	*/
	Move(dst File) error

	/*
		Deletes the file.

		Returns:
		  - error if any occurred.
	*/
	Delete() error

	/*
		Opens the file if the given mode.

		Parameters:
		  - mode - opening mode.
	*/
	Open(mode FileMode) error

	/*
		Adds the topology flag to the file.
		If the file does not exist, it is created.

		Returns:
		  - error if any occurred.
	*/
	MkDir() error

	/*
		Acquires a tree of files to the given depth starting from the storage root. It is returned as a stream.
		If the depth is 0, the stream contains only the file itself.

		Parameters:
		  - depth - how deep to go in the file tree.

		Returns:
		  - stream of the files,
		  - error if any occurred.
	*/
	Tree(depth Depth) (stream.Producer[File], error)

	/*
		Acquires a current status of the file.

		Returns:
		  - stat structure,
		  - error if any occurred.
	*/
	Stat() (FileStat, error)

	/*
		Manually sets the time when the file was originally created.

		Parameters:
		  - time - time to set.
	*/
	SetOrigTime(time time.Time)
}

A file in the Gonatus File System.

Extends:

  • gonatus.Gobjecter,
  • io.Closer,
  • FileDescriptor.

func NewFile 

func NewFile(conf FileConf) File

File constructor.

Parameters:

  • conf - configuration structure.

Returns:

  • pointer to the created file.

type FileConf 

type FileConf struct {
	Path      Path
	StorageId gonatus.GId
	Flags     FileFlags
	OrigTime  time.Time
}

Configuration structure for the file.

type FileDescriptor 

type FileDescriptor interface {
	io.Reader
	io.ReaderFrom
	io.ReaderAt
	io.Writer
	io.Seeker
}

Gonatus abstraction of the Go file descriptor.

Extends:

  • io.Reader
  • io.ReaderFrom
  • io.Writer
  • io.Seeker

type FileFlags 

type FileFlags uint8

Flag byte for the file. 

const (
	FileUndetermined FileFlags = 0
	FileContent      FileFlags = 1 << (iota - 1)
	FileTopology
)

type FileMode 

type FileMode uint8

Mode in which a file could be opened.

type FileStat 

type FileStat struct {
	Flags     FileFlags
	Size      uint64
	OrigTime  time.Time
	ModifTime time.Time
}

Stat structure for the file.

type Path 

type Path []string

Representation of the file path.

func (Path) Base 

func (ego Path) Base() string

Acquires the name of the file.

Returns:

  • last element of the path.

func (Path) Clone 

func (ego Path) Clone() Path

Creates a copy of the path.

Returns:

  • new path.

func (Path) Dir 

func (ego Path) Dir() Path

Acquires the parent path of the path.

Returns:

  • path without the last element.

func (Path) Equals 

func (ego Path) Equals(another Path) bool

Checks if the path is equal to another path.

Parameters:

  • another - path to compare with.

Returns:

  • true if the paths are equal, false otherwise.

func (Path) Join 

func (ego Path) Join(apendee Path) Path

Joins two paths into a single one.

Parameters:

  • apendee - path to append.

Returns:

  • concated path.

func (Path) String 

func (ego Path) String() string

Acquires a string representation of the path.

Returns:

  • serialized path.

type Storage 

type Storage interface {
	gonatus.Gobjecter

	/*
		Copies all files in the given storage into this storage.

		Parameters:
		  - source - storage to merge.

		Returns:
		  - error if any occurred.
	*/
	Merge(source Storage) error

	/*
		Acquires a tree of files to the given depth starting from this file. It is returned as a stream.
		If the depth is 0, the stream contains only the file itself.

		Parameters:
		  - depth - how deep to go in the file tree.

		Returns:
		  - stream of the files,
		  - error if any occurred.
	*/
	Tree(depth Depth) (stream.Producer[File], error)

	/*
		Changes the current working directory to the given path.

		Parameters:
		  - path - the path to set.

		Returns:
		  - error if any occurred.
	*/
	ChDir(path Path) error

	/*
		Commits the changes.

		Returns:
		  - error if any occurred.
	*/
	Commit() error

	/*
		Deletes all files in the storage.

		Returns:
		  - error if any occurred.
	*/
	Clear() error

	/*
		Acquires the ID of the storage.
		If the storage is not registered, returns 0.

		Returns:
		  - ID of the storage.
	*/
	Id() gonatus.GId
	// contains filtered or unexported methods
}

Abstraction of the Gonatus storage. The actual implementation depends on the storage driver.

Extends:

  • gonatus.Gobjecter.

func NewStorage 

func NewStorage(driver StorageDriver) Storage

Storage constructor. Meant to be used only from inside of storage drivers.

Parameters:

  • driver - driver for the storage.

Returns:

  • pointer to the created storage.

type StorageDriver 

type StorageDriver interface {
	gonatus.Gobjecter

	/*
		Opens a file.

		Parameters:
		  - path - path to the file,
		  - mode - opening mode,
		  - givenFlags - flags to open with (content is implicit),
		  - origTime - time when the file was originally created.

		Returns:
		  - file descriptor,
		  - error if any occurred.
	*/
	Open(path Path, mode FileMode, givenFlags FileFlags, origTime time.Time) (FileDescriptor, error)

	/*
		Closes one descriptor of a file.

		Parameters:
		  - descriptor - descriptor to close,
		  - path - path to the file.

		Returns:
		  - error if any occurred.
	*/
	CloseDescriptor(descriptor FileDescriptor, path Path) error

	/*
		Closes all descriptors of a file.

		Parameters:
		  - path - path to the file.

		Returns:
		  - error if any occurred.
	*/
	CloseFile(path Path) error

	/*
		Adds the topology flag to the file.
		If the file does not exist, it is created.

		Parameters:
		  - path - path to the file,
		  - origTime - time when the file was originally created.

		Returns:
		  - error if any occurred.
	*/
	MkDir(path Path, origTime time.Time) error

	/*
		Performs an same-storage copy.

		Parameters:
		  - srcPath - path to the source file,
		  - dstPath - path to the destination file.

		Returns:
		  - error if any occurred.
	*/
	Copy(srcPath Path, dstPath Path) error

	/*
		Performs an same-storage move.

		Parameters:
		  - srcPath - path to the source file,
		  - dstPath - path to the destination file.

		Returns:
		  - error if any occurred.
	*/
	Move(srcPath Path, dstPath Path) error

	/*
		Deletes a file.

		Parameters:
		  - path - path to the file.

		Returns:
		  - error if any occurred.
	*/
	Delete(path Path) error

	/*
		Performs a tree request over the storage.

		Parameters:
		  - path - path where to start,
		  - depth - how deep to go in the file tree.

		Returns:
		  - stream of the files,
		  - error if any occurred.
	*/
	Tree(path Path, depth Depth) (stream.Producer[File], error)

	/*
		Acquires a size of a file with the given path.

		Parameters:
		  - path - path to the file.

		Returns:
		  - size of the file,
		  - error if any occurred.
	*/
	Size(path Path) (uint64, error)

	/*
		Acquires flags of a file with the given path.

		Parameters:
		  - path - path to the file.

		Returns:
		  - flags of the file,
		  - error if any occurred.
	*/
	Flags(path Path) (FileFlags, error)

	/*
		Acquires real location (native path) of a file with the given path.

		Parameters:
		  - path - path to the file.

		Returns:
		  - native path,
		  - error if any occurred.
	*/
	Location(path Path) (string, error)

	/*
		Commits the changes.

		Returns:
		  - error if any occurred.
	*/
	Commit() error

	/*
		Deletes all files in the storage.

		Returns:
		  - error if any occurred.
	*/
	Clear() error

	/*
		Acquires the feature flag byte of the storage.

		Returns:
		  - feature flags.
	*/
	Features() StorageFeatures

	/*
		Acquires the ID of the storage.
		If the storage is not registered, returns 0.

		Returns:
		  - ID of the storage.
	*/
	Id() gonatus.GId

	/*
		Sets the storage ID.

		Parameters:
		  - ID to set.
	*/
	SetId(id gonatus.GId)
}

Storage driver. Implements the behavior of the storage.

Extends:

  • gonatus.Gobjecter.

type StorageFeatures 

type StorageFeatures uint8

Features of a storage (flag byte). 

const (
	FeatureRead StorageFeatures = 1 << iota
	FeatureWrite
	FeatureLocation
)

type StorageManager 

type StorageManager struct {
	gonatus.Gobject
	// contains filtered or unexported fields
}

A service which keeps track of the storages.

Extends:

  • gonatus.Gobject.

Implements:

  • gonatus.Gobjecter.
var GStorageManager StorageManager = StorageManager{
	// contains filtered or unexported fields
}

Default storage manager

func (*StorageManager) Fetch 

func (ego *StorageManager) Fetch(e gonatus.GId) (Storage, error)

Fetches a storage with the given ID.

Parameters:

  • e - ID of the storage.

Returns:

  • the storage (nil if not found),
  • error if not found.

func (*StorageManager) GetId 

func (ego *StorageManager) GetId(s Storage) (gonatus.GId, error)

Acquires an ID of the given storage.

Parameters:

  • s - the storage.

Returns:

  • ID of the storage (0 if not found),
  • error if not found.

func (*StorageManager) RegisterStorage 

func (ego *StorageManager) RegisterStorage(s Storage) error

Registers a new storage to the manager.

Parameters:

  • s - storage to register.

func (*StorageManager) Serialize 

func (ego *StorageManager) Serialize() gonatus.Conf

func (*StorageManager) UnregisterStorage 

func (ego *StorageManager) UnregisterStorage(s Storage) error

Unregisters a storage from the manager.

Parameters:

  • s - storage to unregister.

Returns:

  • error if any occurred.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.