README
¶
fs
In memory file system, "os" based file system and utilities
This package contains:
- Utilities to help with the io/fs package
- fs.Simple, a writeable in-memory io.FS
- os.FS - an io.FS that uses the os package
Introduction
The io.FS package is a great addition into Go 1.16 . To start with, its main use seems to be around using the embed package.
The embed package allows embedding files at compile time. But it has this negative consquence of really needing to embed the files somewhere close to the root into a single FS object.
I was looking for a way to embed files at each package layer and them merge them into a single FS. This feels more managable to me in that I have each package handling its own embed and then one file just merging the embed FS that is in each package.
I also wanted a way to optionally transform the files if I merged them into the new file system. One of the use cases is that I like to optimize JS an CSS files, but only when I'm not debugging. This provides an optional transform package that allows for me to optimize those files on the fly during startup in non-debug mode.
I also wanted to have a file system I could write files into after compile time and them make read only.
Simple FS
The Simple FS is a simplistic filesystem that has all the basics needed for an fs.FS. As it states, you really want to stay with ASCII names and not try to get fancy with // kind of things.
It comes with an option called WithPearson() that uses a Pearson hash to do real O(1) non-collision file lookups.
This can only be used if you have ASCII characters in your file name.
Creation is easy:
sfs := fs.NewSimple(fs.WithPearson())
Writing a file is simple:
if err := sfs.WriteFile("path/to/file", []byte("hello world"); err != nil {
// Do something
}
Once we are done writing, we simply need to set our Simple FS to ReadOnly:
sfs.RO()
Reading a file is just as easy:
b, err := sfs.Readfile("path/to/file")
if err != nil {
// Do something
}
fmt.Println(string(b))
And there are various other methods.
But what makes Simple FS particularly useful (besides being writable) is using it as a central location to merge other io.FS into a single structure:
if err := fs.Merge(sfs, pkg.FS, "into/sub/directory/"); err != nil {
// Do something
}
The above merge method will add all the content of pkg.FS and store it in a directory from our sfs root "into/sub/directory". This is a recursive walk and will contain all the files.
If you want to modify files before they are copied (compress certain files, optimize them or rewrite them in any way), use the WithTransform() option.
os.FS
Our sub-directory os/ contains a io.FS that uses the underlying os package. I couldn't seem to find a package that provided this and it was simple to add.
This package has all the same features of our Simple fs, but with all the advanced features that a mature package like os provides.
Documentation
¶
Overview ¶
Package fs provides utilities for making use of the new io/fs abstractions and the embed package.
Index ¶
- func Merge(into Writer, from fs.FS, prepend string, options ...MergeOption) error
- type FileTransform
- type MergeOption
- type OFOption
- type OpenFiler
- type Simple
- func (s *Simple) Open(name string) (fs.File, error)
- func (s *Simple) OpenFile(name string, flags int, options ...OFOption) (fs.File, error)
- func (s *Simple) RO()
- func (s *Simple) ReadDir(name string) ([]fs.DirEntry, error)
- func (s *Simple) ReadFile(name string) ([]byte, error)
- func (s *Simple) Stat(name string) (fs.FileInfo, error)
- func (s *Simple) WriteFile(name string, content []byte, perm fs.FileMode) error
- type SimpleOption
- type WRFile
- type Writer
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type FileTransform ¶
FileTransform gives the base name of a file and the content of the file. It returns the content that MAY be transformed in some way.
type MergeOption ¶
type MergeOption func(o *mergeOptions)
MergeOption is an optional argument for Merge().
func WithTransform ¶
func WithTransform(ft FileTransform) MergeOption
WithTransform instructs the Merge() to use a FileTransform on the files it reads before writing them to the destination.
type OFOption ¶
type OFOption func(o interface{}) error
OFOption is an option for the OpenFiler.OpenFile() call. The passed "o" arge is implementation dependent.
type OpenFiler ¶
type OpenFiler interface {
fs.FS
// OpenFile opens the file at name with flags and options. flags can be any subset of the
// flags defined in the fs module (O_CREATE, O_READONLY, ...). The set of options is implementation
// dependent. The fs.File that is returned should be type asserted to gain access to additional
// capabilities. If opening for ReadOnly, generally the standard fs.Open() call is better.
OpenFile(name string, flags int, options ...OFOption) (fs.File, error)
}
OpenFiler provides a more robust method of opening a file that allows for additional capabilities like writing to files. The fs.File and options are generic and implementation specific. To gain access to additional capabilities usually requires type asserting the fs.File to the implementation specific type.
type Simple ¶
type Simple struct {
// contains filtered or unexported fields
}
Simple provides a simple memory structure that implements io/fs.FS and fs.Writer(above). This is great for aggregating several different embeded fs.FS into a single structure using Merge() below. It uses "/" unix separators and doesn't deal with any funky "\/" things. If you want to use this don't start trying to get complicated with your pathing. This structure is safe for concurrent reading or concurrent writing, but not concurrent read/write. Once finished writing files, you should call .RO() to lock it.
func NewSimple ¶
func NewSimple(options ...SimpleOption) *Simple
NewSimple is the constructor for Simple.
func (*Simple) OpenFile ¶
OpenFile implements OpenFiler. Supports flags O_RDONLY, O_WRONLY, O_CREATE, O_TRUNC and O_EXCL. The file returned by OpenFile is not thread-safe.
func (*Simple) ReadFile ¶
ReadFile implememnts ReadFileFS.ReadFile(). The slice returned by ReadFile is not a copy of the file's contents like Open().File.Read() returns. Modifying it will modifiy the content so BE CAREFUL.
type SimpleOption ¶
type SimpleOption func(s *Simple)
SimpleOption provides an optional argument to NewSimple().
func WithPearson ¶
func WithPearson() SimpleOption
WithPearson will create a lookup cache using Pearson hashing to make lookups actually happen at O(1) (after the hash calc) instead of walking the file system tree after various strings splits. When using this, realize that you MUST be using ASCII characters.
type WRFile ¶
type WRFile struct {
// contains filtered or unexported fields
}
WRFile provides an io.WriteCloser implementation.
type Writer ¶
type Writer interface {
OpenFiler
// Writes file with name (full path) a content to the file system. This implementation may
// return fs.ErrExist if the file already exists if the FileSystem is write once. The FileMode
// may or may not be honored, see the implementation details for more information.
WriteFile(name string, data []byte, perm fs.FileMode) error
}
Writer provides a filesystem implememnting OpenFiler with a simple way to write and entire file.
Source Files
Directories