wpk

package module
v1.4.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 28, 2023 License: MIT Imports: 16 Imported by: 1

README

WPK

Library to build and use data files packages.

Go Reference Go Report Card Hits-of-Code

Preamble

Software often uses a lot of data files and needs effective method to manage them and get quick access. Stitch them to single package and then get access by mapped memory is a good resolution.

Package keeps all files together with warranty that no any file will be deleted, moved or changed separately from others during software is running, it's snapshot of entire development workflow. Package saves disk space due to except file system granulation, especially if many small files are packed together. Package allows to lock fragment of single file to get access to mapped memory.

Capabilities

  • Package read and write API.
  • Lua-scripting API.
  • Virtual file system.
  • Set of associated tags with each file.
  • No sensible limits for package size.
  • Access to package by file mapping, as to slice, as to file.
  • Package can be formed by several steps.
  • Package can be used as insert-read database.
  • Can be used union of packages as single file system.

Structure

Library have root wpk module that used by any code working with .wpk packages. And modules to build utilities for packing/unpacking data files to package:

  • wpk Places data files into single package, extracts them, and gives API for access to package.

  • wpk/bulk Wrapper for package to hold WPK-file whole content as a slice. Actual for small packages (size is much less than the amount of RAM).

  • wpk/mmap Wrapper for package to get access to nested files as to memory mapped blocks. Actual for medium size packages (size correlates with the RAM amount).

  • wpk/fsys Wrapper for package to get access to nested files by OS files. Actual for large packages (size is much exceeds the amount of RAM) or large nested files.

  • wpk/luawpk Package writer with package building process scripting using Lua 5.1. Typical script workflow is to create package for writing, setup some options, put group of files to package, and finalize it.

  • wpk/testdata Contains some Lua-scripts to test wpk/luawpk module and learn scripting API opportunities.

  • wpk/util/pack Small simple utility designed to pack a directory, or a list of directories into an package.

  • wpk/util/extract Small simple utility designed to extract all packed files from package, or list of packages to given directory.

  • wpk/util/build Utility for the packages programmable building, based on wpk/luawpk module.

Compiled binaries of utilities can be downloaded in Releases section.

How to use

At first, install Golang minimum 1.18 version for last version of this package, and get this package:

go get github.com/schwarzlichtbezirk/wpk

Then you can make simple package with files at testdata/media directory by command:

go run github.com/schwarzlichtbezirk/wpk/util/pack --src=${GOPATH}/src/github.com/schwarzlichtbezirk/wpk/testdata/media --dst=${GOPATH}/bin/media.wpk

It's runs utility that receives source directory full path and destination package full path. ${GOPATH} at command line directory path replaced by GOPATH environment variable value. To place any other environment variable VAR you can by ${VAR}. In this sample package placed into bin directory with other compiled golang binary files.

To extract files from this media.wpk package run command:

go run github.com/schwarzlichtbezirk/wpk/util/extract --md --src=${GOPATH}/bin/media.wpk --dst=${GOPATH}/bin/media

and see files in directory ${GOPATH}/bin/media.

To build package at development workflow you can by build utility. It can put files into package from any different paths with given names, and bind addition tags to each file, such as MIME types, keywords, CRC, MD5, SHA256 and others. Run this command to see how its work:

go run github.com/schwarzlichtbezirk/wpk/util/build ${GOPATH}/src/github.com/schwarzlichtbezirk/wpk/testdata/build.lua

and see build.wpk file in binary directory near compiled output.

WPK-format

Package consist of 3 sections:

  1. Header, constantly 64 bytes. Starts with signature (24 bytes), then follow 8 bytes with used at package types sizes, file tags table offset and table size (8+8 bytes), and bata block offset and size (8+8 bytes).

  2. Bare data files blocks.

  3. File tags set table. Contains list of tagset for each file alias. Each tagset must contain some requered fields: it's ID, file size, file offset in package, file name (path), creation time. Package can have common description stored as tagset with empty name. This tagset is placed as first record in file tags table.

Existing package can be opened to append new files, in this case new files blocks will be posted to tags sets old place.

Package can be splitted in two files: 1) file with header and tags table, .wpt-file, it's a short file in most common, and 2) file with data files block, typically .wpf-file. In this case package is able for reading during new files packing to package. If process of packing new files will be broken by any case, package remains accessible with information pointed at last header record.

Lua-scripting API

build utility receives one or more Lua-scripts that maneges package building workflow. Typical sequence is to create new package, setup common properties, put files and add aliases with some tags if it necessary, and complete package building. See whole script API documentation in header comment of api.lua script, and sample package building algorithm below.

step1.lua and step2.lua scripts shows sample how to create new package at step1:

go run github.com/schwarzlichtbezirk/wpk/util/build ${GOPATH}/src/github.com/schwarzlichtbezirk/wpk/testdata/step1.lua

and later append to exising package new files at step2 call:

go run github.com/schwarzlichtbezirk/wpk/util/build ${GOPATH}/src/github.com/schwarzlichtbezirk/wpk/testdata/step2.lua

packdir.lua script has function that can be used to put to package directory with original tree hierarchy.

WPK API usage

See godoc with API description, and wpk_test.go for usage samples.

On your program initialisation open prepared wpk-package by Package.OpenPackage call. It reads tags sets of package at once, then you can get access to filenames and it's tags. TagsetRaw structure helps you to get tags associated to files, and also it provides file information by standard interfaces implementation. To get access to package nested files, create some Tagger object. Modules wpk/bulk, wpk/mmap and wpk/fsys provides this access by different ways. Package object have all io/fs file system interfaces implementations, and can be used by anyway where they needed.

Documentation

Index

Examples

Constants

View Source 
const (
	SignSize   = 24 // signature field size.
	HeaderSize = 64 // package header size in bytes.

	SignReady = "Whirlwind 3.4 Package   " // package is ready for use
	SignBuild = "Whirlwind 3.4 Prebuild  " // package is in building progress
)
View Source 
const (
	TIDnone = 0

	TIDoffset = 1  // required, uint
	TIDsize   = 2  // required, uint
	TIDpath   = 3  // required, unique, string
	TIDfid    = 4  // unique, uint
	TIDmtime  = 5  // required for files, 8/12 bytes (mod-time)
	TIDatime  = 6  // 8/12 bytes (access-time)
	TIDctime  = 7  // 8/12 bytes (change-time)
	TIDbtime  = 8  // 8/12 bytes (birth-time)
	TIDattr   = 9  // uint32
	TIDmime   = 10 // string

	TIDcrc32ieee = 11 // uint32, CRC-32-IEEE 802.3, poly = 0x04C11DB7, init = -1
	TIDcrc32c    = 12 // uint32, (Castagnoli), poly = 0x1EDC6F41, init = -1
	TIDcrc32k    = 13 // uint32, (Koopman), poly = 0x741B8CD7, init = -1
	TIDcrc64iso  = 14 // uint64, poly = 0xD800000000000000, init = -1

	TIDmd5    = 20 // [16]byte
	TIDsha1   = 21 // [20]byte
	TIDsha224 = 22 // [28]byte
	TIDsha256 = 23 // [32]byte
	TIDsha384 = 24 // [48]byte
	TIDsha512 = 25 // [64]byte

	TIDtmbjpeg  = 100 // []byte, thumbnail image (icon) in JPEG format
	TIDtmbwebp  = 101 // []byte, thumbnail image (icon) in WebP format
	TIDlabel    = 110 // string
	TIDlink     = 111 // string
	TIDkeywords = 112 // string
	TIDcategory = 113 // string
	TIDversion  = 114 // string
	TIDauthor   = 115 // string
	TIDcomment  = 116 // string
)

List of predefined tags IDs.

View Source 
const (
	PTStidsz  = 2 // "tag ID" type size.
	PTStagsz  = 2 // "tag size" type size.
	PTStssize = 2 // "tagset size" type size.

)

Package types sizes.

View Source 
const (
	InfoName = "@info" // package info tagset name
	PackName = "@pack" // package content reference
)

Variables

View Source 
var (
	ErrSignPre = errors.New("package is not ready")
	ErrSignBad = errors.New("signature does not pass")
	ErrSignFTT = errors.New("header contains incorrect data")

	ErrRangeTSSize = errors.New("tagset size value is exceeds out of the type dimension")

	ErrNoTag    = errors.New("tag with given ID not found")
	ErrNoPath   = errors.New("file name is absent")
	ErrNoOffset = errors.New("file offset is absent")
	ErrOutOff   = errors.New("file offset is out of bounds")
	ErrNoSize   = errors.New("file size is absent")
	ErrOutSize  = errors.New("file size is out of bounds")

	ErrOtherSubdir = errors.New("directory refers to other workspace")
)

Errors on WPK-API.

View Source 
var Normalize = ToSlash

Normalize brings file path to normalized form. Normalized path is the key to FTT map.

Functions

func B2S added in v1.4.3 

func B2S(b []byte) string

func Envfmt added in v1.1.0 

func Envfmt(p string) string

Envfmt replaces environment variables entries in file path to there values. Environment variables must be enclosed as ${...} in string.

func MakeDataPath added in v1.3.0 

func MakeDataPath(fpath string) string

MakeDataPath receives file path and returns it with ".wpf" extension. It hepls to open splitted package.

func MakeTagsPath added in v1.3.0 

func MakeTagsPath(fpath string) string

MakeTagsPath receives file path and returns it with ".wpt" extension. It hepls to open splitted package.

func PathExists added in v1.1.0 

func PathExists(path string) (bool, error)

PathExists check up file or directory existence.

func S2B added in v1.4.3 

func S2B(s string) (b []byte)

func TempPath added in v1.4.0 

func TempPath(fname string) string

TempPath returns filename located at temporary directory.

func ToSlash 

func ToSlash(s string) string

ToSlash brings filenames to true slashes without superfluous allocations if it possible.

func WriteUint added in v1.3.2 

func WriteUint(w io.Writer, data Uint, l byte) (err error)

WriteUint writes to stream given unsigned integer with given size in bytes. Size can be 1, 2, 4, 8.

func WriteUintBuf added in v1.3.2 

func WriteUintBuf(b []byte, v Uint)

WriteUintBuf writes unsigned integer into buffer with predefined size. Size of buffer can be 1, 2, 4, 8.

Types

type CompleteFS added in v1.4.0 

type CompleteFS interface {
	fs.SubFS
	fs.StatFS
	fs.GlobFS
	fs.ReadDirFS
	fs.ReadFileFS
}

CompleteFS includes all FS interfaces.

type ErrTag 

type ErrTag struct {
	What error  // error message
	Key  string // normalized file name
	TID  Uint   // tag ID
}

ErrTag is error on some field of tags set.

func (*ErrTag) Error 

func (e *ErrTag) Error() string

func (*ErrTag) Unwrap 

func (e *ErrTag) Unwrap() error

type FTT added in v1.4.0 

type FTT struct {
	// contains filtered or unexported fields
}

File tags table.

func (*FTT) Append added in v1.4.0 

func (ftt *FTT) Append(wpt, wpf io.WriteSeeker) (err error)

Append writes prebuild header for previously opened package to append new files.

func (*FTT) Begin added in v1.4.0 

func (ftt *FTT) Begin(wpt, wpf io.WriteSeeker) (err error)

Begin writes prebuild header for new empty package.

func (*FTT) CheckTagset added in v1.4.1 

func (ftt *FTT) CheckTagset(ts TagsetRaw) (fpath string, err error)

CheckTagset tests path & offset & size tags existence and checks that size & offset is are in the bounds.

func (*FTT) DataSize added in v1.4.1 

func (ftt *FTT) DataSize() Uint

DataSize returns actual package data size from files tags table.

func (*FTT) GetInfo added in v1.4.4 

func (ftt *FTT) GetInfo() (TagsetRaw, bool)

GetInfo returns package information tagset if it present.

Example 
package main

import (
	"fmt"
	"log"
	"strings"

	"github.com/schwarzlichtbezirk/wpk"
)

func main() {
	var err error

	// Open package files tags table
	var pkg *wpk.Package
	if pkg, err = wpk.OpenPackage("example.wpk"); err != nil {
		log.Fatal(err)
	}

	// How many records in package
	var m = map[wpk.Uint]wpk.Void{}
	var n = 0
	pkg.Enum(func(fkey string, ts wpk.TagsetRaw) bool {
		if offset, ok := ts.TagUint(wpk.TIDoffset); ok {
			m[offset] = wpk.Void{} // count unique offsets
		}
		n++
		return true
	})

	// Format package information
	var items = []string{
		fmt.Sprintf("files: %d", len(m)),
		fmt.Sprintf("aliases: %d", n-len(m)),
		fmt.Sprintf("datasize: %d", pkg.DataSize()),
	}
	if ts, ok := pkg.GetInfo(); ok { // get package info if it present
		if str, ok := ts.TagStr(wpk.TIDlabel); ok {
			items = append(items, fmt.Sprintf("label: %s", str))
		}
		if str, ok := ts.TagStr(wpk.TIDlink); ok {
			items = append(items, fmt.Sprintf("link: %s", str))
		}
	}
	log.Println(strings.Join(items, ", "))
}

Output:

func (*FTT) Init added in v1.4.0 

func (ftt *FTT) Init(c int)

Init performs initialization for given Package structure.

func (*FTT) IsSplitted added in v1.4.0 

func (ftt *FTT) IsSplitted() bool

IsSplitted returns true if package is splitted on tags and data files.

func (*FTT) OpenDir added in v1.4.0 

func (ftt *FTT) OpenDir(fulldir string) (fs.ReadDirFile, error)

OpenDir returns PackDirFile structure associated with group of files in package pooled with common directory prefix. Usable to implement fs.FileSystem interface.

func (*FTT) Parse added in v1.4.4 

func (ftt *FTT) Parse(buf []byte) (n int64, err error)

Parse makes table from given byte slice.

func (*FTT) ReadDirN added in v1.4.0 

func (ftt *FTT) ReadDirN(fulldir string, n int) (list []fs.DirEntry, err error)

ReadDirN returns fs.DirEntry array with nested into given package directory presentation. It's core function for ReadDirFile and ReadDirFS structures.

func (*FTT) ReadFTT added in v1.4.0 

func (ftt *FTT) ReadFTT(r io.ReadSeeker) (err error)

ReadFTT opens package for reading. At first it checkups file signature, then reads records table, and reads file tagset table. Tags set for each file must contain at least file offset, file size, file ID and file name.

func (*FTT) ReadFrom added in v1.4.0 

func (ftt *FTT) ReadFrom(r io.Reader) (n int64, err error)

ReadFrom reads file tags table whole content from the given stream.

func (*FTT) SetInfo added in v1.4.0 

func (ftt *FTT) SetInfo(ts TagsetRaw)

SetInfo puts given tagset as package information tagset with "@info" path tag.

func (*FTT) Sync added in v1.4.0 

func (ftt *FTT) Sync(wpt, wpf io.WriteSeeker) (err error)

Sync writes actual file tags table and true signature with settings.

func (*FTT) WriteTo added in v1.4.0 

func (ftt *FTT) WriteTo(w io.Writer) (n int64, err error)

WriteTo writes file tags table whole content to the given stream.

type FileReader added in v1.2.0 

type FileReader interface {
	io.Reader
	io.ReaderAt
	io.Seeker
	Size() int64
}

FileReader is interface for nested package files access. 

type Header struct {
	// contains filtered or unexported fields
}

Header - package header.

func (*Header) IsReady added in v1.3.0 

func (hdr *Header) IsReady() error

IsReady determines that package is ready for read the data.

func (*Header) ReadFrom added in v1.1.0 

func (hdr *Header) ReadFrom(r io.Reader) (n int64, err error)

ReadFrom reads header from stream as binary data of constant length in little endian order.

func (*Header) WriteTo added in v1.1.0 

func (hdr *Header) WriteTo(w io.Writer) (n int64, err error)

WriteTo writes header to stream as binary data of constant length in little endian order.

type PackDirFile added in v1.4.0 

type PackDirFile struct {
	TagsetRaw // has fs.FileInfo interface
	// contains filtered or unexported fields
}

PackDirFile is a directory file whose entries can be read with the ReadDir method. fs.ReadDirFile interface implementation.

func (*PackDirFile) Close added in v1.4.0 

func (f *PackDirFile) Close() error

fs.ReadDirFile interface implementation.

func (*PackDirFile) Read added in v1.4.0 

func (f *PackDirFile) Read(b []byte) (n int, err error)

fs.ReadDirFile interface implementation.

func (*PackDirFile) ReadDir added in v1.4.0 

func (f *PackDirFile) ReadDir(n int) (matches []fs.DirEntry, err error)

fs.ReadDirFile interface implementation.

func (*PackDirFile) Stat added in v1.4.0 

func (f *PackDirFile) Stat() (fs.FileInfo, error)

fs.ReadDirFile interface implementation.

type PackDirLogger added in v1.3.0 

type PackDirLogger func(pkg *Package, r io.ReadSeeker, ts TagsetRaw) error

PackDirLogger is function called during PackDir processing after each file with OS file object and inserted tagset, that can be modified.

type Package 

type Package struct {
	*FTT
	Tagger
	Workspace string
}

Package structure contains file tags table, tagger object to get access to nested files, and subdirectory workspace.

func NewPackage added in v1.3.2 

func NewPackage() *Package

NewPackage returns pointer to new initialized Package filesystem structure. Tagger should be set later if access to nested files is needed.

func OpenPackage added in v1.4.0 

func OpenPackage(fpath string) (pkg *Package, err error)

OpenPackage creates Package objects and reads package file tags table from the file with given name. Tagger should be set later if access to nested files is needed.

func (*Package) BaseTagset added in v1.3.3 

func (pkg *Package) BaseTagset(offset, size Uint, fpath string) TagsetRaw

BaseTagset returns new tagset based on predefined TID type size and tag size type, and puts file offset and file size into tagset with predefined sizes.

func (*Package) DelTagset added in v1.4.0 

func (pkg *Package) DelTagset(fkey string)

DelTagset deletes tagset with given filename key.

func (*Package) Enum 

func (pkg *Package) Enum(f func(string, TagsetRaw) bool)

Enum calls given closure for each tagset in package. Skips package info.

Example 
package main

import (
	"log"

	"github.com/schwarzlichtbezirk/wpk"
)

func main() {
	var err error

	// Open package files tags table
	var pkg *wpk.Package
	if pkg, err = wpk.OpenPackage("example.wpk"); err != nil {
		log.Fatal(err)
	}

	// How many records in package
	var n = 0
	pkg.Enum(func(fkey string, ts wpk.TagsetRaw) bool {
		if n < 5 { // print not more than 5 file names from package
			log.Println(fkey)
		}
		n++
		return true
	})
	log.Printf("total %d records in package files tags table", n)
}

Output:

func (*Package) FullPath added in v1.4.0 

func (pkg *Package) FullPath(fpath string) string

FullPath returns concatenation of workspace and relative path.

func (*Package) GetDelTagset added in v1.4.0 

func (pkg *Package) GetDelTagset(fkey string) (TagsetRaw, bool)

GetDelTagset deletes the tagset for a key, returning the previous tagset if any.

func (*Package) GetTagset added in v1.4.4 

func (pkg *Package) GetTagset(fkey string) (TagsetRaw, bool)

GetTagset returns tagset with given filename key, if it found.

func (*Package) Glob 

func (pkg *Package) Glob(pattern string) (res []string, err error)

Glob returns the names of all files in package matching pattern or nil if there is no matching file. fs.GlobFS interface implementation.

Example 
package main

import (
	"log"

	"github.com/schwarzlichtbezirk/wpk"
)

func main() {
	var err error

	// Open package files tags table
	var pkg *wpk.Package
	if pkg, err = wpk.OpenPackage("example.wpk"); err != nil {
		log.Fatal(err)
	}

	// Get all JPEG-files in subdirectories
	var res []string
	if res, err = pkg.Glob("*/*.jpg"); err != nil {
		log.Fatal(err)
	}
	// and print them
	for _, fname := range res {
		log.Println(fname)
	}
}

Output:

func (*Package) HasTagset added in v1.4.0 

func (pkg *Package) HasTagset(fkey string) bool

HasTagset check up that tagset with given filename key is present.

func (*Package) Open added in v1.4.0 

func (pkg *Package) Open(dir string) (fs.File, error)

Open implements access to nested into package file or directory by filename. fs.FS implementation.

func (*Package) PackData added in v1.3.0 

func (pkg *Package) PackData(w io.WriteSeeker, r io.Reader, fpath string) (ts TagsetRaw, err error)

PackData puts data streamed by given reader into package as a file and associate keyname "kpath" with it.

func (*Package) PackDir added in v1.3.0 

func (pkg *Package) PackDir(w io.WriteSeeker, dirname, prefix string, logger PackDirLogger) (err error)

PackDir puts all files of given folder and it's subfolders into package. Logger function can be nil.

func (*Package) PackFile added in v1.3.0 

func (pkg *Package) PackFile(w io.WriteSeeker, file fs.File, fpath string) (ts TagsetRaw, err error)

PackFile puts file with given file handle into package and associate keyname "fpath" with it.

func (*Package) PutAlias added in v1.3.0 

func (pkg *Package) PutAlias(oldname, newname string) error

PutAlias makes clone tagset with file name 'oldname' and replace name tag in it to 'newname'. Keeps link to original file name.

func (*Package) ReadDir added in v1.4.0 

func (pkg *Package) ReadDir(dir string) ([]fs.DirEntry, error)

ReadDir reads the named directory and returns a list of directory entries sorted by filename. fs.ReadDirFS interface implementation.

func (*Package) ReadFile added in v1.4.0 

func (pkg *Package) ReadFile(fpath string) ([]byte, error)

ReadFile returns slice with nested into package file content. Makes content copy to prevent ambiguous access to closed mapped memory block. fs.ReadFileFS implementation.

func (*Package) Rename added in v1.3.0 

func (pkg *Package) Rename(oldname, newname string) error

Rename tagset with file name 'oldname' to 'newname'. Keeps link to original file name.

func (*Package) RenameDir added in v1.4.3 

func (pkg *Package) RenameDir(olddir, newdir string, skipexist bool) (count int, err error)

RenameDir renames all files in package with 'olddir' path to 'newdir' path.

func (*Package) SetTagset added in v1.4.0 

func (pkg *Package) SetTagset(fkey string, ts TagsetRaw)

SetTagset puts tagset with given filename key.

func (*Package) SetupTagset added in v1.4.4 

func (pkg *Package) SetupTagset(ts TagsetRaw)

SetupTagset puts tagset with filename key stored at tagset.

func (*Package) Stat added in v1.4.0 

func (pkg *Package) Stat(fpath string) (fs.FileInfo, error)

Stat returns a fs.FileInfo describing the file. fs.StatFS interface implementation.

func (*Package) Sub added in v1.4.0 

func (pkg *Package) Sub(dir string) (sub fs.FS, err error)

Sub clones object and gives access to pointed subdirectory. fs.SubFS implementation.

func (*Package) TrimPath added in v1.4.3 

func (pkg *Package) TrimPath(fpath string) string

TrimPath returns trimmed path without workspace prefix.

type PkgFile added in v1.4.4 

type PkgFile interface {
	fs.File
	FileReader
}

PkgFile is interface for access to nested into package files.

type RWMap added in v1.4.4 

type RWMap[K comparable, T any] struct {
	// contains filtered or unexported fields
}

func (*RWMap[K, T]) Delete added in v1.4.4 

func (rwm *RWMap[K, T]) Delete(key K)

func (*RWMap[K, T]) Get added in v1.4.4 

func (rwm *RWMap[K, T]) Get(key K) (ret T, ok bool)

func (*RWMap[K, T]) GetAndDelete added in v1.4.4 

func (rwm *RWMap[K, T]) GetAndDelete(key K) (ret T, ok bool)

func (*RWMap[K, T]) Has added in v1.4.4 

func (rwm *RWMap[K, T]) Has(key K) (ok bool)

func (*RWMap[K, T]) Init added in v1.4.4 

func (rwm *RWMap[K, T]) Init(c int)

func (*RWMap[K, T]) Len added in v1.4.4 

func (rwm *RWMap[K, T]) Len() int

func (*RWMap[K, T]) Range added in v1.4.4 

func (rwm *RWMap[K, T]) Range(f func(K, T) bool)

func (*RWMap[K, T]) Set added in v1.4.4 

func (rwm *RWMap[K, T]) Set(key K, val T)

type TagRaw added in v1.4.0 

type TagRaw []byte

TagRaw - file description item.

func BoolTag added in v1.4.0 

func BoolTag(val bool) TagRaw

BoolTag is boolean tag constructor.

func ByteTag added in v1.4.0 

func ByteTag(val byte) TagRaw

ByteTag is byte tag constructor.

func NumberTag added in v1.4.0 

func NumberTag(val float64) TagRaw

NumberTag is 64-bit float tag constructor.

func StrTag added in v1.4.0 

func StrTag(val string) TagRaw

StrTag is string tag constructor.

func TimeTag added in v1.4.0 

func TimeTag(val time.Time) TagRaw

TimeTag is 12-bytes UNIX time tag constructor.

func Uint16Tag added in v1.4.0 

func Uint16Tag(val uint16) TagRaw

Uint16Tag is 16-bit unsigned int tag constructor.

func Uint32Tag added in v1.4.0 

func Uint32Tag(val uint32) TagRaw

Uint32Tag is 32-bit unsigned int tag constructor.

func Uint64Tag added in v1.4.0 

func Uint64Tag(val uint64) TagRaw

Uint64Tag is 64-bit unsigned int tag constructor.

func UintLenTag added in v1.4.0 

func UintLenTag(val Uint, l byte) TagRaw

UintLenTag is unsigned int tag constructor with specified length in bytes.

func UintTag added in v1.4.0 

func UintTag(val Uint) TagRaw

UintTag is unspecified size unsigned int tag constructor.

func UnixTag added in v1.4.0 

func UnixTag(val time.Time) TagRaw

UnixTag is 8-bytes UNIX time in milliseconds tag constructor.

func (TagRaw) TagBool added in v1.4.0 

func (t TagRaw) TagBool() (bool, bool)

TagBool is boolean tag converter.

func (TagRaw) TagByte added in v1.4.0 

func (t TagRaw) TagByte() (byte, bool)

TagByte tag converter.

func (TagRaw) TagNumber added in v1.4.0 

func (t TagRaw) TagNumber() (float64, bool)

TagNumber is 64-bit float tag converter.

func (TagRaw) TagStr added in v1.4.0 

func (t TagRaw) TagStr() (string, bool)

TagStr tag converter.

func (TagRaw) TagTime added in v1.4.0 

func (t TagRaw) TagTime() (time.Time, bool)

TagTime is 8/12-bytes time tag converter.

func (TagRaw) TagUint added in v1.4.0 

func (t TagRaw) TagUint() (ret Uint, ok bool)

TagUint is unspecified size unsigned int tag converter.

func (TagRaw) TagUint16 added in v1.4.0 

func (t TagRaw) TagUint16() (uint16, bool)

TagUint16 is 16-bit unsigned int tag converter.

func (TagRaw) TagUint32 added in v1.4.0 

func (t TagRaw) TagUint32() (uint32, bool)

TagUint32 is 32-bit unsigned int tag converter.

func (TagRaw) TagUint64 added in v1.4.0 

func (t TagRaw) TagUint64() (uint64, bool)

TagUint64 is 64-bit unsigned int tag converter.

type Tagger 

type Tagger interface {
	OpenTagset(TagsetRaw) (PkgFile, error)
	io.Closer
}

Tagger provides acces to nested files by given tagset of this package.

type TagsetIterator added in v1.3.0 

type TagsetIterator struct {
	TagsetRaw
	// contains filtered or unexported fields
}

TagsetIterator helps to iterate through all tags.

func (*TagsetIterator) Del added in v1.3.0 

func (tsi *TagsetIterator) Del(tid Uint) (TagsetRaw, bool)

Del is the stub to disable any changes to data content of iterator.

func (*TagsetIterator) Failed added in v1.3.0 

func (tsi *TagsetIterator) Failed() bool

Failed points that iterator is finished by broken tagset state.

func (*TagsetIterator) Next added in v1.3.0 

func (tsi *TagsetIterator) Next() (ok bool)

Next carries to the next tag position.

Example 
package main

import (
	"fmt"
	"time"

	"github.com/schwarzlichtbezirk/wpk"
)

func main() {
	var ts = wpk.TagsetRaw{}.
		Put(wpk.TIDpath, wpk.StrTag("picture.jpg")).
		Put(wpk.TIDmtime, wpk.TimeTag(time.Now())).
		Put(wpk.TIDmime, wpk.StrTag("image/jpeg"))
	var tsi = ts.Iterator()
	for tsi.Next() {
		fmt.Printf("tid=%d, len=%d\n", tsi.TID(), tsi.TagLen())
	}
}

Output:

tid=3, len=11
tid=5, len=12
tid=10, len=10

func (*TagsetIterator) Passed added in v1.3.0 

func (tsi *TagsetIterator) Passed() bool

Passed returns true if the end of iterations is successfully reached.

Example 
package main

import (
	"fmt"

	"github.com/schwarzlichtbezirk/wpk"
)

func main() {
	var slice = []byte{
		3, 0, 4, 0, 10, 0, 0, 0,
		4, 0, 12, 0, 115, 111, 109, 101, 102, 105, 108, 101, 46, 100, 97, 116,
	}
	var tsi = wpk.TagsetRaw(slice).Iterator()
	for tsi.Next() {
		// place some handler code here
	}
	fmt.Println(tsi.Passed())
}

Output:

true

func (*TagsetIterator) Put added in v1.3.0 

func (tsi *TagsetIterator) Put(tid Uint, tag TagRaw) TagsetRaw

Put is the stub to disable any changes to data content of iterator.

func (*TagsetIterator) Reset added in v1.3.0 

func (tsi *TagsetIterator) Reset()

Reset restarts iterator for new iterations loop.

func (*TagsetIterator) Set added in v1.3.0 

func (tsi *TagsetIterator) Set(tid Uint, tag TagRaw) (TagsetRaw, bool)

Set is the stub to disable any changes to data content of iterator.

func (*TagsetIterator) TID added in v1.3.0 

func (tsi *TagsetIterator) TID() Uint

TID returns the tag ID of the last readed tag.

func (*TagsetIterator) Tag added in v1.3.0 

func (tsi *TagsetIterator) Tag() TagRaw

Tag returns tag slice of the last readed tag content.

func (*TagsetIterator) TagLen added in v1.3.0 

func (tsi *TagsetIterator) TagLen() int

TagLen returns length of last readed tag content.

type TagsetRaw added in v1.4.0 

type TagsetRaw []byte

TagsetRaw is slice of bytes with tags set. Each tag should be with unique ID. fs.FileInfo interface implementation.

func GetPackageInfo added in v1.3.0 

func GetPackageInfo(r io.ReadSeeker) (ts TagsetRaw, err error)

GetPackageInfo returns tagset with package information. It's a quick function to get info from the file.

func (TagsetRaw) AccessTime added in v1.4.0 

func (ts TagsetRaw) AccessTime() time.Time

AccessTime returns file access timestamp of nested into package file. times.Timespec implementation.

func (TagsetRaw) BirthTime added in v1.4.0 

func (ts TagsetRaw) BirthTime() time.Time

BirthTime returns file access timestamp of nested into package file. times.Timespec implementation.

func (TagsetRaw) ChangeTime added in v1.4.0 

func (ts TagsetRaw) ChangeTime() time.Time

ChangeTime returns file change timestamp of nested into package file. times.Timespec implementation.

func (TagsetRaw) Del added in v1.4.0 

func (ts TagsetRaw) Del(tid Uint) (TagsetRaw, bool)

Del deletes tag with given ID.

func (TagsetRaw) Get added in v1.4.0 

func (ts TagsetRaw) Get(tid Uint) (TagRaw, bool)

GetTag returns TagRaw with given identifier. If tag is not found, slice content is broken, returns false.

func (TagsetRaw) Has added in v1.4.0 

func (ts TagsetRaw) Has(tid Uint) bool

Has checks existence of tag with given ID.

func (TagsetRaw) HasBirthTime added in v1.4.0 

func (ts TagsetRaw) HasBirthTime() bool

HasBirthTime is times.Timespec interface implementation. Returns whether birth timestamp is present.

func (TagsetRaw) HasChangeTime added in v1.4.0 

func (ts TagsetRaw) HasChangeTime() bool

HasChangeTime is times.Timespec interface implementation. Returns whether change timestamp is present.

func (TagsetRaw) Info added in v1.4.0 

func (ts TagsetRaw) Info() (fs.FileInfo, error)

Info returns the FileInfo for the file or subdirectory described by the entry. fs.DirEntry interface implementation.

func (TagsetRaw) IsDir added in v1.4.0 

func (ts TagsetRaw) IsDir() bool

IsDir detects that object presents a directory. Directory can not have file ID. fs.FileInfo implementation.

func (TagsetRaw) Iterator added in v1.4.0 

func (ts TagsetRaw) Iterator() TagsetIterator

Iterator clones this tagset to iterate through all tags.

func (TagsetRaw) ModTime added in v1.4.0 

func (ts TagsetRaw) ModTime() time.Time

ModTime returns file modification timestamp of nested into package file. fs.FileInfo & times.Timespec implementation.

func (TagsetRaw) Mode added in v1.4.0 

func (ts TagsetRaw) Mode() fs.FileMode

Mode is for fs.FileInfo interface compatibility.

func (TagsetRaw) Name added in v1.4.0 

func (ts TagsetRaw) Name() string

Name returns base name of nested into package file. fs.FileInfo implementation.

func (TagsetRaw) Num added in v1.4.0 

func (ts TagsetRaw) Num() (n int)

Num returns number of tags in tagset.

func (TagsetRaw) Path added in v1.4.0 

func (ts TagsetRaw) Path() string

Path returns path of nested into package file. Path required to be present in any tagset.

func (TagsetRaw) Pos added in v1.4.0 

func (ts TagsetRaw) Pos() (offset, size Uint)

Pos returns file offset and file size in package. Those values required to be present in any tagset.

func (TagsetRaw) Put added in v1.4.0 

func (ts TagsetRaw) Put(tid Uint, tag TagRaw) TagsetRaw

Put appends new tag to tagset. Can be used in chain calls at initialization.

func (TagsetRaw) Set added in v1.4.0 

func (ts TagsetRaw) Set(tid Uint, tag TagRaw) (TagsetRaw, bool)

Set replaces tag with given ID and equal size, or appends it to tagset. Returns true if new one added.

func (TagsetRaw) Size added in v1.4.0 

func (ts TagsetRaw) Size() int64

Size returns size of nested into package file. fs.FileInfo implementation.

func (TagsetRaw) Sys added in v1.4.0 

func (ts TagsetRaw) Sys() interface{}

Sys is for fs.FileInfo interface compatibility.

func (TagsetRaw) TagBool added in v1.4.0 

func (ts TagsetRaw) TagBool(tid Uint) (bool, bool)

TagBool is boolean tag getter.

func (TagsetRaw) TagByte added in v1.4.0 

func (ts TagsetRaw) TagByte(tid Uint) (byte, bool)

TagByte tag getter.

func (TagsetRaw) TagNumber added in v1.4.0 

func (ts TagsetRaw) TagNumber(tid Uint) (float64, bool)

TagNumber is 64-bit float tag getter.

func (TagsetRaw) TagStr added in v1.4.0 

func (ts TagsetRaw) TagStr(tid Uint) (string, bool)

TagStr tag getter.

func (TagsetRaw) TagTime added in v1.4.0 

func (ts TagsetRaw) TagTime(tid Uint) (time.Time, bool)

TagTime is time tag getter.

func (TagsetRaw) TagUint added in v1.4.0 

func (ts TagsetRaw) TagUint(tid Uint) (Uint, bool)

TagUint is unspecified size unsigned int tag getter.

func (TagsetRaw) TagUint16 added in v1.4.0 

func (ts TagsetRaw) TagUint16(tid Uint) (uint16, bool)

TagUint16 is 16-bit unsigned int tag getter. Conversion can be used to get signed 16-bit integers.

func (TagsetRaw) TagUint32 added in v1.4.0 

func (ts TagsetRaw) TagUint32(tid Uint) (uint32, bool)

TagUint32 is 32-bit unsigned int tag getter. Conversion can be used to get signed 32-bit integers.

func (TagsetRaw) TagUint64 added in v1.4.0 

func (ts TagsetRaw) TagUint64(tid Uint) (uint64, bool)

TagUint64 is 64-bit unsigned int tag getter. Conversion can be used to get signed 64-bit integers.

func (TagsetRaw) Type added in v1.4.0 

func (ts TagsetRaw) Type() fs.FileMode

Type is for fs.DirEntry interface compatibility.

type Uint added in v1.4.0 

type Uint uint64

Uint can hold unsigned integer of any size on any platform.

func ReadUint added in v1.3.2 

func ReadUint(r io.Reader, l byte) (data Uint, err error)

ReadUint reads from stream unsigned integer with given size in bytes. Size can be 1, 2, 4, 8.

func ReadUintBuf added in v1.3.2 

func ReadUintBuf(b []byte) (r Uint)

ReadUintBuf reads unsigned integer from buffer of predefined size. Dimension of integer depended from size of buffer, size can be 1, 2, 4, 8.

type Union added in v1.4.0 

type Union struct {
	List []*Package
}

Union glues list of packages into single filesystem.

func (*Union) AllKeys added in v1.4.0 

func (u *Union) AllKeys() (res []string)

AllKeys returns list of all accessible files in union of packages. If union have more than one file with the same name, only first entry will be included to result.

func (*Union) Close added in v1.4.0 

func (u *Union) Close() (err error)

Close call Close-function for all included into the union packages. io.Closer implementation.

func (*Union) Glob added in v1.4.0 

func (u *Union) Glob(pattern string) (res []string, err error)

Glob returns the names of all files in union matching pattern or nil if there is no matching file.

func (*Union) Open added in v1.4.0 

func (u *Union) Open(dir string) (fs.File, error)

Open implements access to nested into union of packages file or directory by keyname. If union have more than one file with the same name, first will be returned. fs.FS implementation.

func (*Union) ReadDir added in v1.4.0 

func (u *Union) ReadDir(dir string) ([]fs.DirEntry, error)

ReadDir reads the named directory and returns a list of directory entries sorted by filename. fs.ReadDirFS interface implementation.

func (*Union) ReadDirN added in v1.4.0 

func (u *Union) ReadDirN(dir string, n int) (list []fs.DirEntry, err error)

ReadDir reads the named directory and returns a list of directory entries sorted by filename.

func (*Union) ReadFile added in v1.4.0 

func (u *Union) ReadFile(fpath string) ([]byte, error)

ReadFile returns slice with nested into union of packages file content. If union have more than one file with the same name, first will be returned. fs.ReadFileFS implementation.

func (*Union) Stat added in v1.4.0 

func (u *Union) Stat(fpath string) (fs.FileInfo, error)

Stat returns a fs.FileInfo describing the file. If union have more than one file with the same name, info of the first will be returned. fs.StatFS implementation.

func (*Union) Sub added in v1.4.0 

func (u *Union) Sub(dir string) (fs.FS, error)

Sub clones object and gives access to pointed subdirectory. fs.SubFS implementation.

type UnionDir added in v1.4.0 

type UnionDir struct {
	TagsetRaw
	*Union
}

UnionDir is helper to get access to union directories, that contains files from all packages with same dir if it present.

func (*UnionDir) Close added in v1.4.0 

func (f *UnionDir) Close() error

fs.ReadDirFile interface implementation.

func (*UnionDir) Read added in v1.4.0 

func (f *UnionDir) Read([]byte) (int, error)

fs.ReadDirFile interface implementation.

func (*UnionDir) ReadDir added in v1.4.0 

func (f *UnionDir) ReadDir(n int) ([]fs.DirEntry, error)

fs.ReadDirFile interface implementation.

func (*UnionDir) Stat added in v1.4.0 

func (f *UnionDir) Stat() (fs.FileInfo, error)

fs.ReadDirFile interface implementation.

type Void added in v1.4.0 

type Void = struct{}

Void is empty structure to release of the set of keys.

type WriteSeekCloser added in v1.3.0 

type WriteSeekCloser interface {
	io.Writer
	io.Seeker
	io.Closer
}

WriteSeekCloser is typical useful interface for package writing.

Source Files

View all Source files

Directories

Path Synopsis
util
build
extract
pack

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.