xam

package module
v0.0.0-...-e2f0145 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 5, 2022 License: GPL-2.0 Imports: 14 Imported by: 0

README 

#+TITLE: xam

* Introduction
XAM is a filesystem metadatabase. It allows you to query a set of files for their metadata, such as asking which files are a specific size or which match a given hash.

As a command line utility, it generates a xam.csv file, which is sort of like md5deep.

* Dependencies
** Runtime
None!
** Development
[[https://golang.org/dl/][go]]
[[https://github.com/constabulary/gb][gb]]
* Installation
** Source
#+begin_src bash
# assumes a working Go installation
go get github.com/constabulary/gb/...
git clone github.com/hut8/xam
cd xam
gb vendor restore
gb build
mv bin/xam $WHEREVER_IN_PATH
#+end_src

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ComputeHashes 

func ComputeHashes(
	output chan FileData,
	input chan FileData)

ComputeHashes loops over file data from input, hashes each, and passes it down the output channel

func HashFile 

func HashFile(path string) ([]byte, error)

HashFile hashes a file with SHA1 and returns the hash as a byte slice

func HashFileHex 

func HashFileHex(path string) (string, error)

HashFileHex hashes a file with SHA1 and returns the hash as a hex string

func WalkFSTree 

func WalkFSTree(fileDataChan chan FileData, rootPath string)

WalkFSTree passes each file encountered while walking tree into fileDataChan rootPath should be an absolute path

func WriteCSV 

func WriteCSV(
	fileData chan FileData,
	csvFile io.Writer,
	doneChan chan struct{})

WriteCSV serializeds xam.FileData instances to be read by ReadCSV

Types

type FileDB 

type FileDB struct {
	// contains filtered or unexported fields
}

FileDB provides an in-memory database for querying duplicate files by size and SHA1

func NewFileDB 

func NewFileDB(fd []*FileData) *FileDB

NewFileDB constructs a new FileDB instance given a slice of FileData instances

func (*FileDB) Add 

func (db *FileDB) Add(fd *FileData)

Add adds a new FileData object to the database if not already present

func (*FileDB) FindBySHA1 

func (db *FileDB) FindBySHA1(sha1 string) []*FileData

FindBySHA1 returns all files in FileDB with matching SHA1

func (*FileDB) FindBySize 

func (db *FileDB) FindBySize(size int64) []*FileData

FindBySize returns all files in FileDB with matching size

type FileData 

type FileData struct {
	Path    string      `csv:"path"`
	Err     error       `csv:"err"`
	Size    int64       `csv:"size"`
	Mode    os.FileMode `csv:"-"`
	SHA1    string      `csv:"sha1"`
	ModTime Time        `csv:"modified"`
}

FileData represents attributes of file +gen * slice:"Where"

func ReadCSV 

func ReadCSV(csvPath string) ([]*FileData, error)

ReadCSV returns entries serialized by WriteCSV

type FileDataSlice 

type FileDataSlice []*FileData

FileDataSlice is a slice of type *FileData. Use it where you would use []*FileData.

func (FileDataSlice) Where 

func (rcv FileDataSlice) Where(fn func(*FileData) bool) (result FileDataSlice)

Where returns a new FileDataSlice whose elements return true for func. See: http://clipperhouse.github.io/gen/#Where

type HashCacheFunc 

type HashCacheFunc func(*FileData) string

HashCacheFunc allows hashes to be cached in a file. The function should return "" if it's unsure of the hash given the information in the FileData instance. The instance passed in will never have the SHA1 set to a non-zero value.

type Index 

type Index struct {
	Set IndexEntrySet
	// contains filtered or unexported fields
}

func NewIndex 

func NewIndex() *Index

func (*Index) FromFileData 

func (i *Index) FromFileData(source chan FileData)

func (*Index) HasHash 

func (i *Index) HasHash(hash string) bool

func (*Index) HasSize 

func (i *Index) HasSize(sz int64) bool

func (*Index) LoadEntries 

func (i *Index) LoadEntries(source chan IndexEntry)

func (*Index) LoadFromFile 

func (i *Index) LoadFromFile(path string) error

func (*Index) Write 

func (i *Index) Write(w io.Writer)

type IndexEntry 

type IndexEntry struct {
	Hash string
	Size int64
}

IndexEntry is a (Hash, Size) Tuple +gen set

func NewIndexEntry 

func NewIndexEntry(fd FileData) IndexEntry

func (*IndexEntry) String 

func (e *IndexEntry) String() string

type IndexEntrySet 

type IndexEntrySet map[IndexEntry]struct{}

IndexEntrySet is the primary type that represents a set

func NewIndexEntrySet 

func NewIndexEntrySet(a ...IndexEntry) IndexEntrySet

NewIndexEntrySet creates and returns a reference to an empty set.

func (IndexEntrySet) Add 

func (set IndexEntrySet) Add(i IndexEntry) bool

Add adds an item to the current set if it doesn't already exist in the set.

func (IndexEntrySet) Cardinality 

func (set IndexEntrySet) Cardinality() int

Cardinality returns how many items are currently in the set.

func (*IndexEntrySet) Clear 

func (set *IndexEntrySet) Clear()

Clear clears the entire set to be the empty set.

func (IndexEntrySet) Clone 

func (set IndexEntrySet) Clone() IndexEntrySet

Clone returns a clone of the set. Does NOT clone the underlying elements.

func (IndexEntrySet) Contains 

func (set IndexEntrySet) Contains(i IndexEntry) bool

Contains determines if a given item is already in the set.

func (IndexEntrySet) ContainsAll 

func (set IndexEntrySet) ContainsAll(i ...IndexEntry) bool

ContainsAll determines if the given items are all in the set

func (IndexEntrySet) Difference 

func (set IndexEntrySet) Difference(other IndexEntrySet) IndexEntrySet

Difference returns a new set with items in the current set but not in the other set

func (IndexEntrySet) Equal 

func (set IndexEntrySet) Equal(other IndexEntrySet) bool

Equal determines if two sets are equal to each other. If they both are the same size and have the same items they are considered equal. Order of items is not relevent for sets to be equal.

func (IndexEntrySet) Intersect 

func (set IndexEntrySet) Intersect(other IndexEntrySet) IndexEntrySet

Intersect returns a new set with items that exist only in both sets.

func (IndexEntrySet) IsSubset 

func (set IndexEntrySet) IsSubset(other IndexEntrySet) bool

IsSubset determines if every item in the other set is in this set.

func (IndexEntrySet) IsSuperset 

func (set IndexEntrySet) IsSuperset(other IndexEntrySet) bool

IsSuperset determines if every item of this set is in the other set.

func (IndexEntrySet) Iter 

func (set IndexEntrySet) Iter() <-chan IndexEntry

Iter returns a channel of type IndexEntry that you can range over.

func (IndexEntrySet) Remove 

func (set IndexEntrySet) Remove(i IndexEntry)

Remove allows the removal of a single item in the set.

func (IndexEntrySet) SymmetricDifference 

func (set IndexEntrySet) SymmetricDifference(other IndexEntrySet) IndexEntrySet

SymmetricDifference returns a new set with items in the current set or the other set but not in both.

func (IndexEntrySet) ToSlice 

func (set IndexEntrySet) ToSlice() []IndexEntry

ToSlice returns the elements of the current set as a slice

func (IndexEntrySet) Union 

func (set IndexEntrySet) Union(other IndexEntrySet) IndexEntrySet

Union returns a new set with all items in both sets.

type Time 

type Time struct {
	time.Time
}

func (*Time) MarshalCSV 

func (t *Time) MarshalCSV() (string, error)

MarshalCSV is used by github.com/gocarina/gocsv

func (*Time) UnmarshalCSV 

func (t *Time) UnmarshalCSV(csv string) (err error)

UnmarshalCSV is used by github.com/gocarina/gocsv

Source Files

View all Source files

Directories

Path Synopsis
cmd
xam

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.