collect

Documentation

Index

• func E(err error, s string)
• func EB(err bool, s string)
• type Collect
  • func NewCollect(outputName string, build bool) (collect *Collect, err error)
  • func (c *Collect) BuildHashFile() (err error)
  • func (c *Collect) Close() (err error)
  • func (c *Collect) Fetch(KeySought interface{}) (transaction, hash []byte, err error)
  • func (c *Collect) GetHash() (hash []byte)
  • func (c *Collect) NumberHashes() (hashCnt int, err error)
  • func (c *Collect) Open(filename string) (err error)
  • func (c *Collect) SortIndexes() (err error)
  • func (c *Collect) WriteHash(hash []byte) (err error)
  • func (c *Collect) WriteTx(tx []byte) (err error)
• type Record

Functions

func E

func E(err error, s string)

E Cheap throw an error because programmer is lazy

func EB

func EB(err bool, s string)

EB Cheap error thrown against a boolean

Types

type Collect

type Collect struct {
	TxCount       int     // Transactions written
	GuessCnt      int     // Just Info.  Count of guesses made to find hash
	OffsetToIndex [8]byte // Offset to the Index Table
	Offset        int64   // Keep the offset to Index Table as a uint64 as well
	OffsetEnd     int64   // Offset end of file
	Filename      string  // Name of the output file
	TmpDirName    string  // Name of a temporary directory where we put the tmp files
	// contains filtered or unexported fields
}

func NewCollect

func NewCollect(outputName string, build bool) (collect *Collect, err error)

NewCollect The outputName is the full path of the desired output file. If it exists, the outputName will be deleted.

If build is true, then structures to build a snapshot are initialized. If false, then the snapshot is simply opened to allow queries

A directory of the name outputName.tmp will be created. After successful processing the directory will be deleted. If one exists when NewCollect is called, that directory will first be deleted.

func (*Collect) BuildHashFile

func (c *Collect) BuildHashFile() (err error)

BuildHashFile Collect all the hashes from the byte hash files and build one sorted Hash File Leaves the file pointer at the start of Hash file for easy serial access

func (*Collect) Close

func (c *Collect) Close() (err error)

Close Close all the files and clean up

func (*Collect) Fetch

func (c *Collect) Fetch(KeySought interface{}) (transaction, hash []byte, err error)

Fetch Find "KeySought" in the transactions. It is easy if the KeySought is an index. It is a bit harder if the KeySought is a hash, because we have to search Anything else is an error.

func (*Collect) GetHash

func (c *Collect) GetHash() (hash []byte)

GetHash Get the next hash out of the hash file. After all the hashes have been put into collect, the user then does a BuildHashFile(). Afterwards, the caller can walk serially through the hashes by making repeated calls to GetHash. Returns nil on EOF

GetHash will panic on any error other than EOF

func (*Collect) NumberHashes

func (c *Collect) NumberHashes() (hashCnt int, err error)

NumberHashes Returns the number of Hashes in the temporary hashes file Note this call is only valid after calling BuildHshFile()

func (*Collect) Open

func (c *Collect) Open(filename string) (err error)

Open Opens the output file for read access after building a transaction file

func (*Collect) SortIndexes

func (c *Collect) SortIndexes() (err error)

Sort Indexes Now all the transactions are written to the transactions file, and all the hash/index records sorted into 256 bins. It would be easy enough to process each bin by the next byte and so forth, keeping our memory usage way down. However, one byte is likely enough for what we are doing.

func (*Collect) WriteHash

func (c *Collect) WriteHash(hash []byte) (err error)

WriteHash Sorts Hashes into buckets. Allows them to be sorted, deduplicated, then written all into the hash file

func (*Collect) WriteTx

func (c *Collect) WriteTx(tx []byte) (err error)

WriteTx Takes a transaction to write to c.Filename. Computes the hash and the length of the transaction Writes the length as a 4 byte value (limit 4 GB per transaction) Writes the transaction after the length Writes the index to the length to the transaction indexes

type Record

type Record struct {
	// contains filtered or unexported fields
}

Record each entry in the index for the transactions