zoossh

package module
v0.0.0-...-0156201 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2023 License: BSD-2-Clause Imports: 14 Imported by: 0

README

Build Status

Overview

Zoossh is a Go parsing library for Tor-specific data formats. It can parse consensuses and server descriptors. I originally wrote zoossh as a speedy fundament for sybilhunter, a tool to find Sybils in the Tor network. I have no need for other file types, so if zoossh doesn't provide what you need, check out the libraries below.

Supported file formats

Zoossh partially supports the following two file formats:

  • Server descriptors (@type server-descriptor 1.0)
  • Network status consensuses (@type network-status-consensus-3 1.0)

For more information about file formats, have a look at CollecTor.

Examples

Here's how you can parse a network status document and iterate over all relay statuses:

consensus, err := zoossh.ParseConsensusFile(fileName)
if err != nil {
    // Handle error.
}

for status := range consensus.Iterate(nil) {
    fmt.Println(status)
}

Similarly, here's how you can parse a file containing server descriptors:

descriptors, err := zoossh.ParseDescriptorFile(fileName)
if err != nil {
    // Handle error.
}

for desc := range descriptors.Iterate(nil) {
    fmt.Println(desc)
}

For more details, have a look at zoossh's GoDoc page.

Alternatives

Check out the Python library Stem or the Java library metrics-lib. Both have more comprehensive support for data formats. There is also a comparison available online between Stem, metrics-lib, and zoossh.

Contact

Contact: Philipp Winter phw@nymity.ch
OpenPGP fingerprint: B369 E7A2 18FE CEAD EB96 8C73 CF70 89E3 D7FD C0D0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var DescCache = make(map[string]*RouterDescriptor)

DescCache maps a descriptor's digest to its router descriptor.

Functions

func Base64ToString 

func Base64ToString(encoded string) (string, error)

Base64ToString decodes the given Base64-encoded string and returns the resulting string. If there are errors during decoding, an error string is returned.

func CheckAnnotation 

func CheckAnnotation(fd *os.File, expected map[Annotation]bool) error

CheckAnnotation checks the type annotation in the given file. The Annotation struct determines what we want to see in the file. If we don't see the expected annotation, an error string is returned.

func DissectFile 

func DissectFile(r io.Reader, extractor bufio.SplitFunc, queue chan QueueUnit)

Dissects the given file into string chunks by using the given string extraction function. The resulting string chunks are then written to the given queue where the receiving end parses them.

func LazyParseRawDescriptor 

func LazyParseRawDescriptor(rawDescriptor string) (Fingerprint, GetDescriptor, error)

LazyParseRawDescriptor lazily parses a raw router descriptor (in string format) and returns the descriptor's fingerprint, a function returning the descriptor, and an error if the descriptor could not be parsed. Parsing is delayed until the router descriptor is accessed.

func LazyParseRawStatus 

func LazyParseRawStatus(rawStatus string) (Fingerprint, GetStatus, error)

LazyParseRawStatus parses a raw router status (in string format) and returns the router's fingerprint, a function which returns a RouterStatus, and an error if there were any during parsing. Parsing of the given string is delayed until the returned function is executed.

func ParseRawDescriptor 

func ParseRawDescriptor(rawDescriptor string) (Fingerprint, GetDescriptor, error)

ParseRawDescriptor parses a raw router descriptor (in string format) and returns the descriptor's fingerprint, a function returning the descriptor, and an error if the descriptor could not be parsed. In contrast to LazyParseRawDescriptor, parsing is *not* delayed.

func ParseRawStatus 

func ParseRawStatus(rawStatus string) (Fingerprint, GetStatus, error)

ParseRawStatus parses a raw router status (in string format) and returns the router's fingerprint, a function which returns a RouterStatus, and an error if there were any during parsing.

func StringToPort 

func StringToPort(portStr string) uint16

Convert the given port string to an unsigned 16-bit integer. If the conversion fails or the number cannot be represented in 16 bits, 0 is returned.

Types

type Annotation 

type Annotation struct {
	Type  string
	Major string
	Minor string
}

func GetAnnotation 

func GetAnnotation(fileName string) (*Annotation, error)

GetAnnotation obtains and returns the given file's annotation. If anything fails in the process, an error string is returned.

func (*Annotation) Equals 

func (a *Annotation) Equals(b *Annotation) bool

Equals checks whether the two given annotations have the same content.

func (*Annotation) String 

func (a *Annotation) String() string

type Consensus 

type Consensus struct {
	// Generic map of consensus metadata
	MetaInfo map[string][]byte

	// Document validity period
	ValidAfter time.Time
	FreshUntil time.Time
	ValidUntil time.Time

	// Shared randomness
	SharedRandPrevious []byte
	SharedRandCurrent  []byte

	// A map from relay fingerprint to a function which returns the relay
	// status.
	RouterStatuses map[Fingerprint]GetStatus
}

func LazilyParseConsensusFile 

func LazilyParseConsensusFile(fileName string) (*Consensus, error)

LazilyParseConsensusFile parses the given file and returns a network consensus if parsing was successful. If there were any errors, an error string is returned. Parsing of the router statuses is delayed until they are accessed using the Get method. As a result, this function is recommended as long as you won't access more than ~50% of all statuses.

func LazilyParseUnsafeConsensusFile 

func LazilyParseUnsafeConsensusFile(fileName string) (*Consensus, error)

LazilyParseUnsafeConsensusFile parses the given file without checking the annotations and returns a network consensus if parsing was successful. If there were any errors, consensus if parsing was successful. If there were any errors, an error string is returned. Parsing of the router statuses is delayed until they are accessed using the Get method. As a result, this function is recommended as long as you won't access more than ~50% of all statuses.

func NewConsensus 

func NewConsensus() *Consensus

NewConsensus serves as a constructor and returns a pointer to a freshly allocated and empty Consensus.

func ParseConsensusFile 

func ParseConsensusFile(fileName string) (*Consensus, error)

ParseConsensusFile parses the given file and returns a network consensus if parsing was successful. If there were any errors, an error string is returned. In contrast to LazilyParseConsensusFile, parsing of router statuses is *not* delayed. As a result, this function is recommended as long as you will access most of all statuses.

func ParseRawConsensus 

func ParseRawConsensus(rawConsensus string, lazy bool) (*Consensus, error)

ParseRawConsensus parses a raw consensus (in string format) and returns a network consensus if parsing was successful.

func ParseRawUnsafeConsensus 

func ParseRawUnsafeConsensus(rawConsensus string, lazy bool) (*Consensus, error)

ParseRawUnsafeConsensus parses a raw consensus (in string format) and returns a network consensus if parsing was successful.

func ParseUnsafeConsensusFile 

func ParseUnsafeConsensusFile(fileName string) (*Consensus, error)

ParseUnsafeConsensusFile parses the given file without checking the annotations and returns a network consensus if parsing was successful. If there were any errors, an error string is returned. In contrast to LazilyParseConsensusFile, parsing of router statuses is *not* delayed. As a result, this function is recommended as long as you will access most of all statuses.

func (*Consensus) Get 

func (c *Consensus) Get(fingerprint Fingerprint) (*RouterStatus, bool)

Get returns the router status for the given fingerprint and a boolean value indicating if the status could be found in the consensus.

func (*Consensus) GetObject 

func (c *Consensus) GetObject(fingerprint Fingerprint) (Object, bool)

GetObject implements the ObjectSet interface. It returns the object identified by the given fingerprint. If the object is not present in the set, false is returned, otherwise true.

func (*Consensus) Intersect 

func (c *Consensus) Intersect(b *Consensus) *Consensus

Intersect determines the intersection between the given consensus b and consensus a. It returns a new consensus which is the intersection of both given consensuses.

func (*Consensus) Iterate 

func (c *Consensus) Iterate(filter *ObjectFilter) <-chan Object

Iterate implements the ObjectSet interface. Using a channel, it iterates over and returns all router statuses. The given object filter can be used to filter router statuses, e.g., by fingerprint.

func (*Consensus) Length 

func (c *Consensus) Length() int

Length implements the ObjectSet interface. It returns the length of the consensus.

func (*Consensus) Merge 

func (c *Consensus) Merge(objs ObjectSet)

Merge merges the given object set with itself.

func (*Consensus) Set 

func (c *Consensus) Set(fingerprint Fingerprint, status *RouterStatus)

Set adds a new fingerprint mapping to a function returning the router status to the consensus.

func (*Consensus) Subtract 

func (c *Consensus) Subtract(b *Consensus) *Consensus

Subtract removes all routers which are part of the given consensus b from consensus a. It returns a new consensus which is the result of the subtraction.

func (*Consensus) ToSlice 

func (c *Consensus) ToSlice() []GetStatus

ToSlice converts the given consensus to a slice. Consensus meta information is lost.

type ExitPattern 

type ExitPattern struct {
	AddressSpec string
	PortSpec    string
}

An exitpattern as defined in dirspec.txt, Section 2.1.3.

type Fingerprint 

type Fingerprint string

Fingerprint represents a relay's fingerprint as 40 hex digits.

func SanitiseFingerprint 

func SanitiseFingerprint(fingerprint Fingerprint) Fingerprint

SanitiseFingerprint returns a sanitised version of the given fingerprint by making it upper case and removing leading and trailing white spaces.

type GetDescriptor 

type GetDescriptor func() *RouterDescriptor

type GetStatus 

type GetStatus func() *RouterStatus

type Object 

type Object interface {
	String() string
	GetFingerprint() Fingerprint
}

Object defines functions that should be supported by a data element, e.g., a router descriptor, or a router status in a consensus.

type ObjectFilter 

type ObjectFilter struct {
	Fingerprints map[Fingerprint]struct{}
	IPAddrs      map[string]struct{}
	Nicknames    map[string]struct{}
}

ObjectFilter holds sets that consist of objects that should pass object set filtering.

func NewObjectFilter 

func NewObjectFilter() *ObjectFilter

NewObjectFilter returns a newly allocated object filter instance.

func (*ObjectFilter) AddFingerprint 

func (filter *ObjectFilter) AddFingerprint(fpr Fingerprint)

AddFingerprint adds the given fingerprint to the object filter.

func (*ObjectFilter) AddIPAddr 

func (filter *ObjectFilter) AddIPAddr(addr net.IP)

AddIPAddr adds the given IP address to the object filter.

func (*ObjectFilter) AddNickname 

func (filter *ObjectFilter) AddNickname(nickname string)

AddNickname adds the given nickname to the object filter.

func (*ObjectFilter) HasFingerprint 

func (filter *ObjectFilter) HasFingerprint(fpr Fingerprint) bool

HasFingerprint returns true if the given fingerprint is present in the object filter.

func (*ObjectFilter) HasIPAddr 

func (filter *ObjectFilter) HasIPAddr(addr net.IP) bool

HasIPAddr returns true if the given IP address is present in the object filter.

func (*ObjectFilter) HasNickname 

func (filter *ObjectFilter) HasNickname(nickname string) bool

HasNickname returns true if the given nickname is present in the object filter.

func (*ObjectFilter) IsEmpty 

func (filter *ObjectFilter) IsEmpty() bool

IsEmpty returns true if the object filter is empty.

func (*ObjectFilter) MatchesRouterDescriptor 

func (filter *ObjectFilter) MatchesRouterDescriptor(desc *RouterDescriptor) bool

MatchesRouterDescriptor returns true if fields of the given router descriptor are present in the object filter, e.g., the descriptor's nickname is part of the object filter.

func (*ObjectFilter) MatchesRouterStatus 

func (filter *ObjectFilter) MatchesRouterStatus(status *RouterStatus) bool

MatchesRouterStatus returns true if fields of the given router status are present in the object filter, e.g., the router's nickname is part of the object filter.

type ObjectSet 

type ObjectSet interface {
	Length() int
	Iterate(*ObjectFilter) <-chan Object
	GetObject(Fingerprint) (Object, bool)
	Merge(ObjectSet)
}

ObjectSet defines functions that should be supported by a set of objects.

func ParseUnknown 

func ParseUnknown(r io.Reader) (ObjectSet, error)

ParseUnknown first reads a type annotation and passes it along with the rest of the input to parseWithAnnotation.

func ParseUnknownFile 

func ParseUnknownFile(fileName string) (ObjectSet, error)

ParseUnknownFile attempts to parse a file whose content we don't know. We try to use the right parser by looking at the file's annotation. An ObjectSet is returned if parsing was successful.

type QueueUnit 

type QueueUnit struct {
	Blurb string
	Err   error
}

type RouterAddress 

type RouterAddress struct {
	IPv4Address net.IP
	IPv4ORPort  uint16
	IPv4DirPort uint16

	IPv6Address net.IP
	IPv6ORPort  uint16
}

func (RouterAddress) String 

func (address RouterAddress) String() string

Implement the Stringer interface for pretty printing.

type RouterDescriptor 

type RouterDescriptor struct {

	// The single fields of a "router" line.
	Nickname  string
	Address   net.IP
	ORPort    uint16
	SOCKSPort uint16
	DirPort   uint16

	// The single fields of a "bandwidth" line.  All bandwidth values are in
	// bytes per second.
	BandwidthAvg   uint64
	BandwidthBurst uint64
	BandwidthObs   uint64

	// The single fields of a "platform" line.
	OperatingSystem string
	TorVersion      string

	// The single fields of a "published" line.
	Published time.Time

	// The single fields of an "uptime" line.
	Uptime uint64

	// The single fields of a "fingerprint" line.
	Fingerprint Fingerprint

	// The single fields of a "hibernating" line.
	Hibernating bool

	// The single fields of a "family" line.
	Family map[Fingerprint]bool

	// The single fields of a "contact" line.
	Contact string

	// The "hidden-service-dir" line.
	HiddenServiceDir bool

	// The "bridge-distribution-request" line
	// it only exist on the bridge-descriptors
	BridgeDistributionRequest string

	OnionKey     string
	NTorOnionKey string
	SigningKey   string

	RawAccept     string
	RawReject     string
	RawExitPolicy string

	Accept []*ExitPattern
	Reject []*ExitPattern
}

An (incomplete) router descriptor as defined in dirspec.txt, Section 2.1.1.

func LoadDescriptorFromDigest 

func LoadDescriptorFromDigest(descriptorDir, digest string, date time.Time) (*RouterDescriptor, error)

LoadDescriptorFromDigest takes as input the descriptor directory, a descriptor's digest, and the date the digest was created. It then attempts to parse and return the descriptor referenced by the digest. The descriptor directory expects to contain CollecTor server descriptor archives such as: server-descriptors-2015-03/ server-descriptors-2015-04/ ...

func NewRouterDescriptor 

func NewRouterDescriptor() *RouterDescriptor

NewRouterDescriptor serves as a constructor and returns a pointer to a freshly allocated and empty RouterDescriptor struct.

func (*RouterDescriptor) GetFingerprint 

func (rd *RouterDescriptor) GetFingerprint() Fingerprint

GetFingerprint implements the Object interface. It returns the descriptor's fingerprint.

func (*RouterDescriptor) HasFamily 

func (rd *RouterDescriptor) HasFamily(fingerprint Fingerprint) bool

HasFamily returns true if the given relay identified by its fingerprint is part of this relay's family.

func (*RouterDescriptor) String 

func (rd *RouterDescriptor) String() string

String implements the String as well as the Object interface. It returns the descriptor's string representation.

type RouterDescriptors 

type RouterDescriptors struct {

	// A map from relay fingerprint to a function which returns the router
	// descriptor.
	RouterDescriptors map[Fingerprint]GetDescriptor
}

func LazilyParseDescriptorFile 

func LazilyParseDescriptorFile(fileName string) (*RouterDescriptors, error)

LazilyParseDescriptorFile parses the given file and returns a pointer to RouterDescriptors containing the router descriptors. If there were any errors, an error string is returned. Note that parsing is done lazily which means that it is delayed until a given router descriptor is accessed. That pays off when you know that you will not parse most router descriptors.

func LazilyParseUnsafeDescriptorFile 

func LazilyParseUnsafeDescriptorFile(fileName string) (*RouterDescriptors, error)

LazilyParseDescriptorFile parses the given file without checking the annotations and returns a pointer to RouterDescriptors containing the router descriptors. If there were any errors, an error string is returned. Note that parsing is done lazily which means that it is delayed until a given router descriptor is accessed. That pays off when you know that you will not parse most router descriptors.

func NewRouterDescriptors 

func NewRouterDescriptors() *RouterDescriptors

NewRouterDescriptors serves as a constructor and returns a pointer to a freshly allocated and empty RouterDescriptors struct.

func ParseDescriptorFile 

func ParseDescriptorFile(fileName string) (*RouterDescriptors, error)

ParseDescriptorFile parses the given file and returns a pointer to RouterDescriptors containing the router descriptors. If there were any errors, an error string is returned. Note that in contrast to LazilyParseDescriptorFile, parsing is *not* delayed. That pays off when you know that you will parse most router descriptors.

func ParseUnsafeDescriptorFile 

func ParseUnsafeDescriptorFile(fileName string) (*RouterDescriptors, error)

ParseDescriptorFile parses the given file without checking the annotations and returns a pointer to RouterDescriptors containing the router descriptors. If there were any errors, an error string is returned. Note that in contrast to LazilyParseDescriptorFile, parsing is *not* delayed. That pays off when you know that you will parse most router descriptors.

func (*RouterDescriptors) Get 

func (rds *RouterDescriptors) Get(fingerprint Fingerprint) (*RouterDescriptor, bool)

Get returns the router descriptor for the given fingerprint and a boolean value indicating if the descriptor could be found.

func (*RouterDescriptors) GetObject 

func (rds *RouterDescriptors) GetObject(fingerprint Fingerprint) (Object, bool)

GetObject implements the ObjectSet interface. It returns the object identified by the given fingerprint. If the object is not present in the set, false is returned, otherwise true.

func (*RouterDescriptors) Iterate 

func (rds *RouterDescriptors) Iterate(filter *ObjectFilter) <-chan Object

Iterate implements the ObjectSet interface. Using a channel, it iterates over and returns all router descriptors. The given object filter can be used to filter descriptors, e.g., by fingerprint.

func (*RouterDescriptors) Length 

func (rds *RouterDescriptors) Length() int

Length implements the ObjectSet interface. It returns the length of the router descriptors.

func (*RouterDescriptors) Merge 

func (rds *RouterDescriptors) Merge(objs ObjectSet)

Merge merges the given object set with itself.

func (*RouterDescriptors) Set 

func (rds *RouterDescriptors) Set(fingerprint Fingerprint, descriptor *RouterDescriptor)

Set adds a new fingerprint mapping to a function returning the router descriptor.

func (*RouterDescriptors) ToSlice 

func (rds *RouterDescriptors) ToSlice() []GetDescriptor

ToSlice converts the given router descriptors to a slice.

type RouterFlags 

type RouterFlags struct {
	Authority bool
	BadExit   bool
	Exit      bool
	Fast      bool
	Guard     bool
	HSDir     bool
	Named     bool
	Stable    bool
	Running   bool
	Unnamed   bool
	Valid     bool
	V2Dir     bool
}

func (RouterFlags) String 

func (flags RouterFlags) String() string

Implement the Stringer interface for pretty printing.

type RouterStatus 

type RouterStatus struct {

	// The single fields of an "r" line.
	Nickname    string
	Fingerprint Fingerprint
	Digest      string
	Publication time.Time

	// The IPv4 and IPv6 fields of "a" line
	Address RouterAddress

	// The single fields of an "s" line.
	Flags RouterFlags

	// The single fields of a "v" line.
	TorVersion string

	// The single fields of a "w" line.
	Bandwidth  uint64
	Measured   uint64
	Unmeasured bool

	// The single fields of a "p" line.
	Accept   bool
	PortList string
}

func (*RouterStatus) GetFingerprint 

func (s *RouterStatus) GetFingerprint() Fingerprint

GetFingerprint implements the Object interface. It returns the router status' fingerprint.

func (*RouterStatus) String 

func (s *RouterStatus) String() string

String implements the String as well as the Object interface. It returns the status' string representation.

type StringExtractor 

type StringExtractor func(string) (string, bool, error)

Extracts a string unit from an archive file that can be readily thrown into the respective parser.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.