Documentation ¶
Overview ¶
TODO: Add into the configuration option a list of allowed file extnsions, default being all, -conf option only
Index ¶
- Variables
- func DumpWorker(dc *DumpConfig, wg *sync.WaitGroup)
- func GetMRTScanner(fd *os.File) *bufio.Scanner
- func NewMlFormatter() mlFormatter
- type ConfigFile
- type DayFormatter
- type DirectorySource
- type DiscardCloser
- type DumpConfig
- type Formatter
- type IdentityFormatter
- type JSONFormatter
- type MBSInfo
- type MultiWriteFile
- type PrefixEvent
- type PrefixHistory
- type PrefixWalker
- type StringArray
- type TextFormatter
- type UniquePrefixList
- type UniquePrefixSeries
Constants ¶
This section is empty.
Variables ¶
var (
DEBUG bool
)
var EOP error = fmt.Errorf("End of paths")
This is the normal error returned by a stringsource, indicating there were no failures, there are just no more strings to recieve
Functions ¶
func DumpWorker ¶
func DumpWorker(dc *DumpConfig, wg *sync.WaitGroup)
Simple worker function, launched in a new goroutine. Reads from stringsource and launches dumpfile
func NewMlFormatter ¶
func NewMlFormatter() mlFormatter
Types ¶
type ConfigFile ¶
type ConfigFile struct { Collist []string //List of collectors Start string // Start month These first three are only used in configuration option, which is why they don't have flags End string //end month Lo string //Log output So string //Stat output Do string //dump output Wc int //worker count Fmtr string //output format Conf bool //get config from a file Srcas string `json:"Srcas,omitempty"` Destas string `json:"Destas,omitempty"` PrefList string `json:"Prefixes,omitempty"` Debug bool // sets the global debug flag for the package }
This is a struct to store all options in. This is just convenient so it can be read as a json object
type DayFormatter ¶
type DayFormatter struct {
// contains filtered or unexported fields
}
func NewDayFormatter ¶
func NewDayFormatter(fd io.Writer) *DayFormatter
type DirectorySource ¶
type DirectorySource struct {
// contains filtered or unexported fields
}
func NewDirectorySource ¶
func NewDirectorySource(dirs []string) *DirectorySource
func (*DirectorySource) Next ¶
func (ds *DirectorySource) Next() (string, error)
type DiscardCloser ¶
type DiscardCloser struct{}
func (DiscardCloser) Close ¶
func (d DiscardCloser) Close() error
type DumpConfig ¶
type DumpConfig struct {
// contains filtered or unexported fields
}
This struct is the complete parameter set for a file dump.
func GetDumpConfig ¶
func GetDumpConfig(configFile ConfigFile) (*DumpConfig, error)
func (*DumpConfig) CloseAll ¶
func (dc *DumpConfig) CloseAll()
func (*DumpConfig) GetWorkers ¶
func (dc *DumpConfig) GetWorkers() int
func (*DumpConfig) SummarizeAndClose ¶
func (dc *DumpConfig) SummarizeAndClose(start time.Time)
type Formatter ¶
type Formatter interface {
// contains filtered or unexported methods
}
A Formatter takes the bufferstack and the underlying buffer and returns a representation of the data to be written to the dump file. The underlying buffer is necessary for the ID formatter
type IdentityFormatter ¶
type IdentityFormatter struct{}
Applies no formatting to the data But data is decompressed, may need to fix that However, golang bz2 doesn't have compression features
func NewIdentityFormatter ¶
func NewIdentityFormatter() IdentityFormatter
type JSONFormatter ¶
type JSONFormatter struct{}
Formats each update as a JSON message
func NewJSONFormatter ¶
func NewJSONFormatter() JSONFormatter
type MBSInfo ¶
type MBSInfo struct {
// contains filtered or unexported fields
}
This contains miscellaneous info for the MBS structure
func NewMBSInfo ¶
type MultiWriteFile ¶
type MultiWriteFile struct {
// contains filtered or unexported fields
}
The dump, stat, and log files are all accessed by multiple goroutines. This is a simple file wrapper to lock on a write, and unlock once the write is complete
func NewMultiWriteFile ¶
func NewMultiWriteFile(w io.WriteCloser) *MultiWriteFile
func (*MultiWriteFile) Close ¶
func (mwf *MultiWriteFile) Close() error
func (*MultiWriteFile) WriteString ¶
func (mwf *MultiWriteFile) WriteString(s string) (n int, err error)
type PrefixHistory ¶
type PrefixHistory struct { Pref string Events []PrefixEvent // contains filtered or unexported fields }
func NewPrefixHistory ¶
func (*PrefixHistory) String ¶
func (ph *PrefixHistory) String() string
type PrefixWalker ¶
type PrefixWalker struct {
// contains filtered or unexported fields
}
type StringArray ¶
type StringArray struct {
// contains filtered or unexported fields
}
Simple wrapper for a string array, so it can be accessed concurrently, and in the same way as a DirectorySource
func NewStringArray ¶
func NewStringArray(buf []string) *StringArray
func (*StringArray) Next ¶
func (sa *StringArray) Next() (string, error)
type TextFormatter ¶
type TextFormatter struct {
// contains filtered or unexported fields
}
A simple text representation for the dump. The only formatter that needs the msgnum
func NewTextFormatter ¶
func NewTextFormatter() *TextFormatter
type UniquePrefixList ¶
type UniquePrefixList struct {
// contains filtered or unexported fields
}
UniquePrefixList will look at all incoming messages, and output only the top level prefixes seen.
func NewUniquePrefixList ¶
func NewUniquePrefixList(fd io.Writer) *UniquePrefixList
type UniquePrefixSeries ¶
type UniquePrefixSeries struct {
// contains filtered or unexported fields
}
UniquePrefixSeries does the same thing as UniquePrefixList, but rather than just a list, it will output a gob file containing each prefix and every event seen associated with that prefix
func NewUniquePrefixSeries ¶
func NewUniquePrefixSeries(fd io.Writer) *UniquePrefixSeries