format

Documentation

Overview

Package format contains functions for printing and formatting output

Index

• func ASCIIEscapeUnicode(str string) string
• func CreateVerboseStats(freq map[string]int) string
• func DeASCIIEscapeUnicode(str string) string
• func DecodeInputMap(input map[string]int, bypass bool, debug bool) map[string]int
• func DecodeString(s string) (string, string, string)
• func DehexMap(input map[string]int, bypass bool, debug bool) map[string]int
• func EncodeInputMap(input map[string]int, bypass bool, debug bool) map[string]int
• func EncodeString(s string) (string, string, string)
• func HexEncodeMap(input map[string]int, bypass bool, debug bool) map[string]int
• func PrintArrayToSTDOUT(freq map[string]int, verbose bool)
• func PrintStatsToSTDOUT(freq map[string]int, verbose bool, max int)
• func RemoveLengthRange(freq map[string]int, start int, end int) map[string]int
• func RemoveMinimumFrequency(freq map[string]int, min int) map[string]int
• func RetainRemove(textMap map[string]int, retainMap map[string]int, removeMap map[string]int) (map[string]int, error)
• func SaveArrayToJSON(path string, freq map[string]int) error
• func StatClassifyToken(s string) []string

Functions

func ASCIIEscapeUnicode

func ASCIIEscapeUnicode(str string) string

ASCIIEscapeUnicode will convert a string into an unicode escaped format

Args:

str (string): String to escape

Returns:

escapedRunes (string): Converted runes in string format

func CreateVerboseStats

func CreateVerboseStats(freq map[string]int) string

CreateVerboseStats creates a string of verbose statistics about the frequency map including several statistics about the frequency map.

Args:

freq (map[string]int): A map of item frequencies

Returns:

string: A string of verbose statistics

func DeASCIIEscapeUnicode

func DeASCIIEscapeUnicode(str string) string

DeASCIIEscapeUnicode will convert a string from an unicode escaped format to its string representation. This function will only convert the unicode escaped characters if they are in the format \uXXXX where X is a hexadecimal digit.

Args:

str (string): String to unescape

Returns:

unescapedRunes (string): Converted runes in string format

func DecodeInputMap

func DecodeInputMap(input map[string]int, bypass bool, debug bool) map[string]int

DecodeInputMap will decode a map of URL, HTML, and unicode escaped strings where possible and return a new map of decoded strings.

Args:

input (map[string]int): A map of input strings
bypass (bool): If true, the map is not used for output or filtering
debug (bool): If true, print additional debug information to stderr

Returns:

(map[string]int): A new map of decoded strings

func DecodeString

func DecodeString(s string) (string, string, string)

DecodeString is used to URL and HTML decode a string where possible this will only return the decoded string if it is different from the input string

Args:

s (string): Input string

Returns:

urlDecoded (string): Input string URL decoded
htmlDecoded (string): Input string HTML decoded
escapedDecoded (string): Input string unicode escaped decoded

func DehexMap

func DehexMap(input map[string]int, bypass bool, debug bool) map[string]int

DehexMap will decode a map of hex encoded strings and return a new map of decoded strings where possible

Args:

input (map[string]int): A map of hex encoded strings
bypass (bool): If true, the map is not used for output or filtering
debug (bool): If true, print additional debug information to stderr

Returns:

(map[string]int): A new map of decoded strings

func EncodeInputMap

func EncodeInputMap(input map[string]int, bypass bool, debug bool) map[string]int

EncodeInputMap will encode a map of strings to URL, HTML, and unicode escaped strings where possible and return a new map of encoded strings

Args:

input (map[string]int): A map of input strings
bypass (bool): If true, the map is not used for output or filtering
debug (bool): If true, print additional debug information to stderr

Returns:

(map[string]int): A new map of encoded strings

func EncodeString

func EncodeString(s string) (string, string, string)

EncodeString is used to URL and HTML encode a string where possible this will only return the encoded string if it is different from the input string

Args:

s (string): Input string

Returns:

urlEncoded (string): Input string URL encoded
htmlEncoded (string): Input string HTML encoded
escapedEncoded (string): Input string unicode escaped encoded

func HexEncodeMap

func HexEncodeMap(input map[string]int, bypass bool, debug bool) map[string]int

HexEncodeMap will encode a map of strings to hex encoded strings where possible and return a new map of encoded strings

Args:

input (map[string]int): A map of input strings
bypass (bool): If true, the map is not used for output or filtering
debug (bool): If true, print additional debug information to stderr

Returns:

(map[string]int): A new map of encoded strings

func PrintArrayToSTDOUT

func PrintArrayToSTDOUT(freq map[string]int, verbose bool)

PrintArrayToSTDOUT prints an array of items to stdout

Args:

items (map[string]int): A map of item frequencies
verbose (bool): If true, the count is printed

Returns:

None

func PrintStatsToSTDOUT

func PrintStatsToSTDOUT(freq map[string]int, verbose bool, max int)

PrintStatsToSTDOUT prints statistics about the frequency map to stdout including several statistics about the frequency map. If verbose is true, additional information is printed and increased number of items are printed. Items are printed in graph format with a # for each unit of item.

Args:

freq (map[string]int): A map of item frequencies
verbose (bool): If true, additional information is printed
max (int): The maximum number of items to print

Returns:

None

func RemoveLengthRange

func RemoveLengthRange(freq map[string]int, start int, end int) map[string]int

RemoveLengthRange removes items from a map that are outside of a length range or not equal to the start of the end if no end is provided. The length of the range is inclusive.

Args:

freq (map[string]int): A map of item frequencies
start (int): The start of the length range
end (int): The end of the length range

Returns:

(map[string]int): A new map of item frequencies within the length range

func RemoveMinimumFrequency

func RemoveMinimumFrequency(freq map[string]int, min int) map[string]int

RemoveMinimumFrequency removes items from a map that are below a minimum frequency threshold and returns a new map

Args:

freq (map[string]int): A map of item frequencies
min (int): The minimum frequency threshold

Returns:

(map[string]int): A new map of item frequencies above the minimum threshold

func RetainRemove

func RetainRemove(textMap map[string]int, retainMap map[string]int, removeMap map[string]int) (map[string]int, error)

RetainRemove compares a string against a list of words to retain and remove

Args:

text (map[string]int): The text to process
retainMap (map[string]int): A map of words to retain
removeMap (map[string]int): A map of words to remove

Returns:

(map[string]int): A new map of words that have been retained and removed
(error): An error if the text map is empty

func SaveArrayToJSON

func SaveArrayToJSON(path string, freq map[string]int) error

SaveArrayToJSON saves an array of items to a JSON file at the specified path with a generated filename. The filename is generated with the format "ptt-<timestamp>.json" where the timestamp is the current time in RFC3339 format.

Args:

path (string): The path to save the JSON file
items (map[string]int): A map of item frequencies

Returns:

error: An error if the file cannot be saved

func StatClassifyToken

func StatClassifyToken(s string) []string

StatClassifyToken classifies a token into a set of categories based on the token's content. "Short" and "long" are relative to ten characters currently.

Args:

s (string): The token to classify

Returns:

[]string: A list of categories that the token belongs to