sanitize

Documentation

Index

• Constants
• func CleanCSVFile(dirtyPath string, cleanPath string)
• func CleanJSONZip(dirtyZipPath string, cleanZipPath string)
• func CleanTIFSHAFiles(dirtyPaths []string, dirtyBaseDir string, cleanBaseDir string)
• func Initialize(seed []byte)
• func IsTIFSHAFilename(filename string) bool

Constants

const (
	CSV_NUM_HEADER_ROWS = 4
	CSV_DATA_ROWS_START = CSV_NUM_HEADER_ROWS

	CSV_CVR_NUM_INDEX      = 0
	CSV_TABULATOR_ID_INDEX = 1
	CSV_BATCH_ID_INDEX     = 2
	CSV_RECORD_ID_INDEX    = 3
)

const (
	PATH_TABULATOR_ID_INDEX = 0
	PATH_BATCH_ID_INDEX     = 1
	PATH_RECORD_ID_INDEX    = 2
)

const (
	ONE_MILLION = 1000000
)

Functions

func CleanCSVFile

func CleanCSVFile(dirtyPath string, cleanPath string)

Read the dirtyPath CSV in either standard CSV or odd-ball CSV (="0" style fields) format and then:

• Sanitize the record ID column
• Re-order rows based on the new record ID
• Tabulator and batch IDs are maintaine
• Re-number the CVR Numbers of the newly-ordered rows
• Write to cleanPath with the same format that was read (standard or odd-ball)

func CleanJSONZip

func CleanJSONZip(dirtyZipPath string, cleanZipPath string)

func CleanTIFSHAFiles

func CleanTIFSHAFiles(dirtyPaths []string, dirtyBaseDir string, cleanBaseDir string)

func Initialize

func Initialize(seed []byte)

func IsTIFSHAFilename

func IsTIFSHAFilename(filename string) bool