fileutil

Documentation

Index

• func CountLines(path string) (int, error)
• func EnsureDirExists(dirPath string) (string, error)
• func EnsureDirPathExists(dirPath string) error
• func FileExists(path string) bool
• func GetHourlyDirs(prefixDir string, start, end time.Time) []string
• func Must(s string, err error) string
• func MustOpen(name string) *os.File
• func MustReaddir(dir *os.File) []os.FileInfo
• func ReadAllFilesRecursive(dir string, excludeDirs []string) (map[string][]byte, error)
• func ReadAndSha1File(path string) (contents, hash string, err error)
• func TwoLevelRadixPath(pathSegments ...string) string

Functions

func CountLines

func CountLines(path string) (int, error)

CountLines opens the given path and counts the number of lines in the file. Returns -1 with a non-nil error if an error is encountered.

func EnsureDirExists

func EnsureDirExists(dirPath string) (string, error)

EnsureDirExists checks whether the given path to a directory exits and creates it if necessary. Returns the absolute path that corresponds to the input path and an error indicating a problem.

func EnsureDirPathExists

func EnsureDirPathExists(dirPath string) error

EnsureDirPathExists checks whether the directories of the given file path exist and creates them if necessary. Returns an error if there was a problem creating the path.

func FileExists

func FileExists(path string) bool

FileExists returns true if the given path exists and false otherwise. If there is an error it will return false and log the error message.

func GetHourlyDirs

func GetHourlyDirs(prefixDir string, start, end time.Time) []string

GetHourlyDirs generates paths with the given prefix for all the paths of the form 'prefix/<year>/<month>/<day>/<hour>'. All path segments are padded with zeros where necessary, e.g. prefix/2018/01/02/09. All hours are from 0-23. This is intended to enumerate the directories that should be polled when ingesting data. startTS and endTS are seconds since the Unix epoch. Both time stamps are inclusive. The generated directories are based on UTC being the locale. If the prefix is empty string, there will be no slash before the year.

func Must

func Must(s string, err error) string

Must checks whether err in the provided pair (s, err) is nil. If so it returns s otherwise it cause the program to stop with the error message.

func MustOpen

func MustOpen(name string) *os.File

MustOpen opens the file of the given name, returning an *os.File on success, or causing the program to stop with the error message.

func MustReaddir

func MustReaddir(dir *os.File) []os.FileInfo

MustReaddir returns a slice of os.FileInfo for every file in the given dir. This is equivalent to calling dir.Readdir(-1), except this call will stop the program on an error

func ReadAllFilesRecursive

func ReadAllFilesRecursive(dir string, excludeDirs []string) (map[string][]byte, error)

ReadAllFilesRecursive recursively reads all files in the given dir and returns a map of filename to contents.

func ReadAndSha1File

func ReadAndSha1File(path string) (contents, hash string, err error)

ReadAndSha1File opens a file, reads the contents, hashes them using the sha1 hashing algorithm and returns the contents and hash. If the file doesn't exist, the err will be nil and both contents and hash will be empty string.

func TwoLevelRadixPath

func TwoLevelRadixPath(pathSegments ...string) string

TwoLevelRadixPath expands a path (defined by one or more path segments) by adding two additional directories based on the filename in the last segment. i.e. TwoLevelRadixPath("/some", "path", "to", "abcdefgh.txt") will return "/some/path/to/ab/cd/abcdefgh.txt". If the filename does not have at least four characters before a period the return values is equivalent to filepath.Join(pathSegments...).