dups

Documentation

Index

• Constants
• func CleanPath(path string) string
• func CollectHashes(fileGroups map[int][]FileInfo, singleThread bool, algorithm string, flat bool, ...) map[string][]FileInfo
• func GetAlgorithm(al string) string
• func GetDuplicates(hashes map[string][]FileInfo) ([][]FileInfo, int, int)
• func GetFileHash(path, algorithm string) (string, error)
• func GroupFiles(files []FileInfo, minSize int) (map[int][]FileInfo, int)
• func RemoveDuplicates(fileSets [][]FileInfo) (int, int, error)
• type FileInfo
  • func GetFiles(root string, full bool) ([]FileInfo, error)

Constants

const (
	// XXHash represents XXHash algorithm
	XXHash = "xxhash"
	// MD5 represents XXHash algorithm
	MD5 = "md5"
	// SHA256 represents XXHash algorithm
	SHA256 = "sha256"
)

Functions

func CleanPath

func CleanPath(path string) string

CleanPath replaces \ with / in a path

func CollectHashes

func CollectHashes(fileGroups map[int][]FileInfo, singleThread bool, algorithm string, flat bool, fileCount int) map[string][]FileInfo

CollectHashes returns hashes for the given group files if there is more than one file with the same size A hash will be the key and a list of FileInfo for files that share the hash as the value "singleThread=false" will force all the function to use one thread only minSize is the minimum file size to scan "flat=true" will tell the function not to print out any data other than the path to duplicate files algorithm is the algorithm to calculate the hash with

func GetAlgorithm

func GetAlgorithm(al string) string

GetAlgorithm matches the given string to one of the supported algorithms Returns md5 if a match wasn't found

func GetDuplicates

func GetDuplicates(hashes map[string][]FileInfo) ([][]FileInfo, int, int)

GetDuplicates scans the given map of hashes and finds the one with duplicates It will return a slice containing slices with each slice containing paths to duplicate files It will also returns the total of duplicate files and the total of files that have duplicates

func GetFileHash

func GetFileHash(path, algorithm string) (string, error)

GetFileHash returns given file hash using the provided algorithm Default: md5

func GroupFiles

func GroupFiles(files []FileInfo, minSize int) (map[int][]FileInfo, int)

GroupFiles groups files based on their file size This will help avoid unnecessary hash calculations since files with different file sizes can't be duplicates

func RemoveDuplicates

func RemoveDuplicates(fileSets [][]FileInfo) (int, int, error)

RemoveDuplicates removes duplicates It will keep the first file in a duplicate set and removes any other files in the set It will return the sum of deleted file sizes and total number of deleted files

Types

type FileInfo

type FileInfo struct {
	Path string
	Info os.FileInfo
}

FileInfo represents a file containing os.FileInfo and file path

func GetFiles

func GetFiles(root string, full bool) ([]FileInfo, error)

GetFiles finds and returns all the files in the given path It will also returns any file in sub-directories if "full=true"