download

Documentation

Index

• func DownloadAndUnzip(url, outputDir string) error
• func EnumerateAndRemoveZIPFiles(dir string) error
• func FetchAndParseJSON(url string, filePath string) ([]types.Item, error)
• func ParallelCheckAndRemoveEmptyTXTFiles(dir string) error
• func ParallelMergeURLFiles(dir, outputFile string) error
• type Item

Functions

func DownloadAndUnzip

func DownloadAndUnzip(url, outputDir string) error

DownloadAndUnzip downloads and unzips the file

func EnumerateAndRemoveZIPFiles

func EnumerateAndRemoveZIPFiles(dir string) error

EnumerateAndRemoveZIPFiles enumerates and removes .zip files in the directory

func FetchAndParseJSON

func FetchAndParseJSON(url string, filePath string) ([]types.Item, error)

FetchAndParseJSON fetches and parses JSON data from the specified URL

func ParallelCheckAndRemoveEmptyTXTFiles

func ParallelCheckAndRemoveEmptyTXTFiles(dir string) error

ParallelCheckAndRemoveEmptyTXTFiles checks and removes empty .txt files in parallel

func ParallelMergeURLFiles

func ParallelMergeURLFiles(dir, outputFile string) error

ParallelMergeURLFiles merges URL files in parallel

Types

type Item

type Item struct {
	Name        string    `json:"name"`
	ProgramURL  string    `json:"program_url"`
	URL         string    `json:"URL"`
	Count       int       `json:"count"`
	Change      int       `json:"change"`
	IsNew       bool      `json:"is_new"`
	Platform    string    `json:"platform"`
	Bounty      bool      `json:"bounty"`
	LastUpdated time.Time `json:"last_updated"`
}

Item represents the structure of each object in the JSON array