scraper

Documentation

Overview

Package scraper provides a function to Scrape UGC info from a JSON file.

A couple of variables can be set to make it function properly. The results are saved in a JSON file or XLSX file according according to the variable resultFormat.

Index

• Constants
• func Scrape(scrapedJSONFile string) error
• func ScrapeUnscraped(filename string) error
• func Search(hashtags string) error
• func SetFromTo(f, t int)
• func SetHeadless(h bool)
• func SetLimit(l uint)
• func SetRecentVideosNum(r uint)
• func SetResultFormat(f string)
• func SetVerbose(v bool)

Constants

const (
	TIKTOK        = "https://www.tiktok.com"
	TIKTOK_SEARCH = TIKTOK + "/search?q="
)

Functions

func Scrape

func Scrape(scrapedJSONFile string) error

Scrape scrapes UGC info according to a JSON file providing their unique IDs.

It will supposingly save results anyway whether the process has finished successfully or not.

func ScrapeUnscraped

func ScrapeUnscraped(filename string) error

TODO comments

func Search

func Search(hashtags string) error

func SetFromTo

func SetFromTo(f, t int)

func SetHeadless

func SetHeadless(h bool)

func SetLimit

func SetLimit(l uint)

func SetRecentVideosNum

func SetRecentVideosNum(r uint)

func SetResultFormat

func SetResultFormat(f string)

func SetVerbose

func SetVerbose(v bool)