Documentation ¶
Overview ¶
loadfavicon module includes a package and a command tool to downloads favicons of a given website.
Index ¶
- func DownloadAll(client *http.Client, websiteURL string, toDir string, onlymissing bool) (n int, err error)
- func DownloadOne(client *http.Client, websiteURL string, toDir string, onlymissing bool) (iconfilename string, err error)
- func Slugify(prefix1 string, prefix2 string, filename string, ext string) string
- func ToPixels(size string) int64
- type Favicon
- func Download(client *http.Client, websiteURL string, toDir string, size string, ...) (favicons []Favicon, err error)
- func GetFaviconLinks(client *http.Client, websiteURL string) (favicons []Favicon, errX error)
- func Read(client *http.Client, websiteURL string, sz ...string) (favicons []Favicon, err error)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DownloadAll ¶ added in v2.1.0
func DownloadAll(client *http.Client, websiteURL string, toDir string, onlymissing bool) (n int, err error)
Download downloads favicons related to a website and save them locally to the 'toDir' directory. The directory is created if it does not exist. The written file is suffixed with the icon file name. Download does not overwrite existing file on disk if onlymissing parameter is turned on.
You need to provide an http.Client for example with a timeout like this :
client := &http.Client{Timeout: time.Second * 5}
Returns the number of successfull downloads.
func DownloadOne ¶ added in v2.1.0
func DownloadOne(client *http.Client, websiteURL string, toDir string, onlymissing bool) (iconfilename string, err error)
DownloadOne downloads the favicon of a website and save it locally to the 'toDir' directory. The directory is created if it does not exist. DownloadOne downloads the svg version if it exists otherwize looks for one with the highest resolution. The written file is not suffixed with the icon file name. DownloadOne does not overwrite existing file on disk if onlymissing parameter is turned on.
You need to provide an http.Client for example with a timeout like this :
client := &http.Client{Timeout: time.Second * 5}
Returns the filename of the downloaded icon. If none icons are found the returned filename is empty with no error.
func Slugify ¶ added in v2.2.0
Slugify returns the slugified name base on a simple filename with an extension, and a prefix. The returned string is ready to be used as a filename.
The returned string pattern is:
[{prefix1}+][{prefix2}+][{filename}].{ext}
where each part is slugified. Any query and fragment are ignored.
Returns only the file name, not an absolute path.
Types ¶
type Favicon ¶
type Favicon struct { WebsiteURL *url.URL // The absolute URL of the favicon's host website WebIconURL *url.URL // The URL of the favicon's file Image []byte // The loaded raw image Color string // Color specfications if any specified in the <link> node Size string // Size specfications if any specified in the <link> node MimeType string // The mimetype detected when loading the image. }
func Download ¶
func Download(client *http.Client, websiteURL string, toDir string, size string, onlymissing bool, suffix bool) (favicons []Favicon, err error)
Download downloads favicons related to a website and save it locally to the 'toDir' directory. The directory is created if it does not exist. Download downloads the svg version if it exists otherwize looks for one in the given size. If no favicon exists in te given size, DownloadOne downloads the closest favicon with an upper number of pixels.
Parameters:
size = {width}x{height} or maxres // If the size is empty all favicons found are downloaded. If the size is 'maxres' only the favicon with the maximum resolution is downloaded. onlymissing = true // downloads if the file is missing on the disk, do not overwrite. suffix = true // written file name will be suffixed with the icon file name.
You need to provide an http.Client for example with a timeout like this :
client := &http.Client{Timeout: time.Second * 5}
Returns the slice of downloaded Favicons. If none icons were found the returned slice is empty with no error.
Example ¶
verbose.IsOn = false os.RemoveAll("./examples") client := &http.Client{Timeout: time.Second * 5} DownloadOne(client, "https://lolorenzo777.github.io/website4tests-2", "./examples", false) DownloadAll(client, "https://laurent.lourenco.pro", "./examples", false) DownloadOne(client, "https://web.archive.org/", "./examples", false) DownloadOne(client, "https://wikipedia.org", "./examples", false) files, _ := os.ReadDir("./examples") for _, file := range files { fmt.Println(file.Name()) }
Output: laurent-lourenco-pro+16x16+favicon-16x16.png laurent-lourenco-pro+180x180+apple-touch-icon.png laurent-lourenco-pro+32x32+favicon-32x32.png lolorenzo777-github-io+32x32+.png web-archive-org+32x32+.ico wikipedia-org+160x160+.png
func GetFaviconLinks ¶
GetFaviconLinks returns a list of favicons urls for websiteURL.
GetFaviconLinks extracts the list of links declared in the <head> section of the site that may correspond to favicon. favicon.ico is added to the list if the list is empty.
Color and Size favicon properties are filled with data found in the <head> items. The Size and the MimeType will be updated when loading the Image.
You need to provide an http.Client for example with a timeout like this :
client := &http.Client{Timeout: time.Second * 5}
Example (Second) ¶
client := &http.Client{Timeout: time.Second * 5} for _, w := range _gwebsitesOK { fmt.Println("GetFaviconLinks:", w) icons, err := GetFaviconLinks(client, w) if err != nil { fmt.Println(err) } else { printlist(icons) } fmt.Println() }
Output: GetFaviconLinks: https://lolorenzo777.github.io/website4tests-2 https://lolorenzo777.github.io/website4tests-2 -- https://lolorenzo777.github.io/website4tests-2/test-32x32.png GetFaviconLinks: https://laurent.lourenco.pro https://laurent.lourenco.pro -- https://laurent.lourenco.pro/favicon-32x32.png https://laurent.lourenco.pro -- https://laurent.lourenco.pro/favicon-16x16.png https://laurent.lourenco.pro -- https://laurent.lourenco.pro/apple-touch-icon.png GetFaviconLinks: https://go.dev/ https://go.dev/ -- https://go.dev/images/favicon-gopher.png https://go.dev/ -- https://go.dev/images/favicon-gopher-plain.png https://go.dev/ -- https://go.dev/images/favicon-gopher.svg GetFaviconLinks: https://brave.com/ https://brave.com/ -- https://brave.com/static-assets/images/brave-favicon.png https://brave.com/ -- https://brave.com/static-assets/images/cropped-brave_appicon_release-32x32.png https://brave.com/ -- https://brave.com/static-assets/images/cropped-brave_appicon_release-192x192.png https://brave.com/ -- https://brave.com/static-assets/images/cropped-brave_appicon_release-180x180.png GetFaviconLinks: https://github.com/ https://github.com/ -- https://github.githubassets.com/pinned-octocat.svg https://github.com/ -- https://github.githubassets.com/favicons/favicon.svg GetFaviconLinks: https://www.sncf.com/ https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon.ico https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon.png https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/apple-touch-icon.png https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon-32x32.png https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon-16x16.png GetFaviconLinks: https://protonmail.com/ https://protonmail.com/ -- https://proton.me/favicons/apple-touch-icon.png https://protonmail.com/ -- https://proton.me/favicons/favicon-32x32.png https://protonmail.com/ -- https://proton.me/favicons/favicon-16x16.png https://protonmail.com/ -- https://proton.me/favicons/safari-pinned-tab.svg https://protonmail.com/ -- https://proton.me/favicons/favicon.ico GetFaviconLinks: https://getbootstrap.com/ https://getbootstrap.com/ -- https://getbootstrap.com/docs/5.3/assets/img/favicons/apple-touch-icon.png https://getbootstrap.com/ -- https://getbootstrap.com/docs/5.3/assets/img/favicons/favicon-32x32.png https://getbootstrap.com/ -- https://getbootstrap.com/docs/5.3/assets/img/favicons/favicon-16x16.png https://getbootstrap.com/ -- https://getbootstrap.com/docs/5.3/assets/img/favicons/safari-pinned-tab.svg https://getbootstrap.com/ -- https://getbootstrap.com/docs/5.3/assets/img/favicons/favicon.ico GetFaviconLinks: https://www.cloudflare.com/ https://www.cloudflare.com/ -- https://www.cloudflare.com/favicon.ico GetFaviconLinks: https://www.docker.com/ https://www.docker.com/ -- https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-32x32.png https://www.docker.com/ -- https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-192x192.png https://www.docker.com/ -- https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-180x180.png
Example (Third) ¶
// verbose.IsOn = true // verbose.IsDebugging = true client := &http.Client{Timeout: time.Second * 5} for _, w := range _gwebsitesTricky { fmt.Println("GetFaviconLinks:", w) icons, err := GetFaviconLinks(client, w) if err != nil { fmt.Println(err) } else { printlist(icons) } fmt.Println() }
Output: GetFaviconLinks: https://bitcoin.org/bitcoin.pdf https://bitcoin.org -- https://bitcoin.org/favicon.png?1687792074 https://bitcoin.org -- https://bitcoin.org/img/icons/logo_ios.png?1687792074 GetFaviconLinks: https://twitter.com/ GetFaviconLinks: Get "/": stopped after 10 redirects GetFaviconLinks: https://mail.proton.me/u/0/inbox/ https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/favicon.ico https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/favicon.782cda472f79b5eed726.svg https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-57x57.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-60x60.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-72x72.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-76x76.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-114x114.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-120x120.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-144x144.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-152x152.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-167x167.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-180x180.png https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/apple-touch-icon-1024x1024.png GetFaviconLinks: https://www.amazon.com/ https://www.amazon.com/ -- https://www.amazon.com/favicon.ico GetFaviconLinks: https://getemoji.com/ https://getemoji.com/ -- https://getemoji.com/ico/favicon.png https://getemoji.com/ -- https://getemoji.com/ico/apple-touch-icon.png GetFaviconLinks: https://www.linkedin.com/in/laurentlourenco GetFaviconLinks: "https://www.linkedin.com/in/laurentlourenco" returned status 999 GetFaviconLinks: https://mail.google.com/mail/u/0/#inbox https://mail.google.com/mail/u/0/#inbox -- https://mail.google.com/favicon.ico
func Read ¶
Read reads favicons of a website and returns them in a slice. The returned slice is sorted from the highest icon resolution to the lowest one, starting with SVG ones if any.
The size parameter must follow this pattern:
size = {width}x{height} or maxres
If the size is empty all favicons found are read. If the size is 'maxres' only the favicon with the maximum resolution is downloaded.
You need to provide an http.Client for example with a timeout like this :
client := &http.Client{Timeout: time.Second * 5}
The returned slice contains valid images only. (see Favicon.ReadImage)
Example ¶
client := &http.Client{Timeout: time.Second * 5} for _, w := range _gwebsitesOK { fmt.Println("Read:", w) icons, err := Read(client, w) if err != nil { fmt.Println(err) } else { printlist(icons) } fmt.Println() }
Output: Read: https://lolorenzo777.github.io/website4tests-2 https://lolorenzo777.github.io/website4tests-2 -- https://lolorenzo777.github.io/website4tests-2/test-32x32.png Read: https://laurent.lourenco.pro https://laurent.lourenco.pro -- https://laurent.lourenco.pro/apple-touch-icon.png https://laurent.lourenco.pro -- https://laurent.lourenco.pro/favicon-32x32.png https://laurent.lourenco.pro -- https://laurent.lourenco.pro/favicon-16x16.png Read: https://go.dev/ https://go.dev/ -- https://go.dev/images/favicon-gopher.svg Read: https://brave.com/ https://brave.com/ -- https://brave.com/static-assets/images/cropped-brave_appicon_release-192x192.png https://brave.com/ -- https://brave.com/static-assets/images/cropped-brave_appicon_release-180x180.png https://brave.com/ -- https://brave.com/static-assets/images/brave-favicon.png https://brave.com/ -- https://brave.com/static-assets/images/cropped-brave_appicon_release-32x32.png Read: https://github.com/ https://github.com/ -- https://github.githubassets.com/pinned-octocat.svg Read: https://www.sncf.com/ https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon.png https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/apple-touch-icon.png https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon.ico https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon-32x32.png https://www.sncf.com/ -- https://www.sncf.com/themes/sncfcom/img/favicon-16x16.png Read: https://protonmail.com/ https://protonmail.com/ -- https://proton.me/favicons/safari-pinned-tab.svg Read: https://getbootstrap.com/ https://getbootstrap.com/ -- https://getbootstrap.com/docs/5.3/assets/img/favicons/safari-pinned-tab.svg Read: https://www.cloudflare.com/ https://www.cloudflare.com/ -- https://www.cloudflare.com/favicon.ico Read: https://www.docker.com/ https://www.docker.com/ -- https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-192x192.png https://www.docker.com/ -- https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-180x180.png https://www.docker.com/ -- https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-32x32.png
Example (Second) ¶
// verbose.IsOn = true // verbose.IsDebugging = true client := &http.Client{Timeout: time.Second * 5} for _, w := range _gwebsitesTricky { fmt.Println("Read:", w) icons, err := Read(client, w) if err != nil { fmt.Println(err) } else { printlist(icons) } fmt.Println() }
Output: Read: https://bitcoin.org/bitcoin.pdf https://bitcoin.org -- https://bitcoin.org/img/icons/logo_ios.png?1687792074 https://bitcoin.org -- https://bitcoin.org/favicon.png?1687792074 Read: https://twitter.com/ https://twitter.com/ -- https://twitter.com/favicon.ico Read: https://mail.proton.me/u/0/inbox/ https://mail.proton.me/u/0/inbox/ -- https://mail.proton.me/assets/favicon.782cda472f79b5eed726.svg Read: https://www.amazon.com/ https://www.amazon.com/ -- https://www.amazon.com/favicon.ico Read: https://getemoji.com/ https://getemoji.com/ -- https://getemoji.com/favicon.ico Read: https://www.linkedin.com/in/laurentlourenco https://www.linkedin.com/in/laurentlourenco -- https://www.linkedin.com/favicon.ico Read: https://mail.google.com/mail/u/0/#inbox https://mail.google.com/mail/u/0/#inbox -- https://mail.google.com/favicon.ico
func (Favicon) IsFaviconIco ¶ added in v2.2.1
IsFaviconIco true if the icon file name is "favicon.ico"
func (Favicon) Pixels ¶
Pixels returns the total number of pixels. Returns MaxInt for SVG. Returns -1 if unable to get the image size or if the image is not loaded yet.
func (Favicon) Slugify ¶ added in v2.2.0
Slugify returns a slugified file name based on the WebsiteURL, the Size and WebIconURL.
The DiskFileName pattern is:
{WebsiteURL.Host}+{size}[+{base WebIconURL}].{ext}
where each part is slugified. Any query and fragment are ignored.
Returns only the file name, not an absolute path.
Example ¶
type Test struct { web string icon string sz string } data := []Test{ {web: "https://www.dummy.com", icon: "favicon.ico", }, {web: "https://www.dummy.com", icon: "favicon.ico", sz: "16X16", }, {web: "https://www.dummy.com/website/", icon: "https://cdn.com/dummy/favicon.png", sz: "64x64", }, {web: "https://www.dummy.com/website/", icon: "/assets/favicon.png", sz: "128X128", }, {web: "https://www.dummy.com/website/", icon: "./assets/favicon.png?1234567", sz: "256x256", }, } for _, d := range data { for f := 0; f <= 1; f++ { icon := new(Favicon) icon.WebsiteURL, _ = url.Parse(d.web) icon.WebIconURL, _ = url.Parse(d.icon) icon.Size = d.sz fmt.Println(icon.Slugify(f == 1)) } }
Output: www-dummy-com+.ico www-dummy-com+favicon.ico www-dummy-com+16x16+.ico www-dummy-com+16x16+favicon.ico www-dummy-com+64x64+.png www-dummy-com+64x64+favicon.png www-dummy-com+128x128+.png www-dummy-com+128x128+favicon.png www-dummy-com+256x256+.png www-dummy-com+256x256+favicon.png