util

Documentation

Index

• func Dedup(original []string) []string
• func ExtractHostname(rawUrl string) (string, error)
• func FilterNewLines(s string) string
• func FilterSubdomains(hostname string) (*regexp.Regexp, error)
• func FixURL(base *url.URL, path string) string
• func GetExtension(rawUrl string) string
• func IsAbsoluteURL(rawUrl string) bool
• func IsScriptOrJSON(link string) bool
• func IsSwaggerSchema(resp *colly.Response) bool
• func StripQueryParams(u *url.URL) string

Examples

• GetExtension

Functions

func Dedup

func Dedup(original []string) []string

字符串切片去重

func ExtractHostname

func ExtractHostname(rawUrl string) (string, error)

ExtractHostname() extracts the hostname from a URL

func FilterNewLines

func FilterNewLines(s string) string

func FilterSubdomains

func FilterSubdomains(hostname string) (*regexp.Regexp, error)

FIXME: 目前要求输入的 hostname 必须为二级子域名 比如，输入 www.example.com 可以匹配 123.example.com 和 123.xxx.example.com 等下级子域名 但输入 123.xxx.example.com 或 example.com 都无法匹配 www.example.com

func FixURL

func FixURL(base *url.URL, path string) string

func GetExtension

func GetExtension(rawUrl string) string

Example

fmt.Println(GetExtension("http://www.example.com/path/test.jsp"))
fmt.Println(GetExtension("http://www.example.com/path/test.php"))

Output:

.jsp
.php

func IsAbsoluteURL

func IsAbsoluteURL(rawUrl string) bool

func IsScriptOrJSON

func IsScriptOrJSON(link string) bool

func IsSwaggerSchema

func IsSwaggerSchema(resp *colly.Response) bool

func StripQueryParams

func StripQueryParams(u *url.URL) string