xurls

README

xurls

Extract urls from text using regular expressions.

go get github.com/mvdan/xurls

import "github.com/mvdan/xurls"

func main() {
	xurls.Relaxed.FindString("Do gophers live in golang.org?")
	// "golang.org"
	xurls.Relaxed.FindAllString("foo.com is http://foo.com/.", -1)
	// []string{"foo.com", "http://foo.com/"}
	xurls.Strict.FindAllString("foo.com is http://foo.com/.", -1)
	// []string{"http://foo.com/"}
}

cmd/xurls

Reads text and prints one url per line.

go get github.com/mvdan/xurls/cmd/xurls

$ echo "Do gophers live in http://golang.org?" | xurls
http://golang.org

Documentation

Overview

Package xurls extracts urls from plain text using regular expressions.

Index

• Variables
• func StrictMatchingScheme(exp string) (*regexp.Regexp, error)

Variables

var (
	// Relaxed matches all the urls it can find.
	Relaxed = regexp.MustCompile(relaxed)
	// Strict only matches urls with a scheme to avoid false positives.
	Strict = regexp.MustCompile(strict)
)

var PseudoTLDs = []string{
	`bit`,
	`example`,
	`exit`,
	`gnu`,
	`i2p`,
	`invalid`,
	`local`,
	`localhost`,
	`onion`,
	`test`,
	`zkey`,
}

PseudoTLDs is a sorted list of some widely used unofficial TLDs.

Sources:

• https://en.wikipedia.org/wiki/Pseudo-top-level_domain
• https://en.wikipedia.org/wiki/Category:Pseudo-top-level_domains
• https://tools.ietf.org/html/draft-grothoff-iesg-special-use-p2p-names-00
• https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml

var SchemesNoAuthority = []string{
	`bitcoin`,
	`file`,
	`magnet`,
	`mailto`,
	`sms`,
	`tel`,
	`xmpp`,
}

SchemesNoAuthority is a sorted list of some well-known url schemes that are followed by ":" instead of "://". Since these are more prone to false positives, we limit their matching.

var TLDs = []string{}/* 1544 elements not displayed */

TLDs is a sorted list of all public top-level domains.

Sources:

• https://data.iana.org/TLD/tlds-alpha-by-domain.txt
• https://publicsuffix.org/list/effective_tld_names.dat

Functions

func StrictMatchingScheme

func StrictMatchingScheme(exp string) (*regexp.Regexp, error)

StrictMatchingScheme produces a regexp that matches urls like Strict but whose scheme matches the given regular expression.