robots

Documentation

Index

• func Robots(options *Options) func(r *request.Request)
• type Listing
• type Options

Functions

func Robots

func Robots(options *Options) func(r *request.Request)

Robots returns a handler that generates a robots.txt file.

Types

type Listing

type Listing struct {
	Allow      []string
	Disallow   []string
	UserAgent  string
	CrawlDelay int
}

Item in the robots.txt file.

type Options

type Options struct {
	Rules   []*Listing
	SiteMap string
}

Options for generating the robots.txt file.