sitemap

README

sitemap

Mark routes to put into the sitemap as they get registered.

This packages builds on github.com/gorilla/mux to make sitemap creation easy.

License

polyglottis/sitemap is licensed under the MIT License.

gorilla/mux is licensed under the New BSD License.

Documentation

Overview

Package sitemap allows dynamic generation of a sitemap. This package allows to:

1. Register routes when binding handlers (embedding a github.com/gorilla/mux.Router),
2. Create a sitemap on request (lazy-creation),
3. Update the sitemap regularly and thread-safely.

The sitemap is stored (=cached) in XML format on disk, and served directly from there.

Example of use:

1. Create the router:

r := sitemap.NewRouter(mux.NewRouter(), "http://example.com", "local/path/to/sitemaps/cache")

2. Static route handler:

r.Register("/my/static/route").Handler(handler)

... or a secret route (i.e. not appearing in the sitemap):

r.HandleFunc("/my/secret/route", f)

3. Parameterized route:

documents := []struct {
  Category string
  Id       string
}{{
  Category: "book",
  Id:       "AAA",
}, {
  Category: "book",
  Id:       "BBB",
}, {
  Category: "html",
  Id:       "WWW",
}}

r.RegisterParam("/documents/{category}/{id:[A-Z]+}", func(cb func(...string) error) error {
  for _, doc := range documents {
    err := cb("category", doc.Category, "id", doc.Id)
    if err != nil {
      return err
    }
  }
  return nil
}).Handler(h)

4. Handle sitemap requests:

r.HandleSitemaps()
http.Handle("/", r)

So that an http GET on (r.Options.ServerPath + "sitemapindex.xml") returns:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd">
  <sitemap>
    <loc>http://example.com/sitemap_1.xml</loc>
  </sitemap>
</sitemapindex>

and (r.Options.ServerPath + "sitemap_1.xml") returns:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
  <url>
    <loc>http://example.com/my/static/route</loc>
    <priority>0.5</priority>
  </url>
  <url>
    <loc>http://example.com/documents/book/AAA</loc>
    <priority>0.5</priority>
  </url>
  <url>
    <loc>http://example.com/documents/book/BBB</loc>
    <priority>0.5</priority>
  </url>
  <url>
    <loc>http://example.com/documents/html/WWW</loc>
    <priority>0.5</priority>
  </url>
</urlset>

Index

• Constants
• Variables
• type Buffer
  • func NewBuffer(domain, path string) *Buffer
  • func (b *Buffer) AddEntry(e *Entry) error
  • func (b *Buffer) Flush() error
• type ChangeFrequency
• type Entry
• type FileReference
• type Options
• type Router
  • func NewRouter(router *mux.Router, domain, localPath string) *Router
  • func (r *Router) GenerateSitemaps() ([]string, error)
  • func (r *Router) HandleSitemaps() http.Handler
  • func (r *Router) Register(pattern string) *mux.Route
  • func (r *Router) RegisterParam(pattern string, enum VariableEnumerator) *mux.Route
  • func (r *Router) SitemapHandler() http.Handler
• type Schema
• type Sitemap
  • func NewSitemap() *Sitemap
  • func (s *Sitemap) IsEmpty() bool
  • func (s *Sitemap) IsFull() bool
  • func (s *Sitemap) WriteToFile(path string) error
• type SitemapIndex
  • func NewSitemapIndex(sitemapUrls []string) *SitemapIndex
  • func (s *SitemapIndex) WriteToFile(path string) error
• type VariableEnumerator

Constants

const (
	Always  ChangeFrequency = "always"
	Hourly                  = "hourly"
	Daily                   = "daily"
	Weekly                  = "weekly"
	Monthly                 = "monthly"
	Yearly                  = "yearly"
	Never                   = "never"
)

Variables

var DefaultOptions = &Options{
	ServerPath:      "/",
	DefaultPriority: 0.5,
}

DefaultOptions is the default options used when calling NewRouter().

var SitemapIndexSchema = &Schema{
	Xmlns:             "http://www.sitemaps.org/schemas/sitemap/0.9",
	XmlnsXsi:          "http://www.w3.org/2001/XMLSchema-instance",
	XsiSchemaLocation: "http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd",
}

SitemapIndexSchema is the XML schema used for sitemap indexes.

var SitemapSchema = &Schema{
	Xmlns:             "http://www.sitemaps.org/schemas/sitemap/0.9",
	XmlnsXsi:          "http://www.w3.org/2001/XMLSchema-instance",
	XsiSchemaLocation: "http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd",
}

SitemapSchema is the XML schema used for sitemaps.

Types

type Buffer

type Buffer struct {
	Locations []string // Relative path of serialized sitemaps.
	// contains filtered or unexported fields
}

Buffer is a sitemap buffer.

When the current sitemap is full, it is offloaded to disk, and a new empty sitemap is created.

func NewBuffer

func NewBuffer(domain, path string) *Buffer

NewBuffer creates a new buffer for sitemaps on the given domain. The path variable is the location for serialization on Flush().

func (*Buffer) AddEntry

func (b *Buffer) AddEntry(e *Entry) error

AddEntry adds an entry to the buffer. If the sitemap buffer is full, it calls Flush() before inserting the entry to a new Sitemap.

func (*Buffer) Flush

func (b *Buffer) Flush() error

Flush writes the content of the buffer to a sitemap file and adds the file to the list of locations. This occurs only if the buffer is non-empty. Calling Flush on an empty buffer is a no-op.

type ChangeFrequency

type ChangeFrequency string

ChangeFrequency is an optional attribute for sitemap entries.

type Entry

type Entry struct {
	*FileReference
	ChangeFrequency ChangeFrequency `xml:"changefreq,omitempty"` // optional
	Priority        *float64        `xml:"priority,omitempty"`   // optional
}

Entry is a sitemap entry (a url block in the XML file).

type FileReference

type FileReference struct {
	Location         string     `xml:"loc"`
	LastModification *time.Time `xml:"lastmod,omitempty"` // optional
}

FileReference is a reference to a file (given by full URL) and the last modification.

type Options

type Options struct {
	CachePath       string  // path of a directory, to store sitemaps on disk
	ServerPath      string  // server path for sitemaps
	DefaultPriority float64 // default priority for sitemap entries
	Domain          string  // domain for entries in the sitemap (multiple domains are not supported)
}

Options is used by Router.

type Router

type Router struct {
	*mux.Router

	Options *Options
	// contains filtered or unexported fields
}

Router has an embedded github.com/gorilla/mux.Router. It retains all functionalities of the router (unchanged) and has a few extra methods to register the routes which should belong to the sitemap.

See Register(), RegisterParam() and HandleSitemaps().

func NewRouter

func NewRouter(router *mux.Router, domain, localPath string) *Router

NewRouter wraps router into a new Router, ready to register sitemap urls for the given domain.

localPath is the path where to store the sitemaps when created.

Change the routers options if you want more control on the sitemap creation:

r := NewRouter(router, "example.com", "cache/sitemaps")
r.Options.DefaultPriority = 1
r.ServerPath = "/sitemaps/" // don't forget the trailing slash!

func (*Router) GenerateSitemaps

func (r *Router) GenerateSitemaps() ([]string, error)

GenerateSitemaps creates sitemapindex.xml and as many sitemaps as needed. Since there are size restrictions on a sitemap, there may be more than one. In this case they are named sitemap_1.xml, sitemap_2.xml, and so on.

All files created is returned (paths relative to r.Options.CachePath).

It is safe to call GenerateSitemaps() even when they are served due to a call to HandleSitemaps(). A read-write lock takes care of queueing requests until the sitemaps are generated.

func (*Router) HandleSitemaps

func (r *Router) HandleSitemaps() http.Handler

HandleSitemaps register routes to serve the sitemap files on the router. The http handler is returned.

All routes registered are:

r.Options.ServerPath + "sitemapindex.xml"
r.Options.ServerPath + "sitemap_%d.xml" // where %d is a replaced by a positive integer.

func (*Router) Register

func (r *Router) Register(pattern string) *mux.Route

Register creates a static route (no variables in the path) and adds it to the sitemap.

func (*Router) RegisterParam

func (r *Router) RegisterParam(pattern string, enum VariableEnumerator) *mux.Route

RegisterParam creates a route with parameters (=variables) in the path. Each time the sitemap is (re-)created, enum is called to get the list of allowed variable values.

See the package's main documentation for an example.

func (*Router) SitemapHandler

func (r *Router) SitemapHandler() http.Handler

SitemapHandler creates and returns a new http.Handler for sitemaps. It expects to serve r,Options.ServerPath + `{file:sitemap(index|_\d+)\.xml}`.

type Schema

type Schema struct {
	Xmlns             string `xml:"xmlns,attr"`
	XmlnsXsi          string `xml:"xmlns:xsi,attr"`
	XsiSchemaLocation string `xml:"xsi:schemaLocation,attr"`
}

Schema represents an XML schema.

type Sitemap

type Sitemap struct {
	XMLName xml.Name `xml:"urlset"`
	*Schema
	Entries []*Entry `xml:"url"`
}

Sitemap is a sitemap, with xml-encoding attributes.

Call NewSitemap() to get a new sitemap with the correct XML schema, ready to get encoded.

func NewSitemap

func NewSitemap() *Sitemap

NewSitemap creates an empty sitemap with the schema set as SitemapSchema.

func (*Sitemap) IsEmpty

func (s *Sitemap) IsEmpty() bool

IsEmpty returns true if the sitemap is empty or nil.

func (*Sitemap) IsFull

func (s *Sitemap) IsFull() bool

IsFull retruns true if the sitemap has reached the maximum number of entries allowed.

func (*Sitemap) WriteToFile

func (s *Sitemap) WriteToFile(path string) error

WriteToFile encodes the sitemap in XML format into path.

type SitemapIndex

type SitemapIndex struct {
	XMLName xml.Name `xml:"sitemapindex"`
	*Schema
	SitemapRefs []*FileReference `xml:"sitemap"`
}

SitemapIndex is a sitemap index with xml-encoding attributes.

func NewSitemapIndex

func NewSitemapIndex(sitemapUrls []string) *SitemapIndex

NewSitemapIndex creates a sitemap index with the default schema and all sitemap urls given.

func (*SitemapIndex) WriteToFile

func (s *SitemapIndex) WriteToFile(path string) error

WriteToFile writes the sitemap index in XML into path.

type VariableEnumerator

type VariableEnumerator func(callback func(pairs ...string) error) error

VariableEnumerator calls the callback as many times as there are routes allowed.

The arguments passed to the callback should be as needed by github.com/gorilla/mux.Route.URL().

See the package's main documentation for an example.