stm

package
v0.0.0-...-a820dfc Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 10, 2018 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package stm has almost the same specific condition for sitemap xml.

sitemap guidelines: https://support.google.com/webmasters/answer/183668

Number of URLs = 50,000

File size ( uncompressed ) = 50MB

http://godoc.org/github.com/ikeikeikeike/go-sitemap-generator/stm

Index

Constants

View Source 
const (
	// MaxSitemapFiles defines max sitemap links per index file
	MaxSitemapFiles = 50000
	// MaxSitemapLinks defines max links per sitemap
	MaxSitemapLinks = 50000
	// MaxSitemapImages defines max images per url
	MaxSitemapImages = 1000
	// MaxSitemapNews defines max news sitemap per index_file
	MaxSitemapNews = 1000
	// MaxSitemapFilesize defines file size for sitemap.
	MaxSitemapFilesize = 50000000 // bytes
)
View Source 
const (
	// SchemaGeo exists for geo sitemap
	SchemaGeo = "http://www.google.com/geo/schemas/sitemap/1.0"
	// SchemaImage exists for image sitemap
	SchemaImage = "http://www.google.com/schemas/sitemap-image/1.1"
	// SchemaMobile exists for mobile sitemap
	SchemaMobile = "http://www.google.com/schemas/sitemap-mobile/1.0"
	// SchemaNews exists for news sitemap
	SchemaNews = "http://www.google.com/schemas/sitemap-news/0.9"
	// SchemaPagemap exists for pagemap sitemap
	SchemaPagemap = "http://www.google.com/schemas/sitemap-pagemap/1.0"
	// SchemaVideo exists for video sitemap
	SchemaVideo = "http://www.google.com/schemas/sitemap-video/1.1"
)

Variables

View Source 
var (
	// IndexXMLHeader exists for create sitemap xml as a specific sitemap document.
	IndexXMLHeader = []byte(`<?xml version="1.0" encoding="UTF-8"?>
      <sitemapindex
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
        http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd"
      xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
    >`)
	// IndexXMLFooter and IndexXMLHeader will used from user together .
	IndexXMLFooter = []byte("</sitemapindex>")
)
View Source 
var (
	// XMLHeader exists for create sitemap xml as a specific sitemap document.
	XMLHeader = []byte(`<?xml version="1.0" encoding="UTF-8"?>
      <urlset
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
          http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
        xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:image="` + SchemaImage + `"
        xmlns:video="` + SchemaVideo + `"
        xmlns:geo="` + SchemaGeo + `"
        xmlns:news="` + SchemaNews + `"
        xmlns:mobile="` + SchemaMobile + `"
        xmlns:pagemap="` + SchemaPagemap + `"
        xmlns:xhtml="http://www.w3.org/1999/xhtml"
    >`)
	// XMLFooter and XMLHeader will used from user together .
	XMLFooter = []byte("</urlset>")
)
View Source 
var GzipPtn = regexp.MustCompile(".gz$")

GzipPtn determines gzip file.

Functions

func MergeMap 

func MergeMap(src, dst [][]interface{}) [][]interface{}

MergeMap TODO: Slow function: It wants to change fast function

func PingSearchEngines 

func PingSearchEngines(opts *Options, urls ...string)

PingSearchEngines requests some ping server from it calls Sitemap.PingSearchEngines.

func SetBuilderElementValue 

func SetBuilderElementValue(elm *etree.Element, data [][]interface{}, basekey string) (*etree.Element, bool)

SetBuilderElementValue if it will change to struct from map if the future's author is feeling a bothersome in this function.

func ToLowerString 

func ToLowerString(befores []string) (afters []string)

ToLowerString converts lower strings from including capital or upper strings.

func URLJoin 

func URLJoin(src string, joins ...string) string

URLJoin TODO: Too slowly

Types

type Adapter 

type Adapter interface {
	Write(loc *Location, data []byte)
}

Adapter provides interface for writes some kind of sitemap.

type Attr 

type Attr map[string]string

Attr defines for xml attribute.

type Attrs 

type Attrs []interface{}

Attrs defines for xml attribute.

type BufferPool 

type BufferPool struct {
	sync.Pool
}

BufferPool is

func NewBufferPool 

func NewBufferPool() *BufferPool

NewBufferPool is

func (*BufferPool) Get 

func (bp *BufferPool) Get() *bytes.Buffer

Get is

func (*BufferPool) Put 

func (bp *BufferPool) Put(b *bytes.Buffer)

Put is

type Builder 

type Builder interface {
	XMLContent() []byte
	Content() []byte
	Add(interface{}) BuilderError
	Write()
}

Builder provides interface for adds some kind of url sitemap.

type BuilderError 

type BuilderError interface {
	error
	FullError() bool
}

BuilderError provides interface for it can confirm the error in some difference.

type BuilderFile 

type BuilderFile struct {
	// contains filtered or unexported fields
}

BuilderFile provides implementation for the Builder interface.

func NewBuilderFile 

func NewBuilderFile(opts *Options, loc *Location) *BuilderFile

NewBuilderFile returns the created the BuilderFile's pointer

func (*BuilderFile) Add 

func (b *BuilderFile) Add(url interface{}) BuilderError

Add method joins old bytes with creates bytes by it calls from Sitemap.Add method.

func (*BuilderFile) Content 

func (b *BuilderFile) Content() []byte

Content will return pooled bytes on content attribute.

func (*BuilderFile) Write 

func (b *BuilderFile) Write()

Write will write pooled bytes with header and footer to Location path for output sitemap file.

func (*BuilderFile) XMLContent 

func (b *BuilderFile) XMLContent() []byte

XMLContent will return an XML of the sitemap built

type BuilderIndexfile 

type BuilderIndexfile struct {
	// contains filtered or unexported fields
}

BuilderIndexfile provides implementation for the Builder interface.

func NewBuilderIndexfile 

func NewBuilderIndexfile(opts *Options, loc *Location) *BuilderIndexfile

NewBuilderIndexfile returns the created the BuilderIndexfile's pointer

func (*BuilderIndexfile) Add 

func (b *BuilderIndexfile) Add(link interface{}) BuilderError

Add method joins old bytes with creates bytes by it calls from Sitemap.Finalize method.

func (*BuilderIndexfile) Content 

func (b *BuilderIndexfile) Content() []byte

Content and BuilderFile.Content are almost the same behavior.

func (*BuilderIndexfile) Write 

func (b *BuilderIndexfile) Write()

Write and Builderfile.Write are almost the same behavior.

func (*BuilderIndexfile) XMLContent 

func (b *BuilderIndexfile) XMLContent() []byte

XMLContent and BuilderFile.XMLContent share almost the same behavior.

type FileAdapter 

type FileAdapter struct{}

FileAdapter provides implementation for the Adapter interface.

func NewFileAdapter 

func NewFileAdapter() *FileAdapter

NewFileAdapter returns the created the FileAdapter's pointer

func (*FileAdapter) Write 

func (adp *FileAdapter) Write(loc *Location, data []byte)

Write will create sitemap xml file into the file systems.

type Location 

type Location struct {
	// contains filtered or unexported fields
}

Location provides sitemap's path and filename on file systems and it provides proxy for Adapter interface also.

func NewLocation 

func NewLocation(opts *Options) *Location

NewLocation returns created the Location's pointer

func (*Location) Directory 

func (loc *Location) Directory() string

Directory returns path to combine publicPath and sitemapsPath on file systems. It also indicates where sitemap files are.

func (*Location) Filename 

func (loc *Location) Filename() string

Filename returns sitemap filename.

func (*Location) Filesize 

func (loc *Location) Filesize() int64

Filesize returns file size this struct has.

func (*Location) IsReservedName 

func (loc *Location) IsReservedName() bool

IsReservedName confirms that keeps filename on Location.filename.

func (*Location) IsVerbose 

func (loc *Location) IsVerbose() bool

IsVerbose returns boolean about verbosed summary.

func (*Location) Namer 

func (loc *Location) Namer() *Namer

Namer returns the Namer's pointer that Options struct has.

func (*Location) Path 

func (loc *Location) Path() string

Path returns path to combine publicPath, sitemapsPath and Filename on file systems. It also indicates where sitemap name is.

func (*Location) PathInPublic 

func (loc *Location) PathInPublic() string

PathInPublic returns path to combine sitemapsPath and Filename on website. It also indicates where url file path is.

func (*Location) ReserveName 

func (loc *Location) ReserveName() string

ReserveName returns that sets filename if this struct didn't keep filename and it returns reserved filename if this struct keeps filename also.

func (*Location) Summary 

func (loc *Location) Summary(linkCount int) string

Summary outputs to generated file summary for console.

func (*Location) URL 

func (loc *Location) URL() string

URL returns path to combine SitemapsHost, sitemapsPath and Filename on website with it uses ResolveReference.

func (*Location) Write 

func (loc *Location) Write(data []byte, linkCount int)

Write writes sitemap and index files that used from Adapter interface.

type NOpts 

type NOpts struct {
	// contains filtered or unexported fields
}

NOpts Namer's option

type Namer 

type Namer struct {
	// contains filtered or unexported fields
}

Namer provides sitemap's filename as a file number counter.

func NewNamer 

func NewNamer(opts *NOpts) *Namer

NewNamer returns created the Namer's pointer

func (*Namer) IsStart 

func (n *Namer) IsStart() bool

IsStart confirms that this struct has zero value.

func (*Namer) Next 

func (n *Namer) Next() *Namer

Next is going to go to next index for filename.

func (*Namer) Previous 

func (n *Namer) Previous() *Namer

Previous is going to go to previous index for filename.

func (*Namer) Reset 

func (n *Namer) Reset()

Reset will initialize to zero value on Namer's counter.

func (*Namer) String 

func (n *Namer) String() string

String returns that combines filename base and file extension.

type Options 

type Options struct {
	// contains filtered or unexported fields
}

Options exists for the Sitemap struct.

func NewOptions 

func NewOptions() *Options

NewOptions returns the created the Options's pointer

func (*Options) Clone 

func (opts *Options) Clone() *Options

Clone method returns it copied myself.

func (*Options) IndexLocation 

func (opts *Options) IndexLocation() *Location

IndexLocation returns the Location's pointer with set option to arguments for BuilderIndexfile struct.

func (*Options) Location 

func (opts *Options) Location() *Location

Location returns the Location's pointer with set option to arguments for Builderfile struct.

func (*Options) Namer 

func (opts *Options) Namer() *Namer

Namer returns Namer's pointer cache. If didn't create that yet, It also returns created Namer's pointer.

func (*Options) SetAdapter 

func (opts *Options) SetAdapter(adp Adapter)

SetAdapter sets that arg from Sitemap.SetAdapter method

func (*Options) SetCompress 

func (opts *Options) SetCompress(compress bool)

SetCompress sets that arg from Sitemap.SetCompress method

func (*Options) SetDefaultHost 

func (opts *Options) SetDefaultHost(host string)

SetDefaultHost sets that arg from Sitemap.Finalize method

func (*Options) SetFilename 

func (opts *Options) SetFilename(filename string)

SetFilename sets that arg from Sitemap.SetFilename method

func (*Options) SetPretty 

func (opts *Options) SetPretty(pretty bool)

SetPretty option sets pretty option to Options struct which allows pretty formatting to output files.

func (*Options) SetPublicPath 

func (opts *Options) SetPublicPath(path string)

SetPublicPath sets that arg from Sitemap.SetPublicPath method

func (*Options) SetSitemapsHost 

func (opts *Options) SetSitemapsHost(host string)

SetSitemapsHost sets that arg from Sitemap.SetSitemapsHost method

func (*Options) SetSitemapsPath 

func (opts *Options) SetSitemapsPath(path string)

SetSitemapsPath sets that arg from Sitemap.SetSitemapsPath method.

func (*Options) SetVerbose 

func (opts *Options) SetVerbose(verbose bool)

SetVerbose sets that arg from Sitemap.SetVerbose method

func (*Options) SitemapsHost 

func (opts *Options) SitemapsHost() string

SitemapsHost sets that arg from Sitemap.SitemapsHost method

type Sitemap 

type Sitemap struct {
	// contains filtered or unexported fields
}

Sitemap provides interface for create sitemap xml file and that has convenient interface. And also needs to use first this struct if it wants to use this package.

func NewSitemap 

func NewSitemap(maxProc int) *Sitemap

NewSitemap returns the created the Sitemap's pointer

func (*Sitemap) Add 

func (sm *Sitemap) Add(url interface{}) *Sitemap

Add Should call this after call to Create method on this struct.

func (*Sitemap) Create 

func (sm *Sitemap) Create() *Sitemap

Create method must be that calls first this method in that before call to Add method on this struct.

func (*Sitemap) Finalize 

func (sm *Sitemap) Finalize() *Sitemap

Finalize writes sitemap and index files if it had some specific condition in BuilderFile struct.

func (*Sitemap) PingSearchEngines 

func (sm *Sitemap) PingSearchEngines(urls ...string)

PingSearchEngines requests some ping server. It also has that includes PingSearchEngines function.

func (*Sitemap) SetAdapter 

func (sm *Sitemap) SetAdapter(adp Adapter)

SetAdapter can switch output file storage. We have S3Adapter and FileAdapter (default: FileAdapter)

func (*Sitemap) SetCompress 

func (sm *Sitemap) SetCompress(compress bool)

SetCompress can switch compress for the output file.

func (*Sitemap) SetDefaultHost 

func (sm *Sitemap) SetDefaultHost(host string)

SetDefaultHost is your website's host name

func (*Sitemap) SetFilename 

func (sm *Sitemap) SetFilename(filename string)

SetFilename can apply any name in this method if you wants to change output file name

func (*Sitemap) SetPretty 

func (sm *Sitemap) SetPretty(pretty bool)

SetPretty option allows pretty formating to the output files.

func (*Sitemap) SetPublicPath 

func (sm *Sitemap) SetPublicPath(path string)

SetPublicPath is the directory to write sitemaps to locally

func (*Sitemap) SetSitemapsHost 

func (sm *Sitemap) SetSitemapsHost(host string)

SetSitemapsHost is the remote host where your sitemaps will be hosted

func (*Sitemap) SetSitemapsPath 

func (sm *Sitemap) SetSitemapsPath(path string)

SetSitemapsPath sets this to a directory/path if you don't want to upload to the root of your `SitemapsHost`

func (*Sitemap) SetVerbose 

func (sm *Sitemap) SetVerbose(verbose bool)

SetVerbose can switch verbose output to console.

func (*Sitemap) XMLContent 

func (sm *Sitemap) XMLContent() []byte

XMLContent returns the XML content of the sitemap

type SitemapURL 

type SitemapURL interface {
	XML() []byte
}

SitemapURL provides generated xml interface.

func NewSitemapIndexURL 

func NewSitemapIndexURL(opts *Options, url URL) SitemapURL

NewSitemapIndexURL and NewSitemapURL are almost the same behavior.

func NewSitemapURL 

func NewSitemapURL(opts *Options, url URL) (SitemapURL, error)

NewSitemapURL returns the created the SitemapURL's pointer and it validates URL types error.

type URL 

type URL [][]interface{}

URL User should use this typedef in main func.

func (*URL) BungURLJoinBy 

func (u *URL) BungURLJoinBy(key string, joins ...string)

BungURLJoinBy that's convenient. Though, this is Bung method.

func (URL) URLJoinBy 

func (u URL) URLJoinBy(key string, joins ...string) URL

URLJoinBy that's convenient.

type URLModel 

type URLModel struct {
	Priority   float64                `valid:"float,length(0.0|1.0)"`
	Changefreq string                 `valid:"alpha(always|hourly|daily|weekly|monthly|yearly|never)"`
	Lastmod    time.Time              `valid:"-"`
	Expires    time.Time              `valid:"-"`
	Host       string                 `valid:"ipv4"`
	Loc        string                 `valid:"url"`
	Image      string                 `valid:"url"`
	Video      string                 `valid:"url"`
	Tag        string                 `valid:""`
	Geo        string                 `valid:""`
	News       string                 `valid:"-"`
	Mobile     bool                   `valid:"-"`
	Alternate  string                 `valid:"-"`
	Alternates map[string]interface{} `valid:"-"`
	Pagemap    map[string]interface{} `valid:"-"`
}

URLModel is specific sample model for valuedate. http://www.sitemaps.org/protocol.html https://support.google.com/webmasters/answer/178636

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.