sitemapparser

README

sitemap-parser

Simple XML sitemap parser in Go. Supports both regular sitemaps and sitemap index files.

Install

go get github.com/choirulanwar/sitemap-parser

Usage

package main

import (
    "fmt"
    "log"
    "github.com/choirulanwar/sitemap-parser"
)

func main() {
    urls, err := sitemapparser.ExtractURLs("https://example.com/sitemap.xml")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Found %d URLs\n", len(urls))
    for _, url := range urls {
        fmt.Println(url)
    }
}

License

MIT

Documentation

Overview

Package sitemapparser provides functionality to parse XML sitemaps and extract URLs. It supports both regular sitemaps and sitemap index files with concurrent processing.

Index

• func ExtractURLs(url string) ([]string, error)
• type Sitemap
• type SitemapIndex
• type URL
• type URLSet

Functions

func ExtractURLs

func ExtractURLs(url string) ([]string, error)

ExtractURLs processes a sitemap URL and returns all URLs found It handles both sitemap index files and regular sitemaps For sitemap index files, it processes all sub-sitemaps concurrently

Types

type Sitemap

type Sitemap struct {
	Loc string `xml:"loc"` // URL location of the sitemap
}

Sitemap represents a single sitemap entry within a sitemap index

type SitemapIndex

type SitemapIndex struct {
	XMLName  xml.Name  `xml:"sitemapindex"`
	Sitemaps []Sitemap `xml:"sitemap"`
}

SitemapIndex represents the XML structure of a sitemap index file that contains references to other sitemap files

type URL

type URL struct {
	Loc string `xml:"loc"` // URL location of the page
}

URL represents a single URL entry in a sitemap

type URLSet

type URLSet struct {
	XMLName xml.Name `xml:"urlset"`
	URLs    []URL    `xml:"url"`
}

URLSet represents the XML structure of a regular sitemap file containing a collection of URLs