rss

README

rss

RSS is a small library for simplifying the parsing of RSS and Atom feeds. The package could do with more testing, but it conforms to the RSS 1.0, 2.0, and Atom 1.0 specifications, to the best of my ability. I've tested it with about 15 different feeds, and it seems to work fine with them.

If anyone has any problems with feeds being parsed incorrectly, please let me know so that I can debug and improve the package.

Dependencies:

go get github.com/axgle/mahonia

Example usage:

package main

import "github.com/SlyMarbo/rss"

func main() {
	feed, err := rss.Fetch("http://example.com/rss")
	if err != nil {
		// handle error.
	}
	
	// ... Some time later ...
	
	err = feed.Update()
	if err != nil {
		// handle error.
	}
}

The output structure is pretty much as you'd expect:

type Feed struct {
	Nickname    string              // This is not set by the package, but could be helpful.
	Title       string
	Description string
	Link        string              // Link to the creator's website.
	UpdateURL   string              // URL of the feed itself.
	Image       *Image              // Feed icon.
	Items       []*Item
	ItemMap     map[string]struct{} // Used in checking whether an item has been seen before.
	Refresh     time.Time           // Earliest time this feed should next be checked.
	Unread      uint32              // Number of unread items. Used by aggregators.
}

type Item struct {
	Title   string
	Summary string
	Content string
	Link    string
	Date    time.Time
	ID      string
	Read    bool
}

type Image struct {
	Title   string
	Url     string
	Height  uint32
	Width   uint32
}

The library does its best to follow the appropriate specifications and not to set the Refresh time too soon. It currently follows all update time management methods in the RSS 1.0, 2.0, and Atom 1.0 specifications. If one is not provided, it defaults to 10 minute intervals. If you are having issues with feed providors dropping connections, please let me know and I can increase this default, or you can increase the Refresh time manually. The Feed.Update method uses this Refresh time, so if Update seems to be returning very quickly with no new items, it's likely not making a request due to the provider's Refresh interval.

This is seeing thorough use in RS3, but development is still active.

Documentation

Overview

Package RSS is a small library for simplifying the parsing of RSS and Atom feeds.

The package could do with more testing, but it conforms to the RSS 1.0, 2.0, and Atom 1.0 specifications, to the best of my ability. I've tested it with about 15 different feeds, and it seems to work fine with them.

If anyone has any problems with feeds being parsed incorrectly, please let me know so that I can debug and improve the package.

Example usage: ```go package main

import "github.com/SlyMarbo/rss"

func main() {
	feed, err := rss.Fetch("http://example.com/rss")
	if err != nil {
		// handle error.
	}

	// ... Some time later ...

	err = feed.Update()
	if err != nil {
		// handle error.
	}
}

```

The output structure is pretty much as you'd expect: ```go

type Feed struct {
	Nickname    string // This is not set by the package, but could be helpful.
	Title       string
	Description string
	Link        string // Link to the creator's website.
	UpdateURL   string // URL of the feed itself.
	Image       *Image // Feed icon.
	Items       []*Item
	ItemMap     map[string]struct{} // Used in checking whether an item has been seen before.
	Refresh     time.Time           // Earliest time this feed should next be checked.
	Unread      uint32              // Number of unread items. Used by aggregators.
}

type Item struct {
	Title   string
	Summary string
	Content string
	Link    string
	Date    time.Time
	ID      string
	Read    bool
}

type Image struct {
	Title   string
	Url     string
	Height  uint32
	Width   uint32
}

```

The library does its best to follow the appropriate specifications and not to set the Refresh time too soon. It currently follows all update time management methods in the RSS 1.0, 2.0, and Atom 1.0 specifications. If one is not provided, it defaults to 10 minute intervals. If you are having issues with feed providors dropping connections, please let me know and I can increase this default, or you can increase the Refresh time manually. The Feed.Update method uses this Refresh time, so if Update seems to be returning very quickly with no new items, it's likely not making a request due to the provider's Refresh interval.

Index

• Constants
• Variables
• func CacheParsedItemIDs(flag bool) (didCache bool)
• func NewDB() *db
• type Enclosure
  • func (e *Enclosure) Get() (io.ReadCloser, error)
• type Feed
  • func Fetch(url string) (*Feed, error)
  • func FetchByClient(url string, client *http.Client) (*Feed, error)
  • func FetchByFunc(fetchFunc FetchFunc, url string) (*Feed, error)
  • func Parse(data []byte) (*Feed, error)
  • func (f *Feed) String() string
  • func (f *Feed) Update() error
• type FetchFunc
• type Image
  • func (i *Image) Get() (io.ReadCloser, error)
  • func (i *Image) String() string
• type Item
  • func (i *Item) Format(indent int) string
  • func (i *Item) String() string

Constants

const DATE = "15:04:05 MST 02/01/2006"

Variables

var TimeLayouts = []string{
	"Mon, _2 Jan 2006 15:04:05 MST",
	"Mon, _2 Jan 2006 15:04:05 -0700",
	time.ANSIC,
	time.UnixDate,
	time.RubyDate,
	time.RFC822,
	time.RFC822Z,
	time.RFC850,
	time.RFC1123,
	time.RFC1123Z,
	time.RFC3339,
	time.RFC3339Nano,
}

TimeLayouts is contains a list of time.Parse() layouts that are used in attempts to convert item.Date and item.PubDate string to time.Time values. The layouts are attempted in ascending order until either time.Parse() does not return an error or all layouts are attempted.

Functions

func CacheParsedItemIDs

func CacheParsedItemIDs(flag bool) (didCache bool)

CacheParsedItemIDs enables or disable Item.ID caching when parsing feeds. Returns whether Item.ID were cached prior to function call.

func NewDB

func NewDB() *db

Types

type Enclosure

type Enclosure struct {
	Url    string
	Type   string
	Rel    string
	Length int
}

func (*Enclosure) Get

func (e *Enclosure) Get() (io.ReadCloser, error)

type Feed

type Feed struct {
	Nickname    string // This is not set by the package, but could be helpful.
	Title       string
	Description string
	Link        string // Link to the creator's website.
	UpdateURL   string // URL of the feed itself.
	Image       *Image // Feed icon.
	Items       []*Item
	ItemMap     map[string]struct{} // Used in checking whether an item has been seen before.
	Refresh     time.Time           // Earliest time this feed should next be checked.
	Unread      uint32              // Number of unread items. Used by aggregators.
}

Feed is the top-level structure.

func Fetch

func Fetch(url string) (*Feed, error)

Fetch downloads and parses the RSS feed at the given URL

func FetchByClient

func FetchByClient(url string, client *http.Client) (*Feed, error)

func FetchByFunc

func FetchByFunc(fetchFunc FetchFunc, url string) (*Feed, error)

func Parse

func Parse(data []byte) (*Feed, error)

Parse RSS or Atom data.

func (*Feed) String

func (f *Feed) String() string

func (*Feed) Update

func (f *Feed) Update() error

Update fetches any new items and updates f.

type FetchFunc

type FetchFunc func() (resp *http.Response, err error)

type Image

type Image struct {
	Title  string
	Url    string
	Height uint32
	Width  uint32
}

func (*Image) Get

func (i *Image) Get() (io.ReadCloser, error)

func (*Image) String

func (i *Image) String() string

type Item

type Item struct {
	Title      string
	Summary    string
	Content    string
	Link       string
	Date       time.Time
	ID         string
	Enclosures []*Enclosure
	Read       bool
}

Item represents a single story.

func (*Item) Format

func (i *Item) Format(indent int) string

func (*Item) String

func (i *Item) String() string