feed2json

README

feed2json

Given an Atom or RSS feed, creates a comparable JSON feed.

Installation

First install Go.

If you just want to install the binary to your current directory and don't care about the source code, run

GOBIN="$(pwd)" GOPATH="$(mktemp -d)" go get github.com/carlmjohnson/feed2json/...

Screenshots

$ feed2json -h
feed2json converts an Atom or RSS feed into a JSON feed.

    feed2json [opts]

Options:
  -dst file
        destination file (default stdout)
  -src file or URL
        source file or URL (default stdin)
  -timeout duration
        timeout for URL sources (default 5s)

$ feed2json -src 'https://jsonfeed.org/xml/rss.xml' | json-tidy
{
    "description": "JSON Feed is a pragmatic syndication format for blogs, microblogs, and other time-based content.",
    "home_page_url": "https://jsonfeed.org/",
    "items": [
        {
            "content_html": "<p>We — Manton Reece and Brent Simmons — have noticed that JSON has become the developers’ choice for APIs, and that developers will often go out of their way to avoid XML. JSON is simpler to read and write, and it’s less prone to bugs.</p>\n\n<p>So we developed JSON Feed, a format similar to <a href=\"http://cyber.harvard.edu/rss/rss.html\">RSS</a> and <a href=\"https://tools.ietf.org/html/rfc4287\">Atom</a> but in JSON. It reflects the lessons learned from our years of work reading and publishing feeds.</p>\n\n<p><a href=\"https://jsonfeed.org/version/1\">See the spec</a>. It’s at version 1, which may be the only version ever needed. If future versions are needed, version 1 feeds will still be valid feeds.</p>\n\n<h4>Notes</h4>\n\n<p>We have a <a href=\"https://github.com/manton/jsonfeed-wp\">WordPress plugin</a> and, coming soon, a JSON Feed Parser for Swift. As more code is written, by us and others, we’ll update the <a href=\"https://jsonfeed.org/code\">code</a> page.</p>\n\n<p>See <a href=\"https://jsonfeed.org/mappingrssandatom\">Mapping RSS and Atom to JSON Feed</a> for more on the similarities between the formats.</p>\n\n<p>This website — the Markdown files and supporting resources — <a href=\"https://github.com/brentsimmons/JSONFeed\">is up on GitHub</a>, and you’re welcome to comment there.</p>\n\n<p>This website is also a blog, and you can subscribe to the <a href=\"https://jsonfeed.org/xml/rss.xml\">RSS feed</a> or the <a href=\"https://jsonfeed.org/feed.json\">JSON feed</a> (if your reader supports it).</p>\n\n<p>We worked with a number of people on this over the course of several months. We list them, and thank them, at the bottom of the <a href=\"https://jsonfeed.org/version/1\">spec</a>. But — most importantly — <a href=\"http://furbo.org/\">Craig Hockenberry</a> spent a little time making it look pretty. :)</p>",
            "date_published": "2017-05-17T15:02:12Z",
            "id": "https://jsonfeed.org/2017/05/17/announcing_json_feed",
            "title": "Announcing JSON Feed",
            "url": "https://jsonfeed.org/2017/05/17/announcing_json_feed"
        }
    ],
    "title": "JSON Feed",
    "version": "https://jsonfeed.org/version/1"
}

Documentation

Overview

Package feed2json converts Atom and RSS feeds to JSON feeds.

Index

• func CLI(args []string) (err error)
• func Convert(from, to *bytes.Buffer) (err error)
• func ConvertObject(feed *gofeed.Feed) *feeds.JSONFeed
• func Handler(x URLExtractor, v URLValidator, c *http.Client, l Logger, ms ...Middleware) http.Handler
• type Logger
• type Middleware
• type URLExtractor
  • func ExtractURLFromParam(name string) URLExtractor
• type URLValidator
  • func ValidateHost(names ...string) URLValidator

Examples

• Convert

Functions

func CLI

func CLI(args []string) (err error)

func Convert

func Convert(from, to *bytes.Buffer) (err error)

Convert takes an XML feed from one buffer and turns it into a JSON feed in the other buffer.

Example

package main

import (
	"bytes"
	"encoding/json"
	"fmt"

	"github.com/carlmjohnson/feed2json"
)

func main() {
	var from, to bytes.Buffer
	from.WriteString(`
<?xml version="1.0"?>
<rss version="2.0">
   <channel>
      <title>Liftoff News</title>
      <link>http://liftoff.msfc.nasa.gov/</link>
      <description>Liftoff to Space Exploration.</description>
      <language>en-us</language>
      <pubDate>Tue, 10 Jun 2003 04:00:00 GMT</pubDate>
      <lastBuildDate>Tue, 10 Jun 2003 09:41:01 GMT</lastBuildDate>
      <docs>http://blogs.law.harvard.edu/tech/rss</docs>
      <generator>Weblog Editor 2.0</generator>
      <managingEditor>editor@example.com</managingEditor>
      <webMaster>webmaster@example.com</webMaster>
      <item>
         <title>Star City</title>
         <link>http://liftoff.msfc.nasa.gov/news/2003/news-starcity.asp</link>
         <description>How do Americans get ready to work with Russians aboard the International Space Station? They take a crash course in culture, language and protocol at Russia's &lt;a href="http://howe.iki.rssi.ru/GCTC/gctc_e.htm"&gt;Star City&lt;/a&gt;.</description>
         <pubDate>Tue, 03 Jun 2003 09:39:21 GMT</pubDate>
         <guid>http://liftoff.msfc.nasa.gov/2003/06/03.html#item573</guid>
      </item>
   </channel>
</rss>
    `)
	if err := feed2json.Convert(&from, &to); err == nil {
		from.Reset()
		json.Indent(&from, to.Bytes(), "", "  ")
		fmt.Println(from.String())
	}
}

Output:

{
  "version": "https://jsonfeed.org/version/1",
  "title": "Liftoff News",
  "home_page_url": "http://liftoff.msfc.nasa.gov/",
  "description": "Liftoff to Space Exploration.",
  "author": {},
  "items": [
    {
      "id": "http://liftoff.msfc.nasa.gov/2003/06/03.html#item573",
      "url": "http://liftoff.msfc.nasa.gov/news/2003/news-starcity.asp",
      "title": "Star City",
      "content_html": "How do Americans get ready to work with Russians aboard the International Space Station? They take a crash course in culture, language and protocol at Russia's <a href=\"http://howe.iki.rssi.ru/GCTC/gctc_e.htm\">Star City</a>.",
      "date_published": "2003-06-03T09:39:21Z"
    }
  ]
}

func ConvertObject

func ConvertObject(feed *gofeed.Feed) *feeds.JSONFeed

ConvertObject converts a gofeed.Feed (used by the parser) into a gorilla/feeds.JSONFeed (used by the JSONFeed emitter).

func Handler

func Handler(x URLExtractor, v URLValidator, c *http.Client, l Logger, ms ...Middleware) http.Handler

Types

type Logger

type Logger = func(format string, v ...interface{})

type Middleware

type Middleware = func(http.Handler) http.Handler

type URLExtractor

type URLExtractor = func(*http.Request) *url.URL

func ExtractURLFromParam

func ExtractURLFromParam(name string) URLExtractor

type URLValidator

type URLValidator = func(*url.URL) bool

func ValidateHost

func ValidateHost(names ...string) URLValidator