feedloggr

README

 ,-.
( O )`~-~-~-~-~-~-~-~-~-,
|`-'|  -- feedloggr --  |
|   |     v0.5.0        |
 `-' `~-~-~-~-~-~-~-~-~-'

Collect news from your favourite Atom/RSS/JSON feeds and generate static web pages for easy browsing.

Purpose

I wanted the most basic feed reader that would consolidate all my feeds onto a single, static and read-only page that I could host for myself. I also wanted to do some extra tricks, for example (a feature list):

• Customising and styling my own web template.
• Prefer linking to comment pages, for the more social feeds.
• Parse and find news from non-feed pages.
• Inject any custom information.

Status

The project has successfully been running since June 2021 without any major issues on my own sites. The only minor issues discovered so far has been with external feeds with poor behaviour.

Installation

go install code.larus.se/lmas/feedloggr@latest

Usage

For a list of available flags and commands:

feedloggr help

You can create a new configuration file by running:

feedloggr example > .feedloggr/conf.yml

You should then edit conf.yml and add your own feeds to it.

When you're done editing the configuration, you can test it and make sure there's no errors:

feedloggr test

If no errors are shown, you're good to go!

Now that you have a working configuration, it's time to start collect news from your feeds and export them to a web page:

feedloggr

When it's done you should be able to browse the newly generated page, found inside the output directory that was specified in the configuration file.

Command line options

Usage of feedloggr:

Flags
  -conf string
        Path to conf file (default ".feedloggr/conf.yml")
  -verbose
        Print debug messages while running

Commands
  discover
    Try discover feeds from <site link>
  example
    Print example config
  help
    Print this help message and exit
  regexp
    Try parsing items from <site link> using <regexp> rule
  run
    Update feeds and output new page
  test
    Try loading config
  version
    Print version information

Configuration

Configuration is by default loaded from the file .feedloggr/conf.yml in the current directory, but can be overridden with the -conf flag.

settings

Global configuration settings.

output

Output directory where generated pages and link filter are stored.

maxdays

Optional max amount of days to keep the generated pages for.

maxitems

Max amount of new items to fetch from each feed.

timeout

Max time (in seconds) to try download a feed, before timing out.

jitter

Max time (in seconds) to randomly apply, as a wait time, between each feed update.

verbose

Show verbose output when running.

feeds

The list of Atom/RSS/JSON feeds to be fetched.

title

Custom title for the feed.

link

Site link where new updates will be fetched from.

regexp

Optional regexp rule for fetching items from a non-feed page.
The rule must define two capture groups called "title" and "link".
A third, optional capture group "content" allows for capturing any extra text
that can be used and displayed in the output template.

pipecommands

An optional list of shell commands that can be run when loading the web template. If a command writes a JSON object to stdout, the object's root keys will be inserted into the template, as unique variables prefixed with .Piped.. See the template example for a how to use.

Example

Running feedloggr example should show you:

settings:
    output: ./feeds/
    maxdays: 30
    maxitems: 20
    timeout: 30
    jitter: 2
    verbose: true
pipecommands:
    - echo '{"msg":"hello world!"}'
    - echo "{\"date\":\"$(date)\"}"
feeds:
    - title: Lemmy.world
      link: https://lemmy.world/feeds/local.xml?sort=TopDay
    - title: Lobsters
      link: https://lobste.rs/top.rss
      regexp: (?sU)<item>.*<title>(?P<title>[^<]+)</title>.*<comments>(?P<link>[^<]+)</comments>.*</item>

Output Template

By default template.html is attempted to be loaded from the config directory, but if it's not existing a built in template will be used instead.

The templating system used is html/template, found in Go's standard library.

Template Variables

.Today

Current time. Can be customised with the template functions available down below.

.Generator.Name

.Generator.Version

.Generator.Link

Basic information about this tool.

.Feeds

A list of feeds, as defined in the config file.
Can be iterated easily.

.Feeds.Conf.Title

.Feeds.Conf.Link

Basic information about current feed.

.Feeds.Items

A list of unique items for the feed.
Can be iterated easily.

.Feeds.Items.Title

.Feeds.Items.Link

.Feeds.Items.Content

Basic information about current item.

.Feeds.Error

In case an error was encountered while trying to update the feed,
this variable will contain the error message.

.Piped.

Any JSON object returned from pipecommands will have the object's root keys
inserted under this variable.

Template Functions

shortdate

Can be used to shorten a long time value down to, for example: 2006-01-02.

prevday

Can be used to subtract a day from a time value.

nextday

Can be used to add a day to a time value.

Example

Minimal example template, based on the built in default (without CSS styling):

<!doctype html>
<html>
<head>
    <meta charset="utf-8">
    <title>News for {{.Today | shortdate}}</title>
    <!-- Optional stylesheet to overwrite the default style -->
    <link href="style.css" rel="stylesheet" type="text/css">
</head>
<body>
    <header>
        <h1>{{.Today | shortdate}}</h1>
        <a class="bar" href="./{{.Today | prevday | shortdate}}.html">Previous</a>
        <a class="bar" href="./index.html">Latest</a>
        <a class="bar" href="./{{.Today | nextday | shortdate}}.html">Next</a>
    </header>
    <h1>{{.Piped.msg}}</h1>
    {{range .Feeds}}
    <section>
        <h2 class="bar"><a href="{{.Conf.Link}}">{{.Conf.Title}}</a></h2>
        {{if .Error}}
            <p>Error while updating feed:<br />{{.Error}}</p>
        {{else}}
            <ul>{{range .Items}}
                <li><a href="{{.Link}}" rel="nofollow">{{.Title}}</a></li>
            {{end}}</ul>
        {{end}}
    </section>
    {{else}}
        <p>Sorry, no news for today!</p>
    {{end}}
    <footer>
        Generated with <a href="{{.Generator.Link}}">{{.Generator.Name}} {{.Generator.Version}}</a>
    </footer>
</body>
</html>

License

GPL, See the LICENSE file for details.