skimmer

README

skimmer

skimmer is a lightweight feed reader inspired by newsboat and yarnc. skimmer is minimal and lacks features. That is skimmer's best feature. skimmer can do two things.

• fetch a list of URLs and download their items to an SQLite3 database
• Display the contents of the SQLite3 database in reverse chronological order

That's it. No paging, now UI other than the command line options and what is sent to standard output.

The URL file format is based on newsboat's URL list. That is because I use Newsboat for my interactive feed reading. The skimmer application stores the list of URLs and the SQLite 3 database in your home directory under .skimmer.

OPTIONS

-help : display a help page

-license : display license

-version : display version number and build hash

-fetch : Download items from the list of URLs

-display : Display the contents of the SQLite 3 database

-limit N : Limit the display the N most recent items

-prune TIMESTAMP : The deletes items from the database that are older than TIMESTAMP. TIMESTAMP can be "now","today", a day in YYYY-MM-DD format or a full timestamp in YYYY-MM-DD HH:MM:SS format.

Examples

Fetch and read some news

skimmer

Download some news to read later

skimmer -fetch

Display the downloaded news

skimmer -display

Limit the number of items sent to the screen.

skimmer -display -limit 25

Or my favorite is to run the output through Pandoc and page with less.

skimmer -display -limit 25 | \
    pandoc -f markdown -t plain | \
    less -R

Prune the items in the database older than today.

skimmer -prune today

Installation instructions

• INSTALL contains the general steps to install binary releases
• You can download a release from https://github.com/rsdoiel/stngo/releases

Installation From Source

Requirements

skimmer is an experimental. The precompiled binaries are not tested. To compile from source you need to have git, make, Pandoc SQLite3 and Go.

• Git >= 2
• Make >= 3.8 (GNU Make)
• Pandoc > 3
• Go >= 1.21.1
• SQLite3 > 3.4

Steps to compile and install

Installation process I used to setup skimmer on a new machine.

git clone https://github.com/rsdoiel/skimmer
cd skimmer
make
make install

A default URLs list is provided as an example URLs list. The first time you run skimmer. You should edit $HOME/.skimmer/skimmer.urls to fit your needs.

Acknowledgments

This experiment would not be possible with the authors of newsboat, SQLite3, Pandoc and the gofeed package.

Documentation

Index

• Constants
• func CheckWaitInterval(iTime time.Time, wait time.Duration) (time.Time, bool)
• func FmtHelp(src string, appName string, version string, releaseDate string, ...) string
• func ParseURLList(fName string, src []byte) (map[string]string, error)
• func ProgressETA(t0 time.Time, i int, tot int) string
• func ProgressIPS(t0 time.Time, i int, timeUnit time.Duration) string
• type Skimmer
  • func NewSkimmer(out io.Writer, eout io.Writer, appName string) (*Skimmer, error)
  • func (app *Skimmer) Download() error
  • func (app *Skimmer) ItemCount() (int, error)
  • func (app *Skimmer) PruneItems(dt time.Time) error
  • func (app *Skimmer) ReadUrls() error
  • func (app *Skimmer) Run(out io.Writer, eout io.Writer, args []string) error
  • func (app *Skimmer) Setup(appDir string) error
  • func (app *Skimmer) Write(out io.Writer) error

Constants

const (
	SkimmerURLs   = "skimmer.urls"
	SkimmerScheme = "skimmer.scheme"
	SkimmerDB     = "skimmer.db"
)

const (
	// Version number of release
	Version = "0.0.2"

	// ReleaseDate, the date version.go was generated
	ReleaseDate = "2023-10-06"

	// ReleaseHash, the Git hash when version.go was generated
	ReleaseHash = "68d0b7c"

	LicenseText = `` /* 1025-byte string literal not displayed */

)

Functions

func CheckWaitInterval

func CheckWaitInterval(iTime time.Time, wait time.Duration) (time.Time, bool)

CheckWaitInterval checks to see if an interval of time has been met or exceeded. It returns the remaining time interval (possibly reset) and a boolean. The boolean is true when the time interval has been met or exceeded, false otherwise.

``` tot := len(something) // calculate the total number of items to process t0 := time.Now() iTime := time.Now() reportProgress := false

for i, key := range records {
    // ... process stuff ...
    if iTime, reportProgress = CheckWaitInterval(rptTime, (30 * time.Second)); reportProgress {
        log.Printf("%s", ProgressETA(t0, i, tot))
    }
}

```

func FmtHelp

func FmtHelp(src string, appName string, version string, releaseDate string, releaseHash string) string

FmtHelp lets you process a text block with simple curly brace markup.

func ParseURLList

func ParseURLList(fName string, src []byte) (map[string]string, error)

ParseURLList takes a filename and byte slice source, parses the contents returning a map of urls to labels and an error value.

func ProgressETA

func ProgressETA(t0 time.Time, i int, tot int) string

ProgressETA returns a string with the percentage processed and estimated time remaining. It requires the a counter of records processed, the total count of records and a time zero value.

``` tot := len(something) // calculate the total number of items to process t0 := time.Now() iTime := time.Now() reportProgress := false

for i, key := range records {
    // ... process stuff ...
    if iTime, reportProgress = CheckWaitInterval(rptTime, (30 * time.Second)); reportProgress {
        log.Printf("%s", ProgressETA(t0, i, tot))
    }
}

```

func ProgressIPS

func ProgressIPS(t0 time.Time, i int, timeUnit time.Duration) string

ProgressIPS returns a string with the elapsed time and increments per second. Takes a time zero, a counter and time unit. Returns a string with count, running time and increments per time unit. ``` t0 := time.Now() iTime := time.Now() reportProgress := false

for i, key := range records {
    // ... process stuff ...
    if iTime, reportProgress = CheckWaitInterval(rptTime, (30 * time.Second)); reportProgress || i = 0 {
        log.Printf("%s", ProgressIPS(t0, i, time.Second))
    }
}

```

Types

type Skimmer

type Skimmer struct {
	// AppName holds the name of the application
	AppName string `json:"app_name,omitempty"`
	// AppDir holds the path to the directory where the urls and feed_items.db are held
	AppDir string `json:"app_dir,omitempty"`
	// Fetch indicates that items need to be retrieved from the url list and stored in the database
	Fetch bool `json:"fetch,omitempty"`
	// Display indicates the contents of the database needs to be streamed to output
	Display bool `json:"read,omitempty"`
	// Urls are the map of urls to labels to be fetched or read
	Urls map[string]string `json:"urls,omitempty"`

	// Limit contrains the number of items shown
	Limit int `json:"limit,omitempty"`

	// Prune contains the date to use to prune the database.
	Prune string `json:"prune,omitempty"`
	// contains filtered or unexported fields
}

Skimmer is the application structure that holds configuration and ties the app to the runner for the cli.

func NewSkimmer

func NewSkimmer(out io.Writer, eout io.Writer, appName string) (*Skimmer, error)

func (*Skimmer) Download

func (app *Skimmer) Download() error

Download the contents from app.Urls

func (*Skimmer) ItemCount

func (app *Skimmer) ItemCount() (int, error)

ItemCount returns the total number items in the database.

func (*Skimmer) PruneItems

func (app *Skimmer) PruneItems(dt time.Time) error

PruneItems takes a timestamp and performs a row delete on the table for items that are older than the timestamp.

func (*Skimmer) ReadUrls

func (app *Skimmer) ReadUrls() error

ReadUrls reads the $HOME/.skimmer/urls file and populates the app.Urls map. Newsboat's url file format is `<URL><SPACE>"~<LABEL>"` one entry per line The hash mark, "#" at the start of the line indicates a comment line.

func (*Skimmer) Run

func (app *Skimmer) Run(out io.Writer, eout io.Writer, args []string) error

Run provides the runner for skimmer. It allows for testing of much of the cli functionality

func (*Skimmer) Setup

func (app *Skimmer) Setup(appDir string) error

Setup checks to see if anything needs to be setup (or fixed) for skimmer to run.

func (*Skimmer) Write

func (app *Skimmer) Write(out io.Writer) error

Display the contents from database