reporter

README

uptime-reporter

uptime-reporter is a Go package and a set of command-line tools for reporting on website check data from the Uptime.com monitoring service.

What does it do?

If you'd like to know how many outages each of your site checks had, and how much downtime each of them had in a given period, for example, uptime-reporter can produce that information for you.

It can also produce statistics, such as the average number of outages (both mean and median), average downtime, and so on. If your checks have tags (to classify them by site or business area, for example), uptime-reporter can produce statistics for each tag, as well as overall.

How do I use it?

First, you'll need an Uptime.com account (if you don't have one yet, sign up for a free trial). Then, you'll need an API token.

Clone this repository if you haven't already. Then run the following command to set your API token, substituting your token for XXX:

export UPTIME_API_TOKEN="XXX"

Downloading data

Now you're ready to start downloading your check data. The downloader will fetch data for the period you specify, so you will need to specify the exact start and end times of this period in the following format: YYYY-MM-DDTHH:MM:SSZ.

Run this command, substituting the start and end times of the period you're interested in:

go run cmd/download/main.go 2019-01-01T00:00:00Z 2020-01-01T00:00:00Z |tee data.csv

If all is well, you'll start to see CSV data printed out, one line for each of your checks:

My Main Website,https://example.com/,Sites,0,0
...

Occasionally you may see a message that the downloader is being rate-limited by the Uptime.com API. That's okay, just wait a few seconds and it will keep trying:

rate-limited; sleeping 5s before retry
My Secondary Website,https://subdomain.example.com,Sites,0,0

Resuming interrupted downloads

If you have a large number of checks in your account, and something interrupts your statistics download, don't worry—you don't have to start again from scratch. Just specify the ID of the last check you successfully downloaded, as an extra parameter on the command line:

go run cmd/download/main.go 2019-01-01T00:00:00Z 2020-01-01T00:00:00Z 21999 |tee -a data.csv

The downloader will skip all the checks up to and including ID 21999, and begin downloading data from the next check onwards.

Analysing data

Once the downloader has finished running, your data will be stored in the file data.csv. To produce statistics on it, run:

go run cmd/analyse/main.go <data.csv

You should see some output like this:

Sector: All (160 sites)
Outages: Total 60.0 Min 0.0 Max 19.0 Median 0.0 Mean 0.4 Standard deviation 1.6
Downtimes: Total 1831856.0 Min 0.0 Max 1160496.0 Median 0.0 Mean 11449.1 Standard deviation 98558.7

Sites with most downtime:
Rank  Name     Outages  Downtime
1     Site A   19       322h21m36s
2     Site B   2        131h30m18s
3     Site C   2        14h4m1s
...

Filtering excessive downtime

The downtime figure for some sites may not be meaningful. For example, if you're analysing 90 days of data and a given site only existed or was only monitored for the last day of that period, it would appear to have 89 days of downtime, which is incorrect.

To eliminate these outliers, you can specify a 'maximum downtime' argument to the analyser:

go run cmd/analyse/main.go 100h <data.csv

The 100h here represents a maximum downtime of 100 hours, and sites with more downtime than this will not be included in the statistics or league tables. Try different numbers to find the best one to eliminate the noise from the dataset you're working with.

You can specify the maximum downtime duration as, for example, 5h10m, or 90s, or any combination of units supported by Go's time library.

Using the Go package

If you want to do your own analysis, or download additional data, you can write your own Go programs which use the uptime-reporter library. This provides some useful functions such as a rate-limit-aware API client, CSV input and output, data types, and so on.

import reporter "github.com/bitfield/uptime-reporter"

Documentation

Index

• func WriteCSV(output io.Writer, site Site) error
• type Reporter
  • func New(apiToken string) (Reporter, error)
  • func (r Reporter) GetDowntimes(id int, startDate, endDate string) (Site, error)
  • func (r Reporter) GetDowntimesWithRetry(id int, startDate, endDate string) (Site, error)
  • func (r Reporter) GetSiteIDs() ([]int, error)
• type Site
  • func SiteFromCheck(c uptime.Check, s uptime.CheckStatsResponse) Site
• type SiteSet
  • func ReadCSV(input io.Reader) (SiteSet, error)
  • func (ss SiteSet) BySector() map[string]SiteSet
  • func (ss SiteSet) FilterDowntimeOver(limit int64) SiteSet
  • func (ss SiteSet) SortByDowntime()
• type Summary
  • func StatsSummary(input stats.Float64Data) (Summary, error)
  • func (s Summary) String() string

Functions

func WriteCSV

func WriteCSV(output io.Writer, site Site) error

WriteCSV takes a Site object and prints a CSV-formatted version of it to the supplied writer.

Types

type Reporter

type Reporter struct {
	// contains filtered or unexported fields
}

Reporter stores the Uptime.com client configuration.

func New

func New(apiToken string) (Reporter, error)

New takes an Uptime.com API token and returns a Reporter object which can then be used to query the Uptime API.

func (Reporter) GetDowntimes

func (r Reporter) GetDowntimes(id int, startDate, endDate string) (Site, error)

GetDowntimes takes the ID of a check, and two time values indicating the start and end of the period to query. It returns a Site object containing metadata about the site, plus the number of outages in the period, and the total amount of downtime in the period.

func (Reporter) GetDowntimesWithRetry

func (r Reporter) GetDowntimesWithRetry(id int, startDate, endDate string) (Site, error)

GetDowntimesWithRetry calls GetDowntimes for the given ID. If there is an API rate limit error, it sleeps for a while and tries again, and keeps trying forever.

func (Reporter) GetSiteIDs

func (r Reporter) GetSiteIDs() ([]int, error)

GetSiteIDs returns a slice of check IDs, one for each check in the account associated with the Reporter's API token.

type Site

type Site struct {
	ID                int
	Name, URL, Sector string
	Outages           int
	DowntimeSecs      int64
}

Site represents metadata about an Uptime.com check, and can also store data on its outages and downtime within a specified period.

func SiteFromCheck

func SiteFromCheck(c uptime.Check, s uptime.CheckStatsResponse) Site

SiteFromCheck translates from an uptime.Check and uptime.CheckStats object to a Site object containing the data from both objects.

type SiteSet

type SiteSet []Site

SiteSet represents a slice of Sites.

func ReadCSV

func ReadCSV(input io.Reader) (SiteSet, error)

ReadCSV reads CSV data representing a group of Sites, one per line, from the given input.

func (SiteSet) BySector

func (ss SiteSet) BySector() map[string]SiteSet

BySector operates on a SiteSet and returns a map of sectors to sites (that is, the map key is the sector name, and the corresponding value is the SiteSet of all the sites in that sector).

func (SiteSet) FilterDowntimeOver

func (ss SiteSet) FilterDowntimeOver(limit int64) SiteSet

FilterDowntimeOver returns the set of sites with less than or equal to the specified amount of downtime, in seconds.

func (SiteSet) SortByDowntime

func (ss SiteSet) SortByDowntime()

SortByDowntime sorts the SiteSet by downtime, highest first, then by outages, most outages first, and then by name alphabetically.

type Summary

type Summary struct {
	Sum, Mean, Dev, Min, Max, Median, Q1, Q3 float64
}

Summary represents the statistical summary data for a group of Sites.

func StatsSummary

func StatsSummary(input stats.Float64Data) (Summary, error)

StatsSummary takes a dataset of floating-point values and calculates various statistical values for them, returning a Summary object containing the computed data.

func (Summary) String

func (s Summary) String() string

String returns a formatted version of the Summary data suitable for printing.