retrieve

Documentation

Overview

Package retrieve contains the retrieve action.

This action retrieves data from remote storage and places them in prescribed directories locally.

Index

• func RunOnceWithUnpacking(session *actions.Session, writer *StatusWriter, start hour.Hour, end hour.Hour, ...) error
• func RunOnceWithoutUnpacking(session *actions.Session, writer *StatusWriter, start hour.Hour, end hour.Hour, ...) error
• type StatusWriter
  • func NewStatusWriter(feeds []config.Feed) *StatusWriter
  • func (w *StatusWriter) RecordDownload(feed *config.Feed, err error)
  • func (w *StatusWriter) RecordFinished(feed *config.Feed)
  • func (w *StatusWriter) SetNumArchives(feed *config.Feed, n int)

Functions

func RunOnceWithUnpacking

func RunOnceWithUnpacking(session *actions.Session, writer *StatusWriter,
	start hour.Hour, end hour.Hour, targetDStore storage.WritableDStore) error

RunOnceWithUnpacking retrieves remote data, unpacks the archive, and stores it locally.

func RunOnceWithoutUnpacking

func RunOnceWithoutUnpacking(session *actions.Session, writer *StatusWriter,
	start hour.Hour, end hour.Hour, targetAStore storage.WritableAStore) error

RunOnceWithoutUnpacking retrieves remote data and stores it locally without unpacking the archives. That is, the compressed archive files are just stored.

Types

type StatusWriter

type StatusWriter struct {
	// contains filtered or unexported fields
}

func NewStatusWriter

func NewStatusWriter(feeds []config.Feed) *StatusWriter

func (*StatusWriter) RecordDownload

func (w *StatusWriter) RecordDownload(feed *config.Feed, err error)

func (*StatusWriter) RecordFinished

func (w *StatusWriter) RecordFinished(feed *config.Feed)

func (*StatusWriter) SetNumArchives

func (w *StatusWriter) SetNumArchives(feed *config.Feed, n int)