actors

Documentation

Index

• func Start(a Actor, wgs ...*sync.WaitGroup)
• type Actor
• type Cleaner
  • func (c *Cleaner) Run()
• type FileInfoForCleaner
• type Report
  • func (r *Report) Run()
• type ReportEvent
• type Scraper
  • func (s *Scraper) Run()
• type Transferor
  • func (t *Transferor) Run()

Functions

func Start

func Start(a Actor, wgs ...*sync.WaitGroup)

Start runs the given Actor in its own goroutine.

Types

type Actor

type Actor interface {
	Run()
}

Actor is something that can be run in its own goroutine. This package contains various structs that satisfy this interface, and which make up the bulk of the behavior of swift-http-import.

type Cleaner

type Cleaner struct {
	Context context.Context
	Input   <-chan FileInfoForCleaner
	Report  chan<- ReportEvent
}

Cleaner is an actor that cleans up unknown objects on the target side (i.e. those objects which do not exist on the source side).

func (*Cleaner) Run

func (c *Cleaner) Run()

Run implements the Actor interface.

type FileInfoForCleaner

type FileInfoForCleaner struct {
	objects.File
	Failed bool
}

FileInfoForCleaner contains information about a transferred file for the Cleaner actor.

type Report

type Report struct {
	Input     <-chan ReportEvent
	Statsd    objects.StatsdConfiguration
	StartTime time.Time

	ExitCode int
}

Report is an actor that counts scraped directories and transferred files. It emits StatsD metrics (if desired), logs the final report, and decides whether to exit with an error status.

Events are read from the `Input` channel until it is closed. The `Done` channel can be closed to interrupt the actor. If the `Statter` is not nil, statsd metrics will be emitted. The `StartTime` is used to measure this run's duration at the end. The `ExitCode` can be read after the actor is done.

func (*Report) Run

func (r *Report) Run()

Run implements the Actor interface.

type ReportEvent

type ReportEvent struct {
	IsJob      bool
	JobSkipped bool

	IsDirectory     bool
	DirectoryFailed bool

	IsFile             bool
	FileTransferResult objects.TransferResult
	FileTransferBytes  int64

	IsCleanup            bool
	CleanedUpObjectCount int64
}

ReportEvent counts either a directory that was scraped, or a file that was found (and maybe transferred). It is consumed by the Report actor.

type Scraper

type Scraper struct {
	Context context.Context
	Jobs    []*objects.Job
	Output  chan<- objects.File
	Report  chan<- ReportEvent
}

Scraper is an actor that reads directory listings on the source side to enumerate all files that need to be transferred.

Scraping starts from the root directories of each job in the `Jobs` list. For each input file, a File struct is sent into the `Output` channel. For each directory, a report is sent into the `Report` channel.

func (*Scraper) Run

func (s *Scraper) Run()

Run implements the Actor interface.

type Transferor

type Transferor struct {
	Context context.Context
	Input   <-chan objects.File
	Output  chan<- FileInfoForCleaner
	Report  chan<- ReportEvent
}

Transferor is an actor that transfers files from a Source to a target SwiftLocation.

Files to transfer are read from the `Input` channel until it is closed. For each input file, a report is sent into the `Report` channel.

func (*Transferor) Run

func (t *Transferor) Run()

Run implements the Actor interface.