progress

Documentation

Overview

Package progress is for tracking the progress of long running tasks on the backend in a way that can be reflected in the UI.

We have multiple long running queries like /frame/start and /dryrun/start that start those long running processes and we need to give feedback to the user on how they are proceeding.

The dryrun progress information contains different info with different stages and steps. For example, dryrun progress looks like this:

Step: 1/1
Query: "sub_result=max_rss_mb"
Stage: Looking for regressions in query results.
Commit: 51643
Details: "Filtered Traces: Num Before: 95 Num After: 92 Delta: 3"

Which is just a series of key/value pairs of strings. So our common Progress interface allows for creating a set of key/value pairs to be displayed, along with the Status of the current process, and any results once the process has finished.

Index

• Constants
• Variables
• func New() *progress
• func NewTracker(basePath string) (*tracker, error)
• type Message
• type Progress
• type SerializedProgress
• type Status
• type Tracker

Constants

const ErrorMessageKey = "Error"

ErrorMessageKey is the key in Messages used to store the string passed to Error().

Variables

var AllStatus = []Status{Running, Finished, Error}

AllStatus contains all values of type State.

Functions

func New

func New() *progress

New returns a new Progress in the Running state.

func NewTracker

func NewTracker(basePath string) (*tracker, error)

NewTracker returns a new Tracker instance.

The basePath is the base of the URL path that Progress results will be served from. It must end in a '/' and will have the Progress id appended to it for each Progress. The tracker.Handler() must be set up to receive all requests for that basePath.

Example:

// During init:
singleTrackerInstance := progress.NewTracker("/_/status/")
router.HandleFunc("/_/status/{id:.+}", t.Handler).Methods("GET")

Then in any http handler that starts a long running progress:

prog := StartNewLongRunningProcess()
singleTrackerInstance.Add(prog)
if err := prog.JSON(w); err != nil {
  sklog.Error(err)
}

The serialized Progress contains the URL to make requests back to the app to query the status of the long running process, which will contain the final result when the long running process completes.

Types

type Message

type Message struct {
	Key   string `json:"key"`
	Value string `json:"value"`
}

Message is a key value pair of strings, used in SerializedProgress.

type Progress

type Progress interface {
	// Message adds or updates a message in a progress recorder. If the key
	// matches an existing message it will replace that key's value.
	Message(key, value string)

	// Results is called with the Results that are to be serialized via
	// SerializedProgress. Use this to store intermediate results or if results
	// are accumulated incrementally before the process is Finished.
	Results(interface{})

	// Error sets the Progress status to Error.
	//
	// The passed in string is stored at ErrorMessageKey in Messages.
	Error(string)

	// Finished sets the Progress status to Finished. Should only be used if
	// Results() has been called to fill in intermediate results, otherwise use
	// FinishedWithResults() to avoid race conditions.
	Finished()

	// FinishedWithResults sets the Progress status to Finished with the given
	// result.
	FinishedWithResults(interface{})

	// Status returns the current Status.
	Status() Status

	// URL sets the URL for the next progress update.
	URL(string)

	// JSON writes the data serialized as JSON. The shape is SerializedProgress.
	JSON(w io.Writer) error
}

Progress is the interface for reporting on the progress of a long running process.

Once a Progress has left the Running status it can no longer be modified, and modifying methods like Error() and Results() will panic.

A Progress should only be finalized, by calling Error(), Finished(), or FinishedWithResults() at the outermost calling level. For example, in an HTTP handler function you can kick off a long running process like this:

prog := new Progress()
go func() {
    err, value := SomeLongRunningFuncThatOnlyReturnsWhenItsDone(ctx, prog)
    if err != nil {
        prog.Error("Some failure message")
    } else {
        prog.FinishedWithResults(value)
    }
}()

type SerializedProgress

type SerializedProgress struct {
	Status    Status      `json:"status"`
	Messsages []*Message  `json:"messages" go2ts:"ignorenil"`
	Results   interface{} `json:"results,omitempty"`

	// URL to use in the next polling step.
	URL string `json:"url"`
}

SerializedProgress is the shape of the JSON emitted from Progress.JSON().

type Status

type Status string

Status of a process.

const (
	// Running mean a process is still running.
	Running Status = "Running"

	// Finished means the process has finished.
	Finished Status = "Finished"

	// Error means the process has finished with and error.
	Error Status = "Error"
)

type Tracker

type Tracker interface {
	// Add a Progress to the tracker. This will update the URL of the Progress.
	Add(prog Progress)

	// Handler for HTTP requests for Progress updates.
	Handler(w http.ResponseWriter, r *http.Request)

	// Start the background cleanup task.
	Start(ctx context.Context)
}

Tracker keeps track of long running processes.

It will cache Progresses for a time after they complete.