spyglass

package
v0.0.0-...-d7867ec Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 30, 2018 License: Apache-2.0 Imports: 20 Imported by: 0

README

[![GoDoc Widget]][GoDoc]

Spyglass

A spyglass is an lensed monocular maritime instrument used to see things that may have been difficult to see otherwise.

Spyglass is a pluggable artifact viewer framework for Prow and a crude metaphor for the real object. It collects artifacts (usually files in a storage bucket) from various sources and distributes them to registered viewers, which are responsible for consuming them and rendering a view.

A typical Spyglass page might look something like this: I'm not a graphic designer I just make the backend

A general Spyglass query will proceed as follows:

  • User provides a job source in the query (usually a job name and build ID).
  • Spyglass finds all artifact names associated with the given job source.
  • Spyglass builds a cache of which artifacts match which lenses via configured regular expressions.
  • Lenses with matching artifacts are pre-rendered in order of descending priority.
  • Spyglass then sends render requests to each registered lens with its matching artifacts.
  • Each lens performs any necessary operations on the artifacts and produces a blob of HTML.
  • Views (HTML) are inserted asynchronously as viewers return.

Lenses

A lens is an set of functions that consume a list of artifacts and produces some HTML.

Lens names are unique, and must much the package name for the lens.

Built-in Viewers

Spyglass comes with some built-in viewers for commonly produced artifacts.

  • Prow Metadata 
    Name: metadata
Title: Metadata
Match: finished.json|started.json
Priority: 0
  • JUnit 
    Name: junit
Title: JUnit
Matches: artifacts/junit.*\.xml
Priority: 5
  • Logs 
    Name: buildlog
Title: Build Log
Matches: build-log.txt|pod-log
Priority: 10
Building your own viewer

Building a viewer consists of three main steps.

Write Boilerplate

First, create a package lensnamehere under prow/spyglass/lenses and import the lenses package.

Implement

Next, implement the necessary functions for a viewer. More specifically, implement the following interface (defined in lenses.go):

type Lens interface {
    // Name returns the name of your lens (which must match the name of the directory it lives in)
	Name() string
	// Title returns a human-readable title for your lens.
	Title() string
	// Priority returns a number that is used to determine the ordering of your lens (lower is more important)
	Priority() int
	// Header is used to inject content into the lens's <head>. It will only ever be called once per load.
	Header(artifacts []Artifact, resourceDir string) string
	// Body is used to generate the contents of the lens's <body>. It will initially be called with empty data, but
	// the lens front-end code may choose to re-render itself with custom data.
	Body(artifacts []Artifact, resourceDir string, data string) string
	// Callback is used for the viewer to exchange arbitrary data with the frontend. It is called with lens-specified
	// data, and returns data to be passed to the lens. JSON encoding is recommended in both directions.
	Callback(artifacts []Artifact, resourceDir string, data string) string
}

In the init method, call lenses.RegisterLens() with an instance of your implementation of the interface. Spyglass should now be aware of your lens.

Additionally, some front-end TypeScript code can be provided. Configure your BUILD.bazel to build it, then emit a

Documentation

Overview

Package spyglass creates views for Prow job artifacts.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrCannotParseSource is returned by newGCSJobSource when an incorrectly formatted source string is passed
	ErrCannotParseSource = errors.New("could not create job source from provided source")
)

Functions

This section is empty.

Types

type GCSArtifact 

type GCSArtifact struct {
	// contains filtered or unexported fields
}

GCSArtifact represents some output of a prow job stored in GCS

func NewGCSArtifact 

func NewGCSArtifact(ctx context.Context, handle artifactHandle, link string, path string, sizeLimit int64) *GCSArtifact

NewGCSArtifact returns a new GCSArtifact with a given handle, canonical link, and path within the job 

func (a *GCSArtifact) CanonicalLink() string

CanonicalLink gets the GCS web address of the artifact

func (*GCSArtifact) JobPath 

func (a *GCSArtifact) JobPath() string

JobPath gets the GCS path of the artifact within the current job

func (*GCSArtifact) ReadAll 

func (a *GCSArtifact) ReadAll() ([]byte, error)

ReadAll will either read the entire file or throw an error if file size is too big

func (*GCSArtifact) ReadAt 

func (a *GCSArtifact) ReadAt(p []byte, off int64) (n int, err error)

ReadAt reads len(p) bytes from a file in GCS at offset off

func (*GCSArtifact) ReadAtMost 

func (a *GCSArtifact) ReadAtMost(n int64) ([]byte, error)

ReadAtMost reads at most n bytes from a file in GCS. If the file is compressed (gzip) in GCS, n bytes of gzipped content will be downloaded and decompressed into potentially GREATER than n bytes of content.

func (*GCSArtifact) ReadTail 

func (a *GCSArtifact) ReadTail(n int64) ([]byte, error)

ReadTail reads the last n bytes from a file in GCS

func (*GCSArtifact) Size 

func (a *GCSArtifact) Size() (int64, error)

Size returns the size of the artifact in GCS

type GCSArtifactFetcher 

type GCSArtifactFetcher struct {
	// contains filtered or unexported fields
}

GCSArtifactFetcher contains information used for fetching artifacts from GCS

func NewGCSArtifactFetcher 

func NewGCSArtifactFetcher(c *storage.Client) *GCSArtifactFetcher

NewGCSArtifactFetcher creates a new ArtifactFetcher with a real GCS Client

type LensRequest 

type LensRequest struct {
	Source    string   `json:"src"`
	Artifacts []string `json:"artifacts"`
}

LensRequest holds data sent by a view

type PodLogArtifact 

type PodLogArtifact struct {
	// contains filtered or unexported fields
}

PodLogArtifact holds data for reading from a specific pod log

func NewPodLogArtifact 

func NewPodLogArtifact(jobName string, buildID string, sizeLimit int64, ja jobAgent) (*PodLogArtifact, error)

NewPodLogArtifact creates a new PodLogArtifact 

func (a *PodLogArtifact) CanonicalLink() string

CanonicalLink returns a link to where pod logs are streamed

func (*PodLogArtifact) JobPath 

func (a *PodLogArtifact) JobPath() string

JobPath gets the path within the job for the pod log. Always returns build-log.txt. This is because the pod log becomes the build log after the job artifact uploads are complete, which should be used instead of the pod log.

func (*PodLogArtifact) ReadAll 

func (a *PodLogArtifact) ReadAll() ([]byte, error)

ReadAll reads all available pod logs, failing if they are too large

func (*PodLogArtifact) ReadAt 

func (a *PodLogArtifact) ReadAt(p []byte, off int64) (n int, err error)

ReadAt implements reading a range of bytes from the pod logs endpoint

func (*PodLogArtifact) ReadAtMost 

func (a *PodLogArtifact) ReadAtMost(n int64) ([]byte, error)

ReadAtMost reads at most n bytes

func (*PodLogArtifact) ReadTail 

func (a *PodLogArtifact) ReadTail(n int64) ([]byte, error)

ReadTail reads the last n bytes of the pod log

func (*PodLogArtifact) Size 

func (a *PodLogArtifact) Size() (int64, error)

Size gets the size of the pod log. Note: this function makes the same network call as reading the entire file.

type PodLogArtifactFetcher 

type PodLogArtifactFetcher struct {
	// contains filtered or unexported fields
}

PodLogArtifactFetcher is used to fetch artifacts from k8s apiserver

func NewPodLogArtifactFetcher 

func NewPodLogArtifactFetcher(ja jobAgent) *PodLogArtifactFetcher

NewPodLogArtifactFetcher returns a PodLogArtifactFetcher using the given job agent as storage

type Spyglass 

type Spyglass struct {
	// JobAgent contains information about the current jobs in deck
	JobAgent *jobs.JobAgent

	// ConfigAgent contains information about the prow configuration
	ConfigAgent configAgent

	*GCSArtifactFetcher
	*PodLogArtifactFetcher
}

Spyglass records which sets of artifacts need views for a Prow job. The metaphor can be understood as follows: A spyglass receives light from a source through an eyepiece, which has a lens that ultimately presents a view of the light source to the observer. Spyglass receives light (artifacts) via a source (src) through the eyepiece (Eyepiece) and presents the view (what you see in your browser) via a lens (Lens).

func New 

func New(ja *jobs.JobAgent, conf configAgent, c *storage.Client) *Spyglass

New constructs a Spyglass object from a JobAgent, a config.Agent, and a storage Client.

func (*Spyglass) FetchArtifacts 

func (s *Spyglass) FetchArtifacts(src string, podName string, sizeLimit int64, artifactNames []string) ([]lenses.Artifact, error)

FetchArtifacts constructs and returns Artifact objects for each artifact name in the list. This includes getting any handles needed for read write operations, direct artifact links, etc.

func (*Spyglass) JobPath 

func (s *Spyglass) JobPath(src string) (string, error)

JobPath returns a link to the GCS directory for the job specified in src

func (*Spyglass) Lenses 

func (s *Spyglass) Lenses(matchCache map[string][]string) []lenses.Lens

Lenses gets all views of all artifact files matching each regexp with a registered lens

func (*Spyglass) ListArtifacts 

func (s *Spyglass) ListArtifacts(src string) ([]string, error)

ListArtifacts gets the names of all artifacts available from the given source

Source Files

View all Source files

Directories

Path Synopsis
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers
 Package lenses provides interfaces and methods necessary for implementing custom artifact viewers
buildlog
Package buildlog provides a build log viewer for Spyglass
 Package buildlog provides a build log viewer for Spyglass
junit
Package junit provides a junit viewer for Spyglass
 Package junit provides a junit viewer for Spyglass
metadata
Package metadata provides a metadata viewer for Spyglass
 Package metadata provides a metadata viewer for Spyglass

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.