visualizer

README

Jenkins X Pipelines Visualizer

This is a Web UI for Jenkins X, with a clear goal: visualize the pipelines - and their logs.

See the announcement blog post for more details and a demo.

Current Status

This project has been started in September 2020 and shared after a couple of hours of work. It's working - and is deployed and used at Dailymotion - even if the UI is very basic for the moment.

Note that it is being used on GKE with logs stored in a GCS bucket.

Roadmap

• Improve the UI: visualize the pipeline (stages/steps), group logs per stage/step, and so on
• Split the API and the UI, to make it easier to iterate on the UI

Features

• very simple
• very fast: get your logs in milliseconds, not seconds. Yes, I'm looking at you, JXUI.
• either retrieve the build logs from the persistent storage (tested with GCS), or stream them from the running pods if the pipeline is still running.
• retrieve the build logs even for the garbage-collected pipelines (the JXUI just returns 404)
• read-only. Only requires READ permissions on the JX and Tekton Pipelines CRDs
• URLs backward-compatible with JXUI - so that you can easily swap the JXUI URL for the jx-pipelines-visualizer one in the Lighthouse config, and have Lighthouse set links to jx-pipelines-visualizer in GitHub Pull Requests.

Out of scope

• Auth - use a reverse-proxy in front or anything else to handle it
  • for example Vouch and Okta
  • or dex, oauth2-proxy, ...
• Create/Update/Delete operations. It is meant to be a read-only web UI
• Anything in JX which is not related to the pipelines

Installation

Coming soon...

Configuration

See the values.yaml file for the configuration.

If you are not using the Helm Chart, the binary is using CLI flags only - no config files. You can run jx-pipelines-visualizer -h to see all the flags.

Running locally

go run cmd/server/main.go

How It Works

It uses the "informer" Kubernetes pattern to keep a local cache of the Jenkins X PipelineActivities, and index them in an in-memory Bleve index.

It uses part of jx code to retrieve the build logs - mainly the part to stream the build logs from the running pods. It is the same code used by the jx get build logs command.

Credits

Thanks to Dailymotion for creating the original repository and then donate it to the Jenkins X project.

Documentation

Index

• func GetContext(pa *jenkinsv1.PipelineActivity) (string, bool)
• type Informer
  • func (i *Informer) Start(ctx context.Context)
• type Pipeline
  • func PipelineFromPipelineActivity(pa *jenkinsv1.PipelineActivity) Pipeline
• type Pipelines
• type Query
  • func (q Query) ToBleveQuery() query.Query
• type Store
  • func NewStore() (*Store, error)
  • func (s *Store) Add(p Pipeline) error
  • func (s *Store) All() (*Pipelines, error)
  • func (s *Store) Delete(name string) error
  • func (s *Store) Query(q Query) (*Pipelines, error)

Functions

func GetContext

func GetContext(pa *jenkinsv1.PipelineActivity) (string, bool)

GetContext returns the pipeline context using the v2 or v3 labels

Types

type Informer

type Informer struct {
	JXClient       *jxclientset.Clientset
	Namespace      string
	ResyncInterval time.Duration
	Store          *Store
	Logger         *logrus.Logger
}

func (*Informer) Start

func (i *Informer) Start(ctx context.Context)

type Pipeline

type Pipeline struct {
	Name            string
	Owner           string
	Repository      string
	Branch          string
	Build           string
	Context         string
	Author          string
	AuthorAvatarURL string
	Commit          string
	Status          string
	Start           time.Time
	End             time.Time
	Duration        time.Duration
}

func PipelineFromPipelineActivity

func PipelineFromPipelineActivity(pa *jenkinsv1.PipelineActivity) Pipeline

type Pipelines

type Pipelines struct {
	Pipelines []Pipeline
	Counts    struct {
		Owners       map[string]int
		Repositories map[string]int
		Branches     map[string]int
		Builds       map[string]int
	}
}

type Query

type Query struct {
	Owner      string
	Repository string
	Branch     string
}

func (Query) ToBleveQuery

func (q Query) ToBleveQuery() query.Query

type Store

type Store struct {
	Index bleve.Index
}

func NewStore

func NewStore() (*Store, error)

func (*Store) Add

func (s *Store) Add(p Pipeline) error

func (*Store) All

func (s *Store) All() (*Pipelines, error)

func (*Store) Delete

func (s *Store) Delete(name string) error

func (*Store) Query

func (s *Store) Query(q Query) (*Pipelines, error)