visualizer

README

Jenkins X Pipelines Visualizer

This is a Web UI for Jenkins X, with a clear goal: visualize the pipelines - and their logs.

Features

• View the pipelines information: metadata, status, stages and steps with timing
• View the pipelines logs in real-time
• Retrieve the archived pipelines logs from the long-term storage (GCS, S3, ...)
• View all pipelines with their status, and filter/sort them
• Read-only: only requires READ permissions on the Jenkins X and Tekton Pipelines CRDs
• Expose a Shields.io compatible endpoint
• Backward-compatible URLs with the old "JX UI" - so that you can easily swap the JXUI URL for the jx-pipelines-visualizer one in the Lighthouse config, and have Lighthouse set links to jx-pipelines-visualizer in GitHub Pull Requests.
• Work in context of a single namespace or in a cluster context

Screenshots

You can also see the announcement blog post for more details and a demo.

Out of scope

There are a number of features we don't want to include in this project - at least for the moment:

• Everything Auth-related
  • use a reverse-proxy in front or anything else to handle it
    • for example Vouch and Okta
    • or dex, oauth2-proxy, ...
    • or nginx basic-auth - if you are using the nginx ingress controller
• Create/Update/Delete operations
  • it is meant to be a read-only web UI
  • you can use the Octant-based UI for these use-cases.
• Anything in Jenkins X which is not related to the pipelines
  • such as managing repositories, environments, and so on.
  • you can use the Octant-based UI for these use-cases.

Installation

With Jenkins X v3

It's already installed by default with Jenkins X v3.

By default an ingress is created to access the UI using basic authentication. See the documentation for how to access it

You can see the default values here: https://github.com/jenkins-x/jx3-versions/tree/master/charts/jx3/jx-pipelines-visualizer

With Jenkins X v2

In the Git repository for your dev environment:

• Update the env/requirements.yaml file with the following:

  - name: jx-pipelines-visualizer
    repository: https://jenkins-x-charts.github.io/repo
    version: 1.7.2
  

• Create a new file env/jx-pipelines-visualizer/values.tmpl.yaml with the following content:

  {{- if .Requirements.storage.logs.enabled }}
  config:
    archivedLogsURLTemplate: >-
      {{ .Requirements.storage.logs.url }}{{`/jenkins-x/logs/{{.Owner}}/{{.Repository}}/{{if hasPrefix .Branch "pr"}}{{.Branch | upper}}{{else}}{{.Branch}}{{end}}/{{.Build}}.log`}}
  {{- end }}
  
  gitSecretName: ""
  
  ingress:
    enabled: true
    hosts:
      - pipelines{{.Requirements.ingress.namespaceSubDomain}}{{.Requirements.ingress.domain}}
    {{- if .Requirements.ingress.tls.enabled }}
    tls:
      enabled: true
      secrets:
        # re-use the existing tls secret managed by jx
        {{- if .Requirements.ingress.tls.production }}
        tls-{{ .Requirements.ingress.domain | replace "." "-" }}-p: {}
        {{- else }}
        tls-{{ .Requirements.ingress.domain | replace "." "-" }}-s: {}
        {{- end }}
    {{- end }}
    annotations:
      kubernetes.io/ingress.class: nginx
  

  This will expose the UI at pipelines.your.domain.tld - without any auth. You can add basic auth by appending a few additional annotations - re-using the Jenkins X Auth Secret:

  nginx.ingress.kubernetes.io/auth-type: basic
  nginx.ingress.kubernetes.io/auth-secret: jx-basic-auth
  

• If you want Lighthouse to add links to your jx-pipelines-visualizer instance from your Pull/Merge Request checks, update the env/lighthouse-jx/values.tmpl.yaml file and add the following:

  env:
    LIGHTHOUSE_REPORT_URL_BASE: "https://pipelines{{.Requirements.ingress.namespaceSubDomain}}{{.Requirements.ingress.domain}}"
  

With Helm v3

$ helm repo add jx https://jenkins-x-charts.github.io/repo
$ helm install jx-pipelines-visualizer jx/jx-pipelines-Visualizer

With Helm v2

$ helm repo add jx https://jenkins-x-charts.github.io/repo
$ helm repo update
$ helm install --name jx-pipelines-visualizer jx/jx-pipelines-visualizer

Usage

Just go to the homepage, and use the links to view the pipelines logs.

To generate a status badge compatible with shields.io:

• read the shields.io documentation
• the custom endpoint is: https://YOUR_HOST/{owner}/{repo}/{branch}/shields.io - for example https://jx.example.com/my-org/my-repo/master/shields.io. It returns a JSON response with the status of the latest build for the given branch.

Configuration

See the values.yaml file for the configuration.

If you are not using the Helm Chart, the binary is using CLI flags only - no config files. You can run jx-pipelines-visualizer -h to see all the flags.

Running locally

go run cmd/server/main.go

How It Works

It uses the "informer" Kubernetes pattern to keep a local cache of the Jenkins X PipelineActivities, and index them in an in-memory Bleve index.

It uses part of jx code to retrieve the build logs - mainly the part to stream the build logs from the running pods. It is the same code used by the jx get build logs command.

Credits

Thanks to Dailymotion for creating the original repository and then donate it to the Jenkins X project.

Documentation

Index

• type Informer
  • func (i *Informer) Start(ctx context.Context)
• type Pipeline
  • func PipelineFromPipelineActivity(pa *jenkinsv1.PipelineActivity) Pipeline
  • func (p Pipeline) PullRequestNumber() string
• type Pipelines
• type Query
  • func (q Query) ToBleveQuery() query.Query
• type RunningPipeline
  • func RunningPipelinesFromPipelineActivity(pa *jenkinsv1.PipelineActivity) []RunningPipeline
  • func (running RunningPipeline) JSON() string
  • func (running RunningPipeline) String() string
• type RunningPipelines
  • func (pipelines *RunningPipelines) Add(pa *jenkinsv1.PipelineActivity)
  • func (pipelines *RunningPipelines) Get() []RunningPipeline
  • func (pipelines *RunningPipelines) Register(watcher Watcher)
  • func (pipelines *RunningPipelines) UnRegister(watcher Watcher)
• type Store
  • func NewStore() (*Store, error)
  • func (s *Store) Add(p Pipeline) error
  • func (s *Store) All() (*Pipelines, error)
  • func (s *Store) Delete(name string) error
  • func (s *Store) Query(q Query) (*Pipelines, error)
• type Watcher

Types

type Informer

type Informer struct {
	JXClient         *jxclientset.Clientset
	Namespace        string
	ResyncInterval   time.Duration
	Store            *Store
	RunningPipelines *RunningPipelines
	Logger           *logrus.Logger
}

func (*Informer) Start

func (i *Informer) Start(ctx context.Context)

type Pipeline

type Pipeline struct {
	Name            string
	Namespace       string
	Provider        string
	Owner           string
	Repository      string
	Branch          string
	Build           string
	Context         string
	Author          string
	AuthorAvatarURL string
	Commit          string
	Status          string
	Description     string
	Start           time.Time
	End             time.Time
	Duration        time.Duration
	GitUrl          string
}

func PipelineFromPipelineActivity

func PipelineFromPipelineActivity(pa *jenkinsv1.PipelineActivity) Pipeline

func (Pipeline) PullRequestNumber

func (p Pipeline) PullRequestNumber() string

type Pipelines

type Pipelines struct {
	Pipelines []Pipeline
	Counts    struct {
		Statuses     map[string]int
		Repositories map[string]int
		Authors      map[string]int
		Durations    map[string]int
	}
}

type Query

type Query struct {
	Owner      string
	Repository string
	Branch     string
	Query      string
}

func (Query) ToBleveQuery

func (q Query) ToBleveQuery() query.Query

type RunningPipeline

type RunningPipeline struct {
	Pipeline
	Stage          string
	StageStartTime time.Time
	Step           string
	StepStartTime  time.Time
}

func RunningPipelinesFromPipelineActivity

func RunningPipelinesFromPipelineActivity(pa *jenkinsv1.PipelineActivity) []RunningPipeline

func (RunningPipeline) JSON

func (running RunningPipeline) JSON() string

func (RunningPipeline) String

func (running RunningPipeline) String() string

type RunningPipelines

type RunningPipelines struct {
	Logger *logrus.Logger
	// contains filtered or unexported fields
}

func (*RunningPipelines) Add

func (pipelines *RunningPipelines) Add(pa *jenkinsv1.PipelineActivity)

func (*RunningPipelines) Get

func (pipelines *RunningPipelines) Get() []RunningPipeline

func (*RunningPipelines) Register

func (pipelines *RunningPipelines) Register(watcher Watcher)

func (*RunningPipelines) UnRegister

func (pipelines *RunningPipelines) UnRegister(watcher Watcher)

type Store

type Store struct {
	Index bleve.Index
}

func NewStore

func NewStore() (*Store, error)

func (*Store) Add

func (s *Store) Add(p Pipeline) error

func (*Store) All

func (s *Store) All() (*Pipelines, error)

func (*Store) Delete

func (s *Store) Delete(name string) error

func (*Store) Query

func (s *Store) Query(q Query) (*Pipelines, error)

type Watcher

type Watcher struct {
	Name    string
	Added   chan RunningPipeline
	Deleted chan RunningPipeline
}