libindex

Documentation

Index

• Constants
• type ControllerFactory
• type FetchProxy
  • func (p *FetchProxy) Close() error
  • func (p *FetchProxy) Realize(ctx context.Context, ls []*claircore.Layer) error
  • func (p *FetchProxy) RealizeDescriptions(ctx context.Context, descs []claircore.LayerDescription) ([]claircore.Layer, error)
• type Libindex
  • func New(ctx context.Context, opts *Options, cl *http.Client) (*Libindex, error)
  • func (l *Libindex) AffectedManifests(ctx context.Context, vulns []claircore.Vulnerability) (*claircore.AffectedManifests, error)
  • func (l *Libindex) Close(ctx context.Context) error
  • func (l *Libindex) DeleteManifests(ctx context.Context, d ...claircore.Digest) ([]claircore.Digest, error)
  • func (l *Libindex) Index(ctx context.Context, manifest *claircore.Manifest) (*claircore.IndexReport, error)
  • func (l *Libindex) IndexReport(ctx context.Context, hash claircore.Digest) (*claircore.IndexReport, bool, error)
  • func (l *Libindex) State(ctx context.Context) (string, error)
• type LockSource
• type Options
• type RemoteFetchArena
  • func NewRemoteFetchArena(wc *http.Client, root string) *RemoteFetchArena
  • func (a *RemoteFetchArena) Close(ctx context.Context) error
  • func (a *RemoteFetchArena) Realizer(_ context.Context) indexer.Realizer

Examples

• Libindex

Constants

const (
	DefaultScanLockRetry        = 5 * time.Second
	DefaultLayerScanConcurrency = 10
)

Types

type ControllerFactory

type ControllerFactory func(opts *indexer.Options) *controller.Controller

ControllerFactory is a factory method to return a Controller during libindex runtime.

type FetchProxy

type FetchProxy struct {
	// contains filtered or unexported fields
}

FetchProxy tracks the files fetched for layers.

func (*FetchProxy) Close

func (p *FetchProxy) Close() error

Close marks all the files backing any returned claircore.Layer as unused.

This method may delete the backing files, necessitating them being fetched by a subsequent call to FetchProxy.RealizeDescriptions.

func (*FetchProxy) Realize

func (p *FetchProxy) Realize(ctx context.Context, ls []*claircore.Layer) error

Realize populates all the layers locally.

Deprecated: This method proxies to FetchProxy.RealizeDescriptions via copies and a (potentially expensive) comparison operation. Callers should use FetchProxy.RealizeDescriptions if they already have the claircore.LayerDescription constructed.

func (*FetchProxy) RealizeDescriptions

func (p *FetchProxy) RealizeDescriptions(ctx context.Context, descs []claircore.LayerDescription) ([]claircore.Layer, error)

RealizeDesciptions returns claircore.Layer structs populated according to the passed slice of claircore.LayerDescription.

type Libindex

type Libindex struct {
	// holds dependencies for creating a libindex instance
	*Options
	// contains filtered or unexported fields
}

Libindex implements the method set for scanning and indexing a Manifest.

Example

package main

import (
	"context"
	"net/http"
	"os"

	"github.com/quay/claircore"
	"github.com/quay/claircore/datastore/postgres"
	"github.com/quay/claircore/libindex"
	"github.com/quay/claircore/pkg/ctxlock"
)

func main() {
	ctx := context.TODO()
	pool, err := postgres.Connect(ctx, "connection string", "libindex-test")
	if err != nil {
		panic(err)
	}

	store, err := postgres.InitPostgresIndexerStore(ctx, pool, true)
	if err != nil {
		panic(err)
	}

	ctxLocker, err := ctxlock.New(ctx, pool)
	if err != nil {
		panic(err)
	}

	opts := &libindex.Options{
		Store:      store,
		Locker:     ctxLocker,
		FetchArena: libindex.NewRemoteFetchArena(http.DefaultClient, os.TempDir()),
		// see definition for more configuration options
	}
	lib, err := libindex.New(ctx, opts, http.DefaultClient)
	if err != nil {
		panic(err)
	}
	m := &claircore.Manifest{}

	ir, err := lib.Index(ctx, m)
	if err != nil {
		panic(err)
	}
	if ir.State == "IndexError" {
		panic(ir.Err)
	}
}

func New

func New(ctx context.Context, opts *Options, cl *http.Client) (*Libindex, error)

New creates a new instance of libindex.

The passed http.Client will be used for fetching layers and any HTTP requests made by scanners.

func (*Libindex) AffectedManifests

func (l *Libindex) AffectedManifests(ctx context.Context, vulns []claircore.Vulnerability) (*claircore.AffectedManifests, error)

AffectedManifests retrieves a list of affected manifests when provided a list of vulnerabilities.

func (*Libindex) Close

func (l *Libindex) Close(ctx context.Context) error

Close releases held resources.

func (*Libindex) DeleteManifests

func (l *Libindex) DeleteManifests(ctx context.Context, d ...claircore.Digest) ([]claircore.Digest, error)

DeleteManifests removes manifests specified by the provided digests.

Providing an unknown digest is not an error.

func (*Libindex) Index

func (l *Libindex) Index(ctx context.Context, manifest *claircore.Manifest) (*claircore.IndexReport, error)

Index performs a scan and index of each layer within the provided Manifest.

If the index operation cannot start an error will be returned. If an error occurs during scan the error will be propagated inside the IndexReport.

func (*Libindex) IndexReport

func (l *Libindex) IndexReport(ctx context.Context, hash claircore.Digest) (*claircore.IndexReport, bool, error)

IndexReport retrieves an IndexReport for a particular manifest hash, if it exists.

func (*Libindex) State

func (l *Libindex) State(ctx context.Context) (string, error)

State returns an opaque identifier identifying how the struct is currently configured.

If the identifier has changed, clients should arrange for layers to be re-indexed.

type LockSource

type LockSource interface {
	TryLock(context.Context, string) (context.Context, context.CancelFunc)
	Lock(context.Context, string) (context.Context, context.CancelFunc)
	Close(context.Context) error
}

LockSource abstracts over how locks are implemented.

An online system needs distributed locks, offline use cases can use process-local locks.

type Options

type Options struct {
	// Store is the interface used to persist and retrieve results of indexing.
	Store indexer.Store
	// Locker provides system-wide locks. If the indexing work is distributed the
	// lock should be backed by a distributed store.
	Locker LockSource
	// FetchArena is an interface tied to the lifecycle of LibIndex to enable management
	// of the filesystem while separate processes are dealing with layers, for example:
	// you can reference count downloaded layer files to avoid racing.
	FetchArena indexer.FetchArena
	// ScanLockRetry specifies how often we should try to acquire a lock for scanning a
	// given manifest if lock is taken.
	ScanLockRetry time.Duration
	// LayerScanConcurrency specifies the number of layers to be scanned in parallel.
	LayerScanConcurrency int
	// LayerFetchOpt is unused and kept here for backwards compatibility.
	LayerFetchOpt interface{}
	// NoLayerValidation controls whether layers are checked to actually be
	// content-addressed. With this option toggled off, callers can trigger
	// layers to be indexed repeatedly by changing the identifier in the
	// manifest.
	NoLayerValidation bool
	// ControllerFactory provides an alternative method for creating a scanner during libindex runtime
	// if nil the default factory will be used. useful for testing purposes
	ControllerFactory ControllerFactory
	// Ecosystems a list of ecosystems to use which define which package databases and coalescing methods we use
	Ecosystems []*indexer.Ecosystem
	// ScannerConfig holds functions that can be passed into configurable
	// scanners. They're broken out by kind, and only used if a scanner
	// implements the appropriate interface.
	//
	// Providing a function for a scanner that's not expecting it is not a fatal
	// error.
	ScannerConfig struct {
		Package, Dist, Repo, File map[string]func(interface{}) error
	}
	Resolvers []indexer.Resolver
}

Options are dependencies and options for constructing an instance of libindex

type RemoteFetchArena

type RemoteFetchArena struct {
	// contains filtered or unexported fields
}

RemoteFetchArena uses disk space to track fetched layers, removing them once all users are done with the layers.

func NewRemoteFetchArena

func NewRemoteFetchArena(wc *http.Client, root string) *RemoteFetchArena

NewRemoteFetchArena returns an initialized RemoteFetchArena.

If the "root" parameter is "", the advice in file-hierarchy(7) and "Using /tmp/ and /var/tmp/ Safely" is followed. Specifically, "/var/tmp" is used unless "TMPDIR" is set in the environment, in which case the contents of that variable are interpreted as a path and used.

The RemoteFetchArena attempts to use O_TMPFILE and falls back to os.CreateTemp if that seems to not work. If the filesystem backing "root" does not support O_TMPFILE, files may linger in the event of a process crashing or an unclean shutdown. Operators should either use a different filesystem or arrange for periodic cleanup via systemd-tmpfiles(8) or similar.

In a containerized environment, operators may need to mount a directory or filesystem on "/var/tmp".

On OSX, temporary files are not unlinked from the filesystem upon creation, because an equivalent to Linux's "/proc/self/fd" doesn't seem to exist.

On UNIX-unlike systems, none of the above logic takes place.

func (*RemoteFetchArena) Close

func (a *RemoteFetchArena) Close(ctx context.Context) error

Close forgets all references in the arena.

Any outstanding Layers may cause keys to be forgotten at unpredictable times.

func (*RemoteFetchArena) Realizer

func (a *RemoteFetchArena) Realizer(_ context.Context) indexer.Realizer

Realizer returns an indexer.Realizer.

The concrete return type is *FetchProxy.