libindex

package
v1.5.28 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 13, 2024 License: Apache-2.0 Imports: 41 Imported by: 4

Documentation

Index

Examples

Constants

View Source
const (
	DefaultScanLockRetry        = 5 * time.Second
	DefaultLayerScanConcurrency = 10
)

Variables

This section is empty.

Functions

This section is empty.

Types

type ControllerFactory

type ControllerFactory func(opts *indexer.Options) *controller.Controller

ControllerFactory is a factory method to return a Controller during libindex runtime.

type FetchProxy added in v1.1.0

type FetchProxy struct {
	// contains filtered or unexported fields
}

FetchProxy tracks the files fetched for layers.

func (*FetchProxy) Close added in v1.1.0

func (p *FetchProxy) Close() error

Close marks all the files backing any returned claircore.Layer as unused.

This method may delete the backing files, necessitating them being fetched by a subsequent call to FetchProxy.RealizeDescriptions.

func (*FetchProxy) Realize deprecated added in v1.4.5

func (p *FetchProxy) Realize(ctx context.Context, ls []*claircore.Layer) error

Realize populates all the layers locally.

Deprecated: This method proxies to FetchProxy.RealizeDescriptions via copies and a (potentially expensive) comparison operation. Callers should use FetchProxy.RealizeDescriptions if they already have the claircore.LayerDescription constructed.

func (*FetchProxy) RealizeDescriptions added in v1.5.20

func (p *FetchProxy) RealizeDescriptions(ctx context.Context, descs []claircore.LayerDescription) ([]claircore.Layer, error)

RealizeDesciptions returns claircore.Layer structs populated according to the passed slice of claircore.LayerDescription.

type Libindex

type Libindex struct {
	// holds dependencies for creating a libindex instance
	*Options
	// contains filtered or unexported fields
}

Libindex implements the method set for scanning and indexing a Manifest.

Example
package main

import (
	"context"
	"net/http"
	"os"

	"github.com/quay/claircore"
	"github.com/quay/claircore/datastore/postgres"
	"github.com/quay/claircore/libindex"
	"github.com/quay/claircore/pkg/ctxlock"
)

func main() {
	ctx := context.TODO()
	pool, err := postgres.Connect(ctx, "connection string", "libindex-test")
	if err != nil {
		panic(err)
	}

	store, err := postgres.InitPostgresIndexerStore(ctx, pool, true)
	if err != nil {
		panic(err)
	}

	ctxLocker, err := ctxlock.New(ctx, pool)
	if err != nil {
		panic(err)
	}

	opts := &libindex.Options{
		Store:      store,
		Locker:     ctxLocker,
		FetchArena: libindex.NewRemoteFetchArena(http.DefaultClient, os.TempDir()),
		// see definition for more configuration options
	}
	lib, err := libindex.New(ctx, opts, http.DefaultClient)
	if err != nil {
		panic(err)
	}
	m := &claircore.Manifest{}

	ir, err := lib.Index(ctx, m)
	if err != nil {
		panic(err)
	}
	if ir.State == "IndexError" {
		panic(ir.Err)
	}
}
Output:

func New

func New(ctx context.Context, opts *Options, cl *http.Client) (*Libindex, error)

New creates a new instance of libindex.

The passed http.Client will be used for fetching layers and any HTTP requests made by scanners.

func (*Libindex) AffectedManifests added in v0.0.24

func (l *Libindex) AffectedManifests(ctx context.Context, vulns []claircore.Vulnerability) (*claircore.AffectedManifests, error)

AffectedManifests retrieves a list of affected manifests when provided a list of vulnerabilities.

func (*Libindex) Close added in v0.0.16

func (l *Libindex) Close(ctx context.Context) error

Close releases held resources.

func (*Libindex) DeleteManifests added in v1.2.0

func (l *Libindex) DeleteManifests(ctx context.Context, d ...claircore.Digest) ([]claircore.Digest, error)

DeleteManifests removes manifests specified by the provided digests.

Providing an unknown digest is not an error.

func (*Libindex) Index

func (l *Libindex) Index(ctx context.Context, manifest *claircore.Manifest) (*claircore.IndexReport, error)

Index performs a scan and index of each layer within the provided Manifest.

If the index operation cannot start an error will be returned. If an error occurs during scan the error will be propagated inside the IndexReport.

func (*Libindex) IndexReport

func (l *Libindex) IndexReport(ctx context.Context, hash claircore.Digest) (*claircore.IndexReport, bool, error)

IndexReport retrieves an IndexReport for a particular manifest hash, if it exists.

func (*Libindex) State added in v0.0.10

func (l *Libindex) State(ctx context.Context) (string, error)

State returns an opaque identifier identifying how the struct is currently configured.

If the identifier has changed, clients should arrange for layers to be re-indexed.

type LockSource added in v1.4.5

LockSource abstracts over how locks are implemented.

An online system needs distributed locks, offline use cases can use process-local locks.

type Options added in v1.4.5

type Options struct {
	// Store is the interface used to persist and retrieve results of indexing.
	Store indexer.Store
	// Locker provides system-wide locks. If the indexing work is distributed the
	// lock should be backed by a distributed store.
	Locker LockSource
	// FetchArena is an interface tied to the lifecycle of LibIndex to enable management
	// of the filesystem while separate processes are dealing with layers, for example:
	// you can reference count downloaded layer files to avoid racing.
	FetchArena indexer.FetchArena
	// ScanLockRetry specifies how often we should try to acquire a lock for scanning a
	// given manifest if lock is taken.
	ScanLockRetry time.Duration
	// LayerScanConcurrency specifies the number of layers to be scanned in parallel.
	LayerScanConcurrency int
	// LayerFetchOpt is unused and kept here for backwards compatibility.
	LayerFetchOpt interface{}
	// NoLayerValidation controls whether layers are checked to actually be
	// content-addressed. With this option toggled off, callers can trigger
	// layers to be indexed repeatedly by changing the identifier in the
	// manifest.
	NoLayerValidation bool
	// ControllerFactory provides an alternative method for creating a scanner during libindex runtime
	// if nil the default factory will be used. useful for testing purposes
	ControllerFactory ControllerFactory
	// Ecosystems a list of ecosystems to use which define which package databases and coalescing methods we use
	Ecosystems []*indexer.Ecosystem
	// ScannerConfig holds functions that can be passed into configurable
	// scanners. They're broken out by kind, and only used if a scanner
	// implements the appropriate interface.
	//
	// Providing a function for a scanner that's not expecting it is not a fatal
	// error.
	ScannerConfig struct {
		Package, Dist, Repo, File map[string]func(interface{}) error
	}
	Resolvers []indexer.Resolver
}

Options are dependencies and options for constructing an instance of libindex

type RemoteFetchArena added in v1.4.5

type RemoteFetchArena struct {
	// contains filtered or unexported fields
}

RemoteFetchArena uses disk space to track fetched layers, removing them once all users are done with the layers.

func NewRemoteFetchArena added in v1.4.5

func NewRemoteFetchArena(wc *http.Client, root string) *RemoteFetchArena

NewRemoteFetchArena returns an initialized RemoteFetchArena.

If the "root" parameter is "", the advice in file-hierarchy(7) and "Using /tmp/ and /var/tmp/ Safely" is followed. Specifically, "/var/tmp" is used unless "TMPDIR" is set in the environment, in which case the contents of that variable are interpreted as a path and used.

The RemoteFetchArena attempts to use O_TMPFILE and falls back to os.CreateTemp if that seems to not work. If the filesystem backing "root" does not support O_TMPFILE, files may linger in the event of a process crashing or an unclean shutdown. Operators should either use a different filesystem or arrange for periodic cleanup via systemd-tmpfiles(8) or similar.

In a containerized environment, operators may need to mount a directory or filesystem on "/var/tmp".

On OSX, temporary files are not unlinked from the filesystem upon creation, because an equivalent to Linux's "/proc/self/fd" doesn't seem to exist.

On UNIX-unlike systems, none of the above logic takes place.

func (*RemoteFetchArena) Close added in v1.4.5

func (a *RemoteFetchArena) Close(ctx context.Context) error

Close forgets all references in the arena.

Any outstanding Layers may cause keys to be forgotten at unpredictable times.

func (*RemoteFetchArena) Realizer added in v1.4.5

Realizer returns an indexer.Realizer.

The concrete return type is *FetchProxy.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL