claircore

package module
v0.1.24 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 25, 2021 License: Apache-2.0 Imports: 19 Imported by: 24

README

ClairCore

ClairCore provides a set of go modules which handle scanning container layers for installed packages and reporting any discovered vulnerabilities.
ClairCore is designed to be embedded into a service wrapper.

For a full overview see: ClairCore Book

Usage

Two packages exist libindex and libvuln.
These packages export the methods for indexing an image's contents and matching the results of the index to vulnerabilities respectively.

libindex usage

Creating an instance

opts := &libindex.Opts{
    ConnString: "postgres://host:port",
    Migrations: true,
    // see definition for more configuration options
}
lib := libindex.New(opts)

call libindex with a populated Manifest

m := &claircore.Manifest{
    ...
}

ir, err := lib.Index(m)
if err != nil {
    log.Printf("%v", err)
}
if ir.State == "IndexError" {
    log.Printf("scan failed: %s", sr.Err)
}

libvuln usage

creating an instance

opts := &libvuln.Opts{
    ConnString: "postgres://host:port",
    Migrations: true,
    // see definition for more configuration option
}
lib := libvuln.New(opts)

call libvuln with a populated IndexReport

ir := &claircore.IndexReport{
    ...
}
vr, err := libvuln.Scan(ir)
if err != nil {
    log.Printf("%v", err)
}

Libvuln will first initialize all updaters before returning from its constructor.
Controlling how many updaters initialize in parallel is provided via the libvuln.Opts struct

To further understand how these packages work together see:
Highlevel Architecture
Vulnerability Matching

Local development and testing

The following targets start and stop a local development environment

make local-dev-up
make local-dev-down

If you modify libvuln or libindex code the following make targets will restart the services with your changes

make libindexhttp-restart
make libvulnhttp-restart

With the local development environment up the following make target runs all tests including integration

make integration

The following make target runs unit tests which do not require a database or local development environment

make unit

Documentation

Overview

Package claircore has foundational types for the claircore module.

Additional documentation can be found at http://quay.github.io/claircore/

Index

Constants

View Source 
const (
	SHA256 = "sha256"
	SHA512 = "sha512"
)
View Source 
const (
	BINARY = "binary"
	SOURCE = "source"
)

Variables

View Source 
var ErrNotFound = errors.New("claircore: unable to find any requested files")

ErrNotFound is returned by Layer.Files if none of the requested files are found.

Functions

func VersionSort added in v0.0.16 

func VersionSort(vs []Version) func(int, int) bool

VersionSort returns a function suitable for passing to sort.Slice or sort.SliceStable.

Types

type AffectedManifests added in v0.0.25 

type AffectedManifests struct {

	// map of vulnerabilities keyed by the vulnerability's ID
	Vulnerabilities map[string]*Vulnerability `json:"vulnerabilities"`
	// map associating a list of vulnerability ids keyed by the
	// manifest hash they affect.
	VulnerableManifests map[string][]string `json:"vulnerable_manifests"`
	// contains filtered or unexported fields
}

AffectedManifests describes a set of manifests affected by a set of Vulnerabilities.

func NewAffectedManifests added in v0.0.25 

func NewAffectedManifests() AffectedManifests

NewAffectedManifests initializes a new AffectedManifests struct.

func (*AffectedManifests) Add added in v0.0.25 

func (a *AffectedManifests) Add(v *Vulnerability, digests ...Digest)

Add will add the provided Vulnerability and Manifest digest to the necessary maps.

Add is safe to use by multiple goroutines.

func (*AffectedManifests) Sort added in v0.0.25 

func (a *AffectedManifests) Sort()

Sort will sort each array in the VulnerableManifests map by Vulnerability.NormalizedSeverity in Desc order.

Sort is safe to use by multiple goroutines.

type ArchOp added in v0.1.0 

type ArchOp uint
const (
	OpEquals       ArchOp // equals
	OpNotEquals           // not equals
	OpPatternMatch        // pattern match
)

func (ArchOp) Cmp added in v0.1.0 

func (o ArchOp) Cmp(a, b string) bool

func (ArchOp) MarshalText added in v0.1.0 

func (o ArchOp) MarshalText() (text []byte, err error)

func (*ArchOp) Scan added in v0.1.0 

func (o *ArchOp) Scan(i interface{}) error

func (ArchOp) String added in v0.1.0 

func (i ArchOp) String() string

func (*ArchOp) UnmarshalText added in v0.1.0 

func (o *ArchOp) UnmarshalText(text []byte) error

func (ArchOp) Value added in v0.1.0 

func (o ArchOp) Value() (driver.Value, error)

type Digest added in v0.0.13 

type Digest struct {
	// contains filtered or unexported fields
}

Digest is a type representing the hash of some data.

It's used throughout claircore packages as an attempt to remain independent of a specific hashing algorithm.

func MustParseDigest added in v0.0.19 

func MustParseDigest(digest string) Digest

MustParseDigest works like ParseDigest but panics if the provided string is not well-formed.

func NewDigest added in v0.0.13 

func NewDigest(algo string, sum []byte) (Digest, error)

NewDigest constructs a Digest.

func ParseDigest added in v0.0.13 

func ParseDigest(digest string) (Digest, error)

ParseDigest constructs a Digest from a string, ensuring it's well-formed.

func (Digest) Algorithm added in v0.0.13 

func (d Digest) Algorithm() string

Algorithm returns a string representation of the algorithm used for this digest.

func (Digest) Checksum added in v0.0.13 

func (d Digest) Checksum() []byte

Checksum returns the checksum byte slice.

func (Digest) Hash added in v0.0.14 

func (d Digest) Hash() hash.Hash

Hash returns an instance of the hashing algorithm used for this Digest.

func (Digest) MarshalText added in v0.0.13 

func (d Digest) MarshalText() ([]byte, error)

MarshalText implements encoding.TextMarshaler.

func (*Digest) Scan added in v0.0.13 

func (d *Digest) Scan(i interface{}) error

Scan implements sql.Scanner.

func (Digest) String added in v0.0.13 

func (d Digest) String() string

func (*Digest) UnmarshalText added in v0.0.13 

func (d *Digest) UnmarshalText(t []byte) error

UnmarshalText implements encoding.TextUnmarshaler.

func (Digest) Value added in v0.0.13 

func (d Digest) Value() (driver.Value, error)

Value implements driver.Valuer.

type DigestError added in v0.0.14 

type DigestError struct {
	// contains filtered or unexported fields
}

DigestError is the concrete type backing errors returned from Digest's methods.

func (*DigestError) Error added in v0.0.14 

func (e *DigestError) Error() string

Error implements error.

func (*DigestError) Unwrap added in v0.0.14 

func (e *DigestError) Unwrap() error

Unwrap enables errors.Unwrap.

type Distribution 

type Distribution struct {
	// unique ID of this distribution. this will be created as discovered by the library
	// and used for persistence and hash map indexes.
	ID string `json:"id"`
	// A lower-case string (no spaces or other characters outside of 0–9, a–z, ".", "_" and "-") identifying the operating system, excluding any version information
	// and suitable for processing by scripts or usage in generated filenames. Example: "DID=fedora" or "DID=debian".
	DID string `json:"did"`
	// A string identifying the operating system.
	// example: "Ubuntu"
	Name string `json:"name"`
	// A string identifying the operating system version, excluding any OS name information,
	// possibly including a release code name, and suitable for presentation to the user.
	// example: "16.04.6 LTS (Xenial Xerus)"
	Version string `json:"version"`
	// A lower-case string (no spaces or other characters outside of 0–9, a–z, ".", "_" and "-") identifying the operating system release code name,
	// excluding any OS name information or release version, and suitable for processing by scripts or usage in generated filenames
	// example: "xenial"
	VersionCodeName string `json:"version_code_name"`
	// A lower-case string (mostly numeric, no spaces or other characters outside of 0–9, a–z, ".", "_" and "-")
	// identifying the operating system version, excluding any OS name information or release code name,
	// example: "16.04"
	VersionID string `json:"version_id"`
	// A string identifying the OS architecture
	// example: "x86_64"
	Arch string `json:"arch"`
	// Optional common platform enumeration identifier
	CPE cpe.WFN `json:"cpe"`
	// A pretty operating system name in a format suitable for presentation to the user.
	// May or may not contain a release code name or OS version of some kind, as suitable. If not set, defaults to "PRETTY_NAME="Linux"".
	// example: "PRETTY_NAME="Fedora 17 (Beefy Miracle)"".
	PrettyName string `json:"pretty_name"`
}

Distribution is the accompanying system context of a package. this information aides in CVE detection.

Distribution is modeled after the os-release file found in all linux distributions.

type Environment added in v0.0.10 

type Environment struct {
	// the package database the associated package was discovered in
	PackageDB string `json:"package_db"`
	// the layer in which the associated package was introduced
	IntroducedIn Digest `json:"introduced_in"`
	// the ID of the distribution the package was discovered on
	DistributionID string `json:"distribution_id"`
	// the ID of the repository where this package was downloaded from (currently not used)
	RepositoryIDs []string `json:"repository_ids"`
}

Environment describes the surrounding environment a package was discovered in.

Environment must be accompanied by a parent structure which maps IDs to data models in order to have meaning. In our case this is IndexReport or VulnerabilityReport.

type IndexRecord added in v0.0.6 

type IndexRecord struct {
	Package      *Package
	Distribution *Distribution
	Repository   *Repository
}

IndexRecord is an entry in the IndexReport.

IndexRecords provide full access to contextual package structures such as Distribution and Repository.

A list of these can be thought of as an "unpacked" IndexReport

type IndexReport added in v0.0.6 

type IndexReport struct {
	// the manifest hash this IndexReport is describing
	Hash Digest `json:"manifest_hash"`
	// the current state of the index operation
	State string `json:"state"`
	// all discovered packages in this manifest key'd by package id
	Packages map[string]*Package `json:"packages"`
	// all discovered distributions in this manifest key'd by distribution id
	Distributions map[string]*Distribution `json:"distributions"`
	// all discovered repositories in this manifest key'd by repository id
	Repositories map[string]*Repository `json:"repository"`
	// a list of environment details a package was discovered in key'd by package id
	Environments map[string][]*Environment `json:"environments"`
	// whether the index operation finished successfully
	Success bool `json:"success"`
	// an error string in the case the index did not succeed
	Err string `json:"err"`
}

IndexReport provides a database for discovered artifacts in an image.

IndexReports make heavy usage of lookup maps to associate information without repetition.

func (*IndexReport) IndexRecords added in v0.0.6 

func (report *IndexReport) IndexRecords() []*IndexRecord

IndexRecords returns a list of IndexRecords derived from the IndexReport

type Layer 

type Layer struct {
	// Hash is a content addressable hash uniqely identifying this layer.
	// Libindex will treat layers with this same hash as identical.
	Hash    Digest              `json:"hash"`
	URI     string              `json:"uri"`
	Headers map[string][]string `json:"headers"`
	// contains filtered or unexported fields
}

Layer is a container image filesystem layer. Layers are stacked on top of each other to comprise the final filesystem of the container image.

func (*Layer) Fetched added in v0.0.13 

func (l *Layer) Fetched() bool

func (*Layer) Files 

func (l *Layer) Files(paths ...string) (map[string]*bytes.Buffer, error)

Files retrieves specific files from the layer's tar archive.

An error is returned only if none of the requested files are found.

The returned map may contain more entries than the number of paths requested. All entries in the map are keyed by paths that are relative to the tar-root. For example, requesting paths of "/etc/os-release", "./etc/os-release", and "etc/os-release" will all result in any found content being stored with the key "etc/os-release".

func (*Layer) Reader 

func (l *Layer) Reader() (io.ReadCloser, error)

Reader returns a ReadCloser of the layer.

It should also implement io.Seeker, and should be a tar stream.

func (*Layer) SetLocal added in v0.0.13 

func (l *Layer) SetLocal(f string) error

type Manifest 

type Manifest struct {
	// content addressable hash. should be able to be computed via
	// the hashes of all included layers
	Hash Digest `json:"hash"`
	// an array of filesystem layers indexed in the same order as the cooresponding image
	Layers []*Layer `json:"layers"`
}

Manifest represents a docker image. Layers array MUST be indexed in the order that image layers are stacked.

type Package 

type Package struct {
	// unique ID of this package. this will be created as discovered by the library
	// and used for persistence and hash map indexes
	ID string `json:"id"`
	// the name of the package
	Name string `json:"name"`
	// the version of the package
	Version string `json:"version"`
	// type of package. currently expectations are binary or source
	Kind string `json:"kind,omitempty"`
	// if type is a binary package a source package maybe present which built this binary package.
	// must be a pointer to support recursive type:
	Source *Package `json:"source,omitempty"`
	// the file system path or prefix where this package resides
	PackageDB string `json:"-"`
	// a hint on which repository this package was downloaded from
	RepositoryHint string `json:"-"`
	// NormalizedVersion is a representation of a version string that's
	// correctly ordered when compared with other representations from the same
	// producer.
	NormalizedVersion Version `json:"normalized_version,omitempty"`
	// Module and stream which this package is part of
	Module string `json:"module,omitempty"`
	// Package architecture
	Arch string `json:"arch,omitempty"`
	// CPE name for package
	CPE cpe.WFN `json:"cpe,omitempty"`
}

type Range added in v0.0.16 

type Range struct {
	Lower Version `json:"["`
	Upper Version `json:")"`
}

Range is a half-open interval of two Versions.

In the usual notation, it is: [Lower, Upper)

func (*Range) Contains added in v0.0.16 

func (r *Range) Contains(v *Version) bool

Contains reports whether the Version falls within the Range.

type Repository 

type Repository struct {
	ID   string  `json:"id,omitempty"`
	Name string  `json:"name,omitempty"`
	Key  string  `json:"key,omitempty"`
	URI  string  `json:"uri,omitempty"`
	CPE  cpe.WFN `json:"cpe,omitempty"`
}

Repository is a package repository

type Severity added in v0.0.15 

type Severity uint
const (
	Unknown Severity = iota
	Negligible
	Low
	Medium
	High
	Critical
	Defcon1
)

func (*Severity) MarshalText added in v0.0.22 

func (s *Severity) MarshalText() ([]byte, error)

func (*Severity) Scan added in v0.0.22 

func (s *Severity) Scan(i interface{}) error

func (Severity) String added in v0.0.22 

func (i Severity) String() string

func (*Severity) UnmarshalText added in v0.0.22 

func (s *Severity) UnmarshalText(b []byte) error

func (Severity) Value added in v0.0.22 

func (s Severity) Value() (driver.Value, error)

type Version added in v0.0.16 

type Version struct {
	Kind string
	V    [10]int32
}

Version describes a revision of some sort that is ordered correctly within its "Kind".

Versions of different kinds do not have any sensible ordering.

func (*Version) Compare added in v0.0.16 

func (v *Version) Compare(x *Version) int

Compare returns an integer describing the relationship of two Versions.

The result will be 0 if a==b, -1 if a < b, and +1 if a > b. If the Versions are of different kinds, the Kinds will be compared lexographically.

func (*Version) MarshalText added in v0.0.16 

func (v *Version) MarshalText() ([]byte, error)

MarshalText implments encoding.TextMarshaler.

func (*Version) String added in v0.0.16 

func (v *Version) String() string

func (*Version) UnmarshalText added in v0.0.16 

func (v *Version) UnmarshalText(text []byte) (err error)

UnmarshalText implments encoding.TextUnmarshaler.

type Vulnerability 

type Vulnerability struct {
	// unique ID of this vulnerability. this will be created as discovered by the library
	// and used for persistence and hash map indexes
	ID string `json:"id"`
	// the updater that discovered this vulnerability
	Updater string `json:"updater"`
	// the name of the vulnerability. for example if the vulnerability exists in a CVE database this
	// would the unique CVE name such as CVE-2017-11722
	Name string `json:"name"`
	// the description of the vulnerability
	Description string `json:"description"`
	// the timestamp when vulnerability was issued
	Issued time.Time `json:"issued"`
	// any links to more details about the vulnerability
	Links string `json:"links"`
	// the severity string retrieved from the security database
	Severity string `json:"severity"`
	// a normalized Severity type providing client guaranteed severity information
	NormalizedSeverity Severity `json:"normalized_severity"`
	// the package information associated with the vulnerability. ideally these fields can be matched
	// to packages discovered by libindex PackageScanner structs.
	Package *Package `json:"package"`
	// the distribution information associated with the vulnerability.
	Dist *Distribution `json:"distribution,omitempty"`
	// the repository information associated with the vulnerability
	Repo *Repository `json:"repository,omitempty"`
	// a string specifying the package version the fix was released in
	FixedInVersion string `json:"fixed_in_version"`
	// Range describes the range of versions that are vulnerable.
	Range *Range `json:"range,omitempty"`
	// ArchOperation indicates how the affected Package's "arch" should be
	// compared.
	ArchOperation ArchOp `json:"arch_op,omitempty"`
}

type VulnerabilityReport 

type VulnerabilityReport struct {
	// the manifest hash this vulnerability report is describing
	Hash Digest `json:"manifest_hash"`
	// all discovered packages in this manifest keyed by package id
	Packages map[string]*Package `json:"packages"`
	// all discovered distributions in this manifest keyed by distribution id
	Distributions map[string]*Distribution `json:"distributions"`
	// all discovered repositories in this manifest keyed by repository id
	Repositories map[string]*Repository `json:"repository"`
	// a list of environment details a package was discovered in keyed by package id
	Environments map[string][]*Environment `json:"environments"`
	// all discovered vulnerabilities affecting this manifest
	Vulnerabilities map[string]*Vulnerability `json:"vulnerabilities"`
	// a lookup table associating package ids with 1 or more vulnerability ids. keyed by package id
	PackageVulnerabilities map[string][]string `json:"package_vulnerabilities"`
}

VulnerabilityReport provides a report of packages and their associated vulnerabilities.

Source Files

View all Source files

Directories

Path Synopsis
cmd
cctool
libindexhttp
libvulnhttp
Package docs holds go code for inclusion into the prose documentation.
 Package docs holds go code for inclusion into the prose documentation.
Package dpkg implements a package indexer for dpkg packages.
 Package dpkg implements a package indexer for dpkg packages.
internal
indexer
Package indexer is a generated GoMock package.
 Package indexer is a generated GoMock package.
indexer/controller
indexer/fetcher
indexer/layerscanner
indexer/linux
indexer/postgres
matcher
Package matcher is a generated GoMock package.
 Package matcher is a generated GoMock package.
updater
vulnstore
Package vulnstore is a generated GoMock package.
 Package vulnstore is a generated GoMock package.
vulnstore/jsonblob
vulnstore/postgres
Package libindex is a generated GoMock package.
 Package libindex is a generated GoMock package.
migrations
Package libvuln is a generated GoMock package.
 Package libvuln is a generated GoMock package.
driver
migrations
Package osrelease provides an "os-release" distribution scanner.
 Package osrelease provides an "os-release" distribution scanner.
pkg
cpe
Package cpe provides for handling Common Platform Enumeration (CPE) names.
 Package cpe provides for handling Common Platform Enumeration (CPE) names.
distlock
Package distlock is a generated GoMock package.
 Package distlock is a generated GoMock package.
distlock/postgres
fastesturl
jsonerr
microbatch
omnimatcher
ovalutil
path
pep440
Package pep440 implements types for working with versions as defined in PEP-440.
 Package pep440 implements types for working with versions as defined in PEP-440.
tmp
Package python contains components for interrogating python packages in container layers.
 Package python contains components for interrogating python packages in container layers.
Package pyupio provides an updater for importing pyup vulnerability information.
 Package pyupio provides an updater for importing pyup vulnerability information.
containerapi
contentmanifest
pulp
repo2cpe
scanner
pkgconfig
Package pkgconfig implements a scanner that finds pkg-config files.
 Package pkgconfig implements a scanner that finds pkg-config files.
bisect
Bisect is a git bisect helper.
 Bisect is a git bisect helper.
fetch
integration
Package integration is a helper for running integration tests.
 Package integration is a helper for running integration tests.
log
ovaldebug
Ovaldebug is a helper for debugging the ovalutil package.
 Ovaldebug is a helper for debugging the ovalutil package.
postgres
toolkit module
Package updater holds a registry of default updaters.
 Package updater holds a registry of default updaters.
defaults
Package defaults sets updater defaults.
 Package defaults sets updater defaults.
driver Module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.