assetmon

README

assetmon

This system collects information about assets within containers that are actually running on the system. This type of data (also called SBOM, Software Bill Of Materials) is usually computed at build time, on CI systems. However the ties between float and CI are tenuous, for a number of reasons:

• CI data is usually private, or anyway lacks a common API, which is a problem if you want to use multiple sources, possibly not directly controlled by you.
• it is not possible to ensure that a certain image in CI matches what is running in production: pushes are asynchronous with respect to builds, even in the presence of a Continuous Deployment system there are delays, or rollout failures, etc. This is complicated by the (common) usage of floating un-pinned image references.

There is yet another complication: there is no guarantee that an image that is running somewhere is actually downloadable by anyone else at a later time, as it might have already disappeared from the registry. This forces the container scanning process to happen on the hosts themselves, where we can bypass the registry and just operate on the locally stored images.

Each host will thus run an agent, whose job is to keep the central service up to date with information about what is running on the host.

There are two ways for the agent to upload data:

• a hook that runs on container update (still have to decide on the exact source), which sends an update about a single container image
• periodic full-system reports, to compensate temporary issues with the previous process, and delete references to containers that are no longer running.

All upload requests are a two-step conversation: the client starts by sending a description of one or more images, then the server replies with a list of those it does not already know about. The client will then scan the missing images locally and upload the scan results.

As scans aren't exactly free, we want to minimize them by avoiding scanning the same image multiple times on different hosts. So, notifying the server about an image is equivalent to taking a lock for scanning it. Locks expire after a certain time (if the scan fails for some reason), so that the catch-up process can pick up the work if necessary and fill the gap. This makes the service an odd mix of a database with a (bad) task scheduling system, but maintains a very simple data reliability model.

Status

This package implements the agent and a centralized server for collection and reporting, backed by a simple SQLite database. The agent uses Syft to extract artifacts from container images.

The agent is distributed as the assetmon Debian package, while the server comes in the form of a container image.

The agent only implements the cron-based approach described above, running every 10 minutes.

Why not use X instead?

There are good open-source asset-tracking / vulnerability stacks out there, but they tend to be relatively heavyweight in terms of deployment, and, most importantly, they tend to be very focused on the vulnerability detection side of the issue. Vulnerability detection isn't exactly the primary focus for this project at the moment, furthermore there was the desire to have something lightweight that could be easily integrated in all float deployments by default.

Documentation

Index

• func HTTPAPI(a *AssetMon) http.Handler
• func OpenDB(dburi string) (*sql.DB, error)
• type Artifact
• type ArtifactResult
• type AssetMon
  • func NewAssetMon(db *sql.DB) *AssetMon
  • func (a *AssetMon) Close()
  • func (a *AssetMon) FindArtifacts(ctx context.Context, req *FindArtifactsRequest) (*FindArtifactsResponse, error)
  • func (a *AssetMon) FullUpdate(ctx context.Context, req *FullUpdateRequest) (*UpdateResponse, error)
  • func (a *AssetMon) GetContainerDeployment(ctx context.Context, req *GetContainerDeploymentRequest) ([]*ContainerInfo, error)
  • func (a *AssetMon) GetImageSBOM(ctx context.Context, digest string) (*Image, error)
  • func (a *AssetMon) ScanReport(ctx context.Context, req *ScanReportRequest) error
  • func (a *AssetMon) Update(ctx context.Context, req *UpdateRequest) (*UpdateResponse, error)
• type Client
  • func NewClient(be clientutil.Backend, hostname string) *Client
  • func (c *Client) FullUpdate(ctx context.Context, ctnrs []ContainerSummary) error
  • func (c *Client) GetImageSBOM(ctx context.Context, digest string) (*Image, error)
  • func (c *Client) UpdateContainer(ctx context.Context, cont *ContainerSummary) (bool, error)
• type Container
• type ContainerInfo
• type ContainerSummary
  • func RunningContainers(ctx context.Context) ([]ContainerSummary, error)
• type FindArtifactsRequest
• type FindArtifactsResponse
• type FullUpdateRequest
• type GetContainerDeploymentRequest
• type GetContainerDeploymentResponse
• type GetImageSBOMRequest
• type GetImageSBOMResponse
• type Image
• type Match
• type ScanReportRequest
• type UpdateRequest
• type UpdateResponse

Functions

func HTTPAPI

func HTTPAPI(a *AssetMon) http.Handler

func OpenDB

func OpenDB(dburi string) (*sql.DB, error)

OpenDB opens a SQLite database and runs the database migrations.

Types

type Artifact

type Artifact struct {
	Type    string `json:"type"`
	Name    string `json:"name"`
	Version string `json:"version"`
}

type ArtifactResult

type ArtifactResult struct {
	Artifact
	ContainerInfo
}

type AssetMon

type AssetMon struct {
	// contains filtered or unexported fields
}

AssetMon implements the asset management (tracking) service.

func NewAssetMon

func NewAssetMon(db *sql.DB) *AssetMon

func (*AssetMon) Close

func (a *AssetMon) Close()

func (*AssetMon) FindArtifacts

func (a *AssetMon) FindArtifacts(ctx context.Context, req *FindArtifactsRequest) (*FindArtifactsResponse, error)

func (*AssetMon) FullUpdate

func (a *AssetMon) FullUpdate(ctx context.Context, req *FullUpdateRequest) (*UpdateResponse, error)

func (*AssetMon) GetContainerDeployment

func (a *AssetMon) GetContainerDeployment(ctx context.Context, req *GetContainerDeploymentRequest) ([]*ContainerInfo, error)

func (*AssetMon) GetImageSBOM

func (a *AssetMon) GetImageSBOM(ctx context.Context, digest string) (*Image, error)

func (*AssetMon) ScanReport

func (a *AssetMon) ScanReport(ctx context.Context, req *ScanReportRequest) error

func (*AssetMon) Update

func (a *AssetMon) Update(ctx context.Context, req *UpdateRequest) (*UpdateResponse, error)

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client for AssetMon, which implements the client side of the image scanning workflow. Note that the Context passed to client methods should allow for local execution of the scan of potentially multiple images, not just the network RPC, so we use a shorter timeout for the RPC part.

func NewClient

func NewClient(be clientutil.Backend, hostname string) *Client

func (*Client) FullUpdate

func (c *Client) FullUpdate(ctx context.Context, ctnrs []ContainerSummary) error

func (*Client) GetImageSBOM

func (c *Client) GetImageSBOM(ctx context.Context, digest string) (*Image, error)

func (*Client) UpdateContainer

func (c *Client) UpdateContainer(ctx context.Context, cont *ContainerSummary) (bool, error)

type Container

type Container struct {
	Name  string `json:"name"`
	Image Image  `json:"image"`
}

type ContainerInfo

type ContainerInfo struct {
	ContainerSummary

	Host  string    `json:"host"`
	Since time.Time `json:"running_since"`
}

type ContainerSummary

type ContainerSummary struct {
	Name        string `json:"name"`
	ImageName   string `json:"image_name"`
	ImageDigest string `json:"image_digest"`
}

func RunningContainers

func RunningContainers(ctx context.Context) ([]ContainerSummary, error)

RunningContainers returns the list of currently running (according to Podman) containers, along with their associated image IDs.

type FindArtifactsRequest

type FindArtifactsRequest struct {
	Match []Match `json:"match"`
}

type FindArtifactsResponse

type FindArtifactsResponse struct {
	Results []*ArtifactResult `json:"results"`
}

type FullUpdateRequest

type FullUpdateRequest struct {
	Host       string             `json:"host"`
	Containers []ContainerSummary `json:"containers"`
}

type GetContainerDeploymentRequest

type GetContainerDeploymentRequest struct {
	Name        string `json:"name"`
	Host        string `json:"host"`
	ImageDigest string `json:"image_digest"`
}

type GetContainerDeploymentResponse

type GetContainerDeploymentResponse struct {
	Containers []*ContainerInfo `json:"containers"`
}

type GetImageSBOMRequest

type GetImageSBOMRequest struct {
	Digest string `json:"digest"`
}

type GetImageSBOMResponse

type GetImageSBOMResponse struct {
	Image
}

type Image

type Image struct {
	Digest        string     `json:"digest"`
	Distro        string     `json:"distro"`
	DistroVersion string     `json:"distro_version"`
	Artifacts     []Artifact `json:"artifacts"`
}

type Match

type Match struct {
	Field string `json:"field"`
	Op    string `json:"op"`
	Value string `json:"value"`
}

type ScanReportRequest

type ScanReportRequest struct {
	Image *Image `json:"image"`
}

type UpdateRequest

type UpdateRequest struct {
	Host      string           `json:"host"`
	Container ContainerSummary `json:"container"`
}

type UpdateResponse

type UpdateResponse struct {
	ScanImages []string `json:"scan_images"`
}