securityscanutils

README

Trivy Security Scanning

Trivy is a security scanning tool which we use to scan our images for vulnerabilities. You can run a trivy scan identical to CI on your own command line by installing trivy and running

trivy image --severity HIGH,CRITICAL quay.io/solo-io/<IMAGE>:<VERSION>

Using securityscanutils

Using the utils here is as easy as using the CLI defined in the cli subdirectory. The snippet below shows the output the said CLI's help command.

The GITHUB_TOKEN environment variable must be set for security scanning to work.

go-utils/securityscan/cli % go run ./run_scan.go help                                                                                                                                            
Usage:
   [command]

Available Commands:
  echo-inputs          Prints out all the state of all inputs (including inputted, defaults, and derived) for debugging purposes
  gen-releases         cache github releases for inputted repository. This is its own command to protect against rate-limiting by github by trying to pull releases too much.
  gen-security-scan-md pull down security scan files from gcloud bucket and generate docs markdown file
  help                 Help about any command
  run-security-scan    runs trivy scans on images from repo specified

Flags:
  -c, --CachedReleasesFile string   The name of the file that contains a list of all releases from the given repository. This file is generated by the 'gen-releases' command, and used by the others.
  -i, --CreateIssues                If true, open/update a Github Issue for each version that has images that have vulnerabilities. Defaults to true. (default true)
  -p, --GenerateCachedReleases      If true, then populate the file specified by the CachedReleasesFile flag with all releases from Github. If false, then the command assumes that the file has already been created and populated.  Should be set to false for testing to avoid rate-limiting by Github. Defaults to true. (default true)
  -f, --ImageFile string            Different release versions may have different images to scan.
                                    To deal with this, the run-security-scan command expects a file input that maps version constraints to images
                                    to be scanned if a version matches that constraint. Constraints must be mutually exclusive.
                                    The file is expected to be a csv, where the first element of each line is the constraint, and every subsequent element
                                    in that line is an image to be scanned if that constraint is matched.
                                    Read https://github.com/Masterminds/semver#checking-version-constraints for more about how to use semver constraints.
      --ImageRepo string            The repository where images to scan are located. Defaults to 'quay.io/solo-io' (default "quay.io/solo-io")
  -m, --MinScannedVersion string    The minimum version of images to scan. If set, will scan every image from this to the present, and will scan all images otherwise
      --RepoOwner string            The owner of the repository to scan. Defaults to 'solo-io' (default "solo-io")
  -r, --TargetRepo string           The repository to scan
  -w, --TargetRepoWritten string    Specify the human readable name of the repository to scan for output purposes.
  -u, --UploadToGithub              Setting this to true will upload any generated sarif files to the github repository endpoint, 
                                    e.g. https://github.com/solo-io/gloo/security/code-scanning
                                    read more here: https://docs.github.com/en/rest/reference/code-scanning.
                                    Defaults to false.
  -h, --help                        help for this command

Use " [command] --help" for more information about a command.

Documentation

Index

• Constants
• Variables
• func BuildSecurityScanReportForRepo(tags []string, opts *options) error
• func GetSecurityScanReport(url string) (string, error)
• func GetTemplateFile(trivyTemplate string) (string, error)
• func IsImageNotFoundErr(logs string) bool
• func RootApp(ctx context.Context) *cobra.Command
• func RunTrivyScan(image, version, templateFile, output string) (bool, bool, error)
• type SarifMetadata
• type SecurityScanOpts
• type SecurityScanRepo
  • func (r *SecurityScanRepo) CreateUpdateVulnerabilityIssue(ctx context.Context, client *github.Client, ...) error
  • func (r *SecurityScanRepo) GetImagesToScan(versionToScan *semver.Version) ([]string, error)
  • func (r *SecurityScanRepo) RunGithubSarifScan(versionToScan *semver.Version, sarifTplFile string) error
  • func (r *SecurityScanRepo) RunMarkdownScan(ctx context.Context, client *github.Client, versionToScan *semver.Version, ...) error
  • func (r *SecurityScanRepo) UploadSecurityScanToGithub(fileName, versionTag string) error
• type SecurityScanRepositoryReleasePredicate
  • func (s *SecurityScanRepositoryReleasePredicate) Apply(release *github.RepositoryRelease) bool
• type SecurityScanner
  • func (s *SecurityScanner) GenerateSecurityScans(ctx context.Context) error

Constants

const MarkdownTrivyTemplate = `` /* 505-byte string literal not displayed */

Template for markdown docs

const SarifTrivyTemplate = `` /* 3578-byte string literal not displayed */

Template for Sarif files to be uploaded to Github, which displays results on the 'Security' tab. Taken from https://github.com/aquasecurity/trivy/blob/main/contrib/sarif.tpl

const VulnerabilityFoundStatusCode = 52

Status code returned by Trivy if a vulnerability is found

Variables

var (
	MissingGithubTokenError = func(envVar string) error {
		return eris.Errorf("Must either set GITHUB_TOKEN or set %s environment variable to true", envVar)
	}
	FileNotFoundError = func(path string, branch string) error {
		return eris.Errorf("Could not find file at path %s on branch %s", path, branch)
	}
	MalformedVersionImageConstraintLine = func(line string) error {
		return eris.Errorf("Could not properly split version image constraint line: %s", line)
	}
)

var TrivyLabels = []string{"trivy", "vulnerability"}

Labels that are applied to github issues that security scan generates

Functions

func BuildSecurityScanReportForRepo

func BuildSecurityScanReportForRepo(tags []string, opts *options) error

func GetSecurityScanReport

func GetSecurityScanReport(url string) (string, error)

func GetTemplateFile

func GetTemplateFile(trivyTemplate string) (string, error)

Create tempoarary file that contains the trivy template Trivy CLI only accepts files as input for a template, so this is a workaround

func IsImageNotFoundErr

func IsImageNotFoundErr(logs string) bool

func RootApp

func RootApp(ctx context.Context) *cobra.Command

Configure the CLI, including possible commands and input args.

func RunTrivyScan

func RunTrivyScan(image, version, templateFile, output string) (bool, bool, error)

Runs trivy scan command Returns (trivy scan ran successfully, vulnerabilities found, error running trivy scan)

Types

type SarifMetadata

type SarifMetadata struct {
	Ref       string `json:"ref"`
	CommitSha string `json:"commit_sha"`
	Sarif     string `json:"sarif"`
}

type SecurityScanOpts

type SecurityScanOpts struct {
	// The following directory structure will be created in your output dir.
	/*
	   OUTPUT_DIR/
	   ├─ markdown_results/
	   │  ├─ repo1/
	   │  │  ├─ 1.4.12/
	   │  │  ├─ 1.5.0/
	   │  ├─ repo2/
	   │  │  ├─ 1.4.13/
	   │  │  ├─ 1.5.1/
	   ├─ sarif_results/
	   │  ├─ repo1/
	   │  │  ├─ 1.4.12/
	   │  │  ├─ 1.5.0/
	   │  ├─ repo2/
	   │  │  ├─ 1.4.13/
	   │  │  ├─ 1.5.1/
	*/
	OutputDir string
	// A mapping of version constraints to images scanned.
	// If 1.6 had images "gloo", "discovery" and 1.7 introduced a new image "rate-limit",
	// the map would look like:
	/*
	   ' >= 1.6': ["gloo", "discovery"]
	   ' >= 1.7': ["gloo", "discovery", "rate-limit"]
	*/
	// where the patch number is explicitly not set so that these versions can match all
	// 1.6.x-x releases
	ImagesPerVersion map[string][]string
	// VersionConstraint on releases to security scan
	// any releases that do not pass this constraint will not be security scanned.
	// If left empty, all versions will be scanned
	VersionConstraint *semver.Constraints

	// Required: image repo (quay.io, grc.io, gchr.io)
	ImageRepo string

	// Uploads Sarif file to github security code-scanning results
	// e.g. https://github.com/solo-io/gloo/security/code-scanning
	UploadCodeScanToGithub bool

	// Creates github issue if image vulnerabilities are found
	CreateGithubIssuePerVersion bool
}

type SecurityScanRepo

type SecurityScanRepo struct {
	Repo  string
	Owner string
	Opts  *SecurityScanOpts
	// contains filtered or unexported fields
}

func (*SecurityScanRepo) CreateUpdateVulnerabilityIssue

func (r *SecurityScanRepo) CreateUpdateVulnerabilityIssue(ctx context.Context, client *github.Client, version, vulnerabilityMarkdown string) error

Creates/Updates a Github Issue per image The github issue will have the markdown table report of the image's vulnerabilities example: https://github.com/solo-io/solo-projects/issues/2458

func (*SecurityScanRepo) GetImagesToScan

func (r *SecurityScanRepo) GetImagesToScan(versionToScan *semver.Version) ([]string, error)

func (*SecurityScanRepo) RunGithubSarifScan

func (r *SecurityScanRepo) RunGithubSarifScan(versionToScan *semver.Version, sarifTplFile string) error

func (*SecurityScanRepo) RunMarkdownScan

func (r *SecurityScanRepo) RunMarkdownScan(ctx context.Context, client *github.Client, versionToScan *semver.Version, markdownTplFile string) error

func (*SecurityScanRepo) UploadSecurityScanToGithub

func (r *SecurityScanRepo) UploadSecurityScanToGithub(fileName, versionTag string) error

Uploads Github security scan in .sarif file format to Github Security Tab under "Code Scanning"

type SecurityScanRepositoryReleasePredicate

type SecurityScanRepositoryReleasePredicate struct {
	// contains filtered or unexported fields
}

The SecurityScanRepositoryReleasePredicate is responsible for defining which github.RepositoryRelease artifacts should be included in the bulk security scan At the moment, the two requirements are that: 1. The release is not a pre-release or draft 2. The release matches the configured version constraint

func (*SecurityScanRepositoryReleasePredicate) Apply

func (s *SecurityScanRepositoryReleasePredicate) Apply(release *github.RepositoryRelease) bool

type SecurityScanner

type SecurityScanner struct {
	Repos []*SecurityScanRepo
	// contains filtered or unexported fields
}

func (*SecurityScanner) GenerateSecurityScans

func (s *SecurityScanner) GenerateSecurityScans(ctx context.Context) error

Main method to call on SecurityScanner which generates .md and .sarif files in OutputDir as defined above per repo. If UploadCodeScanToGithub is true, sarif files will be uploaded to the repository's code-scanning endpoint.