store

package
v0.0.0-...-a6dec7d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 29, 2025 License: BSD-3-Clause, CC-BY-4.0 Imports: 19 Imported by: 0

Documentation

Overview

Package store supports permanent data storage for the vuln worker.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CVE4Record 

type CVE4Record struct {
	// ID is the CVE ID, which is the same as the filename base. E.g. "CVE-2020-0034".
	ID string
	// Path is the path to the CVE file in the repo.
	Path string
	// BlobHash is the hash of the CVE's blob in repo, for quick change detection.
	BlobHash string
	// CommitHash is the commit of the cvelist repo from which this information came.
	CommitHash string
	// CommitTime is the time of the above commit.
	// If zero, it has not been populated.
	CommitTime time.Time
	// CVEState is the value of the metadata.STATE field.
	CVEState string
	// TriageState is the state of our triage processing on the CVE.
	TriageState TriageState
	// TriageStateReason is an explanation of TriageState.
	TriageStateReason string

	// Module is the Go module path that might be affected.
	Module string

	// Package is the Go package path that might be affected.
	Package string

	// CVE is a copy of the CVE, for the NeedsIssue triage state.
	CVE *cve4.CVE

	// ReferenceURLs is a list of the URLs in the CVE references,
	// for the FalsePositive triage state.
	ReferenceURLs []string

	// IssueReference is a reference to the GitHub issue that was filed.
	// E.g. golang/vulndb#12345.
	// Set only after a GitHub issue has been successfully created.
	IssueReference string

	// IssueCreatedAt is the time when the issue was created.
	// Set only after a GitHub issue has been successfully created.
	IssueCreatedAt time.Time

	// History holds previous states of a CVE4Record,
	// from most to least recent.
	History []*CVE4RecordSnapshot
}

A CVE4Record contains information about a v4 CVE.

func NewCVE4Record 

func NewCVE4Record(cve *cve4.CVE, path, blobHash string, commit *object.Commit) *CVE4Record

NewCVE4Record creates a CVE4Record from a CVE, its path and its blob hash.

func (*CVE4Record) GetDescription 

func (r *CVE4Record) GetDescription() string

func (*CVE4Record) GetID 

func (r *CVE4Record) GetID() string

func (*CVE4Record) GetIssueCreatedAt 

func (r *CVE4Record) GetIssueCreatedAt() time.Time

func (*CVE4Record) GetIssueReference 

func (r *CVE4Record) GetIssueReference() string

func (*CVE4Record) GetSource 

func (r *CVE4Record) GetSource() report.Source

func (*CVE4Record) GetTriageState 

func (r *CVE4Record) GetTriageState() TriageState

func (*CVE4Record) GetUnit 

func (r *CVE4Record) GetUnit() string

func (*CVE4Record) Snapshot 

func (r *CVE4Record) Snapshot() *CVE4RecordSnapshot

func (*CVE4Record) Validate 

func (r *CVE4Record) Validate() error

Validate returns an error if the CVE4Record is not valid.

type CVE4RecordSnapshot 

type CVE4RecordSnapshot struct {
	CommitHash        string
	CVEState          string
	TriageState       TriageState
	TriageStateReason string
}

CVE4RecordSnapshot holds a previous state of a CVE4Record. The fields mean the same as those of CVE4Record.

type CommitUpdateRecord 

type CommitUpdateRecord struct {
	// The ID of this record in the DB. Needed to modify the record.
	ID string
	// When the update started and completed. If EndedAt is zero,
	// the update is in progress (or it crashed).
	StartedAt, EndedAt time.Time
	// The repo commit hash that this update is working on.
	CommitHash string
	// The time the commit occurred.
	CommitTime time.Time
	// The total number of CVEs being processed in this update.
	NumTotal int
	// The number currently processed. When this equals NumTotal, the
	// update is done.
	NumProcessed int
	// The number of CVEs added to the DB.
	NumAdded int
	// The number of CVEs modified.
	NumModified int
	// The error that stopped the update.
	Error string
	// The last time this record was updated.
	UpdatedAt time.Time `firestore:",serverTimestamp"`
}

A CommitUpdateRecord describes a single update operation, which reconciles a commit in the CVE list repo with the DB state.

type FireStore 

type FireStore struct {
	// contains filtered or unexported fields
}

FireStore is a Store implemented with Google Cloud Firestore.

A Firestore DB is a set of documents. Each document has its own unique ID (primary key). Documents are grouped into collections, and each document can have sub-collections. A document can be referred to by a path of the form top-level-collection/doc/sub-collection/doc/...

In this layout, there is a single top-level collection called Namespaces, with documents for each development environment. Within each namespace, there are some collections: - CVEs for CVE4Records - CommitUpdates for CommitUpdateRecords - DirHashes for directory hashes - GHSAs for LegacyGHSARecords.

func NewFireStore 

func NewFireStore(ctx context.Context, projectID, namespace, impersonate string) (_ *FireStore, err error)

NewFireStore creates a new FireStore, backed by a client to Firestore. Since each project can have only one Firestore database, callers must provide a non-empty namespace to distinguish different virtual databases (e.g. prod and testing). If non-empty, the impersonate argument should be the name of a service account to impersonate.

func (*FireStore) Clear 

func (s *FireStore) Clear(ctx context.Context) (err error)

Clear removes all documents in the namespace.

func (*FireStore) CreateCommitUpdateRecord 

func (fs *FireStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)

CreateCommitUpdateRecord implements Store.CreateCommitUpdateRecord. On successful return, r.ID is set to the record's ID.

func (*FireStore) GetDirectoryHash 

func (fs *FireStore) GetDirectoryHash(ctx context.Context, dir string) (_ string, err error)

GetDirectoryHash implements Transaction.GetDirectoryHash.

func (*FireStore) GetRecord 

func (fs *FireStore) GetRecord(ctx context.Context, id string) (_ Record, err error)

GetRecord implements store.GetRecord.

func (*FireStore) ListCVE4RecordsWithTriageState 

func (fs *FireStore) ListCVE4RecordsWithTriageState(ctx context.Context, ts TriageState) (_ []*CVE4Record, err error)

ListCVE4RecordsWithTriageState implements Store.ListCVE4RecordsWithTriageState.

func (*FireStore) ListCommitUpdateRecords 

func (fs *FireStore) ListCommitUpdateRecords(ctx context.Context, limit int) (_ []*CommitUpdateRecord, err error)

ListCommitUpdateRecords implements Store.ListCommitUpdateRecords.

func (*FireStore) RunTransaction 

func (fs *FireStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) (err error)

RunTransaction implements Store.RunTransaction.

func (*FireStore) SetCommitUpdateRecord 

func (fs *FireStore) SetCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)

SetCommitUpdateRecord implements Store.SetCommitUpdateRecord.

func (*FireStore) SetDirectoryHash 

func (fs *FireStore) SetDirectoryHash(ctx context.Context, dir, hash string) (err error)

SetDirectoryHash implements Transaction.SetDirectoryHash.

type LegacyGHSARecord 

type LegacyGHSARecord struct {
	// GHSA is the advisory.
	GHSA *ghsa.SecurityAdvisory
	// TriageState is the state of our triage processing on the CVE.
	TriageState TriageState
	// TriageStateReason is an explanation of TriageState.
	TriageStateReason string
	// IssueReference is a reference to the GitHub issue that was filed.
	// E.g. golang/vulndb#12345.
	// Set only after a GitHub issue has been successfully created.
	IssueReference string
	// IssueCreatedAt is the time when the issue was created.
	// Set only after a GitHub issue has been successfully created.
	IssueCreatedAt time.Time
}

A LegacyGHSARecord holds information about a GitHub security advisory.

func (*LegacyGHSARecord) GetDescription 

func (r *LegacyGHSARecord) GetDescription() string

func (*LegacyGHSARecord) GetID 

func (r *LegacyGHSARecord) GetID() string

func (*LegacyGHSARecord) GetIssueCreatedAt 

func (r *LegacyGHSARecord) GetIssueCreatedAt() time.Time

func (*LegacyGHSARecord) GetIssueReference 

func (r *LegacyGHSARecord) GetIssueReference() string

func (*LegacyGHSARecord) GetSource 

func (r *LegacyGHSARecord) GetSource() report.Source

func (*LegacyGHSARecord) GetTriageState 

func (r *LegacyGHSARecord) GetTriageState() TriageState

func (*LegacyGHSARecord) GetUnit 

func (r *LegacyGHSARecord) GetUnit() string

func (*LegacyGHSARecord) Validate 

func (r *LegacyGHSARecord) Validate() error

type MemStore 

type MemStore struct {
	// contains filtered or unexported fields
}

MemStore is an in-memory implementation of Store, for testing.

func NewMemStore 

func NewMemStore() *MemStore

NewMemStore creates a new, empty MemStore.

func (*MemStore) CVE4Records 

func (ms *MemStore) CVE4Records() map[string]*CVE4Record

CVE4Records return all the CVE4Records of the store.

func (*MemStore) Clear 

func (ms *MemStore) Clear(context.Context) error

Clear removes all data from the MemStore.

func (*MemStore) CreateCommitUpdateRecord 

func (ms *MemStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) error

CreateCommitUpdateRecord implements Store.CreateCommitUpdateRecord.

func (*MemStore) GetDirectoryHash 

func (ms *MemStore) GetDirectoryHash(_ context.Context, dir string) (string, error)

GetDirectoryHash implements Transaction.GetDirectoryHash.

func (*MemStore) GetRecord 

func (ms *MemStore) GetRecord(_ context.Context, id string) (Record, error)

GetRecord implements store.GetCVE4Record.

func (*MemStore) ListCVE4RecordsWithTriageState 

func (ms *MemStore) ListCVE4RecordsWithTriageState(_ context.Context, ts TriageState) ([]*CVE4Record, error)

ListCVE4RecordsWithTriageState implements Store.ListCVE4RecordsWithTriageState.

func (*MemStore) ListCommitUpdateRecords 

func (ms *MemStore) ListCommitUpdateRecords(_ context.Context, limit int) ([]*CommitUpdateRecord, error)

ListCommitUpdateRecords implements Store.ListCommitUpdateRecords.

func (*MemStore) RunTransaction 

func (ms *MemStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) error

RunTransaction implements Store.RunTransaction. A transaction runs with a single lock on the entire DB.

func (*MemStore) SetCommitUpdateRecord 

func (ms *MemStore) SetCommitUpdateRecord(_ context.Context, r *CommitUpdateRecord) error

SetCommitUpdateRecord implements Store.SetCommitUpdateRecord.

func (*MemStore) SetDirectoryHash 

func (ms *MemStore) SetDirectoryHash(_ context.Context, dir, hash string) error

SetDirectoryHash implements Transaction.SetDirectoryHash.

type Record 

type Record interface {
	GetID() string
	GetSource() report.Source
	GetUnit() string
	GetDescription() string
	GetIssueReference() string
	GetIssueCreatedAt() time.Time
	GetTriageState() TriageState
	Validate() error
}

type Store 

type Store interface {
	// CreateCommitUpdateRecord creates a new CommitUpdateRecord. It should be called at the start
	// of an update. On successful return, the CommitUpdateRecord's ID field will be
	// set to a new, unique ID.
	CreateCommitUpdateRecord(context.Context, *CommitUpdateRecord) error

	// SetCommitUpdateRecord modifies the CommitUpdateRecord. Use the same record passed to
	// CreateCommitUpdateRecord, because it will have the correct ID.
	SetCommitUpdateRecord(context.Context, *CommitUpdateRecord) error

	// ListCommitUpdateRecords returns some of the CommitUpdateRecords in the store, from most to
	// least recent.
	ListCommitUpdateRecords(ctx context.Context, limit int) ([]*CommitUpdateRecord, error)

	// GetRecord returns the Record with the given id. If not found, it returns (nil, nil).
	GetRecord(ctx context.Context, id string) (Record, error)

	// ListCVE4RecordsWithTriageState returns all CVE4Records with the given triage state,
	// ordered by ID.
	ListCVE4RecordsWithTriageState(ctx context.Context, ts TriageState) ([]*CVE4Record, error)

	// GetDirectoryHash returns the hash for the tree object corresponding to dir.
	// If dir isn't found, it succeeds with the empty string.
	GetDirectoryHash(ctx context.Context, dir string) (string, error)

	// SetDirectoryHash sets the hash for the given directory.
	SetDirectoryHash(ctx context.Context, dir, hash string) error

	// RunTransaction runs the function in a transaction.
	RunTransaction(context.Context, func(context.Context, Transaction) error) error
}

A Store is a storage system for the CVE database.

type Transaction 

type Transaction interface {
	// CreateRecord creates a new record.
	// It is an error if one with the same ID already exists.
	CreateRecord(Record) error

	// SetRecord sets the record in the database.
	// It is an error if no such record exists.
	SetRecord(Record) error

	// GetRecord returns a single record by ID.
	// If not found, it returns (nil, nil).
	GetRecord(id string) (Record, error)

	// GetCVE4Records retrieves records for all CVE IDs between startID and
	// endID, inclusive.
	GetCVE4Records(startID, endID string) ([]*CVE4Record, error)

	// GetLegacyGHSARecords returns all the GHSARecords in the database.
	GetLegacyGHSARecords() ([]*LegacyGHSARecord, error)
}

Transaction supports store operations that run inside a transaction.

type TriageState 

type TriageState string

TriageState is the state of our work on the CVE or GHSA. It is implemented as a string rather than an int so that stored values are immune to renumbering. 

const (
	// No action is needed on the CVE or GHSA (perhaps because it is rejected, reserved or invalid).
	TriageStateNoActionNeeded TriageState = "NoActionNeeded"
	// The CVE needs to have an issue created.
	TriageStateNeedsIssue TriageState = "NeedsIssue"
	// An issue has been created in the issue tracker.
	// The IssueReference and IssueCreatedAt fields have more information.
	TriageStateIssueCreated TriageState = "IssueCreated"
	// This vulnerability has already been handled under an alias (i.e., a CVE
	// or GHSA that refers to the same vulnerability).
	TriageStateAlias TriageState = "Alias"
	// The CVE state was changed after the CVE was created.
	TriageStateUpdatedSinceIssueCreation TriageState = "UpdatedSinceIssueCreation"
	// Although the triager might think this CVE is relevant to Go, it is not.
	TriageStateFalsePositive TriageState = "FalsePositive"
	// There is already an entry in the Go vuln DB that covers this CVE.
	TriageStateHasVuln TriageState = "HasVuln"
)

func (TriageState) Validate 

func (s TriageState) Validate() error

Validate returns an error if the TriageState is not one of the above values.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.