Documentation ¶
Overview ¶
Package store supports permanent data storage for the vuln worker.
Index ¶
- type CVERecord
- type CVERecordSnapshot
- type CommitUpdateRecord
- type FireStore
- func (s *FireStore) Clear(ctx context.Context) (err error)
- func (fs *FireStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)
- func (fs *FireStore) CreateModuleScanRecord(ctx context.Context, r *ModuleScanRecord) (err error)
- func (fs *FireStore) GetCVERecord(ctx context.Context, id string) (_ *CVERecord, err error)
- func (fs *FireStore) GetDirectoryHash(ctx context.Context, dir string) (_ string, err error)
- func (fs *FireStore) GetModuleScanRecord(ctx context.Context, path, version string, dbTime time.Time) (_ *ModuleScanRecord, err error)
- func (fs *FireStore) ListCVERecordsWithTriageState(ctx context.Context, ts TriageState) (_ []*CVERecord, err error)
- func (fs *FireStore) ListCommitUpdateRecords(ctx context.Context, limit int) (_ []*CommitUpdateRecord, err error)
- func (fs *FireStore) ListModuleScanRecords(ctx context.Context, limit int) (_ []*ModuleScanRecord, err error)
- func (fs *FireStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) (err error)
- func (fs *FireStore) SetCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)
- func (fs *FireStore) SetDirectoryHash(ctx context.Context, dir, hash string) (err error)
- type GHSARecord
- type MemStore
- func (ms *MemStore) CVERecords() map[string]*CVERecord
- func (ms *MemStore) Clear(context.Context) error
- func (ms *MemStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) error
- func (ms *MemStore) CreateModuleScanRecord(_ context.Context, r *ModuleScanRecord) error
- func (ms *MemStore) GetCVERecord(ctx context.Context, id string) (*CVERecord, error)
- func (ms *MemStore) GetDirectoryHash(_ context.Context, dir string) (string, error)
- func (ms *MemStore) GetModuleScanRecord(_ context.Context, path, version string, dbTime time.Time) (*ModuleScanRecord, error)
- func (ms *MemStore) ListCVERecordsWithTriageState(_ context.Context, ts TriageState) ([]*CVERecord, error)
- func (ms *MemStore) ListCommitUpdateRecords(_ context.Context, limit int) ([]*CommitUpdateRecord, error)
- func (ms *MemStore) ListModuleScanRecords(ctx context.Context, limit int) ([]*ModuleScanRecord, error)
- func (ms *MemStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) error
- func (ms *MemStore) SetCommitUpdateRecord(_ context.Context, r *CommitUpdateRecord) error
- func (ms *MemStore) SetDirectoryHash(_ context.Context, dir, hash string) error
- type ModuleScanRecord
- type Store
- type Transaction
- type TriageState
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CVERecord ¶
type CVERecord struct { // ID is the CVE ID, which is the same as the filename base. E.g. "CVE-2020-0034". ID string // Path is the path to the CVE file in the repo. Path string // Blobhash is the hash of the CVE's blob in repo, for quick change detection. BlobHash string // CommitHash is the commit of the cvelist repo from which this information came. CommitHash string // CommitTime is the time of the above commit. // If zero, it has not been populated. CommitTime time.Time // CVEState is the value of the metadata.STATE field. CVEState string // TriageState is the state of our triage processing on the CVE. TriageState TriageState // TriageStateReason is an explanation of TriageState. TriageStateReason string // Module is the Go module path that might be affected. Module string // Package is the Go package path that might be affected. Package string // CVE is a copy of the CVE, for the NeedsIssue triage state. CVE *cveschema.CVE // ReferenceURLs is a list of the URLs in the CVE references, // for the FalsePositive triage state. ReferenceURLs []string // IssueReference is a reference to the GitHub issue that was filed. // E.g. golang/vulndb#12345. // Set only after a GitHub issue has been successfully created. IssueReference string // IssueCreatedAt is the time when the issue was created. // Set only after a GitHub issue has been successfully created. IssueCreatedAt time.Time // History holds previous states of a CVERecord, // from most to least recent. History []*CVERecordSnapshot }
A CVERecord contains information about a CVE.
func NewCVERecord ¶
NewCVERecord creates a CVERecord from a CVE, its path and its blob hash.
func (*CVERecord) GetIssueCreatedAt ¶
func (*CVERecord) GetIssueReference ¶
func (*CVERecord) Snapshot ¶
func (r *CVERecord) Snapshot() *CVERecordSnapshot
type CVERecordSnapshot ¶
type CVERecordSnapshot struct { CommitHash string CVEState string TriageState TriageState TriageStateReason string }
CVERecordSnapshot holds a previous state of a CVERecord. The fields mean the same as those of CVERecord.
type CommitUpdateRecord ¶
type CommitUpdateRecord struct { // The ID of this record in the DB. Needed to modify the record. ID string // When the update started and completed. If EndedAt is zero, // the update is in progress (or it crashed). StartedAt, EndedAt time.Time // The repo commit hash that this update is working on. CommitHash string // The time the commit occurred. CommitTime time.Time // The total number of CVEs being processed in this update. NumTotal int // The number currently processed. When this equals NumTotal, the // update is done. NumProcessed int // The number of CVEs added to the DB. NumAdded int // The number of CVEs modified. NumModified int // The error that stopped the update. Error string // The last time this record was updated. UpdatedAt time.Time `firestore:",serverTimestamp"` }
A CommitUpdateRecord describes a single update operation, which reconciles a commit in the CVE list repo with the DB state.
type FireStore ¶
type FireStore struct {
// contains filtered or unexported fields
}
FireStore is a Store implemented with Google Cloud Firestore.
A Firestore DB is a set of documents. Each document has its own unique ID (primary key). Documents are grouped into collections, and each document can have sub-collections. A document can be referred to by a path of the form top-level-collection/doc/sub-collection/doc/...
In this layout, there is a single top-level collection called Namespaces, with documents for each development environment. Within each namespace, there are some collections: - CVEs for CVERecords - CommitUpdates for CommitUpdateRecords - DirHashes for directory hashes - GHSAs for GHSARecords. - ModuleScans for ModuleScanRecords.
func NewFireStore ¶
func NewFireStore(ctx context.Context, projectID, namespace, impersonate string) (_ *FireStore, err error)
NewFireStore creates a new FireStore, backed by a client to Firestore. Since each project can have only one Firestore database, callers must provide a non-empty namespace to distinguish different virtual databases (e.g. prod and testing). If non-empty, the impersonate argument should be the name of a service account to impersonate.
func (*FireStore) CreateCommitUpdateRecord ¶
func (fs *FireStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)
CreateCommitUpdateRecord implements Store.CreateCommitUpdateRecord. On successful return, r.ID is set to the record's ID.
func (*FireStore) CreateModuleScanRecord ¶
func (fs *FireStore) CreateModuleScanRecord(ctx context.Context, r *ModuleScanRecord) (err error)
CreateModuleScanRecord implements Store.CreateModuleScanRecord.
func (*FireStore) GetCVERecord ¶
GetCVERecord implements store.GetCVERecord.
func (*FireStore) GetDirectoryHash ¶
GetDirectoryHash implements Transaction.GetDirectoryHash.
func (*FireStore) GetModuleScanRecord ¶
func (fs *FireStore) GetModuleScanRecord(ctx context.Context, path, version string, dbTime time.Time) (_ *ModuleScanRecord, err error)
GetModuleScanRecord implements store.GetModuleScanRecord.
func (*FireStore) ListCVERecordsWithTriageState ¶
func (fs *FireStore) ListCVERecordsWithTriageState(ctx context.Context, ts TriageState) (_ []*CVERecord, err error)
ListCVERecordsWithTriageState implements Store.ListCVERecordsWithTriageState.
func (*FireStore) ListCommitUpdateRecords ¶
func (fs *FireStore) ListCommitUpdateRecords(ctx context.Context, limit int) (_ []*CommitUpdateRecord, err error)
ListCommitUpdateRecords implements Store.ListCommitUpdateRecords.
func (*FireStore) ListModuleScanRecords ¶
func (fs *FireStore) ListModuleScanRecords(ctx context.Context, limit int) (_ []*ModuleScanRecord, err error)
ListModuleScanRecords implements Store.ListModuleScanRecords.
func (*FireStore) RunTransaction ¶
func (fs *FireStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) (err error)
RunTransaction implements Store.RunTransaction.
func (*FireStore) SetCommitUpdateRecord ¶
func (fs *FireStore) SetCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)
SetCommitUpdateRecord implements Store.SetCommitUpdateRecord.
type GHSARecord ¶
type GHSARecord struct { // GHSA is the advisory. GHSA *ghsa.SecurityAdvisory // TriageState is the state of our triage processing on the CVE. TriageState TriageState // TriageStateReason is an explanation of TriageState. TriageStateReason string // IssueReference is a reference to the GitHub issue that was filed. // E.g. golang/vulndb#12345. // Set only after a GitHub issue has been successfully created. IssueReference string // IssueCreatedAt is the time when the issue was created. // Set only after a GitHub issue has been successfully created. IssueCreatedAt time.Time }
A GHSARecord holds information about a GitHub security advisory.
func (*GHSARecord) GetID ¶
func (r *GHSARecord) GetID() string
func (*GHSARecord) GetIssueCreatedAt ¶
func (r *GHSARecord) GetIssueCreatedAt() time.Time
func (*GHSARecord) GetIssueReference ¶
func (r *GHSARecord) GetIssueReference() string
func (*GHSARecord) GetUnit ¶
func (r *GHSARecord) GetUnit() string
type MemStore ¶
type MemStore struct {
// contains filtered or unexported fields
}
MemStore is an in-memory implementation of Store, for testing.
func (*MemStore) CVERecords ¶
CVERecords return all the CVERecords of the store.
func (*MemStore) CreateCommitUpdateRecord ¶
func (ms *MemStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) error
CreateCommitUpdateRecord implements Store.CreateCommitUpdateRecord.
func (*MemStore) CreateModuleScanRecord ¶
func (ms *MemStore) CreateModuleScanRecord(_ context.Context, r *ModuleScanRecord) error
CreateModuleScanRecord implements Store.CreateModuleScanRecord.
func (*MemStore) GetCVERecord ¶
GetCVERecord implements store.GetCVERecord.
func (*MemStore) GetDirectoryHash ¶
GetDirectoryHash implements Transaction.GetDirectoryHash.
func (*MemStore) GetModuleScanRecord ¶
func (ms *MemStore) GetModuleScanRecord(_ context.Context, path, version string, dbTime time.Time) (*ModuleScanRecord, error)
GetModuleScanRecord implements store.GetModuleScanRecord.
func (*MemStore) ListCVERecordsWithTriageState ¶
func (ms *MemStore) ListCVERecordsWithTriageState(_ context.Context, ts TriageState) ([]*CVERecord, error)
ListCVERecordsWithTriageState implements Store.ListCVERecordsWithTriageState.
func (*MemStore) ListCommitUpdateRecords ¶
func (ms *MemStore) ListCommitUpdateRecords(_ context.Context, limit int) ([]*CommitUpdateRecord, error)
ListCommitUpdateRecords implements Store.ListCommitUpdateRecords.
func (*MemStore) ListModuleScanRecords ¶
func (ms *MemStore) ListModuleScanRecords(ctx context.Context, limit int) ([]*ModuleScanRecord, error)
ListModuleScanRecords implements Store.ListModuleScanRecords.
func (*MemStore) RunTransaction ¶
func (ms *MemStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) error
RunTransaction implements Store.RunTransaction. A transaction runs with a single lock on the entire DB.
func (*MemStore) SetCommitUpdateRecord ¶
func (ms *MemStore) SetCommitUpdateRecord(_ context.Context, r *CommitUpdateRecord) error
SetCommitUpdateRecord implements Store.SetCommitUpdateRecord.
type ModuleScanRecord ¶
type ModuleScanRecord struct { Path string Version string DBTime time.Time // last-modified time of the vuln DB Error string // if non-empty, error while scanning VulnIDs []string FinishedAt time.Time // when the scan completed (successfully or not) }
A ModuleScanRecord holds information about a vulnerability scan of a module.
func (*ModuleScanRecord) Validate ¶
func (r *ModuleScanRecord) Validate() error
Validate returns an error if the ModuleScanRecord is not valid.
type Store ¶
type Store interface { // CreateCommitUpdateRecord creates a new CommitUpdateRecord. It should be called at the start // of an update. On successful return, the CommitUpdateRecord's ID field will be // set to a new, unique ID. CreateCommitUpdateRecord(context.Context, *CommitUpdateRecord) error // SetCommitUpdateRecord modifies the CommitUpdateRecord. Use the same record passed to // CreateCommitUpdateRecord, because it will have the correct ID. SetCommitUpdateRecord(context.Context, *CommitUpdateRecord) error // ListCommitUpdateRecords returns some of the CommitUpdateRecords in the store, from most to // least recent. ListCommitUpdateRecords(ctx context.Context, limit int) ([]*CommitUpdateRecord, error) // GetCVERecord returns the CVERecord with the given id. If not found, it returns (nil, nil). GetCVERecord(ctx context.Context, id string) (*CVERecord, error) // ListCVERecordsWithTriageState returns all CVERecords with the given triage state, // ordered by ID. ListCVERecordsWithTriageState(ctx context.Context, ts TriageState) ([]*CVERecord, error) // GetDirectoryHash returns the hash for the tree object corresponding to dir. // If dir isn't found, it succeeds with the empty string. GetDirectoryHash(ctx context.Context, dir string) (string, error) // SetDirectoryHash sets the hash for the given directory. SetDirectoryHash(ctx context.Context, dir, hash string) error // CreateModuleScanRecord adds a ModuleScanRecord to the DB. CreateModuleScanRecord(context.Context, *ModuleScanRecord) error // GetModuleScanRecord returns the most recent ModuleScanRecord matching the // given module path, version and DB time. If not found, it returns (nil, // nil). GetModuleScanRecord(ctx context.Context, path, version string, dbTime time.Time) (*ModuleScanRecord, error) // ListModuleScanRecords returns some of the ModuleScanRecords in the store // from most to least recent. If limit is zero, all records are returned. ListModuleScanRecords(ctx context.Context, limit int) ([]*ModuleScanRecord, error) // RunTransaction runs the function in a transaction. RunTransaction(context.Context, func(context.Context, Transaction) error) error }
A Store is a storage system for the CVE database.
type Transaction ¶
type Transaction interface { // CreateCVERecord creates a new CVERecord. It is an error if one with the same ID // already exists. CreateCVERecord(*CVERecord) error // SetCVERecord sets the CVE record in the database. It is // an error if no such record exists. SetCVERecord(r *CVERecord) error // GetCVERecords retrieves CVERecords for all CVE IDs between startID and // endID, inclusive. GetCVERecords(startID, endID string) ([]*CVERecord, error) // CreateGHSARecord creates a new GHSARecord. It is an error if one with the same ID // already exists. CreateGHSARecord(*GHSARecord) error // SetGHSARecord sets the GHSA record in the database. It is // an error if no such record exists. SetGHSARecord(*GHSARecord) error // GetGHSARecord returns a single GHSARecord by GHSA ID. // If not found, it returns (nil, nil). GetGHSARecord(id string) (*GHSARecord, error) // GetGHSARecords returns all the GHSARecords in the database. GetGHSARecords() ([]*GHSARecord, error) }
Transaction supports store operations that run inside a transaction.
type TriageState ¶
type TriageState string
TriageState is the state of our work on the CVE or GHSA. It is implemented as a string rather than an int so that stored values are immune to renumbering.
const ( // No action is needed on the CVE or GHSA (perhaps because it is rejected, reserved or invalid). TriageStateNoActionNeeded TriageState = "NoActionNeeded" // The CVE needs to have an issue created. TriageStateNeedsIssue TriageState = "NeedsIssue" // An issue has been created in the issue tracker. // The IssueReference and IssueCreatedAt fields have more information. TriageStateIssueCreated TriageState = "IssueCreated" // This vulnerability has already been handled under an alias (i.e., a CVE // or GHSA that refers to the same vulnerability). TriageStateAlias TriageState = "Alias" // The CVE state was changed after the CVE was created. TriageStateUpdatedSinceIssueCreation TriageState = "UpdatedSinceIssueCreation" // Although the triager might think this CVE is relevant to Go, it is not. TriageStateFalsePositive TriageState = "FalsePositive" // There is already an entry in the Go vuln DB that covers this CVE. TriageStateHasVuln TriageState = "HasVuln" )
func (TriageState) Validate ¶
func (s TriageState) Validate() error
Validate returns an error if the TriageState is not one of the above values.