ruleset

README

Analyzers Ruleset Library

This repository contains Go modules for implementing customization of rulesets for GitLab Secure analyzers.

Testing

go test ./...

Certain unit tests require additional credentials in order to test authenticated fetching of private repositories. By default the unit tests will assert an "authentication required" error but fetching can be enabled by passing in the necessary credentials:

GITLAB_AUTH="theoretick:glpat-notarealtoken" go test ./...

Support

This is an internal package used for developing GitLab-specific features. This package is not meant for public consumption, and breaking changes might be introduced at any time. See publicly available internal tooling for more details.

Contributing

Contributions are welcome, see CONTRIBUTING.md for more details.

License

This code is distributed under the MIT Expat license, see the LICENSE file.

Documentation

Overview

Package ruleset provides code for the analyzer to use to load external scanner configurations. These rulesets are loaded from .gitlab/{sast}-ruleset.toml.

Index

• Constants
• func ProcessPassthroughs(config *Config, logger GenericLogger) (string, error)
• type Config
  • func Load(rulesetPath string, analyzer string, logger GenericLogger) (*Config, error)
  • func LoadRelative(rulesetPath string, analyzer string) (*Config, error)
  • func (config *Config) DisabledIdentifiers() (map[string]bool, error)
  • func (config *Config) IdentifiersWithOverrides() (map[string]Ruleset, error)
• type ConfigFileNotFoundError
  • func (e *ConfigFileNotFoundError) Error() string
• type ConfigNotFoundError
  • func (e *ConfigNotFoundError) Error() string
• type GenericLogger
• type Identifier
• type InvalidConfig
  • func (e *InvalidConfig) Error() string
• type NotEnabledError
  • func (e *NotEnabledError) Error() string
• type Override
• type Passthrough
• type PassthroughMode
• type PassthroughType
• type Ruleset
• type Validator

Constants

const (
	// EnvVarGitlabFeatures lists Gitlab features available
	EnvVarGitlabFeatures = "GITLAB_FEATURES"
	// GitlabFeatureCustomRulesetsSAST indicates that sast custom rulesets are enabled
	GitlabFeatureCustomRulesetsSAST = "sast_custom_rulesets"
	// PathSAST is the default path to custom sast rules
	PathSAST = ".gitlab/sast-ruleset.toml"
	// PathSecretDetection is the default path to custom secret detection rulesets
	PathSecretDetection = ".gitlab/secret-detection-ruleset.toml" // #nosec
	// GitReferenceSAST indicates a remote ruleset reference is defined
	GitReferenceSAST = "SAST_RULESET_GIT_REFERENCE"
	// GitReferenceSecretDetection indicates a remote ruleset reference is defined
	GitReferenceSecretDetection = "SECRET_DETECTION_RULESET_GIT_REFERENCE"
	// EnableLocalConfiguration indicates to load custom sast rules and instead of default or remote ruleset
	EnableLocalConfiguration = "SECURE_ENABLE_LOCAL_CONFIGURATION"
	// PassthroughFile should be used when the ruleset passthrough is a file.
	PassthroughFile PassthroughType = "file"
	// PassthroughRaw should be used when the ruleset passthrough is defined inline.
	PassthroughRaw PassthroughType = "raw"
	// PassthroughGit should be used when the ruleset is pulled via git pulled via git
	PassthroughGit PassthroughType = "git"
	// PassthroughFileURL should be used to download files
	PassthroughFileURL PassthroughType = "url"

	// ValidatorJSON is used to validate JSON files
	ValidatorJSON Validator = "json"
	// ValidatorTOML is used to validate TOML files
	ValidatorTOML Validator = "toml"
	// ValidatorYAML is used to validate YAML files
	ValidatorYAML Validator = "yaml"
	// ValidatorXML is used to validate XML files
	ValidatorXML Validator = "xml"
	// ValidatorUnknown if validator is not known
	ValidatorUnknown Validator = ""

	// DefaultOverallTimeout should be used to set the overall timeout to evaluate all passthroughs in combination (seconds)
	DefaultOverallTimeout = 60

	// MaxOverallTimeout is the max allowed overall timeout for performing a sequence of passthroughs
	MaxOverallTimeout = 300

	// MaxPassthroughs limits the number of maximally allowed passthroughs (per
	// configuration)
	MaxPassthroughs = 20

	// MaxPassthroughByteSize limits the number of bytes written per (raw|file) passthrough
	MaxPassthroughByteSize = 10000000 // 10MB

	// MaxTargetByteSize limits the size of the target configuration (dir|file) for rule-pack
	// synthesis
	MaxTargetByteSize = 100000000 // 100MB

)

const (
	// GitProtocol is currently limited to cloning over https only
	GitProtocol = "https://"
)

Functions

func ProcessPassthroughs

func ProcessPassthroughs(config *Config, logger GenericLogger) (string, error)

ProcessPassthroughs processes multiple rulesets and returns the combined results

Types

type Config

type Config struct {
	TargetDir   string        `toml:",omitempty"`
	Description string        `toml:",omitempty"`
	Passthrough []Passthrough `toml:",omitempty"`
	Ruleset     []Ruleset     `toml:",omitempty"`
	Path        string        `toml:",omitempty"`
	Timeout     uint          `toml:",omitempty"`
	Interpolate bool          `toml:",omitempty"`
	Validate    bool          `toml:",omitempty"`
}

Config is used for overriding default scanner configurations for the analyzers.

func Load

func Load(rulesetPath string, analyzer string, logger GenericLogger) (*Config, error)

Load accepts a rulesetPath and analyzer. RulesetPath must point to a valid {sast}-ruleset.toml file. A single analyzer rule will be returned if one is found.

func LoadRelative

func LoadRelative(rulesetPath string, analyzer string) (*Config, error)

LoadRelative accepts a rulesetPath and analyzer. Rulesetpath must point to a valid {sast}-ruleset.toml file. A single analyzer rule will be returned if one is found.

func (*Config) DisabledIdentifiers

func (config *Config) DisabledIdentifiers() (map[string]bool, error)

DisabledIdentifiers uses the config pre-loaded by the analyzer then constructs a list of identifiers that will be ignored when reporting vulnerabilities

func (*Config) IdentifiersWithOverrides

func (config *Config) IdentifiersWithOverrides() (map[string]Ruleset, error)

IdentifiersWithOverrides uses the config pre-loaded by the analyzer then constructs a list of identifiers that will be overridden when reporting vulnerabilities

type ConfigFileNotFoundError

type ConfigFileNotFoundError struct {
	RulesetPath string
}

ConfigFileNotFoundError indicates the config file was not found

func (*ConfigFileNotFoundError) Error

func (e *ConfigFileNotFoundError) Error() string

Error formats and returns a ConfigFileNotFoundError

type ConfigNotFoundError

type ConfigNotFoundError struct {
	Analyzer    string
	RulesetPath string
}

ConfigNotFoundError indicates custom rule config is not found

func (*ConfigNotFoundError) Error

func (e *ConfigNotFoundError) Error() string

Error formats and returns a ConfigNotFoundError

type GenericLogger

type GenericLogger interface {
	Info(...interface{})
	Infof(string, ...interface{})
	Warn(...interface{})
	Warnf(string, ...interface{})
	Error(...interface{})
	Errorf(string, ...interface{})
	Debug(...interface{})
	Debugf(string, ...interface{})
	Fatal(...interface{})
	Fatalf(string, ...interface{})
}

GenericLogger is a simple logger interface

type Identifier

type Identifier struct {
	Type  string
	Value string
}

Identifier is a vulnerability id. Identifier.Value is used to filter or override vulnerability information in the final report.

type InvalidConfig

type InvalidConfig struct {
	RulesetPath string
	Err         error
}

InvalidConfig indicates an invalid toml file

func (*InvalidConfig) Error

func (e *InvalidConfig) Error() string

Error formats and returns an InvalidConfig

type NotEnabledError

type NotEnabledError struct{}

NotEnabledError indicates custom rulesets have not been enabled

func (*NotEnabledError) Error

func (e *NotEnabledError) Error() string

Error formats and returns a NotEnabledError

type Override

type Override struct {
	Name        string `toml:"name,omitempty"`
	Message     string `toml:"message,omitempty"`
	Description string `toml:"description,omitempty"`
	Severity    string `toml:"severity,omitempty"`
}

Override is used to override rules properties

type Passthrough

type Passthrough struct {
	Type PassthroughType
	// Target is used for a target file or directory
	Target string
	// subdir cloning for git passthrough
	Subdir string `toml:",omitempty"`
	// (Optional) Authentication
	Auth  string
	Value string
	// examples:
	//  refs/remotes/origin/develop
	//  97f7686db058e2141c0806a477c1e04835c4f395
	Ref       string          `toml:",omitempty"`
	Mode      PassthroughMode `toml:",omitempty"`
	Validator Validator       `toml:",omitempty"`
}

Passthrough is a struct that analyzers use to load external scanner configurations. Users can define in a project's ruleset file a PassthroughType (file, raw) and a value. Depending on the type, the value will either be a scanner specific file configuration or an inline configuration.

type PassthroughMode

type PassthroughMode string

PassthroughMode determines how the results of passthroughs should be applied: append to file or overwrite

type PassthroughType

type PassthroughType string

PassthroughType determines how the analyzer loads the ruleset which can either be via a file or defined inline.

type Ruleset

type Ruleset struct {
	Identifier Identifier
	Disable    bool `toml:",omitempty"`
	Override   Override
}

Ruleset is used for disabling rules

type Validator

type Validator string

Validator is used to determine which validator to use to check the validatity of a passthrough (intermediate) result