producers

README

Producers

A producer is a program that parses the output of a tool and converts it into Smithy compatible file that can be used by the enrichers and consumers.

Writing Producers

Producers can be written in any language that supports protobufs, we have examples in Golang and Python. They are all structured the same way:

1. Parse program arguments:
   1. in: the raw tool results file location
   2. out: where to place the Smithy compatible output file location
2. Parse the in file into Protobufs (LaunchToolResponse)
3. Add metadata to Protobufs (e.g. git/source-code information)
4. Write the protobuf bytes to the out file

Producer API

For convenience, there are helper functions in the ./producers pkg/module for Golang/Python.

The WriteSmithyOut/write_smithy_out method expects a list of issues to write as the LaunchToolResponse protobuf. Your producer should parse the output of a tool results into Issue protobufs which are then passed into this method.

Documentation

Overview

Package producers provides helper functions for writing Smithy compatible producers that parse tool outputs. Subdirectories in this package have more complete example usages of this package.

Index

• Constants
• Variables
• func EnsureValidFileTarget(fileTarget string) (string, error)
• func EnsureValidPURLTarget(purlTarget string) (string, error)
• func GetFileTarget(filePath string, startLine int, endLine int) string
• func GetPURLTarget(purlType string, namespace string, name string, version string, ...) string
• func GetPartsFromFileTarget(fileTarget string) (*url.URL, int, int, error)
• func ParseFlags() error
• func ParseMultiJSONMessages(in []byte) ([]interface{}, error)
• func ReadInFile() ([]byte, error)
• func TestEndToEnd(t *testing.T, inPath string, expectedPbPath string) error
• func WriteSmithyOut(toolName string, issues []*smithyapiv1.Issue) error

Constants

const (
	SourceDir = "/workspace/output/source-code/"
)

Variables

var (
	// InResults represents incoming tool output.
	InResults string
	// OutFile points to the protobuf file where smithy results will be written.
	OutFile string
	// Append flag will append to the outfile instead of overwriting, useful when there's multiple inresults.
	Append bool
)

Functions

func EnsureValidFileTarget

func EnsureValidFileTarget(fileTarget string) (string, error)

EnsureValidFileTarget takes a file target string from an untrusted source, e.g. a tool output, and ensures it is a valid file target. file:///path/to/file.txt:10-20

func EnsureValidPURLTarget

func EnsureValidPURLTarget(purlTarget string) (string, error)

EnsureValidPURLTarget takes a purl target string from an untrusted source, e.g. a tool output, and ensures it is a valid purl target

func GetFileTarget

func GetFileTarget(filePath string, startLine int, endLine int) string

GetFileTarget returns a file target string for a given file path. This should be used as the `Issue.Target` field of SAST producers. The root of the `filePath` should be the root of the scanned code.

Example: GetFileTarget("src/main.go", 10, 20) Result: "file:///src/main.go:10-20"

func GetPURLTarget

func GetPURLTarget(purlType string, namespace string, name string, version string, qualifiers packageurl.Qualifiers, subpath string) string

GetPURLTarget returns a purl target string for a given package. This should be used as the `Issue.Target` field of SCA producers.

Example: GetPURLTarget("deb", "debian", "curl", "7.68.0", nil, "")

func GetPartsFromFileTarget

func GetPartsFromFileTarget(fileTarget string) (*url.URL, int, int, error)

GetPartsFromFileTarget takes a file target string and returns the parts. file:///path/to/file.txt:10-20 Returns: url.URL, startLine, endLine, error

func ParseFlags

func ParseFlags() error

ParseFlags will parse the input flags for the producer and perform simple validation.

func ParseMultiJSONMessages

func ParseMultiJSONMessages(in []byte) ([]interface{}, error)

ParseMultiJSONMessages provides method to parse tool results in JSON format. It allows for parsing single JSON files with multiple JSON messages in them.

func ReadInFile

func ReadInFile() ([]byte, error)

ReadInFile returns the contents of the file given by InResults.

func TestEndToEnd

func TestEndToEnd(t *testing.T, inPath string, expectedPbPath string) error

TestEndToEnd is a helper function to test the end-to-end functionality of a producer.

func WriteSmithyOut

func WriteSmithyOut(
	toolName string,
	issues []*smithyapiv1.Issue,
) error

WriteSmithyOut provides a generic method to write the resulting protobuf to the output file.