e2e

README

E2E tests

The purpose of this docs is to describe KICS' E2E test suite

Getting Started

There are several ways to execute the E2E tests.

TLDR

This steps will build the kics and then run the E2E using the built binary (placed by default under ${PWD}/bin/kics)

make test-e2e

Step by Step

These steps will build the kics and then run the E2E using the built binary.

go build -o ./bin/kics cmd/console/main.go

If you want to provide a version:

go build -o ./bin/kics -ldflags "-X github.com/Checkmarx/kics/internal/constants.Version=$(git rev-parse --short HEAD) cmd/console/main.go

and then:

E2E_KICS_BINARY=./bin/kics go test "github.com/Checkmarx/kics/e2e" -v

Test Scenarios

Test scenarios are defined as follows:

var tests = []struct {
	name          string
	args          args
	wantStatus    int
	removePayload []string
	validation    Validation
}{
	// E2E-CLI-005 - KICS scan with -- payload-path flag should create a file with the
	// passed name containing the payload of the files scanned
	{
		name: "E2E-CLI-005",
		args: args{
            // These are CLI arguments passed down to the KICS binary
			args: []cmdArgs{
				[]string{"scan", "--silent", "-q", "../assets/queries", "-p", "fixtures/samples/terraform.tf",
					"--payload-path", "fixtures/payload.json", "-q", "../assets/queries"},
			},
             // this is a reference to a fixture placed under e2e/fixtures that contains the expected stdout output
			expectedOut: []string{
				"E2E_CLI_005",
			},
            // this is a reference to a fixture placed under e2e/fixtures that contains the expected KICS' payload
			expectedPayload: []string{
				"E2E_CLI_005_PAYLOAD",
			},
		},
		wantStatus:    0,
        // we should cleanup the payload after running this scenario
		removePayload: []string{"payload.json"},
	},

	// E2E-CLI-006 - KICS generate-id should exhibit
	// a valid UUID in the CLI and return exit code 0
	{
		name: "E2E-CLI-006",
		args: args{
			args: []cmdArgs{
				[]string{"generate-id"},
			},
		},
		wantStatus: 0,
		// When the validation function is used,
		// the expectedOut function can be suppressed

		// The validator function allows the creation of customized functions to
		// perform actions on the output generated by the test.
		validation: func(outputText string) bool {
			// Usually, a validation check for elements present in the output using regex
			uuidRegex := "[a-f0-9]{8}-[a-f0-9]{4}-4{1}[a-f0-9]{3}-[89ab]{1}[a-f0-9]{3}-[a-f0-9]{12}"
			match, _ := regexp.MatchString(uuidRegex, outputText)
			
			// After the custom check, the function should return
			// a boolean indicating if the test has passed or failed.
			return match
		},
	},
}

E2E tests are skiped in short mode:

func Test_E2E_CLI(t *testing.T) {
	kicsPath := getKICSBinaryPath("")

	if testing.Short() {
		t.Skip("skipping E2E tests in short mode.")
	}
//...
}