crosstests

README

cross-tests

Tests to compare and contrast the behavior of a given Terraform provider under Terraform CLI against the behavior of the same provider bridged to Pulumi and used under Pulumi CLI.

To make debugging easier, execution faster and avoid an extra build step, these tests start both Pulumi and Terraform providers in-process and have the CLIs attach to these in-process providers:

  graph LR
      PC[pulumi up]       -- Drives --> BP[BridgedProvider]
      TC[terraform apply] -- Drives --> TP[TFProvider]
      Test                -- Runs   --> PC
      Test                -- Runs   --> TC
      Test                -- Hosts  --> TP
      Test                -- Hosts  --> BP

The exact sequence of operations and asserts depends on the use case, for example cross-testing Diff convergence would exercise state transition by imitating a change in resource inputs and comparing generated plans.

Both traditional test cases and property-based test cases powered by rapid could be built on top of the same harness.

This is prototype-level code currently but expected to grow in importance over time as it can help quickly diagnose bridged provider behavior discrepancies from the by-design behavior exhibited by Terraform, as well as proactively locate gaps.

Documentation

Overview

Adapters for converting morally equivalent typed representations of TF values for integrating with all the libraries cross-testing is using.

Compares the effect of transitioning between two randomly sampled resource configurations.

Copyright 2016-2024, Pulumi Corporation.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and

Define pretty-printers to make test output easier to interpret.

Driver code for running tests against an in-process bridged provider under Pulumi CLI.

Rapid generators for schema and value spaces.

Abstractions to allow tests to work against both *testing.T and rapid.TB.

Helper code to drive Terraform CLI to run tests against an in-process provider.

Helper code to emit Terraform HCL code to drive the Terraform CLI.

Index

• func FailNotEqual(t T, name string, tfVal, pulVal any)
• func WriteHCL(out io.Writer, sch map[string]*schema.Schema, ...) error
• type PFWriter
  • func WritePF(out io.Writer) PFWriter
  • func (w PFWriter) Provider(sch pfproviderschema.Schema, providerName string, config map[string]cty.Value) error
• type T
• type TfResDriver

Functions

func FailNotEqual

func FailNotEqual(t T, name string, tfVal, pulVal any)

func WriteHCL

func WriteHCL(out io.Writer, sch map[string]*schema.Schema, resourceType, resourceName string, config cty.Value) error

Writes a resource delaration. Note that unknowns are not yet supported in cty.Value, it will error out if found.

Types

type PFWriter

type PFWriter struct {
	// contains filtered or unexported fields
}

func WritePF

func WritePF(out io.Writer) PFWriter

func (PFWriter) Provider

func (w PFWriter) Provider(sch pfproviderschema.Schema, providerName string, config map[string]cty.Value) error

type T

type T interface {
	Logf(string, ...any)
	TempDir() string
	Skip(...any)
	require.TestingT
	assert.TestingT
	pulumitest.PT
}

type TfResDriver

type TfResDriver struct {
	// contains filtered or unexported fields
}