recording

package
v1.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2024 License: MIT Imports: 26 Imported by: 0

README

Azure SDK for Go Recorded Test Framework

Build Status

The recording package makes it easy to add recorded tests to your track-2 client package. Below are some examples that walk through setting up a recorded test end to end.

Set Up

The recording package supports three different test modes. These modes are set by setting the AZURE_RECORD_MODE environment variable to one of the three values:

  1. record: Used when making requests against real resources. In this mode new recording will be generated and saved to file.
  2. playback: This mode is for running tests against local recordings.
  3. live: This mode should not be used locally, it is used by the nightly live pipelines to run against real resources and skip any routing to the proxy. This mode closely mimics how our customers will use the libraries.

After you've set the AZURE_RECORD_MODE, set the PROXY_CERT environment variable to:

$ENV:PROXY_CERT="C:/ <path-to-repo> /azure-sdk-for-go/eng/common/testproxy/dotnet-devcert.crt"

Running the test proxy

Recording and playing back tests relies on the Test Proxy to intercept traffic. The recording package can automatically install and run an instance of the test-proxy server per package. The below code needs to be added to test setup and teardown in order to achieve this. The service directory value should correspond to the testdata directory within the directory containing the assets.json config, see examples.

const recordingDirectory = "<path to service directory with assets.json file>/testdata"

func TestMain(m *testing.M) {
	code := run(m)
	os.Exit(code)
}

func run(m *testing.M) int {
	if recording.GetRecordMode() == recording.PlaybackMode || recording.GetRecordMode() == recording.RecordingMode {
        proxy, err := recording.StartTestProxy(recordingDirectory, nil)
        if err != nil {
            panic(err)
        }

        // NOTE: defer should not be used directly within TestMain as it will not be executed due to os.Exit()
		defer func() {
			err := recording.StopTestProxy(proxy)
			if err != nil {
				panic(err)
			}
		}()
    }

    ... all other test code, including proxy recording setup ...
	return m.Run()
}

Routing Traffic

The first step in instrumenting a client to interact with recorded tests is to direct traffic to the proxy through a custom policy. In these examples we'll use testify's require library but you can use the framework of your choice. Each test has to call recording.Start and recording.Stop, the rest is taken care of by the recording library and the test-proxy.

The snippet below demonstrates an example test policy:

type recordingPolicy struct {
	options recording.RecordingOptions
	t       *testing.T
}

func (r recordingPolicy) Host() string {
	if r.options.UseHTTPS {
		return "localhost:5001"
	}
	return "localhost:5000"
}

func (r recordingPolicy) Scheme() string {
	if r.options.UseHTTPS {
		return "https"
	}
	return "http"
}

func NewRecordingPolicy(t *testing.T, o *recording.RecordingOptions) policy.Policy {
	if o == nil {
		o = &recording.RecordingOptions{UseHTTPS: true}
	}
	p := &recordingPolicy{options: *o, t: t}
	return p
}

func (p *recordingPolicy) Do(req *policy.Request) (resp *http.Response, err error) {
	if recording.GetRecordMode() != "live" {
		p.options.ReplaceAuthority(t, req.Raw())
	}
	return req.Next()
}

After creating a recording policy, it has to be added to the client on the ClientOptions.PerCallPolicies option:

func TestSomething(t *testing.T) {
    p := NewRecordingPolicy(t)
    httpClient, err := recording.GetHTTPClient(t)
    require.NoError(t, err)

	options := &ClientOptions{
		ClientOptions: azcore.ClientOptions{
			PerCallPolicies: []policy.Policy{p},
			Transport:       client,
		},
	}

    client, err := NewClient("https://mystorageaccount.table.core.windows.net", myCred, options)
    require.NoError(t, err)
    // Continue test
}

Starting and Stopping a Test

To start and stop your tests use the recording.Start and recording.Stop (make sure to use defer recording.Stop to ensure the proxy cleans up your test on failure) methods:

func TestSomething(t *testing.T) {
    err := recording.Start(t, recordingDirectory, nil)
    defer recording.Stop(t, recordingDirectory, nil)

    // Continue test
}

Using Sanitizers

The recording files generated by the test-proxy are committed along with the code to the public repository. We have to keep our recording files free of secrets that can be used by bad actors to infilitrate services. To do so, the recording package has several sanitizers for taking care of this. Sanitizers are added at the session level (ie. for an entire test run) to apply to all recordings generated during a test run. For example, to replace the account name from a storage url use the recording.AddURISanitizer method:

func TestSomething(t *testing.T) {
    err := recording.AddURISanitizer("fakeaccountname", "my-real-account-name", nil)
    require.NoError(t, err)

    // To remove the sanitizer after this test use the following:
    defer recording.ResetSanitizers(nil)

    err := recording.Start(t, recordingDirectory, nil)
    defer recording.Stop(t, recordingDirectory, nil)

    // Continue test
}

In addition to URI sanitizers, there are sanitizers for headers, response bodies, OAuth responses, continuation tokens, and more. For more information about all the sanitizers check out the source code

Reading Environment Variables

The CI pipelines for PRs do not run against live resources, you will need to make sure that the values that are replaced in the recording files are also replaced in your requests when running in playback. The best way to do this is to use the recording.GetEnvVariable and use the replaced value as the recordedValue argument:

func TestSomething(t *testing.T) {
    accountName := recording.GetEnvVariable(t, "TABLES_PRIMARY_ACCOUNT_NAME", "fakeaccountname")
    if recording.GetRecordMode() = recording.RecordMode {
        err := recording.AddURISanitizer("fakeaccountname", accountName, nil)
        require.NoError(t, err)
    }

    // Continue test
}

In this snippet, if the test is running in live mode and we have the real account name, we want to add a URI sanitizer for the account name to ensure the value does not appear in any recordings.

Documentation

Index

Examples

Constants

View Source 
const (
	RecordingMode     = "record"
	PlaybackMode      = "playback"
	LiveMode          = "live"
	IDHeader          = "x-recording-id"
	ModeHeader        = "x-recording-mode"
	UpstreamURIHeader = "x-recording-upstream-base-uri"
)
View Source 
const (
	ModeEnvironmentVariableName = "AZURE_TEST_MODE"
)
View Source 
const SanitizedBase64Value string = "Kg=="

SanitizedBase64Value is the default placeholder value to be used for sanitized base-64 encoded strings.

View Source 
const SanitizedValue string = "Sanitized"

SanitizedValue is the default placeholder value to be used for sanitized strings.

Variables

This section is empty.

Functions

func AddBodyKeySanitizer 

func AddBodyKeySanitizer(jsonPath, value, regex string, options *RecordingOptions) error

AddBodyKeySanitizer adds a sanitizer for JSON Bodies. jsonPath is the path to the key, value is the value to replace with, and regex is the string to match in the body. If your regex includes a group options.GroupForReplace specifies which group to replace

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddBodyKeySanitizer("$.json.path", "new-value", "regex-to-replace", nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddBodyRegexSanitizer 

func AddBodyRegexSanitizer(value, regex string, options *RecordingOptions) error

AddBodyRegexSanitizer offers regex replace within a returned JSON body. value is the substitution value, regex can be a simple regex or a substitution operation if options.GroupForReplace is set.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddBodyRegexSanitizer("my-new-value", "regex-to-replace", nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddContinuationSanitizer 

func AddContinuationSanitizer(key, method string, resetAfterFirst bool, options *RecordingOptions) error

AddContinuationSanitizer is used to anonymize private keys in response/request pairs. key: the name of the header whos value will be replaced from response -> next request method: the method by which the value of the targeted key will be replaced. Defaults to GUID replacement resetAfterFirt: Do we need multiple pairs replaced? Or do we want to replace each value with the same value.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddContinuationSanitizer("key", "my-new-value", true, nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddGeneralRegexSanitizer 

func AddGeneralRegexSanitizer(value, regex string, options *RecordingOptions) error

AddGeneralRegexSanitizer adds a general regex across request/response Body, Headers, and URI. value is the substitution value, regex can be defined as a simple regex replace or a substition operation if options.GroupForReplace specifies which group to replace.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddGeneralRegexSanitizer("my-new-value", "regex-to-scrub-secret", nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddHeaderRegexSanitizer 

func AddHeaderRegexSanitizer(key, value, regex string, options *RecordingOptions) error

AddHeaderRegexSanitizer can be used to replace a key with a specific value: set regex to "" OR can be used to do a simple regex replace operation by setting key, value, and regex. OR To do a substitution operation if options.GroupForReplace is set. key is the name of the header to operate against. value is the substitution or whole new header value. regex can be defined as a simple regex replace or a substitution operation if options.GroupForReplace is set.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddHeaderRegexSanitizer("header", "my-new-value", "regex-to-scrub-secret", nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddOAuthResponseSanitizer 

func AddOAuthResponseSanitizer(options *RecordingOptions) error

AddOAuthResponseSanitizer cleans all request/response pairs taht match an oauth regex in their URI

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddOAuthResponseSanitizer(nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddRemoveHeaderSanitizer 

func AddRemoveHeaderSanitizer(headersForRemoval []string, options *RecordingOptions) error

AddRemoveHeaderSanitizer removes a list of headers from request/responses.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddRemoveHeaderSanitizer([]string{"header1", "header2"}, nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddURISanitizer 

func AddURISanitizer(value, regex string, options *RecordingOptions) error

AddURISanitizer sanitizes URIs via regex. value is the substition value, regex is either a simple regex or a substitution operation if options.GroupForReplace is defined.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddURISanitizer("my-new-value", "my-secret", nil)
	if err != nil {
		panic(err)
	}
}

Output:

func AddURISubscriptionIDSanitizer 

func AddURISubscriptionIDSanitizer(value string, options *RecordingOptions) error

AddURISubscriptionIDSanitizer replaces real subscriptionIDs within a URI with a default or configured fake value. To use the default value set value to "", otherwise value specifies the replacement value.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.AddURISubscriptionIDSanitizer("0123-4567-...", nil)
	if err != nil {
		panic(err)
	}
}

Output:

func DefaultStringSanitizer deprecated 

func DefaultStringSanitizer(s *string)

Deprecated: the local sanitizer API that uses this function is no longer supported.

func GenerateAlphaNumericID added in v1.2.0 

func GenerateAlphaNumericID(t *testing.T, prefix string, length int, lowercaseOnly bool) (string, error)

GenerateAlphaNumericID will generate a recorded random alpha numeric id. When live mode or the recording has a randomSeed already set, the value will be generated from that seed, else a new random seed will be used.

func GetEnvVariable 

func GetEnvVariable(varName string, recordedValue string) string

GetEnvVariable looks up an environment variable and if it is not found, returns the recordedValue

func GetHTTPClient 

func GetHTTPClient(t *testing.T) (*http.Client, error)

func GetRecordMode 

func GetRecordMode() string

func GetRecordingId 

func GetRecordingId(t *testing.T) string

func GetVariables 

func GetVariables(t *testing.T) map[string]interface{}

GetVariables returns access to the variables stored by the test proxy for a specific test

func IsLiveOnly 

func IsLiveOnly(t *testing.T) bool

func LiveOnly 

func LiveOnly(t *testing.T)

func RemoveRegisteredSanitizers added in v1.7.0 

func RemoveRegisteredSanitizers(sanitizerIDs []string, options *RecordingOptions) error

RemoveRegisteredSanitizers selectively disables sanitizers that are enabled by default such as "AZSDK1001"

func ResetProxy 

func ResetProxy(options *RecordingOptions) error

ResetProxy restores the proxy's default sanitizers, matchers, and transforms

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.ResetProxy(nil)
	if err != nil {
		panic(err)
	}
}

Output:

func SetBodilessMatcher 

func SetBodilessMatcher(t *testing.T, options *MatcherOptions) error

SetBodilessMatcher adjusts the "match" operation to exclude the body when matching a request to a recording's entries. Pass in `nil` for `t` if you want the bodiless matcher to apply everywhere

func SetDefaultMatcher added in v0.9.1 

func SetDefaultMatcher(t *testing.T, options *SetDefaultMatcherOptions) error

SetDefaultMatcher adjusts the "match" operation to exclude the body when matching a request to a recording's entries. Pass in `nil` for `t` if you want the bodiless matcher to apply everywhere

func Sleep 

func Sleep(duration time.Duration)

Sleep during a test for `duration` seconds. This method will only execute when AZURE_RECORD_MODE = "record", if a test is running in playback this will be a noop.

func Start 

func Start(t *testing.T, pathToRecordings string, options *RecordingOptions) error
Example 
package main

import (
	"testing"

	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.Start(&testing.T{}, "path/to/sdk/testdata", nil)
	if err != nil {
		panic(err)
	}
}

Output:
Example (Second) 
package main

import (
	"testing"

	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	func(t *testing.T) {
		err := recording.Start(t, "sdk/internal/recording/testdata", nil)
		if err != nil {
			panic(err)
		}
		defer func() {
			err := recording.Stop(t, nil)
			if err != nil {
				panic(err)
			}
		}()

		// Add single test recordings here if necessary
		accountURL := recording.GetEnvVariable("TABLES_ACCOUNT_URL", "https://fakeurl.tables.core.windows.net")
		err = recording.AddURISanitizer(accountURL, "https://fakeurl.tables.core.windows.net", nil)
		if err != nil {
			panic(err)
		}

		// Test functionality

		// Reset Sanitizers IF only for a single session
		err = recording.ResetProxy(nil)
		if err != nil {
			panic(err)
		}
	}(&testing.T{})
}

Output:

func Stop 

func Stop(t *testing.T, options *RecordingOptions) error

Stop tells the test proxy to stop accepting requests for a given test

Example 
package main

import (
	"testing"

	"github.com/Azure/azure-sdk-for-go/sdk/internal/recording"
)

func main() {
	err := recording.Stop(&testing.T{}, nil)
	if err != nil {
		panic(err)
	}
}

Output:

func StopTestProxy added in v1.4.0 

func StopTestProxy(proxyInstance *TestProxyInstance) error

NOTE: The process will be killed if the user hits ctrl-c mid-way through tests, as go will kill child processes when the main process does not exit cleanly. No os.Interrupt handlers need to be added after starting the proxy server in tests.

Types

type Failer deprecated 

type Failer func(string)

Deprecated: only deprecated methods use this type.

type Logger deprecated 

type Logger func(string)

Deprecated: only deprecated methods use this type.

type MatcherOptions 

type MatcherOptions struct {
	RecordingOptions
}

Optional parameters for the SetBodilessMatcher operation

type Name deprecated 

type Name func() string

Deprecated: only deprecated methods use this type.

type RecordMode 

type RecordMode string
const (
	Record   RecordMode = "record"
	Playback RecordMode = "playback"
	Live     RecordMode = "live"
)

type Recording deprecated 

type Recording struct {
	SessionName   string
	RecordingFile string
	VariablesFile string
	Mode          RecordMode
	Sanitizer     *Sanitizer
	Matcher       *RequestMatcher
}

Deprecated: the local recording API that uses this type is no longer supported. Call Start and Stop to make recordings via the test proxy instead.

func NewRecording 

func NewRecording(c TestContext, mode RecordMode) (*Recording, error)

NewRecording initializes a new Recording instance

func (*Recording) Do 

func (r *Recording) Do(req *http.Request) (*http.Response, error)

Do satisfies the azcore.Transport interface so that Recording can be used as the transport for recorded requests

func (*Recording) GenerateAlphaNumericID 

func (r *Recording) GenerateAlphaNumericID(prefix string, length int, lowercaseOnly bool) (string, error)

GenerateAlphaNumericID will generate a recorded random alpha numeric id if the recording has a randomSeed already set, the value will be generated from that seed, else a new random seed will be used

func (*Recording) GetEnvVar 

func (r *Recording) GetEnvVar(name string, variableType VariableType) (string, error)

GetEnvVar returns a recorded environment variable. If the variable is not found we return an error. variableType determines how the recorded variable will be saved.

func (*Recording) GetOptionalEnvVar 

func (r *Recording) GetOptionalEnvVar(name string, defaultValue string, variableType VariableType) string

GetOptionalEnvVar returns a recorded environment variable with a fallback default value. default Value configures the fallback value to be returned if the environment variable is not set. variableType determines how the recorded variable will be saved.

func (*Recording) Now 

func (r *Recording) Now() time.Time

func (*Recording) Stop 

func (r *Recording) Stop() error

Stop stops the recording and saves them, including any captured variables, to disk

func (*Recording) UUID 

func (r *Recording) UUID() uuid.UUID

type RecordingHTTPClient 

type RecordingHTTPClient struct {
	// contains filtered or unexported fields
}

func NewRecordingHTTPClient 

func NewRecordingHTTPClient(t *testing.T, options *RecordingOptions) (*RecordingHTTPClient, error)

NewRecordingHTTPClient returns a type that implements `azcore.Transporter`. This will automatically route tests on the `Do` call.

func (RecordingHTTPClient) Do 

func (c RecordingHTTPClient) Do(req *http.Request) (*http.Response, error)

type RecordingOptions 

type RecordingOptions struct {
	UseHTTPS        bool
	ProxyPort       int
	GroupForReplace string
	Variables       map[string]interface{}
	TestInstance    *testing.T
	// contains filtered or unexported fields
}

func (RecordingOptions) ReplaceAuthority 

func (r RecordingOptions) ReplaceAuthority(t *testing.T, rawReq *http.Request) *http.Request

type RequestMatcher deprecated 

type RequestMatcher struct {
	// IgnoredHeaders is a map acting as a hash set of the header names that will be ignored for matching.
	// Modifying the keys in the map will affect how headers are matched for recordings.
	IgnoredHeaders map[string]struct{}
}

Deprecated: the local recording API that uses this type is no longer supported. Call SetDefaultMatcher to configure the test proxy's matcher instead.

func (*RequestMatcher) SetBodyMatcher 

func (m *RequestMatcher) SetBodyMatcher(matcher StringMatcher)

SetBodyMatcher replaces the default matching behavior with a custom StringMatcher that compares the string value of the request body payload with the string value of the recorded body payload.

func (*RequestMatcher) SetMethodMatcher 

func (m *RequestMatcher) SetMethodMatcher(matcher StringMatcher)

SetMethodMatcher replaces the default matching behavior with a custom StringMatcher that compares the string value of the request method with the string value of the recorded method

func (*RequestMatcher) SetURLMatcher 

func (m *RequestMatcher) SetURLMatcher(matcher StringMatcher)

SetURLMatcher replaces the default matching behavior with a custom StringMatcher that compares the string value of the request URL with the string value of the recorded URL

type Sanitizer deprecated 

type Sanitizer struct{}

Deprecated: the local recording API that uses this type is no longer supported. Use Start and Stop to make recordings with the test proxy instead, and configure its sanitizers with functions such as AddBodyKeySanitizer and AddRemoveHeaderSanitizer.

func (*Sanitizer) AddBodysanitizer 

func (s *Sanitizer) AddBodysanitizer(sanitizer StringSanitizer)

AddBodysanitizer configures the supplied StringSanitizer to sanitize recording request and response bodies

func (*Sanitizer) AddSanitizedHeaders 

func (s *Sanitizer) AddSanitizedHeaders(headers ...string)

AddSanitizedHeaders adds the supplied header names to the list of headers to be sanitized on request and response recordings.

func (*Sanitizer) AddUrlSanitizer 

func (s *Sanitizer) AddUrlSanitizer(sanitizer StringSanitizer)

AddUriSanitizer configures the supplied StringSanitizer to sanitize recording request and response URLs

type SetDefaultMatcherOptions added in v0.9.1 

type SetDefaultMatcherOptions struct {
	RecordingOptions

	CompareBodies       *bool
	ExcludedHeaders     []string
	IgnoredHeaders      []string
	IgnoreQueryOrdering *bool
}

type StringMatcher deprecated 

type StringMatcher func(reqVal string, recVal string) bool

Deprecated: only deprecated methods use this type.

type StringSanitizer deprecated 

type StringSanitizer func(*string)

StringSanitizer is a func that will modify the string pointed to by the parameter into a sanitized value.

Deprecated: the local sanitizer API that uses this type is no longer supported. Use test proxy sanitizer configuration functions such as AddBodyKeySanitizer and AddRemoveHeaderSanitizer instead.

type TestContext deprecated 

type TestContext interface {
	Fail(string)
	Log(string)
	Name() string
	IsFailed() bool
}

Deprecated: only deprecated methods use this type.

func NewTestContext 

func NewTestContext(failer Failer, logger Logger, name Name) TestContext

NewTestContext initializes a new TestContext

type TestProxyInstance added in v1.4.0 

type TestProxyInstance struct {
	Cmd     *exec.Cmd
	Options *RecordingOptions
}

func StartTestProxy added in v1.4.0 

func StartTestProxy(pathToRecordings string, options *RecordingOptions) (*TestProxyInstance, error)

type VariableType deprecated 

type VariableType string

Deprecated: only deprecated methods use this type. Call Start and Stop to make recordings. 

const (
	// NoSanitization indicates that the recorded value should not be sanitized.
	NoSanitization VariableType = "default"
	// Secret_String indicates that the recorded value should be replaced with a sanitized value.
	Secret_String VariableType = "secret_string"
	// Secret_Base64String indicates that the recorded value should be replaced with a sanitized valid base-64 string value.
	Secret_Base64String VariableType = "secret_base64String"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.