transforms

Documentation

Overview

Package transforms provides a mechanism to define and apply string data transformations, with the transformations defined by regular expressions to match data to be transformed, and regular expression generators to specify additional or replacement data.

Index

• Constants
• Variables
• func WrapDialerWithHTTPTransformer(dialer common.Dialer, params *HTTPTransformerParameters) common.Dialer
• type HTTPNormalizer
  • func NewHTTPNormalizer(conn net.Conn) *HTTPNormalizer
  • func (t *HTTPNormalizer) Close() error
  • func (t *HTTPNormalizer) GetMetrics() common.LogFields
  • func (t *HTTPNormalizer) Read(buffer []byte) (int, error)
  • func (t *HTTPNormalizer) SetDeadline(tt time.Time) error
  • func (t *HTTPNormalizer) SetReadDeadline(tt time.Time) error
  • func (t *HTTPNormalizer) SetWriteDeadline(tt time.Time) error
  • func (t *HTTPNormalizer) Write(b []byte) (n int, err error)
• type HTTPNormalizerListener
  • func WrapListenerWithHTTPNormalizer(listener net.Listener) *HTTPNormalizerListener
  • func (t *HTTPNormalizerListener) Accept() (net.Conn, error)
• type HTTPTransformer
  • func (t *HTTPTransformer) GetMetrics() common.LogFields
  • func (t *HTTPTransformer) Write(b []byte) (int, error)
• type HTTPTransformerParameters
• type ObfuscatorSeedTransformerParameters
  • func (t *ObfuscatorSeedTransformerParameters) Apply(b []byte) error
• type ScopedSpecNames
  • func (scopedSpecs ScopedSpecNames) Validate(specs Specs) error
• type Spec
  • func (spec Spec) Apply(seed *prng.Seed, input []byte) ([]byte, error)
  • func (spec Spec) ApplyPrefix(seed *prng.Seed, minLength int) ([]byte, int, error)
  • func (spec Spec) ApplyString(seed *prng.Seed, input string) (string, error)
• type Specs
  • func (specs Specs) Select(scope string, scopedSpecs ScopedSpecNames) (string, Spec)
  • func (specs Specs) Validate(prefixMode bool) error

Constants

const (
	SCOPE_ANY = ""
)

Variables

var ErrPassthroughActive = std_errors.New("passthrough")

Functions

func WrapDialerWithHTTPTransformer

func WrapDialerWithHTTPTransformer(dialer common.Dialer, params *HTTPTransformerParameters) common.Dialer

Types

type HTTPNormalizer

type HTTPNormalizer struct {

	// ValidateMeekCookieResult stores the result from calling
	// validateMeekCookie.
	ValidateMeekCookieResult []byte

	net.Conn
	// contains filtered or unexported fields
}

HTTPNormalizer wraps a net.Conn, intercepting Read calls, and normalizes any HTTP requests that are read. The HTTP request components preceeding the body are normalized; i.e. the Request-Line and headers.

Each HTTP request read from the underlying net.Conn is normalized and then returned over subsequent Read calls.

HTTPNormalizer is not safe for concurrent use.

func NewHTTPNormalizer

func NewHTTPNormalizer(conn net.Conn) *HTTPNormalizer

func (*HTTPNormalizer) Close

func (t *HTTPNormalizer) Close() error

func (*HTTPNormalizer) GetMetrics

func (t *HTTPNormalizer) GetMetrics() common.LogFields

GetMetrics implements the common.MetricsSource interface.

func (*HTTPNormalizer) Read

func (t *HTTPNormalizer) Read(buffer []byte) (int, error)

Read implements the net.Conn interface.

Note: it is assumed that the underlying transport, net.Conn, is a reliable stream transport, i.e. TCP, therefore it is required that the caller stop calling Read() on an instance of HTTPNormalizer after an error is returned because, following this assumption, the connection will have failed when a Read() call to the underlying net.Conn fails; a new connection must be established, net.Conn, and wrapped with a new HTTPNormalizer.

Warning: Does not handle chunked encoding. Must be called synchronously.

func (*HTTPNormalizer) SetDeadline

func (t *HTTPNormalizer) SetDeadline(tt time.Time) error

func (*HTTPNormalizer) SetReadDeadline

func (t *HTTPNormalizer) SetReadDeadline(tt time.Time) error

func (*HTTPNormalizer) SetWriteDeadline

func (t *HTTPNormalizer) SetWriteDeadline(tt time.Time) error

func (*HTTPNormalizer) Write

func (t *HTTPNormalizer) Write(b []byte) (n int, err error)

type HTTPNormalizerListener

type HTTPNormalizerListener struct {
	HeaderWriteOrder          []string
	MaxReqLineAndHeadersSize  int
	ProhibitedHeaders         []string
	PassthroughAddress        string
	PassthroughDialer         func(network, address string) (net.Conn, error)
	PassthroughLogPassthrough func(clientIP string, tunnelError error, logFields map[string]interface{})
	ValidateMeekCookie        func(clientIP string, rawCookies []byte) ([]byte, error)

	net.Listener
}

Note: all config fields must be set before calling Accept.

func WrapListenerWithHTTPNormalizer

func WrapListenerWithHTTPNormalizer(listener net.Listener) *HTTPNormalizerListener

func (*HTTPNormalizerListener) Accept

func (t *HTTPNormalizerListener) Accept() (net.Conn, error)

type HTTPTransformer

type HTTPTransformer struct {
	net.Conn
	// contains filtered or unexported fields
}

HTTPTransformer wraps a net.Conn, intercepting Write calls and applying the specified protocol transform.

The HTTP request to be written (input to the Write) is converted to a string, transformed, and converted back to binary and then actually written to the underlying net.Conn.

HTTPTransformer is not safe for concurrent use.

func (*HTTPTransformer) GetMetrics

func (t *HTTPTransformer) GetMetrics() common.LogFields

GetMetrics implements the common.MetricsSource interface.

func (*HTTPTransformer) Write

func (t *HTTPTransformer) Write(b []byte) (int, error)

Write implements the net.Conn interface.

Note: it is assumed that the underlying transport, net.Conn, is a reliable stream transport, i.e. TCP, therefore it is required that the caller stop calling Write() on an instance of HTTPTransformer after an error is returned because, following this assumption, the connection will have failed when a Write() call to the underlying net.Conn fails; a new connection must be established, net.Conn, and wrapped with a new HTTPTransformer. For this reason, the return value may be the number of bytes buffered internally and not the number of bytes written to the underlying net.Conn when a non-nil error is returned.

Warning: Does not handle chunked encoding and multiple HTTP requests written in a single Write(). Must be called synchronously.

type HTTPTransformerParameters

type HTTPTransformerParameters struct {
	// ProtocolTransformName specifies the name associated with
	// ProtocolTransformSpec and is used for metrics.
	ProtocolTransformName string

	// ProtocolTransformSpec specifies a transform to apply to the HTTP request.
	// See: "github.com/toidicakhia/psiphon-tunnel/psiphon/common/transforms".
	//
	// HTTP transforms include strategies discovered by the Geneva team,
	// https://geneva.cs.umd.edu.
	ProtocolTransformSpec Spec

	// ProtocolTransformSeed specifies the seed to use for generating random
	// data in the ProtocolTransformSpec transform. To replay a transform,
	// specify the same seed.
	ProtocolTransformSeed *prng.Seed
}

type ObfuscatorSeedTransformerParameters

type ObfuscatorSeedTransformerParameters struct {
	TransformName string
	TransformSpec Spec
	TransformSeed *prng.Seed
}

func (*ObfuscatorSeedTransformerParameters) Apply

func (t *ObfuscatorSeedTransformerParameters) Apply(b []byte) error

Apply applies the transformation in-place to the given slice of bytes. No change is made if the tranformation fails.

type ScopedSpecNames

type ScopedSpecNames map[string][]string

ScopedSpecNames groups a list of Specs, referenced by their Spec name, with the group defined by a scope. The meaning of scope depends on the context in which the transforms are to be used.

For example, in the context of DNS request transforms, the scope is the DNS server for which a specific group of transforms is known to be effective.

The scope name "" is SCOPE_ANY, and matches any input scope name when there is no specific entry for that scope name in ScopedSpecNames.

func (ScopedSpecNames) Validate

func (scopedSpecs ScopedSpecNames) Validate(specs Specs) error

Validate checks that the ScopedSpecNames is well-formed and referenced Spec names are defined in the corresponding input specs.

type Spec

type Spec [][2]string

Spec is a transform spec. A spec is a list of individual transforms to be applied in order. Each transform is defined by two elements: a regular expression to by matched against the input; and a regular expression generator which generates new data. Subgroups from the regular expression may be specified in the regular expression generator, and are populated with the subgroup match, and in this way parts of the original matching data may be retained in the transformed data.

For example, with the transform [2]string{"([a-b])", "\\$\\ {1\\}c"}, substrings consisting of the characters 'a' and 'b' will be transformed into the same substring with a single character 'c' appended.

func (Spec) Apply

func (spec Spec) Apply(seed *prng.Seed, input []byte) ([]byte, error)

Apply applies the Spec to the input bytes, producing the output bytes.

The input seed is used for all random generation. The same seed can be supplied to produce the same output, for replay.

func (Spec) ApplyPrefix

func (spec Spec) ApplyPrefix(seed *prng.Seed, minLength int) ([]byte, int, error)

ApplyPrefix unlike other Apply methods, does not apply the Spec to an input. It instead generates a sequence of bytes according to the Spec, and returns at least minLength bytes if the Spec generates fewer than minLength bytes.

The input seed is used for all random number generation. The same seed can be supplied to produce the same output, for replay.

func (Spec) ApplyString

func (spec Spec) ApplyString(seed *prng.Seed, input string) (string, error)

ApplyString applies the Spec to the input string, producing the output string.

The input seed is used for all random generation. The same seed can be supplied to produce the same output, for replay.

type Specs

type Specs map[string]Spec

Specs is a set of named Specs.

func (Specs) Select

func (specs Specs) Select(scope string, scopedSpecs ScopedSpecNames) (string, Spec)

Select picks a Spec from Specs based on the input scope and scoping rules. If the input scope name is defined in scopedSpecs, that match takes precedence. Otherwise SCOPE_ANY is selected, when present.

After the scope is resolved, Select randomly selects from the matching Spec list.

Select will return "", nil when no selection can be made.

func (Specs) Validate

func (specs Specs) Validate(prefixMode bool) error

Validate checks that all entries in a set of Specs is well-formed, with valid regular expressions.