martian

package module
v0.0.0-...-c724f24 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 18, 2015 License: Apache-2.0 Imports: 18 Imported by: 0

README

Martian Proxy Build Status

Martian Proxy is a programmable HTTP proxy designed to be used for testing.

Martian is a great tool to use if you want to:

  • Verify that all (or some subset) of requests are secure
  • Mock external services at the network layer
  • Inject headers, modify cookies or perform other mutations of HTTP requests and responses
  • Verify that pingbacks happen when you think they should
  • Unwrap encrypted traffic (requires install of CA certificate in browser)

By taking advantage of Go cross-compilation, Martian can be deployed anywhere that Go can target.

Getting Started

Start the Proxy

Running an instance of Martian is as simple as

go run examples/main.go

If you want to see system logs as Martian is running, pass in the verbosity flag:

go run examples/main.go -- -v=0

By default, Martian will be running on port 8080. The port can be specified via a flag:

go run examples/main.go -- --port=9999
Configure

Once Martian is running, you need to configure its behavior. Without configuration, Martian is just proxying without doing anything to the requests or responses. If enabled, logging will take place without additional configuration.

Martian is configured by JSON messages sent over HTTP that take the general form of:

{
  "header.Modifier": {
    "scope": ["response"],
    "name": "Test-Header",
    "value": "true"
  }
}

The above configuration tells Martian to inject a header with the name "Test-Header" and the value "true" on all responses.

Let's break down the parts of this message.

  • [package.Type]: The package.Type of the modifier that you want to use. In this case, it's "header.Modifier", which is the name of the modifier that sets headers (to learn more about the header.Modifier, please refer to the modifier reference.

  • [package.Type].scope: Indicates whether to apply to the modifier to requests, responses or both. This can be an array containing "request", "response", or both.

  • [package.Type].[key]: Modifier specific data. In the case of the header modifier, we need the nameandvalue` of the header.

This is a simple configuration, for more complex configurations, modifiers are combined with groups and filters to compose the desired behavior.

To configure Martian, POST the JSON to host:port/martian/modifiers. You'll want to use whatever mechanism your language of choice provides you to make HTTP requests, but for demo purposes, curl works (assuming your configuration is in a file called modifier.json).

    curl -x localhost:8080 \
         -X POST \
         -H "Content-Type: application/json" \
         -d @modifier.json \
            "http://localhost:8080/martian/modifiers"
Check Verifiers

Let's assume that you've configured Martian to verify the presence a specific header in responses to a specific URL.

Here's a configuration to verify that all requests to example.com return responses with a 200 OK.

      {
        "url.Filter": {
          "scope": ["request", "response"],
          "host" : "example.com",
          "modifier" : {
            "status.Verifier": {
              "scope" : ["response"],
              "statusCode": 200
            }
          }
        }
      }

Once Martian is running, configured and the requests and resultant responses you wish to verify have taken place, you can verify your expectation that you only got back 200 OK responses.

To check verifications, perform

GET host:port/martian/verify

Failed expectations are tracked as errors, and the list of errors are retrieved by making a GET request to host:port/martian/verify, which will return a list of errors:

  {
      "errors" : [
          {
              "message": "response(http://example.com) status code verify failure: got 500, want 200"
          },
          {
              "message": "response(http://example.com/foo) status code verify failure: got 500, want 200"
          }
      ]
  }

Verification errors are held in memory until they are explicitly cleared by

POST host:port/martian/verify/reset

Martian as a Library

Martian can also be included into any Go program and used as a library.

Modifiers All The Way Down

Martian's request and response modification system is designed to be general and extensible. The design objective is to provide individual modifier behaviors that can arranged to build out nearly any desired modification.

When working with Martian to compose behaviors, you'll need to be familiar with these different types of interactions:

  • Modifiers: Changes the state of a request or a response
  • Filters: Conditionally allows a contained Modifier to execute
  • Groups: Bundles multiple modifiers to be executed in the order specified in the group
  • Verifiers: Tracks network traffic against expectations

Modifiers, filters and groups all implement RequestModifer, ResponseModifier or RequestResponseModifier (defined in martian.go).

ModifyRequest(ctx *martian.Context, req *http.Request) error

ModifyResponse(ctx *martian.Context, res *http.Response) error

Throughout the code (and this documentation) you'll see the word "modifier" used as a term that encompasses modifiers, groups and filters. Even though a group does not modify a request or response, we still refer to it as a "modifier".

We refer to anything that implements the modifier interface as a Modifier.

Parser Registration

Each modifier must register its own parser with Martian. The parser is responsible for parsing a JSON message into a Go struct that implements a modifier interface.

Martian holds modifier parsers as a map of strings to functions that is built out at run-time. Each modifier is responsible for registering its parser with a call to parse.Register in init().

Signature of parse.Register:

Register(name string, parseFunc func(b []byte) (interface{}, error))

Register takes in the key as a string in the form package.Type. For instance, cookie_modifier registers itself with the key cookie.Modifier and query_string_filter registers itself as querystring.Filter. This string is the same as the value of name in the JSON configuration message.

In the following configuration message, header.Modifier is how the header modifier is registered in the init() of header_modifier.go.

{
  "header.Modifier": {
    "scope": ["response"],
    "name" : "Test-Header",
    "value" : "true"
  }
}

Example of parser registration from header_modifier.go:

func init() {
  parse.Register("header.Modifier", modifierFromJSON)
}

func modifierFromJSON(b []byte) (interface{}, error) {
  ...
}
Adding Your Own Modifier

If you have a use-case in mind that we have not developed modifiers, filters or verifiers for, you can easily extend Martian to your very specific needs.

There are 2 mandatory parts of a modifier:

  • Implement the modifier interface
  • Register the parser

Any Go struct that implements those interfaces can act as a modifier.

Contact

Google Group (General and Dev): https://groups.google.com/forum/#!forum/martianproxy

Disclaimer

This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.

Documentation

Overview

Package martian provides an HTTP/1.1 proxy with an API for configurable request and response modifiers.

Index

Constants

This section is empty.

Variables

View Source 
var ErrAuthRequired = errors.New("authentication is required")

ErrAuthRequired is the error returned by modifiers when ctx.Auth.ID is required, but empty.

View Source 
var MaxSerialNumber = big.NewInt(0).SetBytes(bytes.Repeat([]byte{255}, 20))

MaxSerialNumber is the upper boundary that is used to create unique serial numbers for the certificate. This can be any unsigned integer up to 20 bytes (2^(8*20)-1).

Functions

func Debugf 

func Debugf(format string, args ...interface{})

Debugf logs a debug message with caller information.

func Errorf 

func Errorf(format string, args ...interface{})

Errorf logs an error message with caller information.

func Infof 

func Infof(format string, args ...interface{})

Infof logs an info message with caller information.

func NewTemplate 

func NewTemplate(org, host string, start, end time.Time, pub interface{}) (*x509.Certificate, error)

NewTemplate returns a new base *x509.Certificate.

Types

type Auth 

type Auth struct {
	// ID is the identifier for a user.
	ID string
	// Error is used to signal that ID is required, but is either
	// blank or invalid per the semantics of the modifier.
	Error error
}

Auth contains per session authentication information.

func (*Auth) Reset 

func (auth *Auth) Reset()

Reset resets the auth fields to their zero values.

type Context 

type Context struct {
	// Auth is the session authentication information.
	Auth *Auth
	// SkipRoundTrip signals to the proxy that it should not send the request over the wire.
	SkipRoundTrip bool
}

Context contains information for a proxy session.

func NewContext 

func NewContext() *Context

NewContext returns an empty martian.Context.

type MITM 

type MITM struct {
	// Authority is the CA certificate used to sign MITM certificates.
	Authority *x509.Certificate
	// PublicKey used to create MITM certificates.
	PublicKey interface{}
	// PrivateKey of the CA used to sign MITM certificates.
	PrivateKey interface{}
	// Validity is the window of time around time.Now() that the
	// certificate will be valid.
	Validity time.Duration
	// Organization that is displayed as the owner of the
	// certificate.
	Organization string
}

MITM is the configuration for using the Proxy as a MITM.

func (*MITM) Hijack 

func (mitm *MITM) Hijack(conn net.Conn, host string) (*tls.Conn, *bufio.ReadWriter, error)

Hijack takes a net.Conn and the host name to create the SSL certificate for and returns a tls.Conn that can read and write to the given host over TLS.

type Proxy 

type Proxy struct {
	// RoundTripper used to make the request from the proxy to the target server.
	RoundTripper http.RoundTripper
	// Timeout is the length of time the connection will be kept open while idle.
	Timeout time.Duration
	// contains filtered or unexported fields
}

Proxy implements an HTTP proxy with CONNECT and TLS MITM support.

func NewProxy 

func NewProxy(mitm *MITM) *Proxy

NewProxy returns an HTTP proxy.

func (*Proxy) ModifyRequest 

func (p *Proxy) ModifyRequest(ctx *Context, req *http.Request) error

ModifyRequest modifies the request.

func (*Proxy) ModifyResponse 

func (p *Proxy) ModifyResponse(ctx *Context, res *http.Response) error

ModifyResponse modifies the response.

func (*Proxy) ServeHTTP 

func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP handles requests from a connection and writes responses.

If a MITM config was provided and a CONNECT request is received, the proxy will generate a fake TLS certificate using the given authority certificate and perform the TLS handshake. The request will then be decrypted and modifiers will be run, followed by the request being re-encrypted and sent to the destination server.

If no MITM config was provided and a CONNECT request is received, the proxy will open a connection to the destination server and copy the encrypted bytes directly, as per normal CONNECT semantics.

func (*Proxy) SetConnectRequestModifier 

func (p *Proxy) SetConnectRequestModifier(creqmod RequestModifier)

SetConnectRequestModifier sets the request modifier for the CONNECT request.

func (*Proxy) SetConnectResponseModifier 

func (p *Proxy) SetConnectResponseModifier(cresmod ResponseModifier)

SetConnectResponseModifier sets the response modifier for the CONNECT response.

func (*Proxy) SetRequestModifier 

func (p *Proxy) SetRequestModifier(reqmod RequestModifier)

SetRequestModifier sets the request modifier for the decrypted request.

func (*Proxy) SetResponseModifier 

func (p *Proxy) SetResponseModifier(resmod ResponseModifier)

SetResponseModifier sets the response modifier for the decrypted response.

type RequestModifier 

type RequestModifier interface {
	// ModifyRequest modifies the request.
	//
	// Modifying the request body is possible, though the req.Body must be
	// replaced with a new io.ReadCloser since rewinding the body is
	// unsupported.
	ModifyRequest(ctx *Context, req *http.Request) error
}

RequestModifier is an interface that defines a request modifier that can be used by a proxy.

type RequestModifierFunc 

type RequestModifierFunc func(ctx *Context, req *http.Request) error

RequestModifierFunc is an adapter for using a function with the given signature as a RequestModifier.

func (RequestModifierFunc) ModifyRequest 

func (f RequestModifierFunc) ModifyRequest(ctx *Context, req *http.Request) error

ModifyRequest modifies the request using the given function.

type RequestResponseModifier 

type RequestResponseModifier interface {
	RequestModifier
	ResponseModifier
}

RequestResponseModifier is an interface that is both a ResponseModifier and a RequestModifier.

type ResponseModifier 

type ResponseModifier interface {
	// ModifyResponse modifies the response.
	//
	// Modifying the response body is possible, though the res.Body must be
	// replaced with a new io.ReadCloser since rewinding the body is
	// unsupported.
	ModifyResponse(ctx *Context, res *http.Response) error
}

ResponseModifier is an interface that defines a response modifier that can be used by a proxy.

type ResponseModifierFunc 

type ResponseModifierFunc func(ctx *Context, res *http.Response) error

ResponseModifierFunc is an adapter for using a function with the given signature as a ResponseModifier.

func (ResponseModifierFunc) ModifyResponse 

func (f ResponseModifierFunc) ModifyResponse(ctx *Context, res *http.Response) error

ModifyResponse modifies the response using the given function.

type RoundTripFunc 

type RoundTripFunc func(*http.Request) (*http.Response, error)

RoundTripFunc is an adapter for using a function with the given signature as an http.RoundTripper.

func (RoundTripFunc) RoundTrip 

func (f RoundTripFunc) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip delegates to the provided RoundTrip function.

Source Files

View all Source files

Directories

Path Synopsis
Package auth provides filtering support for a martian.Proxy based on ctx.Auth.ID.
 Package auth provides filtering support for a martian.Proxy based on ctx.Auth.ID.
Package body allows for the replacement of message body on responses.
 Package body allows for the replacement of message body on responses.
Package cookie allows for the modification of cookies on http requests and responses.
 Package cookie allows for the modification of cookies on http requests and responses.
Package cors provides CORS support for http.Handlers.
 Package cors provides CORS support for http.Handlers.
proxy is a martian.Proxy configurable via HTTP.
 proxy is a martian.Proxy configurable via HTTP.
Package header provides utilities for modifying, filtering, and verifying headers in martian.Proxy.
 Package header provides utilities for modifying, filtering, and verifying headers in martian.Proxy.
Package ipauth provides a martian.Modifier that sets auth based on IP.
 Package ipauth provides a martian.Modifier that sets auth based on IP.
Package martianhttp provides HTTP handlers for managing the state of a martian.Proxy.
 Package martianhttp provides HTTP handlers for managing the state of a martian.Proxy.
Package martianurl provides utilities for modifying, filtering, and verifying URLs in martian.Proxy.
 Package martianurl provides utilities for modifying, filtering, and verifying URLs in martian.Proxy.
Package method provides utilities for verifying method type in martian.Proxy.
 Package method provides utilities for verifying method type in martian.Proxy.
Package parse constructs martian modifiers from JSON messages.
 Package parse constructs martian modifiers from JSON messages.
Package pingback provides verification that specific URLs have been seen by the proxy.
 Package pingback provides verification that specific URLs have been seen by the proxy.
Package priority allows grouping modifiers and applying them in priority order.
 Package priority allows grouping modifiers and applying them in priority order.
Package proxyauth provides authentication support via the Proxy-Authorization header.
 Package proxyauth provides authentication support via the Proxy-Authorization header.
Package proxyutil provides functionality for building proxies.
 Package proxyutil provides functionality for building proxies.
Package querystring contains a modifier to rewrite query strings in a request.
 Package querystring contains a modifier to rewrite query strings in a request.
Package status contains a modifier to rewrite the status code on a response.
 Package status contains a modifier to rewrite the status code on a response.
Package verify provides support for using martian modifiers for request and response verifications.
 Package verify provides support for using martian modifiers for request and response verifications.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.