martian

package module
v2.0.0-beta.2+incompat... Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2015 License: Apache-2.0 Imports: 16 Imported by: 0

README

Martian Proxy Build Status

Martian Proxy is a programmable HTTP proxy designed to be used for testing.

Martian is a great tool to use if you want to:

  • Verify that all (or some subset) of requests are secure
  • Mock external services at the network layer
  • Inject headers, modify cookies or perform other mutations of HTTP requests and responses
  • Verify that pingbacks happen when you think they should
  • Unwrap encrypted traffic (requires install of CA certificate in browser)

By taking advantage of Go cross-compilation, Martian can be deployed anywhere that Go can target.

Requirements

Go 1.5

Getting Started

Installation

Martian Proxy can be installed using go install

go install github.com/google/martian/cmd/proxy
Start the Proxy

Assuming you've installed Martian, running the proxy is as simple as

$GOPATH/bin/proxy

If you want to see system logs as Martian is running, pass in the verbosity flag:

$GOPATH/bin/proxy -v

For logging of requests and responses a logging modifier is available

By default, Martian will be running on port 8080. The port can be specified via a flag:

$GOPATH/bin/proxy -port=9999
Configure

Once Martian is running, you need to configure its behavior. Without configuration, Martian is just proxying without doing anything to the requests or responses. If enabled, logging will take place without additional configuration.

Martian is configured by JSON messages sent over HTTP that take the general form of:

{
  "header.Modifier": {
    "scope": ["response"],
    "name": "Test-Header",
    "value": "true"
  }
}

The above configuration tells Martian to inject a header with the name "Test-Header" and the value "true" on all responses.

Let's break down the parts of this message.

  • [package.Type]: The package.Type of the modifier that you want to use. In this case, it's "header.Modifier", which is the name of the modifier that sets headers (to learn more about the header.Modifier, please refer to the modifier reference.

  • [package.Type].scope: Indicates whether to apply to the modifier to requests, responses or both. This can be an array containing "request", "response", or both.

  • [package.Type].[key]: Modifier specific data. In the case of the header modifier, we need the nameandvalue` of the header.

This is a simple configuration, for more complex configurations, modifiers are combined with groups and filters to compose the desired behavior.

To configure Martian, POST the JSON to http://martian.proxy/modifiers. You'll want to use whatever mechanism your language of choice provides you to make HTTP requests, but for demo purposes, curl works (assuming your configuration is in a file called modifier.json).

    curl -x localhost:8080 \
         -X POST \
         -H "Content-Type: application/json" \
         -d @modifier.json \
            "http://martian.proxy/configure"
Intercepting HTTPS Requests and Responses

Martian supports modifying HTTPS requests and responses if configured to do so.

In order for Martian to intercept HTTPS traffic a custom CA certificate must be installed in the browser so that connection warnings are not shown.

The easiest way to install the CA certificate is to start the proxy with the necessary flags to use a custom CA certificate and private key using the -cert and -key flags, or to have the proxy generate one using the -generate-ca-cert flag.

After the proxy has started, visit http://martian.proxy/authority.cer in the browser configured to use the proxy and a prompt will be displayed to install the certificate.

Several flags are available in examples/main.go to help configure MITM functionality:

-key=""
  PEM encoded private key file of the CA certificate provided in -cert; used
  to sign certificates that are generated on-the-fly
-cert=""
  PEM encoded CA certificate file used to generate certificates
-generate-ca-cert=false
  generates a CA certificate and private key to use for man-in-the-middle;
  most users choosing this option will immediately visit
  http://martian.proxy/authority.cer in the browser whose traffic is to be
  intercepted to install the newly generated CA certificate
-organization="Martian Proxy"
  organization name set on the dynamically-generated certificates during
  man-in-the-middle
-validity="1h"
  window of time around the time of request that the dynamically-generated
  certificate is valid for; the duration is set such that the total valid
  timeframe is double the value of validity (1h before & 1h after)
Check Verifiers

Let's assume that you've configured Martian to verify the presence a specific header in responses to a specific URL.

Here's a configuration to verify that all requests to example.com return responses with a 200 OK.

      {
        "url.Filter": {
          "scope": ["request", "response"],
          "host" : "example.com",
          "modifier" : {
            "status.Verifier": {
              "scope" : ["response"],
              "statusCode": 200
            }
          }
        }
      }

Once Martian is running, configured and the requests and resultant responses you wish to verify have taken place, you can verify your expectation that you only got back 200 OK responses.

To check verifications, perform

GET http://martian.proxy/verify

Failed expectations are tracked as errors, and the list of errors are retrieved by making a GET request to host:port/martian/verify, which will return a list of errors:

  {
      "errors" : [
          {
              "message": "response(http://example.com) status code verify failure: got 500, want 200"
          },
          {
              "message": "response(http://example.com/foo) status code verify failure: got 500, want 200"
          }
      ]
  }

Verification errors are held in memory until they are explicitly cleared by

POST http://martian.proxy/verify/reset

Martian as a Library

Martian can also be included into any Go program and used as a library.

Modifiers All The Way Down

Martian's request and response modification system is designed to be general and extensible. The design objective is to provide individual modifier behaviors that can arranged to build out nearly any desired modification.

When working with Martian to compose behaviors, you'll need to be familiar with these different types of interactions:

  • Modifiers: Changes the state of a request or a response
  • Filters: Conditionally allows a contained Modifier to execute
  • Groups: Bundles multiple modifiers to be executed in the order specified in the group
  • Verifiers: Tracks network traffic against expectations

Modifiers, filters and groups all implement RequestModifer, ResponseModifier or RequestResponseModifier (defined in martian.go).

ModifyRequest(req *http.Request) error

ModifyResponse(res *http.Response) error

Throughout the code (and this documentation) you'll see the word "modifier" used as a term that encompasses modifiers, groups and filters. Even though a group does not modify a request or response, we still refer to it as a "modifier".

We refer to anything that implements the modifier interface as a Modifier.

Parser Registration

Each modifier must register its own parser with Martian. The parser is responsible for parsing a JSON message into a Go struct that implements a modifier interface.

Martian holds modifier parsers as a map of strings to functions that is built out at run-time. Each modifier is responsible for registering its parser with a call to parse.Register in init().

Signature of parse.Register:

Register(name string, parseFunc func(b []byte) (interface{}, error))

Register takes in the key as a string in the form package.Type. For instance, cookie_modifier registers itself with the key cookie.Modifier and query_string_filter registers itself as querystring.Filter. This string is the same as the value of name in the JSON configuration message.

In the following configuration message, header.Modifier is how the header modifier is registered in the init() of header_modifier.go.

{
  "header.Modifier": {
    "scope": ["response"],
    "name" : "Test-Header",
    "value" : "true"
  }
}

Example of parser registration from header_modifier.go:

func init() {
  parse.Register("header.Modifier", modifierFromJSON)
}

func modifierFromJSON(b []byte) (interface{}, error) {
  ...
}
Adding Your Own Modifier

If you have a use-case in mind that we have not developed modifiers, filters or verifiers for, you can easily extend Martian to your very specific needs.

There are 2 mandatory parts of a modifier:

  • Implement the modifier interface
  • Register the parser

Any Go struct that implements those interfaces can act as a modifier.

Contact

For questions and comments on how to use Martian, features announcements, or design discussions check out our public Google Group at https://groups.google.com/forum/#!forum/martianproxy-users.

For security related issues please send a detailed report to our private core group at martianproxy-core@googlegroups.com.

Disclaimer

This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.

Documentation

Overview

Package martian provides an HTTP/1.1 proxy with an API for configurable request and response modifiers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Context 

func Context(req *http.Request) *session.Context

Context returns the associated context for the request.

func RemoveContext 

func RemoveContext(req *http.Request)

RemoveContext removes the context for a given request.

func SetContext 

func SetContext(req *http.Request, ctx *session.Context)

SetContext associates the context with request.

Types

type Proxy 

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy is an HTTP proxy with support for TLS MITM and customizable behavior.

func NewProxy 

func NewProxy() *Proxy

NewProxy returns a new HTTP proxy.

func (*Proxy) Close 

func (p *Proxy) Close()

Close sets the proxying to the closing state and waits for all connections to resolve.

func (*Proxy) Closing 

func (p *Proxy) Closing() bool

Closing returns whether the proxy is in the closing state.

func (*Proxy) Serve 

func (p *Proxy) Serve(l net.Listener) error

Serve accepts connections from the listener and handles the requests.

func (*Proxy) SetDownstreamProxy 

func (p *Proxy) SetDownstreamProxy(proxyURL *url.URL)

SetDownstreamProxy sets the proxy that receives requests from the upstream proxy.

func (*Proxy) SetMITM 

func (p *Proxy) SetMITM(config *mitm.Config)

SetMITM sets the config to use for MITMing of CONNECT requests.

func (*Proxy) SetMux 

func (p *Proxy) SetMux(mux *http.ServeMux)

SetMux sets the http.ServeMux to use for internal proxy requests. Defaults to http.DefaultServeMux.

func (*Proxy) SetRequestModifier 

func (p *Proxy) SetRequestModifier(reqmod RequestModifier)

SetRequestModifier sets the request modifier.

func (*Proxy) SetResponseModifier 

func (p *Proxy) SetResponseModifier(resmod ResponseModifier)

SetResponseModifier sets the response modifier.

func (*Proxy) SetRoundTripper 

func (p *Proxy) SetRoundTripper(rt http.RoundTripper)

SetRoundTripper sets the http.RoundTripper of the proxy.

func (*Proxy) SetTimeout 

func (p *Proxy) SetTimeout(timeout time.Duration)

SetTimeout sets the request timeout of the proxy.

type RequestModifier 

type RequestModifier interface {
	// ModifyRequest modifies the request.
	ModifyRequest(req *http.Request) error
}

RequestModifier is an interface that defines a request modifier that can be used by a proxy.

type RequestModifierFunc 

type RequestModifierFunc func(req *http.Request) error

RequestModifierFunc is an adapter for using a function with the given signature as a RequestModifier.

func (RequestModifierFunc) ModifyRequest 

func (f RequestModifierFunc) ModifyRequest(req *http.Request) error

ModifyRequest modifies the request using the given function.

type RequestResponseModifier 

type RequestResponseModifier interface {
	RequestModifier
	ResponseModifier
}

RequestResponseModifier is an interface that is both a ResponseModifier and a RequestModifier.

func Noop 

func Noop(id string) RequestResponseModifier

Noop returns a modifier that does not change the request or the response.

type ResponseModifier 

type ResponseModifier interface {
	// ModifyResponse modifies the response.
	ModifyResponse(res *http.Response) error
}

ResponseModifier is an interface that defines a response modifier that can be used by a proxy.

type ResponseModifierFunc 

type ResponseModifierFunc func(res *http.Response) error

ResponseModifierFunc is an adapter for using a function with the given signature as a ResponseModifier.

func (ResponseModifierFunc) ModifyResponse 

func (f ResponseModifierFunc) ModifyResponse(res *http.Response) error

ModifyResponse modifies the response using the given function.

Source Files

View all Source files

Directories

Path Synopsis
Package auth provides filtering support for a martian.Proxy based on auth ID.
 Package auth provides filtering support for a martian.Proxy based on auth ID.
Package body allows for the replacement of message body on responses.
 Package body allows for the replacement of message body on responses.
cmd
proxy
proxy is an HTTP/S proxy configurable via an HTTP API.
 proxy is an HTTP/S proxy configurable via an HTTP API.
Package cookie allows for the modification of cookies on http requests and responses.
 Package cookie allows for the modification of cookies on http requests and responses.
Package cors provides CORS support for http.Handlers.
 Package cors provides CORS support for http.Handlers.
Package header provides utilities for modifying, filtering, and verifying headers in martian.Proxy.
 Package header provides utilities for modifying, filtering, and verifying headers in martian.Proxy.
Package ipauth provides a martian.Modifier that sets auth based on IP.
 Package ipauth provides a martian.Modifier that sets auth based on IP.
Package log provides a universal logger for martian packages.
 Package log provides a universal logger for martian packages.
Package martianhttp provides HTTP handlers for managing the state of a martian.Proxy.
 Package martianhttp provides HTTP handlers for managing the state of a martian.Proxy.
Package martianlog provides a Martian modifier that logs the request and response.
 Package martianlog provides a Martian modifier that logs the request and response.
Package martiantest provides helper utilities for testing modifiers.
 Package martiantest provides helper utilities for testing modifiers.
Package martianurl provides utilities for modifying, filtering, and verifying URLs in martian.Proxy.
 Package martianurl provides utilities for modifying, filtering, and verifying URLs in martian.Proxy.
Package messageview provides no-op snapshots for HTTP requests and responses.
 Package messageview provides no-op snapshots for HTTP requests and responses.
Package method provides utilities for verifying method type in martian.Proxy.
 Package method provides utilities for verifying method type in martian.Proxy.
Package mitm provides tooling for MITMing TLS connections.
 Package mitm provides tooling for MITMing TLS connections.
Package parse constructs martian modifiers from JSON messages.
 Package parse constructs martian modifiers from JSON messages.
Package pingback provides verification that specific URLs have been seen by the proxy.
 Package pingback provides verification that specific URLs have been seen by the proxy.
Package priority allows grouping modifiers and applying them in priority order.
 Package priority allows grouping modifiers and applying them in priority order.
Package proxyauth provides authentication support via the Proxy-Authorization header.
 Package proxyauth provides authentication support via the Proxy-Authorization header.
Package proxyutil provides functionality for building proxies.
 Package proxyutil provides functionality for building proxies.
Package querystring contains a modifier to rewrite query strings in a request.
 Package querystring contains a modifier to rewrite query strings in a request.
Package session provides contextual information about a single HTTP/S connection and its associated requests and responses.
 Package session provides contextual information about a single HTTP/S connection and its associated requests and responses.
Package status contains a modifier to rewrite the status code on a response.
 Package status contains a modifier to rewrite the status code on a response.
Package verify provides support for using martian modifiers for request and response verifications.
 Package verify provides support for using martian modifiers for request and response verifications.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.