realclientip

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 3, 2022 License: 0BSD Imports: 4 Imported by: 5

README

GoDoc Go Playground Test coverage license

realclientip-go

X-Forwarded-For and other "real" client IP headers are often used incorrectly, resulting in bugs and security vulnerabilities. This library is an attempt to create a reference implementation of the correct ways to use such headers.

This library is written in Go, but the hope is that it will be reimplemented in other languages. Please open an issue if you would like to create such an implementation.

This library is freely licensed. You may use it as a dependency or copy it or modify it or anything else you want. It has no dependencies, is written in pure Go, and supports Go versions as far back as 1.13.

Usage

This library provides strategies for extracting the desired "real" client IP from various headers or from http.Request.RemoteAddr (the client socket IP).

strategy, err := realclientip.NewRightmostTrustedCountStrategy("X-Forwarded-For", 2)
...
clientIP := strategy.ClientIP(req.Header, req.RemoteAddr)

Try it out in the playground.

There are a number of different strategies available -- the right one will depend on your network configuration. See the documentation to find out what's available and which you should use.

ClientIP is threadsafe for all strategies. The same strategy instance can be used for handling all HTTP requests, for example.

There are examples of use in the documentation and _examples directory.

Strategy failures

The strategy used must be chosen and tuned for your network configuration. This should result in the strategy never returning an empty string -- i.e., never failing to find a candidate for the "real" IP. Consequently, getting an empty-string result should be treated as an application error, perhaps even worthy of panicking.

For example, if you have 2 levels of trusted reverse proxies, you would probably use RightmostTrustedCountStrategy and it should work every time. If you're directly connected to the internet, you would probably use RemoteAddrStrategy or something like ChainStrategy(LeftmostNonPrivateStrategy(...), RemoteAddrStrategy) and you will be sure to get a value every time. If you're behind Cloudflare, you would probably use SingleIPHeaderStrategy("Cf-Connecting-IP") and it should work every time.

So if an empty string is returned, it is either because the strategy choice or configuration is incorrect or your network configuration has changed. In either case, immediate remediation is required.

Headers

Leftmost-ish and rightmost-ish strategies support the X-Forwarded-For and Forwarded headers.

SingleIPHeaderStrategy supports any header containing a single IP address or IP:port. For a list of some common headers, see the Single-IP Headers wiki page.

You must choose exactly the correct header for your configuration. Choosing the wrong header can result in failing to get the client IP or falling victim to IP spoofing.

Do not abuse ChainStrategy to check multiple headers. There is likely only one header you should be checking, and checking more can leave you vulnerable to IP spoofing.

Forwarded header support

Support for the Forwarded header should be sufficient for the vast majority of rightmost-ish uses, but it is not complete and doesn't completely adhere to RFC 7239. See the Test_forwardedHeaderRFCDeviations test for details on deviations.

IPv6 zones

IPv6 zone identifiers are retained in the IP address returned by the strategies. Whether you should keep the zone depends on your specific use case. As a general rule, if you are not immediately using the IP address (for example, if you are appending it to the X-Forwarded-For header and passing it on), then you should include the zone. This allows downstream consumers the option to use it. If your code is the final consumer of the IP address, then keeping the zone will depend on your specific case (for example: if you're logging the IP, then you probably want the zone; if you are rate limiting by IP, then you probably want to discard it).

To split the zone off and discard it, you may use realclientip.SplitHostZone.

Known IP ranges

There is a copy of Cloudflare's IP ranges under ranges.Cloudflare. This can be used with realclientip.RightmostTrustedRangeStrategy. We may add more known cloud provider ranges in the future. Contributions are welcome to add new providers or update existing ones.

(It might be preferable to use provider APIs to retrieve the ranges, as they are guaranteed to be up-to-date.)

Implementation decisions and notes

net vs netip

At the time of writing this library, Go 1.18 was only just released. It made sense to use the older net package rather than the newer netip, so that the required Go version wouldn't be so high as to exclude some users of the library.

In the future we may wish to switch to using netip, but it will require API changes to AddressesAndRangesToIPNets, RightmostTrustedRangeStrategy, and ParseIPAddr.

Disallowed valid IPs

The values 0.0.0.0 (zero) and :: (unspecified) are valid IPs, strictly speaking. However, this library treats them as invalid as they don't make sense to its intended uses. If you have a valid use case for them, please open an issue.

Normalizing IPs

All IPs output by the library are first converted to a structure (like net.IP) and then stringified. This helps normalize the cases where there are multiple ways of encoding the same IP -- like 192.0.2.1 and ::ffff:192.0.2.1, and the various zero-collapsed states of IPv6 (fe80::1 vs fe80::0:0:0:1, etc.).

Input format strictness

Some input is allowed that isn't strictly correct. Some examples:

  • IPv4 with brackets: [2.2.2.2]:1234
  • IPv4 with zone: 2.2.2.2%eth0
  • Non-numeric port values: 2.2.2.2:nope
  • Other Forwarded header deviations

It could be argued that it would be better to be absolutely strict in what is accepted.

Code comments

As this library aspires to be a "reference implementation", the code is heavily commented. Perhaps more than is strictly necessary.

Pre-creating Strategies

Strategies are created by calling a constructor, like NewRightmostTrustedCountStrategy("Forwarded", 2). That can make it awkward to create-and-call at the same time, like NewRightmostTrustedCountStrategy("Forwarded", 2).ClientIP(r.Header, r.RemoteAddr). We could have instead implemented non-pre-created functions, like RightmostTrustedCountStrategy("Forwarded", 2, r.Header, r.RemoteAddr). The reasons for the way we did it include:

  1. A consistent interface. This enables ChainStrategy. It also enables library users to have code paths that aren't strategy-dependent, in case they want the strategy to be configurable.
  2. Pre-creation allows us to put as much of the invariant processing as possible into the creation step. (Although, in practice, so far, this is only the header name canonicalization.)
  3. No error return is required from the strategy ClientIP calls. (Although they can -- but should not -- return empty string.) All error-prone processing is done in the pre-creation.

An alternative approach could be using functions like:

func RightmostTrustedCountStrategy(headerName string, trustedRanges []*net.IPNet, headers http.Header, remoteAddr string) (Strategy, ip, error) {
...
strat, _, err := RightmostTrustedRangeStrategy("Forward", 2, "", "")              // pre-create
_, ip, err := RightmostTrustedRangeStrategy("Forward", 2, r.Header, r.RemoteAddr) // use direct

But perhaps that's no less awkward.

Interfaces vs Functions

A pre-release implementation of this library constructed functions rather than structs that implement an interface. The switch to the latter was made for a few reasons:

  • It seems slightly more Go-idiomatic.
  • It allows for adding new methods in the future without breaking the API. (Such as String().)
  • It allows for configuration information to appear in a printf of a strategy struct. This can be useful for logging.
  • The function approach is still easy to use, with the bound ClientIP method: 
    getClientIP := NewRightmostTrustedCountStrategy("Forwarded", 2).ClientIP

Other language implementations

If you want to reproduce this implementation in another language, please create an issue and we'll make a repo under this organization for you to use.

Documentation

Overview

Package realclientip provides strategies for obtaining the "real" client IP from HTTP requests.

Example (Middleware) 
package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"net/http/httptest"

	"github.com/realclientip/realclientip-go"
)

func main() {
	// Choose the right strategy for our network configuration
	strat, err := realclientip.NewRightmostNonPrivateStrategy("X-Forwarded-For")
	if err != nil {
		log.Fatal("realclientip.NewRightmostNonPrivateStrategy returned error (bad input)")
	}

	// Place our middleware before the handler
	handlerWithMiddleware := clientIPMiddleware(strat, http.HandlerFunc(handler))
	httpServer := httptest.NewServer(handlerWithMiddleware)
	defer httpServer.Close()

	req, _ := http.NewRequest("GET", httpServer.URL, nil)
	req.Header.Add("X-Forwarded-For", "1.1.1.1, 2.2.2.2, 3.3.3.3, 192.168.1.1")

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		log.Fatal(err)
	}

	b, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("%s", b)
}

type clientIPCtxKey struct{}

// Adds the "real" client IP to the request context under the clientIPCtxKey{} key.
// If the client IP couldn't be obtained, the value will be an empty string.
// We could use the RightmostNonPrivateStrategy concrete type, but instead we'll pass
// around the Strategy interface, in case we decide to change our strategy in the future.
func clientIPMiddleware(strat realclientip.Strategy, next http.Handler) http.Handler {
	fn := func(w http.ResponseWriter, r *http.Request) {
		clientIP := strat.ClientIP(r.Header, r.RemoteAddr)
		if clientIP == "" {
			// Write error log. Consider aborting the request depending on use.
			log.Fatal("Failed to find client IP")
		}

		r = r.WithContext(context.WithValue(r.Context(), clientIPCtxKey{}, clientIP))
		next.ServeHTTP(w, r)
	}

	return http.HandlerFunc(fn)
}

func handler(w http.ResponseWriter, r *http.Request) {
	clientIP := r.Context().Value(clientIPCtxKey{})
	fmt.Fprintln(w, "your IP:", clientIP)
}

Output:

 your IP: 3.3.3.3
Example (Playground) 
// We'll make a fake request
req, _ := http.NewRequest("GET", "https://example.com", nil)
req.Header.Add("X-Forwarded-For", "1.1.1.1, 2001:db8:cafe::99%eth0, 3.3.3.3, 192.168.1.1")
req.Header.Add("Forwarded", `For=fe80::abcd;By=fe80::1234, Proto=https;For=::ffff:188.0.2.128, For="[2001:db8:cafe::17]:4848", For=fc00::1`)
req.Header.Add("X-Real-IP", "4.4.4.4")
req.RemoteAddr = "192.168.1.2:8888"

var strat realclientip.Strategy

strat = realclientip.RemoteAddrStrategy{}
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 192.168.1.2

strat, _ = realclientip.NewSingleIPHeaderStrategy("X-Real-IP")
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 4.4.4.4

strat, _ = realclientip.NewLeftmostNonPrivateStrategy("Forwarded")
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 188.0.2.128

strat, _ = realclientip.NewRightmostNonPrivateStrategy("X-Forwarded-For")
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 3.3.3.3

strat, _ = realclientip.NewRightmostTrustedCountStrategy("Forwarded", 2)
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 2001:db8:cafe::17

trustedRanges, _ := realclientip.AddressesAndRangesToIPNets([]string{"192.168.0.0/16", "3.3.3.3"}...)
strat, _ = realclientip.NewRightmostTrustedRangeStrategy("X-Forwarded-For", trustedRanges)
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 2001:db8:cafe::99%eth0
ipAddr, _ := realclientip.ParseIPAddr(strat.ClientIP(req.Header, req.RemoteAddr))
fmt.Println(ipAddr.IP) // 2001:db8:cafe::99

strat = realclientip.NewChainStrategy(
	realclientip.Must(realclientip.NewSingleIPHeaderStrategy("Cf-Connecting-IP")),
	realclientip.RemoteAddrStrategy{},
)
fmt.Printf("\n%+v\n", strat)
fmt.Println(strat.ClientIP(req.Header, req.RemoteAddr)) // 192.168.1.2

Output:

{}
192.168.1.2

{headerName:X-Real-Ip}
4.4.4.4

{headerName:Forwarded}
188.0.2.128

{headerName:X-Forwarded-For}
3.3.3.3

{headerName:Forwarded trustedCount:2}
2001:db8:cafe::17

{headerName:X-Forwarded-For trustedRanges:[192.168.0.0/16 3.3.3.3/32]
2001:db8:cafe::99%eth0
2001:db8:cafe::99

{strategies:[realclientip.SingleIPHeaderStrategy{headerName:Cf-Connecting-Ip} realclientip.RemoteAddrStrategy{}]}
192.168.1.2

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddressesAndRangesToIPNets 

func AddressesAndRangesToIPNets(ranges ...string) ([]net.IPNet, error)

AddressesAndRangesToIPNets converts a slice of strings with IPv4 and IPv6 addresses and CIDR ranges (prefixes) to net.IPNet instances. If net.ParseCIDR or net.ParseIP fail, an error will be returned. Zones in addresses or ranges are not allowed and will result in an error. This is because: a) net.ParseCIDR will fail to parse a range with a zone, and b) netip.ParsePrefix will succeed but silently throw away the zone; then netip.Prefix.Contains will return false for any IP with a zone, causing confusion and bugs.

func MustParseIPAddr 

func MustParseIPAddr(ipStr string) net.IPAddr

MustParseIPAddr panics if ParseIPAddr fails.

func ParseIPAddr 

func ParseIPAddr(ipStr string) (net.IPAddr, error)

ParseIPAddr parses the given string into a net.IPAddr, which is a useful type for dealing with IPs have zones. The Go stdlib net package is lacking such a function. This will also discard any port number from the input.

func SplitHostZone 

func SplitHostZone(s string) (host, zone string)

SplitHostZone splits a "host%zone" string into its components. If there is no zone, host is the original input and zone is empty.

Types

type ChainStrategy added in v1.0.0 

type ChainStrategy struct {
	// contains filtered or unexported fields
}

ChainStrategy attempts to use the given strategies in order. If the first one returns an empty string, the second one is tried, and so on, until a good IP is found or the strategies are exhausted. A common use for this is if a server is both directly connected to the internet and expecting a header to check. It might be called like: 

NewChainStrategy(Must(LeftmostNonPrivateStrategy("X-Forwarded-For")), RemoteAddrStrategy)

func NewChainStrategy added in v1.0.0 

func NewChainStrategy(strategies ...Strategy) ChainStrategy

NewChainStrategy creates a ChainStrategy that attempts to use the given strategies to derive the client IP, stopping when the first one succeeds.

func (ChainStrategy) ClientIP added in v1.0.0 

func (strat ChainStrategy) ClientIP(headers http.Header, remoteAddr string) string

ClientIP derives the client IP using this strategy. headers is expected to be like http.Request.Header. remoteAddr is expected to be like http.Request.RemoteAddr. The returned IP may contain a zone identifier. If all chained strategies fail to derive a valid IP, an empty string is returned.

func (ChainStrategy) String added in v1.0.0 

func (strat ChainStrategy) String() string

type LeftmostNonPrivateStrategy 

type LeftmostNonPrivateStrategy struct {
	// contains filtered or unexported fields
}

LeftmostNonPrivateStrategy derives the client IP from the leftmost valid and non-private IP address in the X-Fowarded-For for Forwarded header. This strategy should be used when a valid, non-private IP closest to the client is desired. Note that this MUST NOT BE USED FOR SECURITY PURPOSES. This IP can be TRIVIALLY SPOOFED.

func NewLeftmostNonPrivateStrategy added in v1.0.0 

func NewLeftmostNonPrivateStrategy(headerName string) (LeftmostNonPrivateStrategy, error)

NewLeftmostNonPrivateStrategy creates a LeftmostNonPrivateStrategy. headerName must be "X-Forwarded-For" or "Forwarded".

func (LeftmostNonPrivateStrategy) ClientIP added in v1.0.0 

func (strat LeftmostNonPrivateStrategy) ClientIP(headers http.Header, _ string) string

ClientIP derives the client IP using this strategy. headers is expected to be like http.Request.Header. The returned IP may contain a zone identifier. If no valid IP can be derived, empty string will be returned.

type RemoteAddrStrategy 

type RemoteAddrStrategy struct{}

RemoteAddrStrategy returns the client socket IP, stripped of port. This strategy should be used if the server accept direct connections, rather than through a reverse proxy.

func (RemoteAddrStrategy) ClientIP added in v1.0.0 

func (strat RemoteAddrStrategy) ClientIP(_ http.Header, remoteAddr string) string

ClientIP derives the client IP using this strategy. remoteAddr is expected to be like http.Request.RemoteAddr. The returned IP may contain a zone identifier. If no valid IP can be derived, empty string will be returned. This should only happen if remoteAddr has been modified to something illegal, or if the server is accepting connections on a Unix domain socket (in which case RemoteAddr is "@").

type RightmostNonPrivateStrategy 

type RightmostNonPrivateStrategy struct {
	// contains filtered or unexported fields
}

RightmostNonPrivateStrategy derives the client IP from the rightmost valid, non-private/non-internal IP address in the X-Fowarded-For for Forwarded header. This strategy should be used when all reverse proxies between the internet and the server have private-space IP addresses.

func NewRightmostNonPrivateStrategy added in v1.0.0 

func NewRightmostNonPrivateStrategy(headerName string) (RightmostNonPrivateStrategy, error)

NewRightmostNonPrivateStrategy creates a RightmostNonPrivateStrategy. headerName must be "X-Forwarded-For" or "Forwarded".

func (RightmostNonPrivateStrategy) ClientIP added in v1.0.0 

func (strat RightmostNonPrivateStrategy) ClientIP(headers http.Header, _ string) string

ClientIP derives the client IP using this strategy. headers is expected to be like http.Request.Header. The returned IP may contain a zone identifier. If no valid IP can be derived, empty string will be returned.

type RightmostTrustedCountStrategy 

type RightmostTrustedCountStrategy struct {
	// contains filtered or unexported fields
}

RightmostTrustedCountStrategy derives the client IP from the valid IP address added by the first trusted reverse proxy to the X-Forwarded-For or Forwarded header. This Strategy should be used when there is a fixed number of trusted reverse proxies that are appending IP addresses to the header.

func NewRightmostTrustedCountStrategy added in v1.0.0 

func NewRightmostTrustedCountStrategy(headerName string, trustedCount int) (RightmostTrustedCountStrategy, error)

NewRightmostTrustedCountStrategy creates a RightmostTrustedCountStrategy. headerName must be "X-Forwarded-For" or "Forwarded". trustedCount is the number of trusted reverse proxies. The IP returned will be the (trustedCount-1)th from the right. For example, if there's only one trusted proxy, this strategy will return the last (rightmost) IP address.

func (RightmostTrustedCountStrategy) ClientIP added in v1.0.0 

func (strat RightmostTrustedCountStrategy) ClientIP(headers http.Header, _ string) string

ClientIP derives the client IP using this strategy. headers is expected to be like http.Request.Header. The returned IP may contain a zone identifier. If no valid IP can be derived, empty string will be returned.

type RightmostTrustedRangeStrategy 

type RightmostTrustedRangeStrategy struct {
	// contains filtered or unexported fields
}

RightmostTrustedRangeStrategy derives the client IP from the rightmost valid IP address in the X-Forwarded-For or Forwarded header which is not in a set of trusted IP ranges. This strategy should be used when the IP ranges of the reverse proxies between the internet and the server are known. If a third-party WAF, CDN, etc., is used, you SHOULD use a method of verifying its access to your origin that is stronger than checking its IP address (e.g., using authenticated pulls). Failure to do so can result in scenarios like: You use AWS CloudFront in front of a server you host elsewhere. An attacker creates a CF distribution that points at your origin server. The attacker uses Lambda@Edge to spoof the Host and X-Forwarded-For headers. Now your "trusted" reverse proxy is no longer trustworthy.

func NewRightmostTrustedRangeStrategy added in v1.0.0 

func NewRightmostTrustedRangeStrategy(headerName string, trustedRanges []net.IPNet) (RightmostTrustedRangeStrategy, error)

NewRightmostTrustedRangeStrategy creates a RightmostTrustedRangeStrategy. headerName must be "X-Forwarded-For" or "Forwarded". trustedRanges must contain all trusted reverse proxies on the path to this server. trustedRanges can be private/internal or external (for example, if a third-party reverse proxy is used).

func (RightmostTrustedRangeStrategy) ClientIP added in v1.0.0 

func (strat RightmostTrustedRangeStrategy) ClientIP(headers http.Header, _ string) string

ClientIP derives the client IP using this strategy. headers is expected to be like http.Request.Header. The returned IP may contain a zone identifier. If no valid IP can be derived, empty string will be returned.

func (RightmostTrustedRangeStrategy) String added in v1.0.0 

func (strat RightmostTrustedRangeStrategy) String() string

type SingleIPHeaderStrategy 

type SingleIPHeaderStrategy struct {
	// contains filtered or unexported fields
}

SingleIPHeaderStrategy derives an IP address from a single-IP header. A non-exhaustive list of such single-IP headers is: X-Real-IP, CF-Connecting-IP, True-Client-IP, Fastly-Client-IP, X-Azure-ClientIP, X-Azure-SocketIP. This strategy should be used when the given header is added by a trusted reverse proxy. You must ensure that this header is not spoofable (as is possible with Akamai's use of True-Client-IP, Fastly's default use of Fastly-Client-IP, and Azure's X-Azure-ClientIP). See the single-IP wiki page for more info: https://github.com/realclientip/realclientip-go/wiki/Single-IP-Headers

func NewSingleIPHeaderStrategy added in v1.0.0 

func NewSingleIPHeaderStrategy(headerName string) (SingleIPHeaderStrategy, error)

NewSingleIPHeaderStrategy creates a SingleIPHeaderStrategy that uses the headerName request header to get the client IP.

func (SingleIPHeaderStrategy) ClientIP added in v1.0.0 

func (strat SingleIPHeaderStrategy) ClientIP(headers http.Header, _ string) string

ClientIP derives the client IP using this strategy. headers is expected to be like http.Request.Header. The returned IP may contain a zone identifier. If no valid IP can be derived, empty string will be returned.

type Strategy 

type Strategy interface {
	// ClientIP returns empty string if there is no derivable IP. In many cases this
	// should be treated as a misconfiguration error, unless the strategy is attempting to
	// get an untrustworthy or optional value.
	// All implementations of this method must be threadsafe.
	ClientIP(headers http.Header, remoteAddr string) string
}

Strategy is satisfied by all of the specific strategies in this package. It can be used instead of the concrete types if the strategy is to be determined at runtime, depending on configuration, for example.

func Must 

func Must(strat Strategy, err error) Strategy

Must panics if err is not nil. This can be used to make sure the strategy-making functions do not return an error. It can also facilitate calling NewChainStrategy(). It can be called like Must(NewSingleIPHeaderStrategy("X-Real-IP")).

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.