freegeoip

README

freegeoip

This is the source code of the freegeoip software. It contains both the web server that empowers freegeoip.net, and a package for the Go programming language that enables any web server to support IP geolocation with a simple and clean API.

See http://en.wikipedia.org/wiki/Geolocation for details about geolocation.

Web Server

The freegeoip web server is a standalone program that serves an HTTP API for searching the geolocation of IP addresses. To serve the API, it uses an IP database that is automatically downloaded and auto-updated from the internet when the server is running.

The API returns data encoded in popular formats such as CSV, XML, JSON and JSONP.

Usage

Run the server:

./freegeoip

Wait for it to download the IP database file for the first time. It does it in background and writes a message to the console when ready. If you'd like to use an alternative database source, see the -db command line flag.

If the server is queried when there is no database available, including this initial first run, it returns HTTP 503 (Service Unavailable), since it can't service requests before a proper database is in place.

Querying

You can use any HTTP client to test the server. The examples below use curl and the environment variable $freegeoip, which must be set to the address of your server, like http://localhost:8080 for example:

export freegeoip=http://localhost:8080

Querying the API is very straightforward: you just have to pick a format of your choice and provide either the IP address or hostname that you'd like to search for. The syntax is as follows:

$freegeoip/{format}/{IP_or_hostname}

Examples:

curl -i $freegeoip/csv/8.8.8.8

curl -i $freegeoip/xml/4.2.2.2

curl -i $freegeoip/json/github.com

If a domain or hostname is passed in the URL, the server will resolve that name to its IP address and lookup the IP instead. If the hostname contains multiple IPs associated to it, the server picks one randomly, which means it could be either IPv4 or IPv6.

If no IP or hostname is provided, then the server queries the IP address of the HTTP client.

Example:

curl -i $freegeoip/json/   (this queries your own IP address)

The JSON endpoint also supports JSONP, by adding a callback argument to the request query.

Example:

curl -i $freegeoip/json/8.8.8.8?callback=f

See http://en.wikipedia.org/wiki/JSONP for details on how JSONP works.

Docker image

Build the docker image:

docker build -t my/freegeoip .

If you want just the API, not the front-end, edit the Dockerfile and comment out the -public command line flag.

Or use the official image:

docker run -d --name freegeoip -p 8080:8080 fiorix/freegeoip

If you need quota then link the container to a Redis container:

docker run -d --name redis -p 6379:6379 dockerfile/redis
docker run -d --name freegeoip --link redis:redis -p 8080:8080 fiorix/freegeoip -redis redis:6379 -quota-max 10000

You can use redis-cli monitor to assure things are working as expected.

freegeoip package for Go

The freegeoip package for the Go programming language provides two APIs:

• A database API that requires zero maintenance of the IP database;
• A geolocation http.Handler that can be used/served by any http server.

tl;dr if all you want is code then see the examples_test.go file.

Otherwise check out the godoc reference.

Features

• Zero maintenance

The DB object alone can download an IP database file from the internet and service lookups to your program right away. It will auto-update the file in background and always magically work.

• DevOps friendly

If you do care about the database and have the commercial version of the MaxMind database, you can update the database file with your program running and the DB object will load it in background. You can focus on your stuff.

• Extensible

Besides the database part, the package provides an http.Handler object that you can add to your HTTP server to service IP geolocation lookups with the same simplistic API of freegeoip.net. There's also an interface for crafting your own HTTP responses encoded in any format.

Install

Install the package:

go get github.com/fiorix/freegeoip

Install the web server:

go install github.com/fiorix/freegeoip/cmd/freegeoip

Test coverage is quite good and may help you find the stuff you need.

Documentation

Overview

Package freegeoip provides an API for searching the geolocation of IP addresses. It uses a database that can be either a local file or a remote resource from a URL.

Local databases are monitored by fsnotify and reloaded when the file is either updated or overwritten.

Remote databases are automatically downloaded and updated in background so you can focus on using the API and not managing the database.

Also, the freegeoip package provides http handlers that any Go http server (net/http) can use. These handlers can process IP geolocation lookup requests and return data in multiple formats like CSV, XML, JSON and JSONP. It has also an API for supporting custom formats.

Index

• Variables
• func ProxyHandler(f http.Handler) http.Handler
• type CSVEncoder
  • func (f *CSVEncoder) Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error
  • func (f *CSVEncoder) NewQuery() Query
• type DB
  • func Open(dsn string) (db *DB, err error)
  • func OpenURL(url string, updateInterval, maxRetryInterval time.Duration) (db *DB, err error)
  • func (db *DB) Close()
  • func (db *DB) Date() time.Time
  • func (db *DB) Lookup(addr net.IP, result interface{}) error
  • func (db *DB) NotifyClose() <-chan struct{}
  • func (db *DB) NotifyError() (errChan <-chan error)
  • func (db *DB) NotifyOpen() (filename <-chan string)
• type Encoder
• type Handler
  • func NewHandler(db *DB, enc Encoder) *Handler
  • func (f *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)
• type JSONEncoder
  • func (f *JSONEncoder) Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error
  • func (f *JSONEncoder) NewQuery() Query
  • func (f *JSONEncoder) P(w http.ResponseWriter, r *http.Request, record *responseRecord, ...) error
• type Query
• type XMLEncoder
  • func (f *XMLEncoder) Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error
  • func (f *XMLEncoder) NewQuery() Query

Variables

var (
	// ErrUnavailable may be returned by DB.Lookup when the database
	// points to a URL and is not yet available because it's being
	// downloaded in background.
	ErrUnavailable = errors.New("No database available")
)

Functions

func ProxyHandler

func ProxyHandler(f http.Handler) http.Handler

ProxyHandler is a wrapper for other http handlers that sets the client IP address in request.RemoteAddr to the first value of a comma separated list of IPs from the X-Forwarded-For request header. It resets the original RemoteAddr back after running the designated handler f.

Types

type CSVEncoder

type CSVEncoder struct {
	UseCRLF bool
}

CSVEncoder encodes the results of an IP lookup as CSV.

func (*CSVEncoder) Encode

func (f *CSVEncoder) Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error

Encode implements the Encoder interface.

func (*CSVEncoder) NewQuery

func (f *CSVEncoder) NewQuery() Query

NewQuery implements the Encoder interface.

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is the IP geolocation database.

func Open

func Open(dsn string) (db *DB, err error)

Open creates and initializes a DB from a local file.

The database file is monitored by fsnotify and automatically reloads when the file is updated or overwritten.

func OpenURL

func OpenURL(url string, updateInterval, maxRetryInterval time.Duration) (db *DB, err error)

OpenURL creates and initializes a DB from a remote file. It automatically downloads and updates the file in background.

func (*DB) Close

func (db *DB) Close()

Close the database.

func (*DB) Date

func (db *DB) Date() time.Time

Date returns the UTC date the database file was last modified. If no database file has been opened the behaviour of Date is undefined.

func (*DB) Lookup

func (db *DB) Lookup(addr net.IP, result interface{}) error

Lookup takes an IP address and a pointer to the result value to decode into. The result value pointed to must be a data value that corresponds to a record in the database. This may include a struct representation of the data, a map capable of holding the data or an empty interface{} value.

If result is a pointer to a struct, the struct need not include a field for every value that may be in the database. If a field is not present in the structure, the decoder will not decode that field, reducing the time required to decode the record.

See https://godoc.org/github.com/oschwald/maxminddb-golang#Reader.Lookup for details.

func (*DB) NotifyClose

func (db *DB) NotifyClose() <-chan struct{}

NotifyClose returns a channel that is closed when the database is closed.

func (*DB) NotifyError

func (db *DB) NotifyError() (errChan <-chan error)

NotifyError returns a channel that notifies when an error occurs while downloading or reloading a DB that points to a URL.

func (*DB) NotifyOpen

func (db *DB) NotifyOpen() (filename <-chan string)

NotifyOpen returns a channel that notifies when a new database is loaded or reloaded. This can be used to monitor background updates when the DB points to a URL.

type Encoder

type Encoder interface {
	// NewQuery returns a query specification that is used to query
	// the IP database. It should be a data structure with tags
	// associated to its fields describing what fields to query in
	// the IP database, such as country and city.
	//
	// See the maxminddb package documentation for details on
	// fields available for the MaxMind database.
	NewQuery() Query

	// Encode writes data to the response of an http request
	// using the results of a query to the IP database.
	//
	// It encodes the query object into a specific format such
	// as XML or JSON and writes to the response.
	//
	// The IP passed to the encoder may be the result of a DNS
	// lookup, and if there are multiple IPs associated to the
	// hostname this will be a random one from the list.
	Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error
}

An Encoder that can provide a query specification to be used for querying the IP database, and later encode the results of that query in a specific format.

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

A Handler provides http handlers that can process requests and return data in multiple formats.

Usage:

handle := NewHandler(db)
http.Handle("/json/", handle.JSON())

Note that the url pattern must end with a trailing slash since the handler looks for IP addresses or hostnames as parameters, for example /json/8.8.8.8 or /json/domain.com.

If no IP or hostname is provided, then the handler will query the IP address of the caller. See the ProxyHandler for more.

func NewHandler

func NewHandler(db *DB, enc Encoder) *Handler

NewHandler creates and initializes a new Handler.

func (*Handler) ServeHTTP

func (f *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface.

type JSONEncoder

type JSONEncoder struct {
	Indent bool
}

JSONEncoder encodes the results of an IP lookup as JSON.

func (*JSONEncoder) Encode

func (f *JSONEncoder) Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error

Encode implements the Encoder interface.

func (*JSONEncoder) NewQuery

func (f *JSONEncoder) NewQuery() Query

NewQuery implements the Encoder interface.

func (*JSONEncoder) P

func (f *JSONEncoder) P(w http.ResponseWriter, r *http.Request, record *responseRecord, callback string) error

P writes JSONP to an http response.

type Query

type Query interface{}

A Query object is used to query the IP database.

Currently the only database supported is MaxMind, and the query is a data structure with tags that are used by the maxminddb.Lookup function.

type XMLEncoder

type XMLEncoder struct {
	Indent bool
}

XMLEncoder encodes the results of an IP lookup as XML.

func (*XMLEncoder) Encode

func (f *XMLEncoder) Encode(w http.ResponseWriter, r *http.Request, q Query, ip net.IP) error

Encode implements the Encoder interface.

func (*XMLEncoder) NewQuery

func (f *XMLEncoder) NewQuery() Query

NewQuery implements the Encoder interface.