mmdb

README

MaxMind DB Reader for Go

This is a Go reader for the MaxMind DB format. Although this can be used to read GeoLite2 and GeoIP2 databases, geoip2 provides a higher-level API for doing so.

This is not an official MaxMind API.

Installation

go get github.com/bloom42/stdx/mmdb

Usage

See GoDoc for documentation and examples.

Examples

See GoDoc or example_test.go for examples.

Contributing

Contributions welcome! Please fork the repository and open a pull request with your changes.

License

This is free software, licensed under the ISC License.

Documentation

Overview

package mmdb provides a reader for the MaxMind DB file format.

Index

• Constants
• func SkipAliasedNetworks(networks *Networks)
• type InvalidDatabaseError
  • func (e InvalidDatabaseError) Error() string
• type Metadata
• type Networks
  • func (n *Networks) Err() error
  • func (n *Networks) Network(result any) (*net.IPNet, error)
  • func (n *Networks) Next() bool
• type NetworksOption
• type Reader
  • func FromBytes(buffer []byte) (*Reader, error)
  • func Open(file string) (*Reader, error)
  • func (r *Reader) Close() error
  • func (r *Reader) Decode(offset uintptr, result any) error
  • func (r *Reader) Lookup(ip net.IP, result any) error
  • func (r *Reader) LookupNetwork(ip net.IP, result any) (network *net.IPNet, ok bool, err error)
  • func (r *Reader) LookupOffset(ip net.IP) (uintptr, error)
  • func (r *Reader) Networks(options ...NetworksOption) *Networks
  • func (r *Reader) NetworksWithin(network *net.IPNet, options ...NetworksOption) *Networks
  • func (r *Reader) Verify() error
• type UnmarshalTypeError
  • func (e UnmarshalTypeError) Error() string

Examples

• Reader.Lookup (Interface)
• Reader.Lookup (Struct)
• Reader.Networks
• Reader.NetworksWithin

Constants

const (
	// NotFound is returned by LookupOffset when a matched root record offset
	// cannot be found.
	NotFound = ^uintptr(0)
)

Functions

func SkipAliasedNetworks

func SkipAliasedNetworks(networks *Networks)

SkipAliasedNetworks is an option for Networks and NetworksWithin that makes them not iterate over aliases of the IPv4 subtree in an IPv6 database, e.g., ::ffff:0:0/96, 2001::/32, and 2002::/16.

You most likely want to set this. The only reason it isn't the default behavior is to provide backwards compatibility to existing users.

Types

type InvalidDatabaseError

type InvalidDatabaseError struct {
	// contains filtered or unexported fields
}

InvalidDatabaseError is returned when the database contains invalid data and cannot be parsed.

func (InvalidDatabaseError) Error

func (e InvalidDatabaseError) Error() string

type Metadata

type Metadata struct {
	Description              map[string]string `maxminddb:"description"`
	DatabaseType             string            `maxminddb:"database_type"`
	Languages                []string          `maxminddb:"languages"`
	BinaryFormatMajorVersion uint              `maxminddb:"binary_format_major_version"`
	BinaryFormatMinorVersion uint              `maxminddb:"binary_format_minor_version"`
	BuildEpoch               uint              `maxminddb:"build_epoch"`
	IPVersion                uint              `maxminddb:"ip_version"`
	NodeCount                uint              `maxminddb:"node_count"`
	RecordSize               uint              `maxminddb:"record_size"`
}

Metadata holds the metadata decoded from the MaxMind DB file. In particular it has the format version, the build time as Unix epoch time, the database type and description, the IP version supported, and a slice of the natural languages included.

type Networks

type Networks struct {
	// contains filtered or unexported fields
}

Networks represents a set of subnets that we are iterating over.

func (*Networks) Err

func (n *Networks) Err() error

Err returns an error, if any, that was encountered during iteration.

func (*Networks) Network

func (n *Networks) Network(result any) (*net.IPNet, error)

Network returns the current network or an error if there is a problem decoding the data for the network. It takes a pointer to a result value to decode the network's data into.

func (*Networks) Next

func (n *Networks) Next() bool

Next prepares the next network for reading with the Network method. It returns true if there is another network to be processed and false if there are no more networks or if there is an error.

type NetworksOption

type NetworksOption func(*Networks)

NetworksOption are options for Networks and NetworksWithin.

type Reader

type Reader struct {
	Metadata Metadata
	// contains filtered or unexported fields
}

Reader holds the data corresponding to the MaxMind DB file. Its only public field is Metadata, which contains the metadata from the MaxMind DB file.

All of the methods on Reader are thread-safe. The struct may be safely shared across goroutines.

func FromBytes

func FromBytes(buffer []byte) (*Reader, error)

FromBytes takes a byte slice corresponding to a MaxMind DB file and returns a Reader structure or an error.

func Open

func Open(file string) (*Reader, error)

Open takes a string path to a MaxMind DB file and returns a Reader structure or an error. The database file is opened using a memory map on supported platforms. On platforms without memory map support, such as WebAssembly or Google App Engine, the database is loaded into memory. Use the Close method on the Reader object to return the resources to the system.

func (*Reader) Close

func (r *Reader) Close() error

Close returns the resources used by the database to the system.

func (*Reader) Decode

func (r *Reader) Decode(offset uintptr, result any) error

Decode the record at |offset| into |result|. The result value pointed to must be a data value that corresponds to a record in the database. This may include a struct representation of the data, a map capable of holding the data or an empty any value.

If result is a pointer to a struct, the struct need not include a field for every value that may be in the database. If a field is not present in the structure, the decoder will not decode that field, reducing the time required to decode the record.

As a special case, a struct field of type uintptr will be used to capture the offset of the value. Decode may later be used to extract the stored value from the offset. MaxMind DBs are highly normalized: for example in the City database, all records of the same country will reference a single representative record for that country. This uintptr behavior allows clients to leverage this normalization in their own sub-record caching.

func (*Reader) Lookup

func (r *Reader) Lookup(ip net.IP, result any) error

Lookup retrieves the database record for ip and stores it in the value pointed to by result. If result is nil or not a pointer, an error is returned. If the data in the database record cannot be stored in result because of type differences, an UnmarshalTypeError is returned. If the database is invalid or otherwise cannot be read, an InvalidDatabaseError is returned.

Example (Interface)

This example demonstrates how to decode to an any.

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/bloom42/stdx/mmdb"
)

func main() {
	db, err := mmdb.Open("test-data/test-data/GeoIP2-City-Test.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	ip := net.ParseIP("81.2.69.142")

	var record any
	err = db.Lookup(ip, &record)
	if err != nil {
		log.Panic(err)
	}
	fmt.Printf("%v", record)
}

Example (Struct)

This example shows how to decode to a struct.

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/bloom42/stdx/mmdb"
)

func main() {
	db, err := mmdb.Open("test-data/test-data/GeoIP2-City-Test.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	ip := net.ParseIP("81.2.69.142")

	var record struct {
		Country struct {
			ISOCode string `maxminddb:"iso_code"`
		} `maxminddb:"country"`
	} // Or any appropriate struct

	err = db.Lookup(ip, &record)
	if err != nil {
		log.Panic(err)
	}
	fmt.Print(record.Country.ISOCode)
}

Output:

GB

func (*Reader) LookupNetwork

func (r *Reader) LookupNetwork(
	ip net.IP,
	result any,
) (network *net.IPNet, ok bool, err error)

LookupNetwork retrieves the database record for ip and stores it in the value pointed to by result. The network returned is the network associated with the data record in the database. The ok return value indicates whether the database contained a record for the ip.

If result is nil or not a pointer, an error is returned. If the data in the database record cannot be stored in result because of type differences, an UnmarshalTypeError is returned. If the database is invalid or otherwise cannot be read, an InvalidDatabaseError is returned.

func (*Reader) LookupOffset

func (r *Reader) LookupOffset(ip net.IP) (uintptr, error)

LookupOffset maps an argument net.IP to a corresponding record offset in the database. NotFound is returned if no such record is found, and a record may otherwise be extracted by passing the returned offset to Decode. LookupOffset is an advanced API, which exists to provide clients with a means to cache previously-decoded records.

func (*Reader) Networks

func (r *Reader) Networks(options ...NetworksOption) *Networks

Networks returns an iterator that can be used to traverse all networks in the database.

Please note that a MaxMind DB may map IPv4 networks into several locations in an IPv6 database. This iterator will iterate over all of these locations separately. To only iterate over the IPv4 networks once, use the SkipAliasedNetworks option.

Example

This example demonstrates how to iterate over all networks in the database.

package main

import (
	"fmt"
	"log"

	"github.com/bloom42/stdx/mmdb"
)

func main() {
	db, err := mmdb.Open("test-data/test-data/GeoIP2-Connection-Type-Test.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	record := struct {
		Domain string `maxminddb:"connection_type"`
	}{}

	networks := db.Networks(mmdb.SkipAliasedNetworks)
	for networks.Next() {
		subnet, err := networks.Network(&record)
		if err != nil {
			log.Panic(err)
		}
		fmt.Printf("%s: %s\n", subnet.String(), record.Domain)
	}
	if networks.Err() != nil {
		log.Panic(networks.Err())
	}
}

Output:

1.0.0.0/24: Cable/DSL
1.0.1.0/24: Cellular
1.0.2.0/23: Cable/DSL
1.0.4.0/22: Cable/DSL
1.0.8.0/21: Cable/DSL
1.0.16.0/20: Cable/DSL
1.0.32.0/19: Cable/DSL
1.0.64.0/18: Cable/DSL
1.0.128.0/17: Cable/DSL
2.125.160.216/29: Cable/DSL
67.43.156.0/24: Cellular
80.214.0.0/20: Cellular
96.1.0.0/16: Cable/DSL
96.10.0.0/15: Cable/DSL
96.69.0.0/16: Cable/DSL
96.94.0.0/15: Cable/DSL
108.96.0.0/11: Cellular
149.101.100.0/28: Cellular
175.16.199.0/24: Cable/DSL
187.156.138.0/24: Cable/DSL
201.243.200.0/24: Corporate
207.179.48.0/20: Cellular
216.160.83.56/29: Corporate
2003::/24: Cable/DSL

func (*Reader) NetworksWithin

func (r *Reader) NetworksWithin(network *net.IPNet, options ...NetworksOption) *Networks

NetworksWithin returns an iterator that can be used to traverse all networks in the database which are contained in a given network.

Please note that a MaxMind DB may map IPv4 networks into several locations in an IPv6 database. This iterator will iterate over all of these locations separately. To only iterate over the IPv4 networks once, use the SkipAliasedNetworks option.

If the provided network is contained within a network in the database, the iterator will iterate over exactly one network, the containing network.

Example

This example demonstrates how to iterate over all networks in the database which are contained within an arbitrary network.

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/bloom42/stdx/mmdb"
)

func main() {
	db, err := mmdb.Open("test-data/test-data/GeoIP2-Connection-Type-Test.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	record := struct {
		Domain string `maxminddb:"connection_type"`
	}{}

	_, network, err := net.ParseCIDR("1.0.0.0/8")
	if err != nil {
		log.Panic(err)
	}

	networks := db.NetworksWithin(network, mmdb.SkipAliasedNetworks)
	for networks.Next() {
		subnet, err := networks.Network(&record)
		if err != nil {
			log.Panic(err)
		}
		fmt.Printf("%s: %s\n", subnet.String(), record.Domain)
	}
	if networks.Err() != nil {
		log.Panic(networks.Err())
	}

}

Output:

1.0.0.0/24: Cable/DSL
1.0.1.0/24: Cellular
1.0.2.0/23: Cable/DSL
1.0.4.0/22: Cable/DSL
1.0.8.0/21: Cable/DSL
1.0.16.0/20: Cable/DSL
1.0.32.0/19: Cable/DSL
1.0.64.0/18: Cable/DSL
1.0.128.0/17: Cable/DSL

func (*Reader) Verify

func (r *Reader) Verify() error

Verify checks that the database is valid. It validates the search tree, the data section, and the metadata section. This verifier is stricter than the specification and may return errors on databases that are readable.

type UnmarshalTypeError

type UnmarshalTypeError struct {
	Type  reflect.Type
	Value string
}

UnmarshalTypeError is returned when the value in the database cannot be assigned to the specified data type.

func (UnmarshalTypeError) Error

func (e UnmarshalTypeError) Error() string