geodbtools

README

geodbtools

geodbtools is a swiss army knife for working with GeoIP databases.

geodbtool CLI

geodbtool is the CLI application that is provided along with the geodbtools library.

Features

• database lookups (lookup command)
• database information (info command)
• database type conversion (convert command)

Installation

You can either grab pre-compiled binaries from the releases page on Github, or install from source:

go get github.com/anexia-it/geodbtools/cmd/geodbtool

Usage

The tool's help command provides you with a list of available commands:

geodbtool help

geodbtools library

The library part of this repository provides functionality for working with GeoIP databases.

Implementation status

• MaxMind DAT format support

  • Country databases
    • Read
    • Write
  • City databases
  • AS number databases

• MaxMind MMDB format support

  • Country databases
    • Read (via https://github.com/oschwald/maxminddb-golang)
    • Write
  • City databases
  • AS number databases

• MaxMind legacy CSV format support

• MaxMind GeoIP2 CSV format support

Contributing

Contributions are always welcome, in every possible way. We are always happy to receive bug reports and pull requests.

License

geodbtools is free software, licensed under the MIT license.

Documentation

Overview

Package geodbtools provides functions for working with GeoIP databases

Index

• Constants
• Variables
• func AreCountryCodesEqual(a, b string) bool
• func CityRecordsEqual(a CityRecord, b Record) bool
• func CountryRecordsEqual(a CountryRecord, b Record) bool
• func FormatNames() []string
• func MustRegisterFormat(f Format)
• func RecordBelongsRightIPv6(b []byte, depth uint) bool
• func RecordsEqual(a, b Record) bool
• func RegisterEquivalentCountryCode(a, b string)
• func RegisterFormat(f Format) (err error)
• func Verify(reader Reader, root *RecordTree, progress chan<- *VerificationProgress) (err error)
• func VersionString() string
• type CityRecord
• type CountryRecord
• type DatabaseType
• type Format
  • func DetectFormat(r ReaderSource) (f Format, err error)
  • func LookupFormat(name string) (f Format, err error)
• type IPVersion
• type Metadata
• type Reader
• type ReaderSource
  • func NewFileReaderSource(path string) (s ReaderSource, err error)
  • func NewReaderSourceWrapper(r io.ReaderAt, size int64) ReaderSource
• type Record
• type RecordBelongsRightFunc
• type RecordTree
  • func NewRecordTree(maxDepth uint, records []Record, belongsRightFunc RecordBelongsRightFunc) (t *RecordTree, err error)
  • func (t *RecordTree) Build(depth int, records []Record, belongsRightFn RecordBelongsRightFunc) (err error)
  • func (t *RecordTree) Leaf() Record
  • func (t *RecordTree) Left() *RecordTree
  • func (t *RecordTree) Records() []Record
  • func (t *RecordTree) Right() *RecordTree
• type VerificationError
  • func (e *VerificationError) Error() string
• type VerificationProgress
• type Writer

Constants

const (
	// VersionMajor defines the major version number of geodbtools
	VersionMajor = 1
	// VersionMinor defines the minor version number of geodbtools
	VersionMinor = 0
	// VersionPatch defines the patch version number of geodbtools
	VersionPatch = 1
)

VersionMajor defines the major version number of geodbtools

Variables

var (
	// ErrUnsupportedDatabaseType indicates that the database type is unsupported
	ErrUnsupportedDatabaseType = errors.New("unsupported database type")

	// ErrDatabaseInvalid indicates that the database's contents are invalid
	ErrDatabaseInvalid = errors.New("database invalid")
	// ErrRecordNotFound indicates that the record for the given lookup could not be found
	ErrRecordNotFound = errors.New("record not found")
	// ErrUnsupportedIPVersion indicates that the desired IP version is not supported by the database
	ErrUnsupportedIPVersion = errors.New("requested IP version not supported by database")
)

var (
	// ErrFormatIsRegistered indicates that a format with the given name has already been registered
	ErrFormatIsRegistered = errors.New("format already registered")

	// ErrFormatNotFound indicates that the format is not known
	ErrFormatNotFound = errors.New("format not found")
)

Functions

func AreCountryCodesEqual

func AreCountryCodesEqual(a, b string) bool

AreCountryCodesEqual checks if two country codes are considered equal

func CityRecordsEqual

func CityRecordsEqual(a CityRecord, b Record) bool

CityRecordsEqual checks if two CityRecord instances are equal

func CountryRecordsEqual

func CountryRecordsEqual(a CountryRecord, b Record) bool

CountryRecordsEqual checks if two CountryRecord instances are equal

func FormatNames

func FormatNames() []string

FormatNames returns the names of all registered formats

func MustRegisterFormat

func MustRegisterFormat(f Format)

MustRegisterFormat registers a database format and panics if the registration fails

func RecordBelongsRightIPv6

func RecordBelongsRightIPv6(b []byte, depth uint) bool

RecordBelongsRightIPv6 defines the "belongs right" test function for IPv6 addresses

func RecordsEqual

func RecordsEqual(a, b Record) bool

RecordsEqual checks if two records are equal

func RegisterEquivalentCountryCode

func RegisterEquivalentCountryCode(a, b string)

RegisterEquivalentCountryCode registers a pair of country codes which are deemed as equivalent during verification

func RegisterFormat

func RegisterFormat(f Format) (err error)

RegisterFormat registers a database format

func Verify

func Verify(reader Reader, root *RecordTree, progress chan<- *VerificationProgress) (err error)

Verify tests if a given reader contains all records defined by the given tree.

func VersionString

func VersionString() string

VersionString returns the complete version as a string

Types

type CityRecord

type CityRecord interface {
	CountryRecord

	// GetCityName returns the name of the city
	GetCityName() string
}

CityRecord describes a database record holding city-specific information

type CountryRecord

type CountryRecord interface {
	Record

	// GetCountryCode returns the 2-character ISO country code
	GetCountryCode() string
}

CountryRecord describes a database record holding country-specific information

type DatabaseType

type DatabaseType string

DatabaseType defines the type of a database

const (
	// DatabaseTypeCountry defines the country database type
	DatabaseTypeCountry DatabaseType = "country"
)

type Format

type Format interface {
	// FormatName returns the format's name
	FormatName() string

	// NewReader returns a new reader instance from the given io.ReaderAt
	// This method will return not only a reader instance, but also the database metadata
	// retrieved from the ReaderAt instance.
	NewReaderAt(r ReaderSource) (reader Reader, meta Metadata, err error)

	// NewWriter returns a new writer instance writing to the given io.Writer
	NewWriter(w io.Writer, dbType DatabaseType, ipVersion IPVersion) (writer Writer, err error)

	// DetectFormat checks if the data represented by the passed io.ReaderAt are of the given format
	DetectFormat(r ReaderSource) (isFormat bool)
}

Format represents a database format

func DetectFormat

func DetectFormat(r ReaderSource) (f Format, err error)

DetectFormat takes an io.ReaderAt and tries to detect the database format

func LookupFormat

func LookupFormat(name string) (f Format, err error)

LookupFormat retrieves a registered format by name

type IPVersion

type IPVersion uint

IPVersion defines an IP version

const (
	// IPVersionUndefined is used for non-IP databases
	IPVersionUndefined IPVersion = 0
	// IPVersion4 is used for IPv4 databases
	IPVersion4 IPVersion = 4
	// IPVersion6 is used for IPv6 databases
	IPVersion6 IPVersion = 6
)

type Metadata

type Metadata struct {
	// Type holds the database type
	Type DatabaseType
	// BuildTime holds the database's build time
	BuildTime time.Time
	// Description holds the human-readable database description
	Description string

	// MajorFormatVersion holds the major version number of the database format
	MajorFormatVersion uint
	// MinorFormatVersion holds the minor version number of the database format
	MinorFormatVersion uint

	// IPVersion holds the IP version represented by the database.
	// This may be IPVersionUndefined for databases non-IP databases
	IPVersion IPVersion
}

Metadata represents a database's metadata

type Reader

type Reader interface {
	// RecordTree returns the database's underlying RecordTree instance, selecting the desired IP version.
	// Note that this is an expensive operation and is not needed for running lookups against the database.
	RecordTree(ipVersion IPVersion) (tree *RecordTree, err error)

	// LookupIP retrieves the record for the given IP address
	LookupIP(ip net.IP) (record Record, err error)
}

Reader represents a database reader

type ReaderSource

type ReaderSource interface {
	io.ReaderAt

	// Size returns the overall size of the underlying ReaderAt
	Size() int64

	// Close frees up memory used by underlying source object
	Close() error
}

ReaderSource defines the interface for reader source

func NewFileReaderSource

func NewFileReaderSource(path string) (s ReaderSource, err error)

NewFileReaderSource returns a new ReaderSource that is backed by a file

func NewReaderSourceWrapper

func NewReaderSourceWrapper(r io.ReaderAt, size int64) ReaderSource

NewReaderSourceWrapper returns a new ReaderSource that wraps a given io.Reader

type Record

type Record interface {
	fmt.Stringer

	// GetNetwork returns the network represented by the record
	GetNetwork() *net.IPNet
}

Record describes a database record

type RecordBelongsRightFunc

type RecordBelongsRightFunc func(b []byte, depth uint) bool

RecordBelongsRightFunc tests if a given record, given the byte-slice representation of its IP address, belongs into the right sub-tree or not. This function is used during build of a RecordTree.

type RecordTree

type RecordTree struct {
	// contains filtered or unexported fields
}

RecordTree represents the rooted binary tree of records Each node inside the tree is either a leaf, or has two children (left and right)

func NewRecordTree

func NewRecordTree(maxDepth uint, records []Record, belongsRightFunc RecordBelongsRightFunc) (t *RecordTree, err error)

NewRecordTree initializes and builds a new RecordTree, given a slice of records

func (*RecordTree) Build

func (t *RecordTree) Build(depth int, records []Record, belongsRightFn RecordBelongsRightFunc) (err error)

Build builds the sub-tree starting at the given depth, a slice of records and a RecordBelongsRightFunc

func (*RecordTree) Leaf

func (t *RecordTree) Leaf() Record

Leaf returns the leaf value of the tree

func (*RecordTree) Left

func (t *RecordTree) Left() *RecordTree

Left returns the left sub-tree

func (*RecordTree) Records

func (t *RecordTree) Records() []Record

Records returns all records the tree node and its children represent

func (*RecordTree) Right

func (t *RecordTree) Right() *RecordTree

Right returns the right sub-tree

type VerificationError

type VerificationError struct {
	ExpectedRecord Record
	Record         Record
	LookupError    error
}

VerificationError holds information relating to a verification error

func (*VerificationError) Error

func (e *VerificationError) Error() string

type VerificationProgress

type VerificationProgress struct {
	TotalRecords   int
	CheckedRecords int
}

VerificationProgress holds information regarding the status of the verification progress

type Writer

type Writer interface {
	// WriteDatabase writes the database given the database metadata and a given record tree
	WriteDatabase(meta Metadata, tree *RecordTree) (err error)
}

Writer defines a database writer instance