ip2x

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 4, 2023 License: MIT Imports: 5 Imported by: 4

README

ip2x

Go Reference test verify

Module ip2x is an idiomatic, efficient, and robust library and command-line tool for querying IP2Location databases.

Compared to github.com/ip2location/ip2location-go/v9 and github.com/ip2location/ip2proxy-go/v3, this library:

  • Supports Go 1.18+.
  • Supports querying using Go 1.18's new net/netip.Addr type, which is much more efficient than parsing the IP from a string every time.
  • Uses native integer types instead of big.Int, which is also much more efficient.
  • Is about 11x faster than this library when querying a single field, and 2x faster for all fields, while making a fraction of the number of allocations (2 for init, 1 for each lookup, plus 1 for each typed field get, or 2 for an untyped one).
  • Has comprehensive built-in documentation, including automatically-generated information about which fields are available in different product types.
  • Supports querying information about the database itself, for example, whether it supports IPv6, and which fields are available.
  • Has a more fluent and flexible API (e.g., record.Get(ip2x.Latitude), record.GetString(ip2x.Latitude), record.GetFloat(ip2x.Latitude))
  • Has built-in support for pretty-printing records as strings or JSON.
  • Supports both IP2Location databases in a single package with a unified API.
  • Uses code generation to simplify adding new products/types/fields/documentation while reducing the likelihood of bugs (input, docs).
  • Is written in idiomatic Go: correct error handling (rather than stuffing error strings into the record struct), useful zero values (an empty record will work properly), proper type names, etc.
  • Has tests to ensure the output is consistent with this library, that a range of IPv4 (and their possible IPv6-mappings) address work correctly, and other things. There are also fuzz tests to ensure IPs can't crash the library and are IPv4/v6-mapped correctly.
  • Has an automated tool to compare the output of this library against the offical ones for every row of any database.

Benchmark

The code for the benchmark can be found in benchmark_test.go.

  • Benchmarks are done using a balanced variety of IP addresses in both small and large subnets, as both IPv4 and IPv6 (native, v4-mapped, 6to4, and teredo). This ensures database indexing and IP parsing/normalization is tested fairly.
  • A test to ensure results from both libraries are the same exists to ensure correctness.
  • The entire DB is loaded into memory to ensure the disk cache does not affect results.
db: IP2Location DB11 2022-10-29 [city,country_code,country_name,latitude,longitude,region,time_zone,zip_code] (IPv4+IPv6)
goos: linux
goarch: amd64
pkg: github.com/pg9182/ip2x/test
cpu: AMD Ryzen 5 5600G with Radeon Graphics         
BenchmarkIP2x_Init-12                       	17850333	        67.91 ns/op	     128 B/op	       2 allocs/op
BenchmarkIP2x_LookupOnly-12                 	18722506	        61.36 ns/op	      48 B/op	       1 allocs/op
BenchmarkIP2x_GetAll-12                     	 1522696	       812.2 ns/op	    1688 B/op	      14 allocs/op
BenchmarkIP2x_GetOneString-12               	 7839385	       144.1 ns/op	     304 B/op	       2 allocs/op
BenchmarkIP2x_GetOneFloat-12                	14312419	        84.16 ns/op	      48 B/op	       1 allocs/op
BenchmarkIP2x_GetTwoString-12               	 4243560	       244.9 ns/op	     560 B/op	       3 allocs/op
BenchmarkIP2x_GetTwoFloat-12                	12198259	       101.1 ns/op	      48 B/op	       1 allocs/op
BenchmarkIP2x_GetNonexistent-12             	14834245	        79.85 ns/op	      48 B/op	       1 allocs/op
BenchmarkIP2LocationV9_Init-12              	  602967	      2191 ns/op	     400 B/op	       7 allocs/op
BenchmarkIP2LocationV9_LookupOnly-12        	 1473849	       782.6 ns/op	     672 B/op	      24 allocs/op
BenchmarkIP2LocationV9_GetAll-12            	  819900	      1324 ns/op	    2268 B/op	      36 allocs/op
BenchmarkIP2LocationV9_GetOneString-12      	 1346534	       889.2 ns/op	     936 B/op	      26 allocs/op
BenchmarkIP2LocationV9_GetOneFloat-12       	 1441219	       795.0 ns/op	     672 B/op	      24 allocs/op
BenchmarkIP2LocationV9_GetTwoString-12      	  546868	      1866 ns/op	    1883 B/op	      53 allocs/op
BenchmarkIP2LocationV9_GetTwoFloat-12       	  693019	      1561 ns/op	    1345 B/op	      49 allocs/op
BenchmarkIP2LocationV9_GetNonexistent-12    	 1399872	       795.5 ns/op	     672 B/op	      24 allocs/op

CLI

ip2x db_path [ip_addr...]
  -compact
        compact output
  -json
        use json output
  -strict
        fail immediately if a record is not found

$ ip2x IP2LOCATION-LITE-DB11.IPV6.BIN 1.1.1.1
IP2Location<DB11>{
  city "Los Angeles"
  country_code "US"
  country_name "United States of America"
  latitude 34.05286
  longitude -118.24357
  region "California"
  time_zone "-07:00"
  zip_code "90001"
}

Library

package main

import (
	"fmt"
	"os"

	"github.com/pg9182/ip2x"
)

func main() {
	f, err := os.Open("IP2LOCATION-LITE-DB11.IPV6.BIN")
	if err != nil {
		panic(err)
	}
	defer f.Close()

	db, err := ip2x.New(f)
	if err != nil {
		panic(err)
	}

	fmt.Println(db)
	fmt.Println()

	r, err := db.LookupString("8.8.8.8")
	if err != nil {
		panic(err)
	}

	// pretty-print
	fmt.Println(r.Format(true, true))
	fmt.Println()

	// get some fields the easy way
	fmt.Println("Test:", r.Get(ip2x.CountryCode), r.Get(ip2x.Region))

	// get the latitude
	{
		fmt.Println()
		fmt.Printf("Get(Latitude): %#v\n", r.Get(ip2x.Latitude))

		latstr, ok := r.GetString(ip2x.Latitude)
		fmt.Printf("GetString(Latitude): %#v, %#v\n", latstr, ok)

		latflt, ok := r.GetFloat32(ip2x.Latitude)
		fmt.Printf("GetFloat32(Latitude): %#v, %#v\n", latflt, ok)
	}

	// get an unsupported field
	{
		fmt.Println()
		fmt.Printf("Get(ISP): %#v\n", r.Get(ip2x.ISP))

		ispstr, ok := r.GetString(ip2x.ISP)
		fmt.Printf("GetString(ISP): %#v, %#v\n", ispstr, ok)

		ispflt, ok := r.GetString(ip2x.ISP)
		fmt.Printf("GetString(ISP): %#v, %#v\n", ispflt, ok)
	}
}
Output: 
IP2Location 2022-10-29 DB11 [city,country_code,country_name,latitude,longitude,region,time_zone,zip_code] (IPv4+IPv6)

IP2Location<DB11>{
  city "Mountain View"
  country_code "US"
  country_name "United States of America"
  latitude 37.40599
  longitude -122.078514
  region "California"
  time_zone "-07:00"
  zip_code "94043"
}

Test: US California

Get(Latitude): 37.40599
GetString(Latitude): "37.40599", true
GetFloat32(Latitude): 37.40599, true

Get(ISP): <nil>
GetString(ISP): "", false
GetString(ISP): "", false

Documentation

Overview

Package ip2x reads IP2Location binary databases.

Index

Constants

This section is empty.

Variables

View Source 
var (
	RecordStringColor     = false
	RecordStringMultiline = false
)

Default options for Record.String.

Functions

This section is empty.

Types

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB reads an IP2Location binary database.

func New 

func New(r io.ReaderAt) (*DB, error)

New opens an IP2Location binary database reading from r.

func (*DB) EachField 

func (db *DB) EachField(fn func(DBField) bool)

EachField calls fn for each column in the database until fn returns false.

func (*DB) Has 

func (db *DB) Has(f DBField) bool

Has returns true if the database contains f.

func (*DB) HasIPv4 

func (db *DB) HasIPv4() bool

HasIPv4 returns true if the database contains IPv4 entries.

func (*DB) HasIPv6 

func (db *DB) HasIPv6() bool

HasIPv6 returns true if the database contains HasIPv6 entries.

func (*DB) Info 

func (db *DB) Info() (p DBProduct, t DBType)

Info returns the database product and type.

func (*DB) Lookup 

func (db *DB) Lookup(a netip.Addr) (r Record, err error)

Lookup looks up a in db. If a is not found, an empty record and nil error is returned. If an i/o error occurs, an empty record and non-nil error is returned.

func (*DB) LookupString 

func (db *DB) LookupString(ip string) (r Record, err error)

LookupString parses and looks up a in db. If a parse error occurs, an empty record and nil error is returned. To catch parse errors, parse it separately using net/netip.ParseAddr, and pass it to DB.Lookup.

func (*DB) String 

func (db *DB) String() string

String returns a human-readable string describing the database.

func (*DB) Version 

func (db *DB) Version() string

Version returns the database version.

type DBField 

type DBField uint

DBField represents a database column. 

const AS DBField = 3

Autonomous system number (ASN).

In DB26, PX7-11. 

const ASN DBField = 4

Autonomous system (AS) name.

In DB26, PX7-11. 

const AddressType DBField = 1

IP address types as defined in Internet Protocol version 4 (IPv4) and Internet Protocol version 6 (IPv6).

  • (A) Anycast - One to the closest
  • (U) Unicast - One to one
  • (M) Multicast - One to multiple
  • (B) Broadcast - One to all

In DB25-26. 

const AreaCode DBField = 2

A varying length number assigned to geographic areas for call between cities.

See https://www.ip2location.com/area-code-coverage.

In DB15-16, DB18, DB20-22, DB24-26. 

const Category DBField = 5

The domain category is based on IAB Tech Lab Content Taxonomy.

These categories are comprised of Tier-1 and Tier-2 (if available) level categories widely used in services like advertising, Internet security and filtering appliances.

See https://www.ip2location.com/free/iab-categories.

In DB25-26. 

const City DBField = 6

City name.

In DB3-26, PX3-11. 

const CountryCode DBField = 7

Two-character country code based on ISO 3166.

In DB1-26, PX1-11. 

const CountryName DBField = 8

Country name based on ISO 3166.

In DB1-26, PX1-11. 

const District DBField = 10

District or county name.

In DB26. 

const Domain DBField = 9

Internet domain name associated with IP address range.

In DB7-8, DB10, DB12, DB14, DB16, DB18-20, DB22-26, PX5-11. 

const Elevation DBField = 11

Average height of city above sea level in meters (m).

In DB21-22, DB24-26. 

const IDDCode DBField = 12

The IDD prefix to call the city from another country.

In DB15-16, DB18, DB20-22, DB24-26. 

const ISP DBField = 13

Internet Service Provider or company's name.

In DB2, DB4, DB6-8, DB10, DB12, DB14, DB16, DB18-20, DB22-26, PX4-11. 

const LastSeen DBField = 14

Proxy last seen in days.

In PX8-11. 

const Latitude DBField = 15

City latitude. Defaults to capital city latitude if city is unknown.

In DB5-6, DB8-26. 

const Longitude DBField = 16

City longitude. Defaults to capital city longitude if city is unknown.

In DB5-6, DB8-26. 

const MCC DBField = 17

Mobile Country Codes (MCC) as defined in ITU E.212 for use in identifying mobile stations in wireless telephone networks, particularly GSM and UMTS networks.

In DB19-20, DB22-26. 

const MNC DBField = 18

Mobile Network Code (MNC) is used in combination with a Mobile Country Code (MCC) to uniquely identify a mobile phone operator or carrier.

In DB19-20, DB22-26. 

const MobileBrand DBField = 19

Commercial brand associated with the mobile carrier.

See https://www.ip2location.com/mobile-carrier-coverage.

In DB19-20, DB22-26. 

const NetSpeed DBField = 20

Internet Connection Type

  • (DIAL) dial up
  • (DSL) broadband/cable/fiber/mobile
  • (COMP) company/T1

In DB13-14, DB16-18, DB20, DB22, DB24-26. 

const Provider DBField = 21

Name of VPN provider if available.

In PX11. 

const ProxyType DBField = 22

Type of proxy.

  • (VPN) Anonymizing VPN services. These services offer users a publicly accessible VPN for the purpose of hiding their IP address. Anonymity: High.
  • (TOR) Tor Exit Nodes. The Tor Project is an open network used by those who wish to maintain anonymity. Anonymity: High.
  • (DCH) Hosting Provider, Data Center or Content Delivery Network. Since hosting providers and data centers can serve to provide anonymity, the Anonymous IP database flags IP addresses associated with them. Anonymity: Low.
  • (PUB) Public Proxies. These are services which make connection requests on a user's behalf. Proxy server software can be configured by the administrator to listen on some specified port. These differ from VPNs in that the proxies usually have limited functions compare to VPNs. Anonymity: High.
  • (WEB) Web Proxies. These are web services which make web requests on a user's behalf. These differ from VPNs or Public Proxies in that they are simple web-based proxies rather than operating at the IP address and other ports level. Anonymity: High.
  • (SES) Search Engine Robots. These are services which perform crawling or scraping to a website, such as, the search engine spider or bots engine. Anonymity: Low.
  • (RES) Residential proxies. These services offer users proxy connections through residential ISP with or without consents of peers to share their idle resources. Only available with PX10 & PX11. Anonymity: Medium.

In PX2-11. 

const Region DBField = 23

Region or state name.

In DB3-26, PX3-11. 

const Threat DBField = 24

Security threat reported.

  • (SPAM) Email and forum spammers
  • (SCANNER) Network security scanners
  • (BOTNET) Malware infected devices

In PX9-11. 

const Timezone DBField = 25

UTC time zone (with DST supported).

In DB11-18, DB20-22, DB24-26. 

const UsageType DBField = 26

Usage type classification of ISP or company.

  • (COM) Commercial
  • (ORG) Organization
  • (GOV) Government
  • (MIL) Military
  • (EDU) University/College/School
  • (LIB) Library
  • (CDN) Content Delivery Network
  • (ISP) Fixed Line ISP
  • (MOB) Mobile ISP
  • (DCH) Data Center/Web Hosting/Transit
  • (SES) Search Engine Spider
  • (RSV) Reserved

In DB23-26, PX6-11. 

const WeatherStationCode DBField = 27

The special code to identify the nearest weather observation station.

In DB17-18, DB20, DB22, DB24-26. 

const WeatherStationName DBField = 28

The name of the nearest weather observation station.

In DB17-18, DB20, DB22, DB24-26. 

const Zipcode DBField = 29

ZIP code or Postal code.

See https://www.ip2location.com/zip-code-coverage.

In DB9-12, DB14-16, DB18, DB20-22, DB24-26.

func (DBField) GoString 

func (f DBField) GoString() string

func (DBField) String 

func (f DBField) String() string

String returns the name of the database column.

type DBProduct 

type DBProduct uint8

DBProduct represents an IP2Location database product. 

const IP2Location DBProduct = 1

IP2Location™ IP Address Geolocation Database provides a solution to deduce the geolocation of a device connected to the Internet and to determine the approximate geographic location of an IP address along with some other useful information like country, region or state, city, latitude and longitude, ZIP/Postal code, time zone, Internet Service Provider (ISP) or company name, domain name, net speed, area code, weather station code, weather station name, mobile country code (MCC), mobile network code (MNC) and carrier brand, elevation, usage type, address type and advertising category. Up to DB26. 

const IP2Proxy DBProduct = 2

IP2Proxy™ Proxy Detection Database contains IP addresses which are used as VPN anonymizer, open proxies, web proxies and Tor exits, data center, web hosting (DCH) range, search engine robots (SES) and residential proxies (RES). Up to PX11.

func (DBProduct) GoString 

func (p DBProduct) GoString() string

func (DBProduct) String 

func (p DBProduct) String() string

String returns the name of the product.

type DBType 

type DBType uint8

DBType represents an IP2Location database variant. Each database type contains different sets of columns.

func (DBType) String 

func (t DBType) String() string

String formats the type as a string.

type Record 

type Record struct {
	// contains filtered or unexported fields
}

Record points to a database row.

func (Record) Format added in v0.3.0 

func (r Record) Format(color, multiline bool) string

Format gets and formats all fields in the record as a human-readable string. Note that this is highly inefficient.

func (Record) Get 

func (r Record) Get(f DBField) any

Get gets f as the default type. If an error occurs or the field is not present, nil is returned. This is slightly less efficient than the more specific getters.

func (Record) GetFloat32 

func (r Record) GetFloat32(f DBField) (float32, bool)

GetFloat32 gets f as a float32, if possible.

func (Record) GetString 

func (r Record) GetString(f DBField) (string, bool)

GetString gets f as a string.

func (Record) IsValid 

func (r Record) IsValid() bool

IsValid checks whether the record is pointing to a database row.

func (Record) MarshalJSON 

func (r Record) MarshalJSON() ([]byte, error)

MarshalJSON encodes the record as JSON.

func (Record) String 

func (r Record) String() string

String gets and formats all fields in the record as a human-readable string. Note that this is highly inefficient.

Source Files

View all Source files

Directories

Path Synopsis
cmd
ip2x
Command ip2x queries an IP2Location binary database.
 Command ip2x queries an IP2Location binary database.
internal
codegen
Package codegen generates ip2x source code for IP2Location binary databases.
 Package codegen generates ip2x source code for IP2Location binary databases.
test module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.