ipcat

README

ipcat: datasets for categorizing IP addresses.

This is a list of IPv4 address that correspond to datacenters, co-location centers, shared and virtual webhosting providers. In other words, ip addresses that end web consumers should not be using.

Licensing -- GPL v3

The data is licensed under GPL v3, see COPYING for details.

Relaxations and commercial licensing are gladly available by request. The use of GPL is to prevent commercial data providers from scooping up this data without compensation or attribution.

This may be changed to another less restrictive license later.

Statistics

Check out the new datacenter stats

What is the file format?

Standard CSV with ip-start, ip-end (inclusive, in dot-notation), name of provider, url of provider. IP ranges are non-overlapping, and in sorted order.

Why is hosting provider XXX is missing?

It might not be. Many providers are resellers of another and will be included under another name or ip range.

Also, as of 16-Oct-2011, many locations from Africa, Latin America, Korea and Japan are missing.

Or, it might just be missing. Please let us know!

Why GitHub + CSV?

The goal of the file format and the use of github was designed to make it really easy for other to send patches or additions. It also provides and easy way of keeping track of changes.

How is this generated?

Manually from users like you, and automatically via proprietary discovery algorithms.

Who made this?

Nick Galbreath. See more at http://www.client9.com/

Documentation

Index

• func CIDR2Range(c string) (string, string, error)
• func DownloadAWS() ([]byte, error)
• func DownloadAzure() ([]byte, error)
• func ToDots(val uint32) string
• func UpdateAWS(ipmap *IntervalSet, body []byte) error
• func UpdateAzure(ipmap *IntervalSet, body []byte) error
• type AWS
• type AWSPrefix
• type AzureIPRange
• type AzurePublicIPAddresses
• type AzureRegion
• type IntervalSet
  • func NewIntervalSet(capacity int) *IntervalSet
  • func (ipset *IntervalSet) AddCIDR(cidr, name, url string) error
  • func (ipset *IntervalSet) AddRange(dotsleft, dotsright, name, url string) error
  • func (ipset IntervalSet) Contains(dots string) (bool, error)
  • func (ipset *IntervalSet) DeleteByName(name string)
  • func (ipset *IntervalSet) ExportCSV(in io.Writer) error
  • func (ipset *IntervalSet) ImportCSV(in io.Reader) error
  • func (ipset IntervalSet) Len() int
  • func (ipset IntervalSet) RankBySize() NameSizeList
• type NameSize
• type NameSizeList
  • func (list NameSizeList) Len() int
  • func (list NameSizeList) Less(i, j int) bool
  • func (list NameSizeList) Swap(i, j int)

Functions

func CIDR2Range

func CIDR2Range(c string) (string, string, error)

CIDR2Range converts a CIDR to a dotted IP address pair, or empty strings and error

generic.. does not care if ipv4 or ipv6

func DownloadAWS

func DownloadAWS() ([]byte, error)

DownloadAWS downloads the latest AWS IP ranges list

func DownloadAzure

func DownloadAzure() ([]byte, error)

DownloadAzure downloads and return raw bytes of the MS Azure ip range list

func ToDots

func ToDots(val uint32) string

ToDots converts a uint32 to a IPv4 Dotted notation

func UpdateAWS

func UpdateAWS(ipmap *IntervalSet, body []byte) error

UpdateAWS parses the AWS IP json file and updates the interval set

func UpdateAzure

func UpdateAzure(ipmap *IntervalSet, body []byte) error

UpdateAzure takes a raw data, parses it and updates the ipmap

Types

type AWS

type AWS struct {
	SyncToken  string      `json:"syncToken"`
	CreateDate string      `json:"createDate"`
	Prefixes   []AWSPrefix `json:"prefixes"`
}

AWS is main record for AWS IP info

type AWSPrefix

type AWSPrefix struct {
	IPPrefix string `json:"ip_prefix"`
	Region   string `json:"region"`
	Service  string `json:"service"`
}

AWSPrefix is AWS prefix in their IP ranges file

type AzureIPRange

type AzureIPRange struct {
	Subnet string `xml:"Subnet,attr"`
}

AzureIPRange is a MS Azure record

type AzurePublicIPAddresses

type AzurePublicIPAddresses struct {
	AzureRegion []AzureRegion
}

AzurePublicIPAddresses is a listing of regions

type AzureRegion

type AzureRegion struct {
	Name    string         `xml:"Name,attr"`
	IPRange []AzureIPRange `xml:"IpRange"`
}

AzureRegion is a MS Region

type IntervalSet

type IntervalSet struct {
	// contains filtered or unexported fields
}

IntervalSet is a mapping of an IP range (the closed interval)

to additional data

func NewIntervalSet

func NewIntervalSet(capacity int) *IntervalSet

NewIntervalSet creates a new set with a capacity

func (*IntervalSet) AddCIDR

func (ipset *IntervalSet) AddCIDR(cidr, name, url string) error

AddCIDR adds an entry based on a CIDR range

func (*IntervalSet) AddRange

func (ipset *IntervalSet) AddRange(dotsleft, dotsright, name, url string) error

AddRange adds an entry based on an IP range

func (IntervalSet) Contains

func (ipset IntervalSet) Contains(dots string) (bool, error)

Contains returns true if the IP address is in some interval

else false or error

func (*IntervalSet) DeleteByName

func (ipset *IntervalSet) DeleteByName(name string)

DeleteByName deletes all entries with the given name

func (*IntervalSet) ExportCSV

func (ipset *IntervalSet) ExportCSV(in io.Writer) error

ExportCSV export data to a CSV file

func (*IntervalSet) ImportCSV

func (ipset *IntervalSet) ImportCSV(in io.Reader) error

ImportCSV imports data from a CSV file

func (IntervalSet) Len

func (ipset IntervalSet) Len() int

Len returns the number of elements in the set

func (IntervalSet) RankBySize

func (ipset IntervalSet) RankBySize() NameSizeList

RankBySize returns a list ISP and how many IPs they have

From this it's easy to compute
  * Lastest providers
  * Number of providers
  * Total number IPs address

type NameSize

type NameSize struct {
	Name string
	Size int
}

NameSize is a tuple mapping name with a size

type NameSizeList

type NameSizeList []NameSize

NameSizeList is a list of NameSize

func (NameSizeList) Len

func (list NameSizeList) Len() int

Len satifies the sort.Sortable interface

func (NameSizeList) Less

func (list NameSizeList) Less(i, j int) bool

Less satifies the sort.Sortable interface THIS IS DESCENDING SORT, the sign is flipped

MORE = FIRST

func (NameSizeList) Swap

func (list NameSizeList) Swap(i, j int)

Swap satifies the sort.Sortable interface