ipcat

README

ipcat: datasets for categorizing IP addresses.

Archived in 2023. Please fork and edit as you wish. It's MIT now. Onward -- nickg

---

This is a list of IPv4 addresses that correspond to datacenters, co-location centers, shared and virtual webhosting providers. In other words, ip addresses that end web consumers should not be using.

Statistics

Check out the new datacenter stats

What is the file format?

Standard CSV with ip-start, ip-end (inclusive, in dot-notation), name of provider, url of provider. IP ranges are non-overlapping, and in sorted order.

Why is hosting provider XXX is missing?

It might not be. Many providers are resellers of another and will be included under another name or ip range.

Also, as of 16-Oct-2011, many locations from Africa, Latin America, Korea and Japan are missing.

Or, it might just be missing. Please let us know!

Why GitHub + CSV?

The goal of the file format and the use of github was designed to make it really easy for other to send patches or additions. It also provides an easy way of keeping track of changes.

How is this generated?

Manually from users like you, and automatically via proprietary discovery algorithms.

Who made this?

Nick Galbreath. See more at http://www.client9.com/

Documentation

Index

• func CIDR2Range(c string) (string, string, error)
• func DownloadAWS() ([]byte, error)
• func DownloadAppEngine() ([]string, error)
• func DownloadAzure() ([]byte, error)
• func DownloadCloudflare() ([]byte, error)
• func ToDots(val uint32) string
• func UpdateAWS(ipmap *IntervalSet, body []byte) error
• func UpdateAppEngine(ipmap *IntervalSet, ranges []string) error
• func UpdateAzure(ipmap *IntervalSet, body []byte) error
• func UpdateCloudflare(ipmap *IntervalSet, body []byte) error
• type AWS
• type AWSPrefix
• type AzureIPRange
• type AzurePublicIPAddresses
• type AzureRegion
• type Interval
• type IntervalSet
  • func NewIntervalSet(capacity int) *IntervalSet
  • func (ipset *IntervalSet) AddCIDR(cidr, name, url string) error
  • func (ipset *IntervalSet) AddRange(dotsleft, dotsright, name, url string) error
  • func (ipset IntervalSet) Contains(dots string) (*Interval, error)
  • func (ipset *IntervalSet) DeleteByName(name string)
  • func (ipset *IntervalSet) ExportCSV(in io.Writer) error
  • func (ipset *IntervalSet) ImportCSV(in io.Reader) error
  • func (ipset IntervalSet) Len() int
  • func (ipset IntervalSet) RankBySize() NameSizeList
• type NameSize
• type NameSizeList

Functions

func CIDR2Range

func CIDR2Range(c string) (string, string, error)

CIDR2Range converts a CIDR to a dotted IP address pair, or empty strings and error

Generic.. does not care if ipv4 or ipv6

func DownloadAWS

func DownloadAWS() ([]byte, error)

DownloadAWS downloads the latest AWS IP ranges list

func DownloadAppEngine

func DownloadAppEngine() ([]string, error)

DownloadAppEngine downloads and returns raw bytes of the Google App Engine ip range list

func DownloadAzure

func DownloadAzure() ([]byte, error)

DownloadAzure downloads and returns raw bytes of the MS Azure ip range list

func DownloadCloudflare

func DownloadCloudflare() ([]byte, error)

DownloadCloudflare downloads the latest Cloudflare IP ranges list

func ToDots

func ToDots(val uint32) string

ToDots converts a uint32 to a IPv4 Dotted notation

func UpdateAWS

func UpdateAWS(ipmap *IntervalSet, body []byte) error

UpdateAWS parses the AWS IP json file and updates the interval set

func UpdateAppEngine

func UpdateAppEngine(ipmap *IntervalSet, ranges []string) error

UpdateAppEngine takes a raw data, parses it and updates the ipmap

func UpdateAzure

func UpdateAzure(ipmap *IntervalSet, body []byte) error

UpdateAzure takes a raw data, parses it and updates the ipmap

func UpdateCloudflare

func UpdateCloudflare(ipmap *IntervalSet, body []byte) error

UpdateCloudflare parses the Cloudflare IP text file and updates the interval set

Types

type AWS

type AWS struct {
	SyncToken  string      `json:"syncToken"`
	CreateDate string      `json:"createDate"`
	Prefixes   []AWSPrefix `json:"prefixes"`
}

AWS is main record for AWS IP info

type AWSPrefix

type AWSPrefix struct {
	IPPrefix string `json:"ip_prefix"`
	Region   string `json:"region"`
	Service  string `json:"service"`
}

AWSPrefix is AWS prefix in their IP ranges file

type AzureIPRange

type AzureIPRange struct {
	Subnet string `xml:"Subnet,attr"`
}

AzureIPRange is a MS Azure record

type AzurePublicIPAddresses

type AzurePublicIPAddresses struct {
	AzureRegion []AzureRegion `xml:"Region"`
}

AzurePublicIPAddresses is a listing of regions

type AzureRegion

type AzureRegion struct {
	Name    string         `xml:"Name,attr"`
	IPRange []AzureIPRange `xml:"IpRange"`
}

AzureRegion is a MS Region

type Interval

type Interval struct {
	Left      uint32
	Right     uint32
	LeftDots  string
	RightDots string
	Name      string
	URL       string
}

Interval is a closed interval [a,b] of an IPv4 range

type IntervalSet

type IntervalSet struct {
	// contains filtered or unexported fields
}

IntervalSet is a mapping of an IP range (the closed interval) to additional data

func NewIntervalSet

func NewIntervalSet(capacity int) *IntervalSet

NewIntervalSet creates a new set with a capacity

func (*IntervalSet) AddCIDR

func (ipset *IntervalSet) AddCIDR(cidr, name, url string) error

AddCIDR adds an entry based on a CIDR range

func (*IntervalSet) AddRange

func (ipset *IntervalSet) AddRange(dotsleft, dotsright, name, url string) error

AddRange adds an entry based on an IP range

func (IntervalSet) Contains

func (ipset IntervalSet) Contains(dots string) (*Interval, error)

Contains returns the internal record if the IP address is in some interval else nil or error. It returns a pointer to the internal record, so be careful.

func (*IntervalSet) DeleteByName

func (ipset *IntervalSet) DeleteByName(name string)

DeleteByName deletes all entries with the given name

func (*IntervalSet) ExportCSV

func (ipset *IntervalSet) ExportCSV(in io.Writer) error

ExportCSV export data to a CSV file

func (*IntervalSet) ImportCSV

func (ipset *IntervalSet) ImportCSV(in io.Reader) error

ImportCSV imports data from a CSV file

func (IntervalSet) Len

func (ipset IntervalSet) Len() int

Len returns the number of elements in the set

func (IntervalSet) RankBySize

func (ipset IntervalSet) RankBySize() NameSizeList

RankBySize returns a list ISP and how many IPs they have From this it's easy to compute:

* Lastest providers

* Number of providers

* Total number IP address

type NameSize

type NameSize struct {
	Name string
	Size int
}

NameSize is a tuple mapping name with a size

type NameSizeList

type NameSizeList []NameSize

NameSizeList is a list of NameSize