zdns

package module
v1.0.0-RC3FIX Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 13, 2020 License: Apache-2.0 Imports: 12 Imported by: 0

README

ZDNS

Build Status Go Report Card

ZDNS is a command-line utility that provides high-speed DNS lookups. For example, the following will perform MX lookups and a secondary A lookup for the IPs of MX servers for the domains in the Alexa Top Million:

cat top-1m.csv | zdns MX --ipv4-lookup --alexa

ZDNS is written in golang and is primarily based on https://github.com/miekg/dns.

Install

ZDNS can be installed by running:

go get github.com/Rich5/zdns/zdns

Usage

ZDNS provides several types of modules.

Raw DNS Modules

The A, AAAA, ANY, AXFR, CAA, CDS, CDNSKEY, CNAME, DMARC, DS, DNSKEY, MX, NAPTR, NS, NSEC, NSEC3, NSEC3PARAM, PTR, RRSIG, SOA, SPF, SRV, TLSA, and TXT modules provide the raw DNS response in JSON form, similar to dig.

For example, the command:

echo "censys.io" | zdns A

returns:

{
  "name": "censys.io",
  "class": "IN",
  "status": "NOERROR",
  "data": {
    "answers": [
      {
        "ttl": 300,
        "type": "A",
        "class": "IN",
        "name": "censys.io",
        "data": "216.239.38.21"
      }
    ],
    "additionals": [
      {
        "ttl": 34563,
        "type": "A",
        "class": "IN",
        "name": "ns-cloud-e1.googledomains.com",
        "data": "216.239.32.110"
      },
    ],
    "authorities": [
      {
        "ttl": 53110,
        "type": "NS",
        "class": "IN",
        "name": "censys.io",
        "data": "ns-cloud-e1.googledomains.com."
      },
    ],
    "protocol": "udp",
    "resolver": "30.128.52.190:53"
  }
}

Lookup Modules

Raw DNS responses frequently do not provide the data you want. For example, an MX response may not include the associated A records in the additionals section requiring an additional lookup. To address this gap and provide a friendlier interface, we also provide several lookup modules: alookup and mxlookup.

mxlookup will additionally do an A lookup for the IP addresses that correspond with an exchange record. alookup acts similar to nslookup and will follow CNAME records.

For example,

echo "censys.io" | ./zdns mxlookup --ipv4-lookup

returns:

{
  "name": "censys.io",
  "status": "NOERROR",
  "data": {
    "exchanges": [
      {
        "name": "aspmx.l.google.com",
        "type": "MX",
        "class": "IN",
        "preference": 1,
        "ipv4_addresses": [
          "74.125.28.26"
        ],
        "ttl": 288
      },
      {
        "name": "alt1.aspmx.l.google.com",
        "type": "MX",
        "class": "IN",
        "preference": 5,
        "ipv4_addresses": [
          "64.233.182.26"
        ],
        "ttl": 288
      }
    ]
  }
}

Local Recursion

ZDNS can either operate against a recursive resolver (e.g., an organizational DNS server) [default behavior] or can perform its own recursion internally. To perform local recursion, run zdns with the --iterative flag. When this flag is used, ZDNS will round-robin between the published root servers (e.g., 198.41.0.4). In iterative mode, you can control the size of the local cache by specifying --cache-size and the timeout for individual iterations by setting --iteration-timeout. The --timeout flag controls the timeout of the entire resolution for a given input (i.e., the sum of all iterative steps).

Output Verbosity

DNS includes a lot of extraneous data that is not always useful. There are four result verbosity levels: short, normal (default), long, and trace:

  • short: Short is the most terse result output. It contains only information about the responses
  • normal: Normal provides everything included in short as well as data about the responding server
  • long: Long outputs everything the server included in the DNS packet, including flags.
  • trace: Trace outputs everything from every step of the recursion process

Users can also include specific additional fields using the --include-fields flag and specifying a list of fields, e.g., --include-fields=flags,resolver. Additional fields are: class, protocol, ttl, resolver, flags.

Running ZDNS

By default, ZDNS will operate with 1,000 light-weight go routines. If you're not careful, this will overwhelm many upstream DNS providers. We suggest that users coordinate with local network administrators before performing any scans. You can control the number of concurrent connections with the --threads and --go-processes command line arguments. Alternate name servers can be specified with --name-servers. ZDNS will rotate through these servers when making requests.

While the number of go routines you use will depend on both hardware and the type of request, we've rarely seen performance increase with more than 5,000 go routines. In most cases, either performance will decrease and/or timeouts will increase beyond that point. We've seen good ZDNS performance by running multiple processes. Running 4 instances of ZDNS each with 2,500 threads is a great place to start testing if you're performing large studies.

Unsupported Types

If zdns encounters a record type it does not support it will generate an output record with the type field set correctly and a representation of the underlying data structure in the unparsed_rr field. Do not rely on the presence or structure of this field. This field (and its existence) may change at any time as we expand support for additional record types. If you find yourself using this field, please consider submitting a pull-request adding parser support.

License

ZDNS Copyright 2020 Regents of the University of Michigan

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See LICENSE for the specific language governing permissions and limitations under the License.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var RootServers = [...]string{
	"198.41.0.4:53",
	"192.228.79.201:53",
	"192.33.4.12:53",
	"199.7.91.13:53",
	"192.203.230.10:53",
	"192.5.5.241:53",
	"192.112.36.4:53",
	"198.97.190.53:53",
	"192.36.148.17:53",
	"192.58.128.30:53",
	"193.0.14.129:53",
	"199.7.83.42:53",
	"202.12.27.33:53"}

Functions

func DoLookups 

func DoLookups(g *GlobalLookupFactory, c *GlobalConf) error

func GetDNSServers 

func GetDNSServers(path string) ([]string, error)

func RegisterInputHandler 

func RegisterInputHandler(name string, h InputHandler)

func RegisterLookup 

func RegisterLookup(name string, s GlobalLookupFactory)

func RegisterOutputHandler 

func RegisterOutputHandler(name string, h OutputHandler)

func ValidlookupsString 

func ValidlookupsString() string

Types

type BaseGlobalLookupFactory 

type BaseGlobalLookupFactory struct {
	GlobalConf *GlobalConf
}

func (*BaseGlobalLookupFactory) AddFlags 

func (s *BaseGlobalLookupFactory) AddFlags(f *flag.FlagSet)

func (*BaseGlobalLookupFactory) AllowStdIn 

func (s *BaseGlobalLookupFactory) AllowStdIn() bool

func (*BaseGlobalLookupFactory) Finalize 

func (f *BaseGlobalLookupFactory) Finalize() error

func (*BaseGlobalLookupFactory) Help 

func (s *BaseGlobalLookupFactory) Help() string

func (*BaseGlobalLookupFactory) Initialize 

func (f *BaseGlobalLookupFactory) Initialize(c *GlobalConf) error

func (*BaseGlobalLookupFactory) RandomNameServer 

func (f *BaseGlobalLookupFactory) RandomNameServer() string

func (*BaseGlobalLookupFactory) ZonefileInput 

func (s *BaseGlobalLookupFactory) ZonefileInput() bool

type BaseLookup 

type BaseLookup struct {
}

func (*BaseLookup) DoLookup 

func (base *BaseLookup) DoLookup(name string, class uint16) (interface{}, Status, error)

func (*BaseLookup) DoZonefileLookup 

func (base *BaseLookup) DoZonefileLookup(record *dns.Token) (interface{}, Status, error)

type GlobalConf 

type GlobalConf struct {
	Threads             int
	Timeout             time.Duration
	IterationTimeout    time.Duration
	Retries             int
	AlexaFormat         bool
	IterativeResolution bool

	ResultVerbosity string
	IncludeInOutput string
	OutputGroups    []string

	MaxDepth             int
	CacheSize            int
	GoMaxProcs           int
	Verbosity            int
	TimeFormat           string
	PassedName           string
	NameServersSpecified bool
	NameServers          []string
	TCPOnly              bool
	UDPOnly              bool

	InputHandler  string
	OutputHandler string

	InputFilePath    string
	OutputFilePath   string
	LogFilePath      string
	MetadataFilePath string

	NamePrefix string

	Module string
	Class  uint16
}

type GlobalLookupFactory 

type GlobalLookupFactory interface {
	// expected to add any necessary commandline flags if being
	// run as a standalone scanner
	AddFlags(flags *flag.FlagSet)
	// global initialization. Gets called once globally
	// This is called after command line flags have been parsed
	Initialize(conf *GlobalConf) error
	Finalize() error
	// We can't set variables on an interface, so write functions
	// that define any settings for the factory
	AllowStdIn() bool
	// Some modules have Zonefile inputs
	ZonefileInput() bool
	// Help text for the CLI
	Help() string
	// Return a single scanner which will scan a single host
	MakeRoutineFactory(int) (RoutineLookupFactory, error)
	RandomNameServer() string
}

one RoutineLookupFactory per execution =====================================

func GetLookup 

func GetLookup(name string) GlobalLookupFactory

type InputHandler 

type InputHandler interface {
	// give the InputHandler access to the global config in case it needs any of the settings
	Initialize(conf *GlobalConf)
	// FeedChannel takes a channel to write domains to, the WaitGroup managing them, and if it's a zonefile input
	FeedChannel(in chan<- interface{}, wg *sync.WaitGroup, zonefileInput bool) error
}

handle domain input

func GetInputHandler 

func GetInputHandler(name string) InputHandler

type Lookup 

type Lookup interface {
	DoLookup(name string) (interface{}, []interface{}, Status, error)
	DoZonefileLookup(record *dns.Token) (interface{}, Status, error)
}

one Lookup per IP/name/connection ==========================================

type Metadata 

type Metadata struct {
	Names       int            `json:"names"`
	Status      map[string]int `json:"statuses"`
	StartTime   string         `json:"start_time"`
	EndTime     string         `json:"end_time"`
	NameServers []string       `json:"name_servers"`
	Timeout     int            `json:"timeout"`
	Retries     int            `json:"retries"`
	Conf        *GlobalConf    `json:"conf"`
}

type OutputHandler 

type OutputHandler interface {
	// give the OutputHandler access to the global config in case it needs any of the settings
	Initialize(conf *GlobalConf)
	// takes a channel (results) to write the query results to, and the WaitGroup managing the handlers
	WriteResults(results <-chan string, wg *sync.WaitGroup) error
}

handle output results

func GetOutputHandler 

func GetOutputHandler(name string) OutputHandler

type Result 

type Result struct {
	AlteredName string        `json:"altered_name,omitempty" groups:"short,normal,long,trace"`
	Name        string        `json:"name,omitempty" groups:"short,normal,long,trace"`
	Nameserver  string        `json:"nameserver,omitempty" groups:"normal,long,trace"`
	Class       string        `json:"class,omitempty" groups:"long,trace"`
	AlexaRank   int           `json:"alexa_rank,omitempty" groups:"short,normal,long,trace"`
	Status      string        `json:"status,omitempty" groups:"short,normal,long,trace"`
	Error       string        `json:"error,omitempty" groups:"short,normal,long,trace"`
	Timestamp   string        `json:"timestamp,omitempty" groups:"short,normal,long,trace"`
	Data        interface{}   `json:"data,omitempty" groups:"short,normal,long,trace"`
	Trace       []interface{} `json:"trace,omitempty" groups:"trace"`
}

type RoutineLookupFactory 

type RoutineLookupFactory interface {
	MakeLookup() (Lookup, error)
}

one RoutineLookupFactory per goroutine =====================================

type Status 

type Status string
const (
	STATUS_NOERROR       Status = "NOERROR"
	STATUS_ERROR         Status = "ERROR"
	STATUS_SERVFAIL      Status = "SERVFAIL"
	STATUS_AUTHFAIL      Status = "AUTHFAIL"
	STATUS_NO_RECORD     Status = "NORECORD"
	STATUS_BLACKLIST     Status = "BLACKLIST"
	STATUS_NO_OUTPUT     Status = "NO_OUTPUT"
	STATUS_NO_ANSWER     Status = "NO_ANSWER"
	STATUS_ILLEGAL_INPUT Status = "ILLEGAL_INPUT"
	STATUS_TIMEOUT       Status = "TIMEOUT"
	STATUS_ITER_TIMEOUT  Status = "ITERATIVE_TIMEOUT"
	STATUS_TEMPORARY     Status = "TEMPORARY"
	STATUS_TRUNCATED     Status = "TRUNCATED"
	STATUS_NXDOMAIN      Status = "NXDOMAIN"
	STATUS_REFUSED       Status = "REFUSED"
)

type TargetedDomain 

type TargetedDomain struct {
	Domain      string   `json:"domain"`
	Nameservers []string `json:"nameservers"`
}

Source Files

View all Source files

Directories

Path Synopsis
iohandlers
file
modules
alookup
axfr
dmarc
miekg
mxlookup
nslookup
spf

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.