go-dnscollector

README

DNS-collector

NOTE: The code before version 1.x is considered beta quality and is subject to breaking changes.

DNS-collector acts as a passive high speed ingestor, aggregator and distributor for your DNS logs with usage indicators and security analysis, written in Golang. The DNS traffic can be collected and aggregated from simultaneously sources like DNStap streams, network interface or log files and relays it to multiple other listeners with some transformations on it (traffic filtering, user privacy, ...).

Additionally, DNS-collector also support

• DNS protocol conversions (to plain text, JSON, and more... )
• DNS parser with Extension Mechanisms for DNS (EDNS) support
• IPv4/v6 defragmentation and TCP reassembly
• Nanoseconds in timestamps

Features

• Collectors

  • Listen for logging traffic with streaming network protocols
    • DNStap with tls|tcp|unix transports support and proxifier
    • PowerDNS streams with full support
    • TZSP protocol support
  • Live capture on a network interface
    • AF_PACKET socket with BPF filter
    • eBPF XDP ingress traffic
  • Read text or binary files as input
    • Read and tail on Plain text files
    • Ingest PCAP or DNSTap files by watching a directory

• Loggers

  • Local storage of your DNS logs in text or binary formats
    • Stdout console in text or binary output
    • File with automatic rotation and compression
  • Provide metrics and API
    • Prometheus metrics
    • Statsd support
    • REST API with swagger to search DNS domains
  • Send to remote host with generic transport protocol
    • TCP
    • Syslog with TLS support
    • DNSTap protobuf messages
  • Send to various sinks
    • Fluentd
    • InfluxDB
    • Loki
    • ElasticSearch
    • Scalyr
    • Redis
    • Kafka
  • Send to security tools
    • Falco

• Transformers

  • Traffic Filtering and Reducer
  • Latency Computing
  • Apply user Privacy
  • Normalize DNS messages
  • Add Geographical metadata
  • Various data Extractor
  • Suspicious traffic Detector and Prediction

Get Started

Download the latest release binary and start the DNS-collector with the provided configuration file. The default configuration listens on tcp/6000 for a DNSTap stream and DNS logs are printed on standard output.

./go-dnscollector -config config.yml

If you prefer run it from docker, follow this guide.

Configuration

The configuration of DNS-collector is done through a file named config.yml. When the DNS-collector starts, it will look for the config.yml from the current working directory.

See the full configuration guide for more details.

Usage examples

The _examples folder from documentation contains a number of various configurations to get you started with the DNS-collector in differentes ways.

Contributing

See the development guide for more information on how to build it yourself.