README
¶
Chapter III: using a custom DNS-over-UDP resolver
In this chapter we learn how to measure sending DNS queries to a DNS server speaking the DNS-over-UDP protocol.
Without further ado, let's describe our example main.go program
and let's use it to better understand this flow.
(This file is auto-generated. Do not edit it directly! To apply
changes you need to modify ./internal/tutorial/measurex/chapter03/main.go.)
main.go
The initial part of the program is pretty much the same as the one used in previous chapters, so I will not add further comments.
package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"time"
"github.com/ooni/probe-cli/v3/internal/legacy/measurex"
"github.com/ooni/probe-cli/v3/internal/runtimex"
)
func main() {
query := flag.String("query", "example.com", "domain to resolve")
address := flag.String("address", "8.8.4.4:53", "DNS-over-UDP server address")
timeout := flag.Duration("timeout", 60*time.Second, "timeout to use")
flag.Parse()
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
defer cancel()
mx := measurex.NewMeasurerWithDefaultSettings()
Using a custom UDP resolver
We now invoke LookupHostUDP. We specify:
-
a context for timeout information;
-
the domain to query for;
-
the address of the DNS-over-UDP server endpoint.
m := mx.LookupHostUDP(ctx, *query, *address)
Also this operation returns a measurement, which we print using the usual three-liner.
data, err := json.Marshal(measurex.NewArchivalDNSMeasurement(m))
runtimex.PanicOnError(err, "json.Marshal failed")
fmt.Printf("%s\n", string(data))
}
Running the example program
As before, let us start off with a vanilla run:
go run -race ./internal/tutorial/measurex/chapter03 | jq
This time we get a much larger JSON, so I will pretend it is actually JavaScript and add comments to explain it inline.
(This is the first case in which we see how a single
method call for measurer causes several events to
be generated and inserted into a Measurement.)
{
"domain": "example.com",
// This block shows the read and write events
// occurred on the sockets (because we control
// in full the implementation of this DNS
// over UDP resolver, we can see these events)
//
// See https://github.com/ooni/spec/blob/master/data-formats/df-008-netevents.md
// for a description of this data format.
"network_events": [
{
"address": "8.8.4.4:53",
"failure": null,
"num_bytes": 29,
"operation": "write",
"proto": "udp",
"t": 0.00048825,
"started": 0.000462917,
"oddity": ""
},
{
"address": "8.8.4.4:53",
"failure": null,
"num_bytes": 45,
"operation": "read",
"proto": "udp",
"t": 0.022081833,
"started": 0.000502625,
"oddity": ""
},
{
"address": "8.8.4.4:53",
"failure": null,
"num_bytes": 29,
"operation": "write",
"proto": "udp",
"t": 0.022433083,
"started": 0.022423875,
"oddity": ""
},
{
"address": "8.8.4.4:53",
"failure": null,
"num_bytes": 57,
"operation": "read",
"proto": "udp",
"t": 0.046706,
"started": 0.022443833,
"oddity": ""
}
],
// This block shows the query we sent (encoded as base64)
// and the response we received. Here we clearly see
// that we perform two DNS "round trip" (i.e., send request
// and receive response) to resolve a domain: one for
// A and the other for AAAA.
//
// We don't have a specification for this data format yet.
"dns_events": [
{
"engine": "udp",
"resolver_address": "8.8.4.4:53",
"raw_query": {
"data": "dGwBAAABAAAAAAAAB2V4YW1wbGUDY29tAAABAAE=",
"format": "base64"
},
"started": 0.000205083,
"t": 0.022141333,
"failure": null,
"raw_reply": {
"data": "dGyBgAABAAEAAAAAB2V4YW1wbGUDY29tAAABAAHADAABAAEAAEuIAARduNgi",
"format": "base64"
}
},
{
"engine": "udp",
"resolver_address": "8.8.4.4:53",
"raw_query": {
"data": "Ts8BAAABAAAAAAAAB2V4YW1wbGUDY29tAAAcAAE=",
"format": "base64"
},
"started": 0.022221417,
"t": 0.046733125,
"failure": null,
"raw_reply": {
"data": "Ts+BgAABAAEAAAAAB2V4YW1wbGUDY29tAAAcAAHADAAcAAEAAFOQABAmBigAAiAAAQJIGJMlyBlG",
"format": "base64"
}
}
],
// This is the same kind of result as before, we
// show the emitted queries and the resolved addrs.
//
// Also note how here the resolver_address is the
// correct endpoint address and the engine tells us
// that we're using DNS over UDP.
"queries": [
{
"answers": [
{
"answer_type": "A",
"ipv4": "93.184.216.34"
}
],
"engine": "udp",
"failure": null,
"hostname": "example.com",
"query_type": "A",
"resolver_address": "8.8.4.4:53",
"t": 0.046766833,
"started": 0.000124375,
"oddity": ""
},
{
"answers": [
{
"answer_type": "AAAA",
"ivp6": "2606:2800:220:1:248:1893:25c8:1946"
}
],
"engine": "udp",
"failure": null,
"hostname": "example.com",
"query_type": "AAAA",
"resolver_address": "8.8.4.4:53",
"t": 0.046766833,
"started": 0.000124375,
"oddity": ""
}
]
}
This data format is really an extension of the LookupHostSystem
one. It just adds more fields that clarify what happened at low
level in terms of socket I/O and queries sent and received.
Let us now try to provoke some errors and see how the output JSON changes because of them.
Measurement with NXDOMAIN
Let us try to get a NXDOMAIN error.
go run -race ./internal/tutorial/measurex/chapter03 -query antani.ooni.org | jq
This produces the following JSON:
{
"domain": "antani.ooni.org",
"network_events": [ /* snip */ ],
"dns_events": [
{
"engine": "udp",
"resolver_address": "8.8.4.4:53",
"raw_query": {
"data": "p7YBAAABAAAAAAAABmFudGFuaQRvb25pA29yZwAAAQAB",
"format": "base64"
},
"started": 0.000152959,
"t": 0.051650917,
"failure": null,
"raw_reply": {
"data": "p7aBgwABAAAAAQAABmFudGFuaQRvb25pA29yZwAAAQABwBMABgABAAAHCAA9BGRuczERcmVnaXN0cmFyLXNlcnZlcnMDY29tAApob3N0bWFzdGVywDJhbwqOAACowAAADhAACTqAAAAOEQ==",
"format": "base64"
}
},
{
"engine": "udp",
"resolver_address": "8.8.4.4:53",
"raw_query": {
"data": "ILkBAAABAAAAAAAABmFudGFuaQRvb25pA29yZwAAHAAB",
"format": "base64"
},
"started": 0.051755209,
"t": 0.101094375,
"failure": null,
"raw_reply": {
"data": "ILmBgwABAAAAAQAABmFudGFuaQRvb25pA29yZwAAHAABwBMABgABAAAHCAA9BGRuczERcmVnaXN0cmFyLXNlcnZlcnMDY29tAApob3N0bWFzdGVywDJhbwqOAACowAAADhAACTqAAAAOEQ==",
"format": "base64"
}
}
],
"queries": [
{
"answers": null,
"engine": "udp",
"failure": "dns_nxdomain_error",
"hostname": "antani.ooni.org",
"query_type": "A",
"resolver_address": "8.8.4.4:53",
"t": 0.101241667,
"started": 8.8e-05,
"oddity": "dns.lookup.nxdomain"
},
{
"answers": null,
"engine": "udp",
"failure": "dns_nxdomain_error",
"hostname": "antani.ooni.org",
"query_type": "AAAA",
"resolver_address": "8.8.4.4:53",
"t": 0.101241667,
"started": 8.8e-05,
"oddity": "dns.lookup.nxdomain"
}
]
}
We indeed get a NXDOMAIN error as the failure in lookup_host.
Let us now decode one of the replies by using this program:
package main
import (
"fmt"
"encoding/base64"
"github.com/miekg/dns"
)
func main() {
const query = "azGBgwABAAAAAQAABmFudGFuaQRvb25pA29yZwAAHAABwBMABgABAAAHCAA9BGRuczERcmVnaXN0cmFyLXNlcnZlcnMDY29tAApob3N0bWFzdGVywDJhABz8AACowAAADhAACTqAAAAOEQ=="
data, _ := base64.StdEncoding.DecodeString(query)
msg := new(dns.Msg)
_ = msg.Unpack(data)
fmt.Printf("%s\n", msg)
}
where query is one of the replies. If we run this program
we get as the output:
;; opcode: QUERY, status: NXDOMAIN, id: 27441
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;antani.ooni.org. IN AAAA
;; AUTHORITY SECTION:
ooni.org. 1800 IN SOA dns1.registrar-servers.com. hostmaster.registrar-servers.com. 1627397372 43200 3600 604800 3601
Measurement with timeout
Let us now query an IP address known for not responding to DNS queries, to get a timeout.
go run -race ./internal/tutorial/measurex/chapter03 -address 182.92.22.222:53
Here's the corresponding JSON:
{
"domain": "example.com",
"network_events": [
{
"address": "182.92.22.222:53",
"failure": null,
"num_bytes": 29,
"operation": "write",
"proto": "udp",
"t": 0.000479583,
"started": 0.00045525,
"oddity": ""
},
{
"address": "182.92.22.222:53",
"failure": "generic_timeout_error", /* <--- */
"operation": "read",
"proto": "udp",
"t": 5.006016292,
"started": 0.000491792,
"oddity": ""
}
],
"dns_events": [
{
"engine": "udp",
"resolver_address": "182.92.22.222:53",
"raw_query": {
"data": "GRUBAAABAAAAAAAAB2V4YW1wbGUDY29tAAABAAE=",
"format": "base64"
},
"started": 0.00018225,
"t": 5.006185667,
"failure": "generic_timeout_error", /* <--- */
"raw_reply": null
}
/* snip */
],
"queries": [
{
"answers": null,
"engine": "udp",
"failure": "generic_timeout_error", /* <--- */
"hostname": "example.com",
"query_type": "A",
"resolver_address": "182.92.22.222:53",
"t": 5.007385458,
"started": 0.000107583,
"oddity": "dns.lookup.timeout"
},
{
"answers": null,
"engine": "udp",
"failure": "generic_timeout_error", /* <--- */
"hostname": "example.com",
"query_type": "AAAA",
"resolver_address": "182.92.22.222:53",
"t": 5.007385458,
"started": 0.000107583,
"oddity": "dns.lookup.timeout"
}
]
}
We see that we fail with a timeout (I have marked some of them
with comments inside the JSON). We see the timeout at three different
levels of abstractions (from lower to higher abstraction): at the socket
layer (network_events), during the DNS round trip (dns_events),
during the DNS lookup (queries).
What we also see is that t's value is ~5s when the read event
fails, which tells us about the socket's read timeout.
Measurement with REFUSED error
Let us now try to get a REFUSED DNS Rcode, again from servers that are, let's say, kind enough to easily help.
go run -race ./internal/tutorial/measurex/chapter03 -address 180.97.36.63:53 | jq
Here's the answer I get:
{
"domain": "example.com",
// The network events look normal this time
"network_events": [
{
"address": "180.97.36.63:53",
"failure": null,
"num_bytes": 29,
"operation": "write",
"proto": "udp",
"t": 0.000492125,
"started": 0.000467042,
"oddity": ""
},
{
"address": "180.97.36.63:53",
"failure": null,
"num_bytes": 29,
"operation": "read",
"proto": "udp",
"t": 0.321373542,
"started": 0.000504833,
"oddity": ""
},
{
"address": "180.97.36.63:53",
"failure": null,
"num_bytes": 29,
"operation": "write",
"proto": "udp",
"t": 0.322500875,
"started": 0.322450042,
"oddity": ""
},
{
"address": "180.97.36.63:53",
"failure": null,
"num_bytes": 29,
"operation": "read",
"proto": "udp",
"t": 0.655514542,
"started": 0.322557667,
"oddity": ""
}
],
// Exercise: do like I did before and decode the messages
"dns_events": [
{
"engine": "udp",
"resolver_address": "180.97.36.63:53",
"raw_query": {
"data": "WcgBAAABAAAAAAAAB2V4YW1wbGUDY29tAAABAAE=",
"format": "base64"
},
"started": 0.000209875,
"t": 0.321504042,
"failure": null,
"raw_reply": {
"data": "WciBBQABAAAAAAAAB2V4YW1wbGUDY29tAAABAAE=",
"format": "base64"
}
},
{
"engine": "udp",
"resolver_address": "180.97.36.63:53",
"raw_query": {
"data": "I9oBAAABAAAAAAAAB2V4YW1wbGUDY29tAAAcAAE=",
"format": "base64"
},
"started": 0.321672042,
"t": 0.655680792,
"failure": null,
"raw_reply": {
"data": "I9qBBQABAAAAAAAAB2V4YW1wbGUDY29tAAAcAAE=",
"format": "base64"
}
}
],
// We see both in the error and in the oddity
// that the response was "REFUSED"
"queries": [
{
"answers": null,
"engine": "udp",
"failure": "dns_refused_error",
"hostname": "example.com",
"query_type": "A",
"resolver_address": "180.97.36.63:53",
"t": 0.655814875,
"started": 0.000107417,
"oddity": "dns.lookup.refused"
},
{
"answers": null,
"engine": "udp",
"failure": "dns_refused_error",
"hostname": "example.com",
"query_type": "AAAA",
"resolver_address": "180.97.36.63:53",
"t": 0.655814875,
"started": 0.000107417,
"oddity": "dns.lookup.refused"
}
]
}
Conclusion
We have seen how to send DNS queries over UDP, measure the results, and what happens on common error conditions.
Documentation
¶
Overview ¶
-=-=- StartHere -=-=-
Chapter III: using a custom DNS-over-UDP resolver ¶
In this chapter we learn how to measure sending DNS queries to a DNS server speaking the DNS-over-UDP protocol.
Without further ado, let's describe our example `main.go` program and let's use it to better understand this flow.
(This file is auto-generated. Do not edit it directly! To apply changes you need to modify `./internal/tutorial/measurex/chapter03/main.go`.)
## main.go
The initial part of the program is pretty much the same as the one used in previous chapters, so I will not add further comments.
```Go
Source Files