netxlite

package
v3.12.0-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 5, 2021 License: GPL-3.0 Imports: 29 Imported by: 0

Documentation

Overview

Package netxlite contains network extensions.

This package is the basic networking building block that you should be using every time you need networking.

You should consider checking the tutorial explaining how to use this package for network measurements: https://github.com/ooni/probe-cli/tree/master/internal/tutorial/netxlite.

Naming and history

Previous versions of this package were called netx. Compared to such versions this package is lightweight because it does not contain code to perform the measurements, hence its name.

Design

We want to potentially be able to observe each low-level operation separately, even though this is not done by this package. This is the use case where we are performing measurements.

We also want to be able to use this package in a more casual way without having to compose each operation separately. This, instead, is the use case where we're communicating with the OONI backend.

We want to optionally provide detailed logging of every operation, thus users can use `-v` to obtain OONI logs.

We also want to mock any underlying dependency for testing.

We also want to map errors to OONI failures, which are described by https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md.

We want to have reasonable watchdog timeouts for each operation.

Operations

This package implements the following operations:

1. establishing a TCP connection;

2. performing a domain name resolution with the "system" resolver (i.e., getaddrinfo on Unix) or custom DNS transports (e.g., DoT, DoH);

3. performing the TLS handshake;

4. performing the QUIC handshake;

5. dialing with TCP, TLS, and QUIC (where in this context dialing means combining domain name resolution and "connecting");

6. performing HTTP, HTTP2, and HTTP3 round trips.

Operations 1, 2, 3, and 4 are used when we perform measurements, while 5 and 6 are mostly used when speaking with our backend.

Index

Constants

View Source 
const (
	DNSNoSuchHostSuffix        = "no such host"
	DNSServerMisbehavingSuffix = "server misbehaving"
	DNSNoAnswerSuffix          = "no answer from DNS server"
)

We use these strings to string-match errors in the standard library and map such errors to OONI failures.

View Source 
const (
	FailureAddressFamilyNotSupported   = "address_family_not_supported"
	FailureAddressInUse                = "address_in_use"
	FailureAddressNotAvailable         = "address_not_available"
	FailureAlreadyConnected            = "already_connected"
	FailureBadAddress                  = "bad_address"
	FailureBadFileDescriptor           = "bad_file_descriptor"
	FailureConnectionAborted           = "connection_aborted"
	FailureConnectionAlreadyClosed     = "connection_already_closed"
	FailureConnectionAlreadyInProgress = "connection_already_in_progress"
	FailureConnectionRefused           = "connection_refused"
	FailureConnectionReset             = "connection_reset"
	FailureDNSBogonError               = "dns_bogon_error"
	FailureDNSNXDOMAINError            = "dns_nxdomain_error"
	FailureDNSNoAnswer                 = "dns_no_answer"
	FailureDNSNonRecoverableFailure    = "dns_non_recoverable_failure"
	FailureDNSRefusedError             = "dns_refused_error"
	FailureDNSServerMisbehaving        = "dns_server_misbehaving"
	FailureDNSTemporaryFailure         = "dns_temporary_failure"
	FailureDestinationAddressRequired  = "destination_address_required"
	FailureEOFError                    = "eof_error"
	FailureGenericTimeoutError         = "generic_timeout_error"
	FailureHostUnreachable             = "host_unreachable"
	FailureInterrupted                 = "interrupted"
	FailureInvalidArgument             = "invalid_argument"
	FailureJSONParseError              = "json_parse_error"
	FailureMessageSize                 = "message_size"
	FailureNetworkDown                 = "network_down"
	FailureNetworkReset                = "network_reset"
	FailureNetworkUnreachable          = "network_unreachable"
	FailureNoBufferSpace               = "no_buffer_space"
	FailureNoProtocolOption            = "no_protocol_option"
	FailureNotASocket                  = "not_a_socket"
	FailureNotConnected                = "not_connected"
	FailureOperationWouldBlock         = "operation_would_block"
	FailurePermissionDenied            = "permission_denied"
	FailureProtocolNotSupported        = "protocol_not_supported"
	FailureQUICIncompatibleVersion     = "quic_incompatible_version"
	FailureSSLFailedHandshake          = "ssl_failed_handshake"
	FailureSSLInvalidCertificate       = "ssl_invalid_certificate"
	FailureSSLInvalidHostname          = "ssl_invalid_hostname"
	FailureSSLUnknownAuthority         = "ssl_unknown_authority"
	FailureTimedOut                    = "timed_out"
	FailureWrongProtocolType           = "wrong_protocol_type"
)

This enumeration lists the failures defined at https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md. Please, refer to that document for more information.

View Source 
const (
	ECONNREFUSED    = unix.ECONNREFUSED
	ECONNRESET      = unix.ECONNRESET
	EHOSTUNREACH    = unix.EHOSTUNREACH
	ETIMEDOUT       = unix.ETIMEDOUT
	EAFNOSUPPORT    = unix.EAFNOSUPPORT
	EADDRINUSE      = unix.EADDRINUSE
	EADDRNOTAVAIL   = unix.EADDRNOTAVAIL
	EISCONN         = unix.EISCONN
	EFAULT          = unix.EFAULT
	EBADF           = unix.EBADF
	ECONNABORTED    = unix.ECONNABORTED
	EALREADY        = unix.EALREADY
	EDESTADDRREQ    = unix.EDESTADDRREQ
	EINTR           = unix.EINTR
	EINVAL          = unix.EINVAL
	EMSGSIZE        = unix.EMSGSIZE
	ENETDOWN        = unix.ENETDOWN
	ENETRESET       = unix.ENETRESET
	ENETUNREACH     = unix.ENETUNREACH
	ENOBUFS         = unix.ENOBUFS
	ENOPROTOOPT     = unix.ENOPROTOOPT
	ENOTSOCK        = unix.ENOTSOCK
	ENOTCONN        = unix.ENOTCONN
	EWOULDBLOCK     = unix.EWOULDBLOCK
	EACCES          = unix.EACCES
	EPROTONOSUPPORT = unix.EPROTONOSUPPORT
	EPROTOTYPE      = unix.EPROTOTYPE
)

This enumeration provides a canonical name for every system-call error we support. Note: this list is system dependent. You're currently looking at the list of errors for linux.

View Source 
const (
	// ResolveOperation is the operation where we resolve a domain name.
	ResolveOperation = "resolve"

	// ConnectOperation is the operation where we do a TCP connect.
	ConnectOperation = "connect"

	// TLSHandshakeOperation is the TLS handshake.
	TLSHandshakeOperation = "tls_handshake"

	// QUICHandshakeOperation is the handshake to setup a QUIC connection.
	QUICHandshakeOperation = "quic_handshake"

	// QUICListenOperation is when we open a listening UDP conn for QUIC.
	QUICListenOperation = "quic_listen"

	// HTTPRoundTripOperation is the HTTP round trip.
	HTTPRoundTripOperation = "http_round_trip"

	// CloseOperation is when we close a socket.
	CloseOperation = "close"

	// ReadOperation is when we read from a socket.
	ReadOperation = "read"

	// WriteOperation is when we write to a socket.
	WriteOperation = "write"

	// ReadFromOperation is when we read from an UDP socket.
	ReadFromOperation = "read_from"

	// WriteToOperation is when we write to an UDP socket.
	WriteToOperation = "write_to"

	// UnknownOperation is when we cannot determine the operation.
	UnknownOperation = "unknown"

	// TopLevelOperation is used when the failure happens at top level. This
	// happens for example with urlgetter with a cancelled context.
	TopLevelOperation = "top_level"
)

Operations that we measure. They are the possible values of the ErrWrapper.Operation field.

Variables

View Source 
var (
	ErrOODNSNoSuchHost  = fmt.Errorf("ooniresolver: %s", DNSNoSuchHostSuffix)
	ErrOODNSRefused     = errors.New("ooniresolver: refused")
	ErrOODNSMisbehaving = fmt.Errorf("ooniresolver: %s", DNSServerMisbehavingSuffix)
	ErrOODNSNoAnswer    = fmt.Errorf("ooniresolver: %s", DNSNoAnswerSuffix)
)

These errors are returned by custom DNSTransport instances (e.g., DNSOverHTTPS and DNSOverUDP). Their suffix matches the equivalent unexported errors used by the Go standard library.

View Source 
var (
	DefaultDialer        = &dialerSystem{}
	DefaultTLSHandshaker = defaultTLSHandshaker
	NewConnUTLS          = newConnUTLS
	DefaultResolver      = &resolverSystem{}
)

These vars export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

View Source 
var ErrDNSBogon = errors.New("dns: detected bogon address")

ErrDNSBogon indicates that we found a bogon address. Code that filters for DNS bogons MUST use this error.

View Source 
var ErrInvalidTLSVersion = errors.New("invalid TLS version")

ErrInvalidTLSVersion indicates that you passed us a string that does not represent a valid TLS version.

View Source 
var ErrNoConnReuse = errors.New("cannot reuse connection")

ErrNoConnReuse is the type of error returned when you create a "single use" dialer or a "single use" TLS dialer and you dial more than once, which is not supported by such a dialer.

View Source 
var ErrNoDNSTransport = errors.New("operation requires a DNS transport")

ErrNoDNSTransport is the error returned when you attempt to perform a DNS operation that requires a custom DNSTransport (e.g., DNSOverHTTPS) but you are using the "system" resolver instead.

View Source 
var ErrNoDialer = errors.New("no configured dialer")

ErrNoDialer is the type of error returned by "null" dialers when you attempt to dial with them.

View Source 
var ErrNoResolver = errors.New("no configured resolver")

ErrNoResolver is the type of error returned by "without resolver" dialer when asked to dial for and endpoint containing a domain name, since they can only dial for endpoints containing IP addresses.

View Source 
var ErrNoTLSDialer = errors.New("no configured TLS dialer")

ErrNoTLSDialer is the type of error returned by "null" TLS dialers when you attempt to dial with them.

View Source 
var ErrNotTLSConn = errors.New("not a TLSConn")

ErrNotTLSConn occur when an interface accepts a net.Conn but internally needs a TLSConn and you pass a net.Conn that doesn't implement TLSConn to such an interface.

View Source 
var ErrUTLSHandshakePanic = errors.New("utls: handshake panic")

ErrUTLSHandshakePanic indicates that there was panic handshaking when we were using the yawning/utls library for parroting. See https://github.com/ooni/probe/issues/1770 for more information.

Functions

func ClassifyGenericError 

func ClassifyGenericError(err error) string

ClassifyGenericError is maps an error occurred during an operation to an OONI failure string. This specific classifier is the most generic one. You usually use it when mapping I/O errors. You should check whether there is a specific classifier for more specific operations (e.g., DNS resolution, TLS handshake).

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

We put inside this classifier:

- system call errors;

- generic errors that can occur in multiple places;

- all the errors that depend on strings.

The more specific classifiers will call this classifier if they fail to find a mapping for the input error.

If everything else fails, this classifier returns a string like "unknown_failure: XXX" where XXX has been scrubbed so to remove any network endpoints from the original error string.

func ClassifyQUICHandshakeError 

func ClassifyQUICHandshakeError(err error) string

ClassifyQUICHandshakeError maps errors during a QUIC handshake to OONI failure strings.

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

If this classifier fails, it calls ClassifyGenericError and returns to the caller its return value.

func ClassifyResolverError 

func ClassifyResolverError(err error) string

ClassifyResolverError maps DNS resolution errors to OONI failure strings.

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

If this classifier fails, it calls ClassifyGenericError and returns to the caller its return value.

func ClassifyTLSHandshakeError 

func ClassifyTLSHandshakeError(err error) string

ClassifyTLSHandshakeError maps an error occurred during the TLS handshake to an OONI failure string.

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

If this classifier fails, it calls ClassifyGenericError and returns to the caller its return value.

func ConfigureTLSVersion 

func ConfigureTLSVersion(config *tls.Config, version string) error

ConfigureTLSVersion configures the correct TLS version into a *tls.Config or returns ErrInvalidTLSVersion.

Recognized strings: TLSv1.3, TLSv1.2, TLSv1.1, TLSv1.0.

func CopyContext 

func CopyContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, error)

CopyContext is like io.Copy but may terminate earlier when the context expires. This function has the same caveats of ReadAllContext regarding the temporary leaking of the background I/O goroutine.

func IsBogon added in v3.14.0 

func IsBogon(address string) bool

IsBogon returns whether if an IP address is bogon. Passing to this function a non-IP address causes it to return true.

func NewDefaultCertPool 

func NewDefaultCertPool() *x509.CertPool

NewDefaultCertPool returns the default x509 certificate pool that we bundle from Mozilla. It's safe to modify the returned value: every invocation returns a distinct *x509.CertPool instance.

func ReadAllContext 

func ReadAllContext(ctx context.Context, r io.Reader) ([]byte, error)

ReadAllContext is like io.ReadAll but reads r in a background goroutine. This function will return earlier if the context is cancelled. In which case we will continue reading from the reader in the background goroutine, and we will discard the result. To stop the long-running goroutine, close the connection bound to the reader. Until such a connection is closed, you're leaking the backround goroutine and doing I/O.

func TLSCipherSuiteString 

func TLSCipherSuiteString(value uint16) string

TLSCipherSuiteString returns the TLS cipher suite as a string. If value is zero, we return the empty string. If we don't know the mapping from the value to a cipher suite name, we return `TLS_CIPHER_SUITE_UNKNOWN_ddd` where `ddd` is the numeric value passed to this function.

func TLSVersionString 

func TLSVersionString(value uint16) string

TLSVersionString returns a TLS version string. If value is zero, we return the empty string. If the value is unknown, we return `TLS_VERSION_UNKNOWN_ddd` where `ddd` is the numeric value passed to this function.

Types

type AddressResolver deprecated 

type AddressResolver = resolverShortCircuitIPAddr

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type Classifier 

type Classifier func(err error) string

Classifier is the type of the function that maps a Go error to a OONI failure string defined at https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md.

type DNSDecoder 

type DNSDecoder interface {
	// DecodeLookupHost decodes an A or AAAA reply.
	//
	// Arguments:
	//
	// - qtype is the query type (e.g., dns.TypeAAAA)
	//
	// - data contains the reply bytes read from a DNSTransport
	//
	// Returns:
	//
	// - on success, a list of IP addrs inside the reply and a nil error
	//
	// - on failure, a nil list and an error.
	//
	// Note that this function will return an error if there is no
	// IP address inside of the reply.
	DecodeLookupHost(qtype uint16, data []byte) ([]string, error)

	// DecodeHTTPS decodes an HTTPS reply.
	//
	// The argument is the reply as read by the DNSTransport.
	//
	// On success, this function returns an HTTPSSvc structure and
	// a nil error. On failure, the HTTPSSvc pointer is nil and
	// the error points to the error that occurred.
	//
	// This function will return an error if the HTTPS reply does not
	// contain at least a valid ALPN entry. It will not return
	// an error, though, when there are no IPv4/IPv6 hints in the reply.
	DecodeHTTPS(data []byte) (*HTTPSSvc, error)
}

The DNSDecoder decodes DNS replies.

type DNSDecoderMiekg 

type DNSDecoderMiekg struct{}

DNSDecoderMiekg uses github.com/miekg/dns to implement the Decoder.

func (*DNSDecoderMiekg) DecodeHTTPS 

func (d *DNSDecoderMiekg) DecodeHTTPS(data []byte) (*HTTPSSvc, error)

func (*DNSDecoderMiekg) DecodeLookupHost 

func (d *DNSDecoderMiekg) DecodeLookupHost(qtype uint16, data []byte) ([]string, error)

type DNSEncoder 

type DNSEncoder interface {
	// Encode transforms its arguments into a serialized DNS query.
	//
	// Arguments:
	//
	// - domain is the domain for the query (e.g., x.org);
	//
	// - qtype is the query type (e.g., dns.TypeA);
	//
	// - padding is whether to add padding to the query.
	//
	// On success, this function returns a valid byte array and
	// a nil error. On failure, we have an error and the byte array is nil.
	Encode(domain string, qtype uint16, padding bool) ([]byte, error)
}

The DNSEncoder encodes DNS queries to bytes

type DNSEncoderMiekg 

type DNSEncoderMiekg struct{}

DNSEncoderMiekg uses github.com/miekg/dns to implement the Encoder.

func (*DNSEncoderMiekg) Encode 

func (e *DNSEncoderMiekg) Encode(domain string, qtype uint16, padding bool) ([]byte, error)

type DNSOverHTTPS 

type DNSOverHTTPS struct {
	// Client is the MANDATORY http client to use.
	Client HTTPClient

	// URL is the MANDATORY URL of the DNS-over-HTTPS server.
	URL string

	// HostOverride is OPTIONAL and allows to override the
	// Host header sent in every request.
	HostOverride string
}

DNSOverHTTPS is a DNS-over-HTTPS DNSTransport.

func NewDNSOverHTTPS 

func NewDNSOverHTTPS(client HTTPClient, URL string) *DNSOverHTTPS

NewDNSOverHTTPS creates a new DNSOverHTTPS instance.

Arguments:

- client in http.Client-like type (e.g., http.DefaultClient);

- URL is the DoH resolver URL (e.g., https://1.1.1.1/dns-query).

func NewDNSOverHTTPSWithHostOverride 

func NewDNSOverHTTPSWithHostOverride(
	client HTTPClient, URL, hostOverride string) *DNSOverHTTPS

NewDNSOverHTTPSWithHostOverride creates a new DNSOverHTTPS with the given Host header override.

func (*DNSOverHTTPS) Address 

func (t *DNSOverHTTPS) Address() string

Address returns the URL we're using for the DoH server.

func (*DNSOverHTTPS) CloseIdleConnections 

func (t *DNSOverHTTPS) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*DNSOverHTTPS) Network 

func (t *DNSOverHTTPS) Network() string

Network returns the transport network, i.e., "doh".

func (*DNSOverHTTPS) RequiresPadding 

func (t *DNSOverHTTPS) RequiresPadding() bool

RequiresPadding returns true for DoH according to RFC8467.

func (*DNSOverHTTPS) RoundTrip 

func (t *DNSOverHTTPS) RoundTrip(ctx context.Context, query []byte) ([]byte, error)

RoundTrip sends a query and receives a reply.

type DNSOverTCP 

type DNSOverTCP struct {
	// contains filtered or unexported fields
}

DNSOverTCP is a DNS-over-{TCP,TLS} DNSTransport.

Bug: this implementation always creates a new connection for each query.

func NewDNSOverTCP 

func NewDNSOverTCP(dial DialContextFunc, address string) *DNSOverTCP

NewDNSOverTCP creates a new DNSOverTCP transport.

Arguments:

- dial is a function with the net.Dialer.DialContext's signature;

- address is the endpoint address (e.g., 8.8.8.8:53).

func NewDNSOverTLS 

func NewDNSOverTLS(dial DialContextFunc, address string) *DNSOverTCP

NewDNSOverTLS creates a new DNSOverTLS transport.

Arguments:

- dial is a function with the net.Dialer.DialContext's signature;

- address is the endpoint address (e.g., 8.8.8.8:853).

func (*DNSOverTCP) Address 

func (t *DNSOverTCP) Address() string

Address returns the upstream server endpoint (e.g., "1.1.1.1:853").

func (*DNSOverTCP) CloseIdleConnections 

func (t *DNSOverTCP) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*DNSOverTCP) Network 

func (t *DNSOverTCP) Network() string

Network returns the transport network, i.e., "dot" or "tcp".

func (*DNSOverTCP) RequiresPadding 

func (t *DNSOverTCP) RequiresPadding() bool

RequiresPadding returns true for DoT and false for TCP according to RFC8467.

func (*DNSOverTCP) RoundTrip 

func (t *DNSOverTCP) RoundTrip(ctx context.Context, query []byte) ([]byte, error)

RoundTrip sends a query and receives a reply.

type DNSOverUDP 

type DNSOverUDP struct {
	// contains filtered or unexported fields
}

DNSOverUDP is a DNS-over-UDP DNSTransport.

func NewDNSOverUDP 

func NewDNSOverUDP(dialer Dialer, address string) *DNSOverUDP

NewDNSOverUDP creates a DNSOverUDP instance.

Arguments:

- dialer is any type that implements the Dialer interface;

- address is the endpoint address (e.g., 8.8.8.8:53).

func (*DNSOverUDP) Address 

func (t *DNSOverUDP) Address() string

Address returns the upstream server address.

func (*DNSOverUDP) CloseIdleConnections 

func (t *DNSOverUDP) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*DNSOverUDP) Network 

func (t *DNSOverUDP) Network() string

Network returns the transport network, i.e., "udp".

func (*DNSOverUDP) RequiresPadding 

func (t *DNSOverUDP) RequiresPadding() bool

RequiresPadding returns false for UDP according to RFC8467.

func (*DNSOverUDP) RoundTrip 

func (t *DNSOverUDP) RoundTrip(ctx context.Context, query []byte) ([]byte, error)

RoundTrip sends a query and receives a reply.

type DNSTransport 

type DNSTransport interface {
	// RoundTrip sends a DNS query and receives the reply.
	RoundTrip(ctx context.Context, query []byte) (reply []byte, err error)

	// RequiresPadding returns whether this transport needs padding.
	RequiresPadding() bool

	// Network is the network of the round tripper (e.g. "dot").
	Network() string

	// Address is the address of the round tripper (e.g. "1.1.1.1:853").
	Address() string

	// CloseIdleConnections closes idle connections, if any.
	CloseIdleConnections()
}

DNSTransport represents an abstract DNS transport.

type DialContextFunc 

type DialContextFunc func(context.Context, string, string) (net.Conn, error)

DialContextFunc is the type of net.Dialer.DialContext.

type Dialer 

type Dialer interface {
	// DialContext behaves like net.Dialer.DialContext.
	DialContext(ctx context.Context, network, address string) (net.Conn, error)

	// CloseIdleConnections closes idle connections, if any.
	CloseIdleConnections()
}

Dialer establishes network connections.

func NewDialerLegacyAdapter deprecated 

func NewDialerLegacyAdapter(d DialerLegacy) Dialer

NewDialerLegacyAdapter adapts a DialerrLegacy to become compatible with the Dialer definition.

Deprecated: do not use this function in new code.

func NewDialerWithResolver 

func NewDialerWithResolver(logger Logger, resolver Resolver) Dialer

NewDialerWithResolver calls WrapDialer for the stdlib dialer.

func NewDialerWithoutResolver 

func NewDialerWithoutResolver(logger Logger) Dialer

NewDialerWithoutResolver calls NewDialerWithResolver with a "null" resolver.

The returned dialer fails with ErrNoResolver if passed a domain name.

func NewNullDialer 

func NewNullDialer() Dialer

NewNullDialer returns a dialer that always fails with ErrNoDialer.

func NewSingleUseDialer 

func NewSingleUseDialer(conn net.Conn) Dialer

NewSingleUseDialer returns a "single use" dialer. The first dial will succed and return conn regardless of the network and address arguments passed to DialContext. Any subsequent dial returns ErrNoConnReuse.

func WrapDialer 

func WrapDialer(logger Logger, resolver Resolver, dialer Dialer) Dialer

WrapDialer creates a new Dialer that wraps the given Dialer. The returned Dialer has the following properties:

1. logs events using the given logger;

2. resolves domain names using the givern resolver;

3. when the resolver is not a "null" resolver, each available enpoint is tried sequentially. On error, the code will return what it believes to be the most representative error in the pack. Most often, the first error that occurred. Choosing the error to return using this logic is a QUIRK that we owe to the original implementation of netx. We cannot change this behavior until we refactor legacy code using it.

Removing this quirk from the codebase is documented as TODO(https://github.com/ooni/probe/issues/1779).

4. wraps errors;

5. has a configured connect timeout;

6. if a dialer wraps a resolver, the dialer will forward the CloseIdleConnection call to its resolver (which is instrumental to manage a DoH resolver connections properly).

In general, do not use WrapDialer directly but try to use more high-level factories, e.g., NewDialerWithResolver.

type DialerLegacy deprecated 

type DialerLegacy interface {
	// DialContext behaves like net.Dialer.DialContext.
	DialContext(ctx context.Context, network, address string) (net.Conn, error)
}

DialerLegacy establishes network connections.

Deprecated: please use Dialer instead.

Existing code in probe-cli can use it until we have finished refactoring it.

type DialerLegacyAdapter 

type DialerLegacyAdapter struct {
	DialerLegacy
}

DialerLegacyAdapter makes a DialerLegacy behave like it was a Dialer type. If DialerLegacy is actually also a Dialer, this adapter will just forward missing calls, otherwise it will implement a sensible default action.

func (*DialerLegacyAdapter) CloseIdleConnections 

func (d *DialerLegacyAdapter) CloseIdleConnections()

CloseIdleConnections implements Dialer.CloseIdleConnections.

type DialerLogger deprecated 

type DialerLogger = dialerLogger

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type DialerResolver deprecated 

type DialerResolver = dialerResolver

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type DialerSystem deprecated 

type DialerSystem = dialerSystem

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type ErrWrapper 

type ErrWrapper struct {
	// Failure is the OONI failure string. The failure strings are
	// loosely backward compatible with Measurement Kit.
	//
	// This is either one of the FailureXXX strings or any other
	// string like `unknown_failure: ...`. The latter represents an
	// error that we have not yet mapped to a failure.
	Failure string

	// Operation is the operation that failed.
	//
	// New code will always nest ErrWrapper and you need to
	// walk the chain to find what happened.
	//
	// The following comment describes the DEPRECATED
	// legacy behavior implements by internal/engine/legacy/errorsx:
	//
	// If possible, the Operation string
	// SHOULD be a _major_ operation. Major operations are:
	//
	// - ResolveOperation: resolving a domain name failed
	// - ConnectOperation: connecting to an IP failed
	// - TLSHandshakeOperation: TLS handshaking failed
	// - HTTPRoundTripOperation: other errors during round trip
	//
	// Because a network connection doesn't necessarily know
	// what is the current major operation we also have the
	// following _minor_ operations:
	//
	// - CloseOperation: CLOSE failed
	// - ReadOperation: READ failed
	// - WriteOperation: WRITE failed
	//
	// If an ErrWrapper referring to a major operation is wrapping
	// another ErrWrapper and such ErrWrapper already refers to
	// a major operation, then the new ErrWrapper should use the
	// child ErrWrapper major operation. Otherwise, it should use
	// its own major operation. This way, the topmost wrapper is
	// supposed to refer to the major operation that failed.
	Operation string

	// WrappedErr is the error that we're wrapping.
	WrappedErr error
}

ErrWrapper is our error wrapper for Go errors. The key objective of this structure is to properly set Failure, which is also returned by the Error() method, to be one of the OONI failure strings.

OONI failure strings are defined in the github.com/ooni/spec repo at https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md.

func NewErrWrapper 

func NewErrWrapper(c Classifier, op string, err error) *ErrWrapper

NewErrWrapper creates a new ErrWrapper using the given classifier, operation name, and underlying error.

This function panics if classifier is nil, or operation is the empty string or error is nil.

func NewTopLevelGenericErrWrapper 

func NewTopLevelGenericErrWrapper(err error) *ErrWrapper

NewTopLevelGenericErrWrapper wraps an error occurring at top level using ClassifyGenericError as classifier.

func (*ErrWrapper) Error 

func (e *ErrWrapper) Error() string

Error returns the OONI failure string for this error.

func (*ErrWrapper) MarshalJSON 

func (e *ErrWrapper) MarshalJSON() ([]byte, error)

MarshalJSON converts an ErrWrapper to a JSON value.

func (*ErrWrapper) Unwrap 

func (e *ErrWrapper) Unwrap() error

Unwrap allows to access the underlying error.

type HTTPClient 

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
	CloseIdleConnections()
}

HTTPClient is an http.Client-like interface.

type HTTPSSvc 

type HTTPSSvc = dnsx.HTTPSSvc

HTTPSSvc is the type returned for HTTPS queries.

type HTTPTransport 

type HTTPTransport interface {
	// RoundTrip performs the HTTP round trip.
	RoundTrip(req *http.Request) (*http.Response, error)

	// CloseIdleConnections closes idle connections.
	CloseIdleConnections()
}

HTTPTransport is an http.Transport-like structure.

func NewHTTP3Transport 

func NewHTTP3Transport(
	logger Logger, dialer QUICDialer, tlsConfig *tls.Config) HTTPTransport

NewHTTP3Transport creates a new HTTPTransport using http3. The dialer argument MUST NOT be nil. If the tlsConfig argument is nil, then the code will use the default TLS configuration.

func NewHTTPTransport 

func NewHTTPTransport(logger Logger, dialer Dialer, tlsDialer TLSDialer) HTTPTransport

NewHTTPTransport combines NewOOHTTPBaseTransport and WrapHTTPTransport.

This factory and NewHTTPTransportStdlib are the recommended ways of creating a new HTTPTransport.

func NewHTTPTransportStdlib 

func NewHTTPTransportStdlib(logger Logger) HTTPTransport

NewHTTPTransportStdlib creates a new HTTPTransport using the stdlib for DNS resolutions and TLS.

This factory calls NewHTTPTransport with suitable dialers.

This factory and NewHTTPTransport are the recommended ways of creating a new HTTPTransport.

func NewOOHTTPBaseTransport 

func NewOOHTTPBaseTransport(dialer Dialer, tlsDialer TLSDialer) HTTPTransport

NewOOHTTPBaseTransport creates an HTTPTransport using the given dialers.

The returned transport will gracefully handle TLS connections created using gitlab.com/yawning/utls.git, if the TLS dialer is a dialer using such library for TLS operations.

The returned transport will not have a configured proxy, not even the proxy configurable from the environment.

The returned transport will disable transparent decompression of compressed response bodies (and will not automatically ask for such compression, though you can always do that manually).

The returned transport will configure TCP and TLS connections created using its dialer and TLS dialer to always have a read watchdog timeout to address https://github.com/ooni/probe/issues/1609.

The returned transport will always enforce 1 connection per host and we cannot get rid of this QUIRK requirement because it is necessary to perform sane measurements with tracing. We will be able to possibly relax this requirement after we change the way in which we perform measurements.

This is a low level factory. Consider not using it directly.

func WrapHTTPTransport 

func WrapHTTPTransport(logger Logger, txp HTTPTransport) HTTPTransport

WrapHTTPTransport creates an HTTPTransport using the given logger.

This is a low level factory. Consider not using it directly.

type HTTPTransportLogger deprecated 

type HTTPTransportLogger = httpTransportLogger

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type Logger 

type Logger interface {
	// Debugf formats and emits a debug message.
	Debugf(format string, v ...interface{})

	// Debug emits a debug message.
	Debug(msg string)
}

Logger is the interface we expect from a logger.

type QUICContextDialer deprecated 

type QUICContextDialer interface {
	// DialContext establishes a new QUIC session using the given
	// network and address. The tlsConfig and the quicConfig arguments
	// MUST NOT be nil. Returns either the session or an error.
	DialContext(ctx context.Context, network, address string,
		tlsConfig *tls.Config, quicConfig *quic.Config) (quic.EarlySession, error)
}

QUICContextDialer is a dialer for QUIC using Context.

Deprecated: new code should use QUICDialer.

Use NewQUICDialerFromContextDialerAdapter if you need to adapt to QUICDialer.

type QUICContextDialerAdapter 

type QUICContextDialerAdapter struct {
	QUICContextDialer
}

QUICContextDialerAdapter adapts a QUICContextDialer to be a QUICDialer.

func (*QUICContextDialerAdapter) CloseIdleConnections 

func (d *QUICContextDialerAdapter) CloseIdleConnections()

CloseIdleConnections implements QUICDialer.CloseIdleConnections.

type QUICDialer 

type QUICDialer interface {
	// DialContext establishes a new QUIC session using the given
	// network and address. The tlsConfig and the quicConfig arguments
	// MUST NOT be nil. Returns either the session or an error.
	//
	// Recommended tlsConfig setup:
	//
	// - set ServerName to be the SNI;
	//
	// - set RootCAs to NewDefaultCertPool();
	//
	// - set NextProtos to []string{"h3"}.
	//
	// Typically, you want to pass `&quic.Config{}` as quicConfig.
	DialContext(ctx context.Context, network, address string,
		tlsConfig *tls.Config, quicConfig *quic.Config) (quic.EarlySession, error)

	// CloseIdleConnections closes idle connections, if any.
	CloseIdleConnections()
}

QUICDialer dials QUIC sessions.

func NewQUICDialerFromContextDialerAdapter 

func NewQUICDialerFromContextDialerAdapter(d QUICContextDialer) QUICDialer

NewQUICDialerFromContextDialerAdapter creates a new QUICDialer from a QUICContextDialer.

func NewQUICDialerWithResolver 

func NewQUICDialerWithResolver(listener QUICListener,
	logger Logger, resolver Resolver) QUICDialer

NewQUICDialerWithResolver returns a QUICDialer using the given QUICListener to create listening connections and the given Resolver to resolve domain names (if needed).

Properties of the dialer:

1. logs events using the given logger;

2. resolves domain names using the givern resolver;

3. when using a resolver, _may_ attempt multiple dials in parallel (happy eyeballs) and _may_ return an aggregate error to the caller;

4. wraps errors;

5. has a configured connect timeout;

6. if a dialer wraps a resolver, the dialer will forward the CloseIdleConnection call to its resolver (which is instrumental to manage a DoH resolver connections properly).

func NewQUICDialerWithoutResolver 

func NewQUICDialerWithoutResolver(listener QUICListener, logger Logger) QUICDialer

NewQUICDialerWithoutResolver is like NewQUICDialerWithResolver except that there is no configured resolver. So, if you pass in an address containing a domain name, the dial will fail with the ErrNoResolver failure.

func NewSingleUseQUICDialer 

func NewSingleUseQUICDialer(sess quic.EarlySession) QUICDialer

NewSingleUseQUICDialer is like NewSingleUseDialer but for QUIC.

type QUICDialerLogger deprecated 

type QUICDialerLogger = quicDialerLogger

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type QUICDialerQUICGo deprecated 

type QUICDialerQUICGo = quicDialerQUICGo

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type QUICDialerResolver deprecated 

type QUICDialerResolver = quicDialerResolver

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type QUICListener 

type QUICListener interface {
	// Listen creates a new listening UDPLikeConn.
	Listen(addr *net.UDPAddr) (UDPLikeConn, error)
}

QUICListener listens for QUIC connections.

func NewQUICListener 

func NewQUICListener() QUICListener

NewQUICListener creates a new QUICListener using the standard library to create listening UDP sockets.

type QUICListenerStdlib deprecated 

type QUICListenerStdlib = quicListenerStdlib

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type Resolver 

type Resolver interface {
	// LookupHost behaves like net.Resolver.LookupHost.
	LookupHost(ctx context.Context, hostname string) (addrs []string, err error)

	// Network returns the resolver type (e.g., system, dot, doh).
	Network() string

	// Address returns the resolver address (e.g., 8.8.8.8:53).
	Address() string

	// CloseIdleConnections closes idle connections, if any.
	CloseIdleConnections()

	// LookupHTTPS issues an HTTPS query for a domain.
	LookupHTTPS(
		ctx context.Context, domain string) (*HTTPSSvc, error)
}

Resolver performs domain name resolutions.

func NewResolverLegacyAdapter 

func NewResolverLegacyAdapter(reso ResolverLegacy) Resolver

NewResolverLegacyAdapter adapts a ResolverLegacy to become compatible with the Resolver definition.

func NewResolverStdlib 

func NewResolverStdlib(logger Logger) Resolver

NewResolverStdlib creates a new Resolver by combining WrapResolver with an internal "system" resolver type.

func NewResolverUDP 

func NewResolverUDP(logger Logger, dialer Dialer, address string) Resolver

NewResolverUDP creates a new Resolver using DNS-over-UDP.

Arguments:

- logger is the logger to use

- dialer is the dialer to create and connect UDP conns

- address is the server address (e.g., 1.1.1.1:53)

func WrapResolver 

func WrapResolver(logger Logger, resolver Resolver) Resolver

WrapResolver creates a new resolver that wraps an existing resolver to add these properties:

1. handles IDNA;

2. performs logging;

3. short-circuits IP addresses like getaddrinfo does (i.e., resolving "1.1.1.1" yields []string{"1.1.1.1"};

4. wraps errors;

5. enforces reasonable timeouts ( see https://github.com/ooni/probe/issues/1726).

This is a low-level factory. Use only if out of alternatives.

type ResolverIDNA deprecated 

type ResolverIDNA = resolverIDNA

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type ResolverLegacy deprecated 

type ResolverLegacy interface {
	// LookupHost behaves like net.Resolver.LookupHost.
	LookupHost(ctx context.Context, hostname string) (addrs []string, err error)
}

ResolverLegacy performs domain name resolutions.

Deprecated: new code should use Resolver.

Existing code in ooni/probe-cli is still using this definition.

type ResolverLegacyAdapter 

type ResolverLegacyAdapter struct {
	ResolverLegacy
}

ResolverLegacyAdapter makes a ResolverLegacy behave like a Resolver.

func (*ResolverLegacyAdapter) Address 

func (r *ResolverLegacyAdapter) Address() string

Address implements Resolver.Address.

func (*ResolverLegacyAdapter) CloseIdleConnections 

func (r *ResolverLegacyAdapter) CloseIdleConnections()

CloseIdleConnections implements Resolver.CloseIdleConnections.

func (*ResolverLegacyAdapter) LookupHTTPS 

func (r *ResolverLegacyAdapter) LookupHTTPS(
	ctx context.Context, domain string) (*HTTPSSvc, error)

LookupHTTPS always returns ErrDNSNoTransport.

func (*ResolverLegacyAdapter) Network 

func (r *ResolverLegacyAdapter) Network() string

Network implements Resolver.Network.

type ResolverLogger deprecated 

type ResolverLogger = resolverLogger

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type ResolverSystem deprecated 

type ResolverSystem = resolverSystem

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type SerialResolver 

type SerialResolver struct {
	// Encoder is the MANDATORY encoder to use.
	Encoder DNSEncoder

	// Decoder is the MANDATORY decoder to use.
	Decoder DNSDecoder

	// NumTimeouts is MANDATORY and counts the number of timeouts.
	NumTimeouts *atomicx.Int64

	// Txp is the underlying DNS transport.
	Txp DNSTransport
}

SerialResolver uses a transport and sends performs a LookupHost operation in a serial fashion (query for A first, wait for response, then query for AAAA, and wait for response), hence its name.

You should probably use NewSerialResolver to create a new instance.

func NewSerialResolver 

func NewSerialResolver(t DNSTransport) *SerialResolver

NewSerialResolver creates a new SerialResolver instance.

func (*SerialResolver) Address 

func (r *SerialResolver) Address() string

Address returns the "address" of the underlying transport.

func (*SerialResolver) CloseIdleConnections 

func (r *SerialResolver) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*SerialResolver) LookupHTTPS 

func (r *SerialResolver) LookupHTTPS(
	ctx context.Context, hostname string) (*HTTPSSvc, error)

LookupHTTPS implements Resolver.LookupHTTPS.

func (*SerialResolver) LookupHost 

func (r *SerialResolver) LookupHost(ctx context.Context, hostname string) ([]string, error)

LookupHost performs an A lookup followed by an AAAA lookup for hostname.

func (*SerialResolver) Network 

func (r *SerialResolver) Network() string

Network returns the "network" of the underlying transport.

func (*SerialResolver) Transport 

func (r *SerialResolver) Transport() DNSTransport

Transport returns the transport being used.

type TLSConn 

type TLSConn = oohttp.TLSConn

TLSConn is the type of connection that oohttp expects from any library that implements TLS functionality. By using this kind of TLSConn we're able to use both the standard library and gitlab.com/yawning/utls.git to perform TLS operations. Note that the stdlib's tls.Conn implements this interface.

type TLSDialer 

type TLSDialer interface {
	// CloseIdleConnections closes idle connections, if any.
	CloseIdleConnections()

	// DialTLSContext dials a TLS connection. This method will always
	// return to you a TLSConn, so you can always safely cast to TLSConn.
	DialTLSContext(ctx context.Context, network, address string) (net.Conn, error)
}

TLSDialer is a Dialer dialing TLS connections.

func NewNullTLSDialer 

func NewNullTLSDialer() TLSDialer

NewNullTLSDialer returns a TLS dialer that always fails with ErrNoTLSDialer.

func NewSingleUseTLSDialer 

func NewSingleUseTLSDialer(conn TLSConn) TLSDialer

NewSingleUseTLSDialer is like NewSingleUseDialer but takes in input a TLSConn rather than a net.Conn.

func NewTLSDialer 

func NewTLSDialer(dialer Dialer, handshaker TLSHandshaker) TLSDialer

NewTLSDialer creates a new TLS dialer using the given dialer and handshaker.

func NewTLSDialerWithConfig 

func NewTLSDialerWithConfig(d Dialer, h TLSHandshaker, c *tls.Config) TLSDialer

NewTLSDialerWithConfig is like NewTLSDialer with an optional config.

type TLSDialerLegacy deprecated 

type TLSDialerLegacy = tlsDialer

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type TLSHandshaker 

type TLSHandshaker interface {
	// Handshake creates a new TLS connection from the given connection and
	// the given config. This function DOES NOT take ownership of the connection
	// and it's your responsibility to close it on failure.
	//
	// Recommended tlsConfig setup:
	//
	// - set ServerName to be the SNI;
	//
	// - set RootCAs to NewDefaultCertPool();
	//
	// - set NextProtos to []string{"h2", "http/1.1"} for HTTPS
	// and []string{"dot"} for DNS-over-TLS.
	//
	// QUIRK: The returned connection will always implement the TLSConn interface
	// exposed by this package. A future version of this interface will instead
	// return directly a TLSConn to avoid unconditional castings.
	Handshake(ctx context.Context, conn net.Conn, tlsConfig *tls.Config) (
		net.Conn, tls.ConnectionState, error)
}

TLSHandshaker is the generic TLS handshaker.

func NewTLSHandshakerStdlib 

func NewTLSHandshakerStdlib(logger Logger) TLSHandshaker

NewTLSHandshakerStdlib creates a new TLS handshaker using the go standard library to manage TLS.

The handshaker guarantees:

1. logging

2. error wrapping

func NewTLSHandshakerUTLS 

func NewTLSHandshakerUTLS(logger Logger, id *utls.ClientHelloID) TLSHandshaker

NewTLSHandshakerUTLS creates a new TLS handshaker using gitlab.com/yawning/utls for TLS.

The id is the address of something like utls.HelloFirefox_55.

The handshaker guarantees:

1. logging

2. error wrapping

Passing a nil `id` will make this function panic.

type TLSHandshakerConfigurable deprecated 

type TLSHandshakerConfigurable = tlsHandshakerConfigurable

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type TLSHandshakerLogger deprecated 

type TLSHandshakerLogger = tlsHandshakerLogger

These types export internal names to legacy ooni/probe-cli code.

Deprecated: do not use these names in new code.

type TProxable 

type TProxable interface {
	// ListenUDP creates a new quicx.UDPLikeConn conn.
	ListenUDP(network string, laddr *net.UDPAddr) (quicx.UDPLikeConn, error)

	// LookupHost lookups a domain using the stdlib resolver.
	LookupHost(ctx context.Context, domain string) ([]string, error)

	// NewTProxyDialer returns a new TProxyDialer.
	NewTProxyDialer(timeout time.Duration) TProxyDialer
}

TProxable is the fundamental type used by the netxlite package to perform low-level network operations for which, by default, we use the stdlib.

The t stands for transparent. By using this type as the fundamental type, we can transparently intercept connections and implement censorship policies. The implementation of this functionality is not part of netxlite: here we only have the basic mechanism to make this possible. 

var TProxy TProxable = &TProxyStdlib{}

TProxy is the fundamental variable controlling how netxlite creates net.Conn and quicx.UDPLikeConn, as well as how it uses the stdlib resolver. By modifying this variable, you can effectively transparently proxy netxlite (and hence OONI) activities to other services. This is quite convenient when performing quality assurance tests.

type TProxyDialer 

type TProxyDialer interface {
	// DialContext behaves like net.Dialer.DialContext.
	DialContext(ctx context.Context, network, address string) (net.Conn, error)
}

TProxyDialer is the dialer type returned by TProxable.NewDialer.

type TProxyStdlib added in v3.14.0 

type TProxyStdlib struct{}

TProxyStdlib is the default TProxable implementation that uses the stdlib in the most obvious way for every functionality.

func (*TProxyStdlib) ListenUDP added in v3.14.0 

func (*TProxyStdlib) ListenUDP(network string, laddr *net.UDPAddr) (quicx.UDPLikeConn, error)

ListenUDP calls net.ListenUDP.

func (*TProxyStdlib) LookupHost added in v3.14.0 

func (*TProxyStdlib) LookupHost(ctx context.Context, domain string) ([]string, error)

LookupHost calls net.DefaultResolver.LookupHost.

func (*TProxyStdlib) NewTProxyDialer 

func (*TProxyStdlib) NewTProxyDialer(timeout time.Duration) TProxyDialer

NewTProxyDialer returns a &net.Dialer{Timeout: timeout} instance.

type UDPLikeConn 

type UDPLikeConn = quicx.UDPLikeConn

UDPLikeConn is the kind of UDP socket used by QUIC.

Source Files

View all Source files

Directories

Path Synopsis
Package dnsx contains DNS extension types.
 Package dnsx contains DNS extension types.
Package filtering allows to implement self-censorship.
 Package filtering allows to implement self-censorship.
internal
gencertifi
generrno
Package mocks contains mocks for netx types.
 Package mocks contains mocks for netx types.
Package quicx contains lucas-clemente/quic-go extensions.
 Package quicx contains lucas-clemente/quic-go extensions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.