webconnectivitylte

package
v3.22.0-alpha Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 14, 2024 License: GPL-3.0 Imports: 28 Imported by: 0

README

webconnectivity

This directory contains a new implementation of Web Connectivity.

As of 2022-09-15, this code is experimental and is not selected by default when you run the websites group. You can select this implementation with miniooni using miniooni web_connectivity@v0.5 from the command line.

Issue #2237 explains the rationale behind writing this new implementation.

Implementation overview

graph TD;
    measurer.go --> dnsresolvers.go;
	dnsresolvers.go --> control.go;
	dnsresolvers.go --> cleartext.go;
	dnsresolvers.go --> secure.go;
	control.go --> cleartext.go;
	control.go --> secure.go;
	cleartext.go --> dnsresolvers.go;
	secure.go --> dnsresolvers.go;
	measurer.go --> analysiscore.go;

Figure I. Relationship between files in this implementation

The experiment measures a single URL at a time. The OONI Engine invokes the Run method inside the measurer.go file.

The first task that Run starts deals with DNS and lives in the dnsresolvers.go file. This task is responsible for resolving the domain inside the URL into 0..N IP addresses. The domain resolution includes the system resolver and a DNS-over-UDP resolver. The implementaion may do more than that, but this is the bare minimum we're feeling like documenting right now. (We need to experiment a bit more to understand what else we can do there, hence the code is probably doing more than just that.)

Once we know the 0..N IP addresses for the domain we do the following:

  1. start a background task to communicate with the Web Connectivity test helper, using code inside control.go;

  2. start an endpoint measurement task for each IP adddress (which of course only happens when we know at least one addr).

Regarding starting endpoint measurements, we follow this policy:

  1. if the original URL is http://... then, for each address, we start an HTTP task using port 80 and an HTTPS task using 443.

  2. if it's https://..., then we only start HTTPS tasks.

HTTP tasks are implemented by cleartextflow.go while the HTTPS tasks live in secureflow.go.

An HTTP task does the following:

  1. TCP connect;

  2. additionally, the first task to establish a connection also performs a GET request to fetch a webpage (we cannot GET for all connections, because that would be websteps and would require a different data format).

An HTTPS task does the following:

  1. TCP connect;

  2. TLS handshake;

  3. additionally, the first task to handshake also performs a GET request to fetch a webpage iff the input URL was https://... (we cannot GET for all connections, because that would be websteps and would require a different data format).

If fetching the webpage returns a redirect, we start a new DNS task passing it the redirect URL as the new URL to measure, thus transferring the control again to dnsresolvers.go. We do not call the test helper again when this happens, though. The Web Connectivity test helper already follows the whole redirect chain, so we would need to change the test helper to get information on each flow. If we fetched more than one webpage per redirect chain, this experiment would be websteps.

Additionally, when the test helper terminates, control.go may run HTTP and/or HTTPS tasks (when applicable) for new IP addresses discovered using the test helper that were previously unknown to the probe, thus collecting extra information.

When several connections are racing to fetch a webpage, we need specific logic to choose which of them to give the permission to actually fetch the webpage. This logic lives inside the priority.go file.

When all tasks complete, either because we reach a final state or because we have followed too many redirects, we use code inside analysiscore.go to compute the top-level test keys. We emit the blocking and accessible keys we emitted before as well as new keys, prefixed by x_ to indicate that they're experimental.

Limitations and next steps

Further changes are probably possible. Departing too radically from the Web Connectivity model, though, will lead us to have a websteps implementation (but then the data model would most likely be different).

Documentation

Overview

Package webconnectivitylte implements the web_connectivity experiment.

Spec: https://github.com/ooni/spec/blob/master/nettests/ts-017-web-connectivity.md.

This implementation, in particular, contains extensions over the original model, which we document at https://github.com/ooni/probe/issues/2237.

Index

Constants

View Source 
const (
	// AnalysisBlockingFlagDNSBlocking indicates there's blocking at the DNS level.
	AnalysisBlockingFlagDNSBlocking = 1 << iota

	// AnalysisBlockingFlagTCPIPBlocking indicates there's blocking at the TCP/IP level.
	AnalysisBlockingFlagTCPIPBlocking

	// AnalysisBlockingFlagTLSBlocking indicates there were TLS issues.
	AnalysisBlockingFlagTLSBlocking

	// AnalysisBlockingFlagHTTPBlocking indicates there was an HTTP failure.
	AnalysisBlockingFlagHTTPBlocking

	// AnalysisBlockingFlagHTTPDiff indicates there's an HTTP diff.
	AnalysisBlockingFlagHTTPDiff

	// AnalysisBlockingFlagSuccess indicates we did not detect any blocking.
	AnalysisBlockingFlagSuccess
)

These flags determine the context of TestKeys.Blocking. However, while .Blocking is an enumeration, these flags allow to describe multiple blocking methods.

View Source 
const (
	// AnalysisFlagNullNullExpectedDNSLookupFailure indicates some of the DNS lookup
	// attempts failed both in the probe and in the test helper.
	AnalysisFlagNullNullExpectedDNSLookupFailure = 1 << iota

	// AnalysisFlagNullNullExpectedTCPConnectFailure indicates that some of the connect
	// attempts failed both in the probe and in the test helper.
	AnalysisFlagNullNullExpectedTCPConnectFailure

	// AnalysisFlagNullNullExpectedTLSHandshakeFailure indicates that we have seen some TLS
	// handshakes failing consistently for both the probe and the TH.
	AnalysisFlagNullNullExpectedTLSHandshakeFailure

	// AnalysisFlagNullNullSuccessfulHTTPS indicates that we had no TH data
	// but all the HTTP requests used always HTTPS and never failed.
	AnalysisFlagNullNullSuccessfulHTTPS

	// AnalysisFlagNullNullUnexpectedDNSLookupSuccess indicates the case
	// where the TH resolved no addresses while the probe did.
	AnalysisFlagNullNullUnexpectedDNSLookupSuccess
)
View Source 
const (
	// AnalysisFlagDNSBogon indicates we got any bogon reply
	AnalysisFlagDNSBogon = 1 << iota

	// AnalysisDNSFlagUnexpectedFailure indicates the TH could
	// resolve a domain while the probe couldn't
	AnalysisDNSFlagUnexpectedFailure

	// AnalysisDNSFlagUnexpectedAddrs indicates the TH resolved
	// different addresses from the probe
	AnalysisDNSFlagUnexpectedAddrs
)
View Source 
const (
	// DNSAddrFlagSystemResolver means we discovered this addr using the system resolver.
	DNSAddrFlagSystemResolver = 1 << iota

	// DNSAddrFlagUDP means we discovered this addr using the UDP resolver.
	DNSAddrFlagUDP

	// DNSAddrFlagHTTPS means we discovered this addr using the DNS-over-HTTPS resolver.
	DNSAddrFlagHTTPS
)

Variables

View Source 
var DNSWhoamiSingleton = webconnectivityalgo.NewDNSWhoamiService(model.DiscardLogger)

DNSWhoamiSingleton is the DNSWhoamiService singleton.

View Source 
var ErrTooManyRedirects = errors.New("stopped after too many redirects")

ErrTooManyRedirects indicates we have seen too many HTTP redirects.

View Source 
var MaybeDelayCleartextFlows = func(index int) {

}

MaybeDelayCleartextFlows is an OPTIONAL hook that possibly delays cleartext flows.

View Source 
var MaybeDelaySecureFlows = func(index int) {

}

MaybeDelaySecureFlows is an OPTIONAL hook that possibly delays secure flows.

View Source 
var MaybeSortAddresses = func(entries []DNSEntry) {

}

MaybeSortAddresses is an OPTIONAL hook that possibly sorts the resolved addresses.

Functions

func NewExperimentMeasurer 

func NewExperimentMeasurer(config *Config) model.ExperimentMeasurer

NewExperimentMeasurer creates a new model.ExperimentMeasurer.

Types

type CleartextFlow 

type CleartextFlow struct {
	// Address is the MANDATORY address to connect to.
	Address string

	// Classic is true if this address was discovered using getaddrinfo.
	Classic bool

	// DNSCache is the MANDATORY DNS cache.
	DNSCache *DNSCache

	// DNSOverHTTPSURLProvider is the MANDATORY provider of DNS-over-HTTPS
	// URLs that arranges for periodic measurements.
	DNSOverHTTPSURLProvider *webconnectivityalgo.OpportunisticDNSOverHTTPSURLProvider

	// Depth is the OPTIONAL current redirect depth.
	Depth int64

	// IDGenerator is the MANDATORY atomic int64 to generate task IDs.
	IDGenerator *IDGenerator

	// Logger is the MANDATORY logger to use.
	Logger model.Logger

	// NumRedirects it the MANDATORY counter of the number of redirects.
	NumRedirects *NumRedirects

	// TestKeys is MANDATORY and contains the TestKeys.
	TestKeys *TestKeys

	// ZeroTime is the MANDATORY measurement's zero time.
	ZeroTime time.Time

	// WaitGroup is the MANDATORY wait group this task belongs to.
	WaitGroup *sync.WaitGroup

	// CookieJar contains the OPTIONAL cookie jar, used for redirects.
	CookieJar http.CookieJar

	// FollowRedirects is OPTIONAL and instructs this flow
	// to follow HTTP redirects (if any).
	FollowRedirects bool

	// HostHeader is the OPTIONAL host header to use.
	HostHeader string

	// PrioSelector is the OPTIONAL priority selector to use to determine
	// whether this flow is allowed to fetch the webpage.
	PrioSelector *prioritySelector

	// Referer contains the OPTIONAL referer, used for redirects.
	Referer string

	// UDPAddress is the OPTIONAL address of the UDP resolver to use. If this
	// field is not set we use a default one (e.g., `8.8.8.8:53`).
	UDPAddress string

	// URLPath is the OPTIONAL URL path.
	URLPath string

	// URLRawQuery is the OPTIONAL URL raw query.
	URLRawQuery string
}

Measures HTTP endpoints.

The zero value of this structure IS NOT valid and you MUST initialize all the fields marked as MANDATORY before using this structure.

func (*CleartextFlow) Run 

func (t *CleartextFlow) Run(parentCtx context.Context, index int64) error

Run runs this task in the current goroutine.

func (*CleartextFlow) Start 

func (t *CleartextFlow) Start(ctx context.Context)

Start starts this task in a background goroutine.

type Config 

type Config struct {
	DNSOverUDPResolver string
}

Config contains webconnectivity experiment configuration.

type ConnPriorityLogEntry 

type ConnPriorityLogEntry struct {
	// Msg is the specific log entry
	Msg string `json:"msg"`

	// T is when this entry was generated
	T float64 `json:"t"`
}

ConnPriorityLogEntry is an entry in the TestKeys.ConnPriorityLog slice.

type Control 

type Control struct {
	// Addresses contains the MANDATORY addresses we've looked up.
	Addresses []string

	// ExtraMeasurementsStarter is MANDATORY and allows this struct to
	// start additional measurements using new TH-discovered addrs.
	ExtraMeasurementsStarter EndpointMeasurementsStarter

	// Logger is the MANDATORY logger to use.
	Logger model.Logger

	// PrioSelector is the OPTIONAL priority selector to use to determine
	// whether we will be allowed to fetch the webpage.
	PrioSelector *prioritySelector

	// TestKeys is MANDATORY and contains the TestKeys.
	TestKeys *TestKeys

	// Session is the MANDATORY session to use.
	Session model.ExperimentSession

	// TestHelpers is the MANDATORY list of test helpers.
	TestHelpers []model.OOAPIService

	// URL is the MANDATORY URL we are measuring.
	URL *url.URL

	// WaitGroup is the MANDATORY wait group this task belongs to.
	WaitGroup *sync.WaitGroup
}

Control issues a Control request and saves the results inside of the experiment's TestKeys.

The zero value of this structure IS NOT valid and you MUST initialize all the fields marked as MANDATORY before using this structure.

func (*Control) Run 

func (c *Control) Run(parentCtx context.Context)

Run runs this task until completion.

func (*Control) Start 

func (c *Control) Start(ctx context.Context)

Start starts this task in a background goroutine.

type DNSCache 

type DNSCache struct {
	// contains filtered or unexported fields
}

DNSCache wraps a model.Resolver to provide DNS caching.

The zero value is invalid; please, use NewDNSCache to construct.

func NewDNSCache 

func NewDNSCache() *DNSCache

NewDNSCache creates a new DNSCache instance.

func (*DNSCache) Get 

func (c *DNSCache) Get(domain string) ([]DNSEntry, bool)

Get gets values from the cache

func (*DNSCache) Set 

func (c *DNSCache) Set(domain string, values []DNSEntry)

Set inserts into the cache

type DNSEntry 

type DNSEntry struct {
	// Addr is the cached address
	Addr string

	// Flags contains flags
	Flags int64
}

DNSEntry is an entry in the DNS cache.

type DNSResolvers 

type DNSResolvers struct {
	// DNSCache is the MANDATORY DNS cache.
	DNSCache *DNSCache

	// DNSOverHTTPSURLProvider is the MANDATORY provider of DNS-over-HTTPS
	// URLs that arranges for periodic measurements.
	DNSOverHTTPSURLProvider *webconnectivityalgo.OpportunisticDNSOverHTTPSURLProvider

	// Domain is the MANDATORY domain to resolve.
	Domain string

	// Depth is the OPTIONAL current redirect depth.
	Depth int64

	// IDGenerator is the MANDATORY atomic int64 to generate task IDs.
	IDGenerator *IDGenerator

	// Logger is the MANDATORY logger to use.
	Logger model.Logger

	// NumRedirects it the MANDATORY counter of the number of redirects.
	NumRedirects *NumRedirects

	// TestKeys is MANDATORY and contains the TestKeys.
	TestKeys *TestKeys

	// URL is the MANDATORY URL we're measuring.
	URL *url.URL

	// ZeroTime is the MANDATORY zero time of the measurement.
	ZeroTime time.Time

	// WaitGroup is the MANDATORY wait group this task belongs to.
	WaitGroup *sync.WaitGroup

	// CookieJar contains the OPTIONAL cookie jar, used for redirects.
	CookieJar http.CookieJar

	// Referer contains the OPTIONAL referer, used for redirects.
	Referer string

	// Session is the OPTIONAL session. If the session is set, we will use
	// it to start the task that issues the control request. This request must
	// only be sent during the first iteration. It would be pointless to
	// issue such a request for subsequent redirects, because the TH will
	// always follow the redirect chain caused by the provided URL.
	Session model.ExperimentSession

	// TestHelpers is the OPTIONAL list of test helpers. If the list is
	// empty, we are not going to try to contact any test helper.
	TestHelpers []model.OOAPIService

	// UDPAddress is the OPTIONAL address of the UDP resolver to use. If this
	// field is not set we use a default one (e.g., `8.8.8.8:53`).
	UDPAddress string
}

Resolves the URL's domain using several resolvers.

The zero value of this structure IS NOT valid and you MUST initialize all the fields marked as MANDATORY before using this structure.

func (*DNSResolvers) Run 

func (t *DNSResolvers) Run(parentCtx context.Context)

Run runs this task in the current goroutine.

func (*DNSResolvers) Start 

func (t *DNSResolvers) Start(ctx context.Context)

Start starts this task in a background goroutine.

type DNSWhoamiInfo 

type DNSWhoamiInfo struct {
	// SystemV4 contains results related to the system resolver using IPv4.
	SystemV4 []webconnectivityalgo.DNSWhoamiInfoEntry `json:"system_v4"`

	// UDPv4 contains results related to an UDP resolver using IPv4.
	UDPv4 map[string][]webconnectivityalgo.DNSWhoamiInfoEntry `json:"udp_v4"`
}

DNSWhoamiInfo contains information about a DNS whoami lookup.

type EndpointMeasurementsStarter 

type EndpointMeasurementsStarter interface {
	// contains filtered or unexported methods
}

EndpointMeasurementsStarter is used by Control to start extra measurements using new IP addrs discovered by the TH.

type IDGenerator added in v3.21.0 

type IDGenerator struct {
	// contains filtered or unexported fields
}

IDGenerator helps with generating IDs that neatly fall into namespaces.

The zero value is invalid, please use NewIDGenerator.

func NewIDGenerator added in v3.21.0 

func NewIDGenerator() *IDGenerator

NewIDGenerator creates a new *IDGenerator instance.

func (*IDGenerator) NewIDForDNSOverHTTPS added in v3.21.0 

func (idgen *IDGenerator) NewIDForDNSOverHTTPS() int64

NewIDForDNSOverHTTPS returns a new ID for a DNS-over-HTTPS lookup.

func (*IDGenerator) NewIDForDNSOverUDP added in v3.21.0 

func (idgen *IDGenerator) NewIDForDNSOverUDP() int64

NewIDForDNSOverUDP returns a new ID for a DNS-over-UDP lookup.

func (*IDGenerator) NewIDForEndpointCleartext added in v3.21.0 

func (idgen *IDGenerator) NewIDForEndpointCleartext() int64

NewIDForEndpointCleartext returns a new ID for a cleartext endpoint operation.

func (*IDGenerator) NewIDForEndpointSecure added in v3.21.0 

func (idgen *IDGenerator) NewIDForEndpointSecure() int64

NewIDForEndpointSecure returns a new ID for a secure endpoint operation.

func (*IDGenerator) NewIDForGetaddrinfo added in v3.21.0 

func (idgen *IDGenerator) NewIDForGetaddrinfo() int64

NewIDForGetaddrinfo returns a new ID for a getaddrinfo lookup.

type Measurer 

type Measurer struct {
	// Contains the experiment's config.
	Config *Config

	// DNSOverHTTPSURLProvider is the MANDATORY provider of DNS-over-HTTPS
	// URLs that arranges for periodic measurements.
	DNSOverHTTPSURLProvider *webconnectivityalgo.OpportunisticDNSOverHTTPSURLProvider
}

Measurer for the web_connectivity experiment.

func (*Measurer) ExperimentName 

func (m *Measurer) ExperimentName() string

ExperimentName implements model.ExperimentMeasurer.

func (*Measurer) ExperimentVersion 

func (m *Measurer) ExperimentVersion() string

ExperimentVersion implements model.ExperimentMeasurer.

func (*Measurer) Run 

func (m *Measurer) Run(ctx context.Context, args *model.ExperimentArgs) error

Run implements model.ExperimentMeasurer.

type NumRedirects 

type NumRedirects struct {
	// contains filtered or unexported fields
}

NumRedirects counts the number of redirects left.

func NewNumRedirects 

func NewNumRedirects(n int64) *NumRedirects

NewNumRedirects creates a new NumRedirects instance.

func (*NumRedirects) CanFollowOneMoreRedirect 

func (nr *NumRedirects) CanFollowOneMoreRedirect() bool

CanFollowOneMoreRedirect returns true if we are allowed to follow one more redirect.

type SecureFlow 

type SecureFlow struct {
	// Address is the MANDATORY address to connect to.
	Address string

	// Classic is true if this address was discovered using getaddrinfo.
	Classic bool

	// DNSCache is the MANDATORY DNS cache.
	DNSCache *DNSCache

	// DNSOverHTTPSURLProvider is the MANDATORY provider of DNS-over-HTTPS
	// URLs that arranges for periodic measurements.
	DNSOverHTTPSURLProvider *webconnectivityalgo.OpportunisticDNSOverHTTPSURLProvider

	// Depth is the OPTIONAL current redirect depth.
	Depth int64

	// IDGenerator is the MANDATORY atomic int64 to generate task IDs.
	IDGenerator *IDGenerator

	// Logger is the MANDATORY logger to use.
	Logger model.Logger

	// NumRedirects it the MANDATORY counter of the number of redirects.
	NumRedirects *NumRedirects

	// TestKeys is MANDATORY and contains the TestKeys.
	TestKeys *TestKeys

	// ZeroTime is the MANDATORY measurement's zero time.
	ZeroTime time.Time

	// WaitGroup is the MANDATORY wait group this task belongs to.
	WaitGroup *sync.WaitGroup

	// ALPN is the OPTIONAL ALPN to use.
	ALPN []string

	// CookieJar contains the OPTIONAL cookie jar, used for redirects.
	CookieJar http.CookieJar

	// FollowRedirects is OPTIONAL and instructs this flow
	// to follow HTTP redirects (if any).
	FollowRedirects bool

	// HostHeader is the OPTIONAL host header to use.
	HostHeader string

	// PrioSelector is the OPTIONAL priority selector to use to determine
	// whether this flow is allowed to fetch the webpage.
	PrioSelector *prioritySelector

	// Referer contains the OPTIONAL referer, used for redirects.
	Referer string

	// SNI is the OPTIONAL SNI to use.
	SNI string

	// UDPAddress is the OPTIONAL address of the UDP resolver to use. If this
	// field is not set we use a default one (e.g., `8.8.8.8:53`).
	UDPAddress string

	// URLPath is the OPTIONAL URL path.
	URLPath string

	// URLRawQuery is the OPTIONAL URL raw query.
	URLRawQuery string
}

Measures HTTPS endpoints.

The zero value of this structure IS NOT valid and you MUST initialize all the fields marked as MANDATORY before using this structure.

func (*SecureFlow) Run 

func (t *SecureFlow) Run(parentCtx context.Context, index int64) error

Run runs this task in the current goroutine.

func (*SecureFlow) Start 

func (t *SecureFlow) Start(ctx context.Context)

Start starts this task in a background goroutine.

type SummaryKeys 

type SummaryKeys struct {
	Accessible bool   `json:"accessible"`
	Blocking   string `json:"blocking"`
	IsAnomaly  bool   `json:"-"`
}

SummaryKeys contains summary keys for this experiment.

func (*SummaryKeys) Anomaly added in v3.21.0 

func (sk *SummaryKeys) Anomaly() bool

Anomaly implements model.MeasurementSummaryKeys.

type TestKeys 

type TestKeys struct {
	// Agent is the HTTP agent we use.
	Agent string `json:"agent"`

	// ClientResolver is the IPv4 of the resolver used by getaddrinfo.
	ClientResolver string `json:"client_resolver"`

	// Retries is a legacy field always set to nil by web_connectivity@v0.4.x
	Retries *int64 `json:"retries"`

	// SOCKSProxy is a legacy field always set to nil by web_connectivity@v0.4.x
	SOCKSProxy *string `json:"socksproxy"`

	// NetworkEvents contains network events.
	NetworkEvents []*model.ArchivalNetworkEvent `json:"network_events"`

	// DNSWhoami contains results of using the DNS whoami functionality for the
	// possibly cleartext resolvers that we're using.
	DNSWoami *DNSWhoamiInfo `json:"x_dns_whoami"`

	// DoH contains ancillary observations collected by DoH resolvers.
	DoH *TestKeysDoH `json:"x_doh"`

	// Do53 contains ancillary observations collected by Do53 resolvers.
	Do53 *TestKeysDo53 `json:"x_do53"`

	// DNSDuplicateResponses contains late/duplicate responses we didn't expect to receive from
	// a resolver (which may raise eyebrows if they're different).
	DNSDuplicateResponses []*model.ArchivalDNSLookupResult `json:"x_dns_duplicate_responses"`

	// Queries contains DNS queries.
	Queries []*model.ArchivalDNSLookupResult `json:"queries"`

	// Requests contains HTTP results.
	Requests []*model.ArchivalHTTPRequestResult `json:"requests"`

	// TCPConnect contains TCP connect results.
	TCPConnect []*model.ArchivalTCPConnectResult `json:"tcp_connect"`

	// TLSHandshakes contains TLS handshakes results.
	TLSHandshakes []*model.ArchivalTLSOrQUICHandshakeResult `json:"tls_handshakes"`

	// ControlRequest is the control request we sent.
	ControlRequest *webconnectivity.ControlRequest `json:"x_control_request"`

	// Control contains the TH's response.
	Control *webconnectivity.ControlResponse `json:"control"`

	// ConnPriorityLog explains why Web Connectivity chose to use a given
	// ready-to-use HTTP(S) connection among many.
	ConnPriorityLog []*ConnPriorityLogEntry `json:"x_conn_priority_log"`

	// ControlFailure contains the failure of the control experiment.
	ControlFailure *string `json:"control_failure"`

	// DNSFlags describes specific DNS anomalies we observed.
	DNSFlags int64 `json:"x_dns_flags"`

	// DNSExperimentFailure indicates whether there was a failure in any
	// of the DNS experiments we performed.
	DNSExperimentFailure *string `json:"dns_experiment_failure"`

	// DNSConsistency indicates whether there is consistency between
	// the TH's DNS results and the probe's DNS results.
	DNSConsistency optional.Value[string] `json:"dns_consistency"`

	// HTTPExperimentFailure indicates whether there was a failure in
	// the final HTTP request that we recorded.
	HTTPExperimentFailure optional.Value[string] `json:"http_experiment_failure"`

	// BlockingFlags explains why we think that the website is blocked.
	BlockingFlags int64 `json:"x_blocking_flags"`

	// NullNullFlags describes what the algorithm to avoid emitting
	// blocking = null, accessible = null measurements did
	NullNullFlags int64 `json:"x_null_null_flags"`

	// BodyProportion is the value used to compute BodyLength.
	BodyProportion float64 `json:"body_proportion"`

	// BodyLength match tells us whether the body length matches.
	BodyLengthMatch optional.Value[bool] `json:"body_length_match"`

	// HeadersMatch tells us whether the headers match.
	HeadersMatch optional.Value[bool] `json:"headers_match"`

	// StatusCodeMatch tells us whether the status code matches.
	StatusCodeMatch optional.Value[bool] `json:"status_code_match"`

	// TitleMatch tells us whether the title matches.
	TitleMatch optional.Value[bool] `json:"title_match"`

	// Blocking indicates the reason for blocking. This is notoriously a bad
	// type because it can be one of the following values:
	//
	// - "tcp_ip"
	// - "dns"
	// - "http-diff"
	// - "http-failure"
	// - false
	// - null
	//
	// In addition to having a ~bad type, this field has the issue that it
	// reduces the reason for blocking to an enum, whereas it's a set of flags,
	// hence we introduced the x_blocking_flags field.
	Blocking any `json:"blocking"`

	// Accessible indicates whether the resource is accessible. Possible
	// values for this field are: nil, true, and false.
	Accessible optional.Value[bool] `json:"accessible"`
	// contains filtered or unexported fields
}

TestKeys contains the results produced by web_connectivity.

func NewTestKeys 

func NewTestKeys() *TestKeys

NewTestKeys creates a new instance of TestKeys.

func (*TestKeys) AppendConnPriorityLogEntry 

func (tk *TestKeys) AppendConnPriorityLogEntry(entry *ConnPriorityLogEntry)

AppendConnPriorityLogEntry appends an entry to ConnPriorityLog.

func (*TestKeys) AppendDNSLateReplies 

func (tk *TestKeys) AppendDNSLateReplies(v ...*model.ArchivalDNSLookupResult)

AppendDNSLateReplies appends to DNSLateReplies.

func (*TestKeys) AppendNetworkEvents 

func (tk *TestKeys) AppendNetworkEvents(v ...*model.ArchivalNetworkEvent)

AppendNetworkEvents appends to NetworkEvents.

func (*TestKeys) AppendQueries 

func (tk *TestKeys) AppendQueries(v ...*model.ArchivalDNSLookupResult)

AppendQueries appends to Queries.

func (*TestKeys) AppendTCPConnectResults 

func (tk *TestKeys) AppendTCPConnectResults(v ...*model.ArchivalTCPConnectResult)

AppendTCPConnectResults appends to TCPConnect.

func (*TestKeys) AppendTLSHandshakes 

func (tk *TestKeys) AppendTLSHandshakes(v ...*model.ArchivalTLSOrQUICHandshakeResult)

AppendTLSHandshakes appends to TLSHandshakes.

func (*TestKeys) Finalize 

func (tk *TestKeys) Finalize(logger model.Logger)

Finalize performs any delayed computation on the test keys. This function must be called from the measurer after all the tasks have completed.

func (*TestKeys) MeasurementSummaryKeys added in v3.21.0 

func (tk *TestKeys) MeasurementSummaryKeys() model.MeasurementSummaryKeys

MeasurementSummaryKeys implements model.MeasurementSummaryKeysProvider.

func (*TestKeys) PrependRequests added in v3.20.0 

func (tk *TestKeys) PrependRequests(v ...*model.ArchivalHTTPRequestResult)

PrependRequests prepends to Requests.

func (*TestKeys) SetControl 

func (tk *TestKeys) SetControl(v *webconnectivity.ControlResponse)

SetControl sets the value of Control.

func (*TestKeys) SetControlFailure 

func (tk *TestKeys) SetControlFailure(err error)

SetControlFailure sets the value of controlFailure.

func (*TestKeys) SetControlRequest 

func (tk *TestKeys) SetControlRequest(v *webconnectivity.ControlRequest)

SetControlRequest sets the value of controlRequest.

func (*TestKeys) SetFundamentalFailure 

func (tk *TestKeys) SetFundamentalFailure(err error)

SetFundamentalFailure sets the value of fundamentalFailure.

func (*TestKeys) WithDNSWhoami 

func (tk *TestKeys) WithDNSWhoami(fun func(*DNSWhoamiInfo))

WithDNSWhoami calls the given function with the mutex locked passing to it as argument the pointer to the DNSWhoami field.

func (*TestKeys) WithTestKeysDo53 

func (tk *TestKeys) WithTestKeysDo53(f func(*TestKeysDo53))

WithTestKeysDo53 calls the given function with the mutex locked passing to it as argument the pointer to the Do53 field.

func (*TestKeys) WithTestKeysDoH 

func (tk *TestKeys) WithTestKeysDoH(f func(*TestKeysDoH))

WithTestKeysDoH calls the given function with the mutex locked passing to it as argument the pointer to the DoH field.

type TestKeysDo53 

type TestKeysDo53 struct {
	// NetworkEvents contains network events.
	NetworkEvents []*model.ArchivalNetworkEvent `json:"network_events"`

	// Queries contains DNS queries.
	Queries []*model.ArchivalDNSLookupResult `json:"queries"`
}

TestKeysDo53 contains ancillary observations collected using Do53.

They are on a separate hierarchy to simplify processing.

type TestKeysDoH 

type TestKeysDoH struct {
	// NetworkEvents contains network events.
	NetworkEvents []*model.ArchivalNetworkEvent `json:"network_events"`

	// Queries contains DNS queries.
	Queries []*model.ArchivalDNSLookupResult `json:"queries"`

	// Requests contains HTTP results.
	Requests []*model.ArchivalHTTPRequestResult `json:"requests"`

	// TCPConnect contains TCP connect results.
	TCPConnect []*model.ArchivalTCPConnectResult `json:"tcp_connect"`

	// TLSHandshakes contains TLS handshakes results.
	TLSHandshakes []*model.ArchivalTLSOrQUICHandshakeResult `json:"tls_handshakes"`
}

TestKeysDoH contains ancillary observations collected using DoH (e.g., the DNS lookups, TCP connects, TLS handshakes caused by given DoH lookups).

They are on a separate hierarchy to simplify processing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.