model

README

Package github.com/ooni/probe-engine/model

Shared data structures and interfaces.

Documentation

Overview

Package model defines shared data structures and interfaces.

Index

• Constants
• Variables
• type CheckInConfig
• type CheckInConfigWebConnectivity
• type CheckInInfo
• type CheckInInfoWebConnectivity
• type ExperimentAsyncTestKeys
• type ExperimentCallbacks
• type ExperimentMeasurer
• type ExperimentMeasurerAsync
• type ExperimentSession
• type KeyValueStore
• type Logger
• type Measurement
  • func (m *Measurement) AddAnnotation(key, value string)
  • func (m *Measurement) AddAnnotations(input map[string]string)
  • func (m *Measurement) MaybeRewriteTestKeys(currentIP string, marshal func(interface{}) ([]byte, error)) error
  • func (m *Measurement) Scrub(probeIP string) (err error)
• type MeasurementTarget
  • func (t MeasurementTarget) MarshalJSON() ([]byte, error)
• type PrinterCallbacks
  • func NewPrinterCallbacks(logger Logger) PrinterCallbacks
  • func (d PrinterCallbacks) OnProgress(percentage float64, message string)
• type Service
• type TorTarget
• type URLInfo
• type URLListConfig

Constants

const (
	// DefaultProbeIP is the default probe IP.
	DefaultProbeIP = geolocate.DefaultProbeIP
)

const Scrubbed = `[scrubbed]`

Scrubbed is the string that replaces IP addresses.

Variables

var ErrInvalidProbeIP = errors.New("model: invalid probe IP")

ErrInvalidProbeIP indicates that we're dealing with a string that is not the valid serialization of an IP address.

Types

type CheckInConfig

type CheckInConfig struct {
	Charging        bool                         `json:"charging"`         // Charging indicate if the phone is actually charging
	OnWiFi          bool                         `json:"on_wifi"`          // OnWiFi indicate if the phone is actually connected to a WiFi network
	Platform        string                       `json:"platform"`         // Platform of the probe
	ProbeASN        string                       `json:"probe_asn"`        // ProbeASN is the probe country code
	ProbeCC         string                       `json:"probe_cc"`         // ProbeCC is the probe country code
	RunType         string                       `json:"run_type"`         // RunType
	SoftwareName    string                       `json:"software_name"`    // SoftwareName of the probe
	SoftwareVersion string                       `json:"software_version"` // SoftwareVersion of the probe
	WebConnectivity CheckInConfigWebConnectivity `json:"web_connectivity"` // WebConnectivity class contain an array of categories
}

CheckInConfig contains configuration for calling the checkin API.

type CheckInConfigWebConnectivity

type CheckInConfigWebConnectivity struct {
	CategoryCodes []string `json:"category_codes"` // CategoryCodes is an array of category codes
}

CheckInConfigWebConnectivity is the configuration for the WebConnectivity test

type CheckInInfo

type CheckInInfo struct {
	WebConnectivity *CheckInInfoWebConnectivity `json:"web_connectivity"`
}

CheckInInfo contains the return test objects from the checkin API

type CheckInInfoWebConnectivity

type CheckInInfoWebConnectivity struct {
	ReportID string    `json:"report_id"`
	URLs     []URLInfo `json:"urls"`
}

CheckInInfoWebConnectivity contains the array of URLs returned by the checkin API

type ExperimentAsyncTestKeys

type ExperimentAsyncTestKeys struct {
	// Extensions contains the extensions used by this experiment.
	Extensions map[string]int64

	// Input is the input this measurement refers to.
	Input MeasurementTarget

	// MeasurementRuntime is the total measurement runtime.
	MeasurementRuntime float64

	// TestKeys contains the actual test keys.
	TestKeys interface{}
}

ExperimentAsyncTestKeys is the type of test keys returned by an experiment when running in async fashion rather than in sync fashion.

type ExperimentCallbacks

type ExperimentCallbacks interface {
	// OnProgress provides information about an experiment progress.
	OnProgress(percentage float64, message string)
}

ExperimentCallbacks contains experiment event-handling callbacks

type ExperimentMeasurer

type ExperimentMeasurer interface {
	// ExperimentName returns the experiment name.
	ExperimentName() string

	// ExperimentVersion returns the experiment version.
	ExperimentVersion() string

	// Run runs the experiment with the specified context, session,
	// measurement, and experiment calbacks. This method should only
	// return an error in case the experiment could not run (e.g.,
	// a required input is missing). Otherwise, the code should just
	// set the relevant OONI error inside of the measurement and
	// return nil. This is important because the caller may not submit
	// the measurement if this method returns an error.
	Run(
		ctx context.Context, sess ExperimentSession,
		measurement *Measurement, callbacks ExperimentCallbacks,
	) error

	// GetSummaryKeys returns summary keys expected by ooni/probe-cli.
	GetSummaryKeys(*Measurement) (interface{}, error)
}

ExperimentMeasurer is the interface that allows to run a measurement for a specific experiment.

type ExperimentMeasurerAsync

type ExperimentMeasurerAsync interface {
	// RunAsync runs the experiment in async fashion.
	//
	// Arguments:
	//
	// - ctx is the context for deadline/timeout/cancellation
	//
	// - sess is the measurement session
	//
	// - input is the input URL to measure
	//
	// - callbacks contains the experiment callbacks
	//
	// Returns either a channel where TestKeys are posted or an error.
	//
	// An error indicates that specific preconditions for running the experiment
	// are not met (e.g., the input URL is invalid).
	//
	// On success, the experiment will post on the channel each new
	// measurement until it is done and closes the channel.
	RunAsync(ctx context.Context, sess ExperimentSession, input string,
		callbacks ExperimentCallbacks) (<-chan *ExperimentAsyncTestKeys, error)
}

ExperimentMeasurerAsync is a measurer that can run in async fashion.

Currently this functionality is optional, but we will likely migrate all experiments to use this functionality in 2022.

type ExperimentSession

type ExperimentSession interface {
	GetTestHelpersByName(name string) ([]Service, bool)
	DefaultHTTPClient() *http.Client
	FetchPsiphonConfig(ctx context.Context) ([]byte, error)
	FetchTorTargets(ctx context.Context, cc string) (map[string]TorTarget, error)
	FetchURLList(ctx context.Context, config URLListConfig) ([]URLInfo, error)
	Logger() Logger
	ProbeCC() string
	ResolverIP() string
	TempDir() string
	TorArgs() []string
	TorBinary() string
	UserAgent() string
}

ExperimentSession is the experiment's view of a session.

type KeyValueStore

type KeyValueStore interface {
	Get(key string) (value []byte, err error)
	Set(key string, value []byte) (err error)
}

KeyValueStore is a key-value store used by the session.

type Logger

type Logger interface {
	// Debug emits a debug message.
	Debug(msg string)

	// Debugf formats and emits a debug message.
	Debugf(format string, v ...interface{})

	// Info emits an informational message.
	Info(msg string)

	// Infof formats and emits an informational message.
	Infof(format string, v ...interface{})

	// Warn emits a warning message.
	Warn(msg string)

	// Warnf formats and emits a warning message.
	Warnf(format string, v ...interface{})
}

Logger defines the common interface that a logger should have. It is out of the box compatible with `log.Log` in `apex/log`.

var DiscardLogger Logger = logDiscarder{}

DiscardLogger is a logger that discards its input

type Measurement

type Measurement struct {
	// Annotations contains results annotations
	Annotations map[string]string `json:"annotations,omitempty"`

	// DataFormatVersion is the version of the data format
	DataFormatVersion string `json:"data_format_version"`

	// Extensions contains information about the extensions included
	// into the test_keys of this measurement.
	Extensions map[string]int64 `json:"extensions,omitempty"`

	// ID is the locally generated measurement ID
	ID string `json:"id,omitempty"`

	// Input is the measurement input
	Input MeasurementTarget `json:"input"`

	// InputHashes contains input hashes
	InputHashes []string `json:"input_hashes,omitempty"`

	// MeasurementStartTime is the time when the measurement started
	MeasurementStartTime string `json:"measurement_start_time"`

	// MeasurementStartTimeSaved is the moment in time when we
	// started the measurement. This is not included into the JSON
	// and is only used within probe-engine as a "zero" time.
	MeasurementStartTimeSaved time.Time `json:"-"`

	// Options contains command line options
	Options []string `json:"options,omitempty"`

	// ProbeASN contains the probe autonomous system number
	ProbeASN string `json:"probe_asn"`

	// ProbeCC contains the probe country code
	ProbeCC string `json:"probe_cc"`

	// ProbeCity contains the probe city
	ProbeCity string `json:"probe_city,omitempty"`

	// ProbeIP contains the probe IP
	ProbeIP string `json:"probe_ip,omitempty"`

	// ProbeNetworkName contains the probe network name
	ProbeNetworkName string `json:"probe_network_name"`

	// ReportID contains the report ID
	ReportID string `json:"report_id"`

	// ResolverASN is the ASN of the resolver
	ResolverASN string `json:"resolver_asn"`

	// ResolverIP is the resolver IP
	ResolverIP string `json:"resolver_ip"`

	// ResolverNetworkName is the network name of the resolver.
	ResolverNetworkName string `json:"resolver_network_name"`

	// SoftwareName contains the software name
	SoftwareName string `json:"software_name"`

	// SoftwareVersion contains the software version
	SoftwareVersion string `json:"software_version"`

	// TestHelpers contains the test helpers. It seems this structure is more
	// complex than we would like. In particular, using a map from string to
	// string does not fit into the web_connectivity use case. Hence, for now
	// we're going to represent this using interface{}. In going forward we
	// may probably want to have more uniform test helpers.
	TestHelpers map[string]interface{} `json:"test_helpers,omitempty"`

	// TestKeys contains the real test result. This field is opaque because
	// each experiment will insert here a different structure.
	TestKeys interface{} `json:"test_keys"`

	// TestName contains the test name
	TestName string `json:"test_name"`

	// MeasurementRuntime contains the measurement runtime. The JSON name
	// is test_runtime because this is the name expected by the OONI backend
	// even though that name is clearly a misleading one.
	MeasurementRuntime float64 `json:"test_runtime"`

	// TestStartTime contains the test start time
	TestStartTime string `json:"test_start_time"`

	// TestVersion contains the test version
	TestVersion string `json:"test_version"`
}

Measurement is a OONI measurement.

This structure is compatible with the definition of the base data format in https://github.com/ooni/spec/blob/master/data-formats/df-000-base.md.

func (*Measurement) AddAnnotation

func (m *Measurement) AddAnnotation(key, value string)

AddAnnotation adds a single annotations to m.Annotations.

func (*Measurement) AddAnnotations

func (m *Measurement) AddAnnotations(input map[string]string)

AddAnnotations adds the annotations from input to m.Annotations.

func (*Measurement) MaybeRewriteTestKeys

func (m *Measurement) MaybeRewriteTestKeys(
	currentIP string, marshal func(interface{}) ([]byte, error)) error

MaybeRewriteTestKeys is the function called by Scrub that ensures that m's serialization doesn't include the IP

func (*Measurement) Scrub

func (m *Measurement) Scrub(probeIP string) (err error)

Scrub scrubs the probeIP out of the measurement.

type MeasurementTarget

type MeasurementTarget string

MeasurementTarget is the target of a OONI measurement.

func (MeasurementTarget) MarshalJSON

func (t MeasurementTarget) MarshalJSON() ([]byte, error)

MarshalJSON serializes the MeasurementTarget.

type PrinterCallbacks

type PrinterCallbacks struct {
	Logger
}

PrinterCallbacks is the default event handler

func NewPrinterCallbacks

func NewPrinterCallbacks(logger Logger) PrinterCallbacks

NewPrinterCallbacks returns a new default callback handler

func (PrinterCallbacks) OnProgress

func (d PrinterCallbacks) OnProgress(percentage float64, message string)

OnProgress provides information about an experiment progress.

type Service

type Service struct {
	// Address is the address of the server.
	Address string `json:"address"`

	// Type is the type of the service.
	Type string `json:"type"`

	// Front is the front to use with "cloudfront" type entries.
	Front string `json:"front,omitempty"`
}

Service describes a backend service.

The fields of this struct have the meaning described in v2.0.0 of the OONI bouncer specification defined by https://github.com/ooni/spec/blob/master/backends/bk-004-bouncer.md.

type TorTarget

type TorTarget struct {
	// Address is the address of the target.
	Address string `json:"address"`

	// Name is the name of the target.
	Name string `json:"name"`

	// Params contains optional params for, e.g., pluggable transports.
	Params map[string][]string `json:"params"`

	// Protocol is the protocol to use with the target.
	Protocol string `json:"protocol"`

	// Source is the source from which we fetched this specific
	// target. Whenever the source is non-empty, we will treat
	// this specific target as a private target.
	Source string `json:"source"`
}

TorTarget is a target for the tor experiment.

type URLInfo

type URLInfo struct {
	CategoryCode string `json:"category_code"`
	CountryCode  string `json:"country_code"`
	URL          string `json:"url"`
}

URLInfo contains info on a test lists URL

type URLListConfig

type URLListConfig struct {
	Categories  []string // Categories to query for (empty means all)
	CountryCode string   // CountryCode is the optional country code
	Limit       int64    // Max number of URLs (<= 0 means no limit)
}

URLListConfig contains configuration for fetching the URL list.