engine

package
v3.19.0-alpha.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 16, 2023 License: GPL-3.0 Imports: 29 Imported by: 0

Documentation

Overview

Package engine contains the engine API.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrNoURLsReturned    = errors.New("no URLs returned")
	ErrDetectedEmptyFile = errors.New("file did not contain any input")
	ErrInputRequired     = errors.New("no input provided")
	ErrNoInputExpected   = errors.New("we did not expect any input")
	ErrNoStaticInput     = errors.New("no static input for this experiment")
)

These errors are returned by the InputLoader.

View Source 
var ErrAllProbeServicesFailed = errors.New("all available probe services failed")

ErrAllProbeServicesFailed indicates all probe services failed.

View Source 
var ErrAlreadyUsingProxy = errors.New(
	"session: cannot create a new tunnel of this kind: we are already using a proxy",
)

ErrAlreadyUsingProxy indicates that we cannot create a tunnel with a specific name because we already configured a proxy.

Functions

func AllExperiments 

func AllExperiments() []string

AllExperiments returns the name of all experiments

func CheckEmbeddedPsiphonConfig added in v3.13.0 

func CheckEmbeddedPsiphonConfig() error

CheckEmbeddedPsiphonConfig checks whether we can load psiphon's config

func StaticBareInputForExperiment added in v3.13.0 

func StaticBareInputForExperiment(name string) ([]string, error)

StaticBareInputForExperiment returns the list of strings an experiment should use as static input. In case there is no static input for this experiment, we return an error.

Types

type InputLoader 

type InputLoader struct {
	// CheckInConfig contains options for the CheckIn API. If
	// not set, then we'll create a default config. If set but
	// there are fields inside it that are not set, then we
	// will set them to a default value.
	CheckInConfig *model.OOAPICheckInConfig

	// ExperimentName is the name of the experiment. This field
	// is only used together with the InputOrStaticDefault policy.
	ExperimentName string

	// InputPolicy specifies the input policy for the
	// current experiment. We will not load any input if
	// the policy says we should not. You MUST fill in
	// this field.
	InputPolicy model.InputPolicy

	// Logger is the optional logger that the InputLoader
	// should be using. If not set, we will use the default
	// logger of github.com/apex/log.
	Logger InputLoaderLogger

	// Session is the current measurement session. You
	// MUST fill in this field.
	Session InputLoaderSession

	// StaticInputs contains optional input to be added
	// to the resulting input list if possible.
	StaticInputs []string

	// SourceFiles contains optional files to read input
	// from. Each file should contain a single input string
	// per line. We will fail if any file is unreadable
	// as well as if any file is empty.
	SourceFiles []string
}

InputLoader loads input according to the specified policy either from command line and input files or from OONI services. The behaviour depends on the input policy as described below.

You MUST NOT change any public field of this structure when in use, because that MAY lead to data races.

InputNone

We fail if there is any StaticInput or any SourceFiles. If there's no input, we return a single, empty entry that causes experiments that don't require input to run once.

InputOptional

We gather input from StaticInput and SourceFiles. If there is input, we return it. Otherwise we return a single, empty entry that causes experiments that don't require input to run once.

InputOrQueryBackend

We gather input from StaticInput and SourceFiles. If there is input, we return it. Otherwise, we use OONI's probe services to gather input using the best API for the task.

InputOrStaticDefault

We gather input from StaticInput and SourceFiles. If there is input, we return it. Otherwise, we return an internal static list of inputs to be used with this experiment.

InputStrictlyRequired

We gather input from StaticInput and SourceFiles. If there is input, we return it. Otherwise, we return an error.

func (*InputLoader) Load 

func (il *InputLoader) Load(ctx context.Context) ([]model.OOAPIURLInfo, error)

Load attempts to load input using the specified input loader. We will return a list of URLs because this is the only input we support.

type InputLoaderLogger added in v3.9.2 

type InputLoaderLogger interface {
	// Warnf formats and emits a warning message.
	Warnf(format string, v ...interface{})
}

InputLoaderLogger is the logger according to an InputLoader.

type InputLoaderSession 

type InputLoaderSession interface {
	CheckIn(ctx context.Context,
		config *model.OOAPICheckInConfig) (*model.OOAPICheckInResultNettests, error)
}

InputLoaderSession is the session according to an InputLoader. We introduce this abstraction because it helps us with testing.

type InputProcessor 

type InputProcessor struct {
	// Annotations contains the measurement annotations
	Annotations map[string]string

	// Experiment is the code that will run the experiment.
	Experiment InputProcessorExperimentWrapper

	// Inputs is the list of inputs to measure.
	Inputs []model.OOAPIURLInfo

	// MaxRuntime is the optional maximum runtime
	// when looping over a list of inputs (e.g. when
	// running Web Connectivity). Zero means that
	// there will be no MaxRuntime limit.
	MaxRuntime time.Duration

	// Options contains command line options for this experiment.
	Options []string

	// Saver is the code that will save measurement results
	// on persistent storage (e.g. the file system).
	Saver InputProcessorSaverWrapper

	// Submitter is the code that will submit measurements
	// to the OONI collector.
	Submitter InputProcessorSubmitterWrapper
}

InputProcessor processes inputs. We perform a Measurement for each input using the given Experiment.

func (*InputProcessor) Run 

func (ip *InputProcessor) Run(ctx context.Context) error

Run processes all the input subject to the duration of the context. The code will perform measurements using the given experiment; submit measurements using the given submitter; save measurements using the given saver.

Annotations and Options will be saved in the measurement.

The default behaviour of this code is that an error while measuring, while submitting, or while saving a measurement is always causing us to break out of the loop. The user though is free to choose different policies by configuring the Experiment, Submitter, and Saver fields properly.

type InputProcessorExperiment 

type InputProcessorExperiment interface {
	MeasureAsync(
		ctx context.Context, input string) (<-chan *model.Measurement, error)
}

InputProcessorExperiment is the Experiment according to InputProcessor.

type InputProcessorExperimentWrapper 

type InputProcessorExperimentWrapper interface {
	MeasureAsync(
		ctx context.Context, input string, idx int) (<-chan *model.Measurement, error)
}

InputProcessorExperimentWrapper is a wrapper for an Experiment that also allow to pass around the input index.

func NewInputProcessorExperimentWrapper 

func NewInputProcessorExperimentWrapper(
	exp InputProcessorExperiment) InputProcessorExperimentWrapper

NewInputProcessorExperimentWrapper creates a new instance of InputProcessorExperimentWrapper.

type InputProcessorSaverWrapper 

type InputProcessorSaverWrapper interface {
	SaveMeasurement(idx int, m *model.Measurement) error
}

InputProcessorSaverWrapper is InputProcessor's wrapper for a Saver implementation.

func NewInputProcessorSaverWrapper 

func NewInputProcessorSaverWrapper(saver Saver) InputProcessorSaverWrapper

NewInputProcessorSaverWrapper wraps a Saver for InputProcessor.

type InputProcessorSubmitterWrapper 

type InputProcessorSubmitterWrapper interface {
	Submit(ctx context.Context, idx int, m *model.Measurement) error
}

InputProcessorSubmitterWrapper is InputProcessor's wrapper for a Submitter implementation.

func NewInputProcessorSubmitterWrapper 

func NewInputProcessorSubmitterWrapper(submitter Submitter) InputProcessorSubmitterWrapper

NewInputProcessorSubmitterWrapper wraps a Submitter for the InputProcessor.

type Saver 

type Saver = model.Saver

Saver is an alias for model.Saver.

func NewSaver 

func NewSaver(config SaverConfig) (Saver, error)

NewSaver creates a new instance of Saver.

type SaverConfig 

type SaverConfig struct {
	// Enabled is true if saving is enabled.
	Enabled bool

	// Experiment is the experiment we're currently running.
	Experiment SaverExperiment

	// FilePath is the filepath where to append the measurement as a
	// serialized JSON followed by a newline character.
	FilePath string

	// Logger is the logger used by the saver.
	Logger model.Logger
}

SaverConfig is the configuration for creating a new Saver.

type SaverExperiment 

type SaverExperiment interface {
	SaveMeasurement(m *model.Measurement, filepath string) error
}

SaverExperiment is an experiment according to the Saver.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session is a measurement session. It contains shared information required to run a measurement session, and it controls the lifecycle of such resources. It is not possible to reuse a Session. You MUST NOT attempt to use a Session again after Session.Close.

func NewSession 

func NewSession(ctx context.Context, config SessionConfig) (*Session, error)

NewSession creates a new session. This factory function will execute the following steps:

1. Make sure the config is sane, apply reasonable defaults where possible, otherwise return an error.

2. Create a temporary directory.

3. Create an instance of the session.

4. If the user requested for a proxy that entails a tunnel (at the moment of writing this note, either psiphon or tor), then start the requested tunnel and configure it as our proxy.

5. Create a compound resolver for the session that will attempt to use a bunch of DoT/DoH servers before falling back to the system resolver if nothing else works (see the engineresolver pkg). This engineresolver will be using the configured proxy, if any.

6. Create the default HTTP transport that we should be using when we communicate with the OONI backends. This transport will be using the configured proxy, if any.

If any of these steps fails, then we cannot create a measurement session and we return an error.

func (*Session) CheckIn added in v3.9.0 

func (s *Session) CheckIn(
	ctx context.Context, config *model.OOAPICheckInConfig) (*model.OOAPICheckInResultNettests, error)

CheckIn calls the check-in API. The input arguments MUST NOT be nil. Before querying the API, this function will ensure that the config structure does not contain any field that SHOULD be initialized and is not initialized. Whenever there is a field that is not initialized, we will attempt to set a reasonable default value for such a field. This list describes the current defaults we'll choose:

- Platform: if empty, set to Session.Platform();

- ProbeASN: if empty, set to Session.ProbeASNString();

- ProbeCC: if empty, set to Session.ProbeCC();

- RunType: if empty, set to model.RunTypeTimed;

- SoftwareName: if empty, set to Session.SoftwareName();

- SoftwareVersion: if empty, set to Session.SoftwareVersion();

- WebConnectivity.CategoryCodes: if nil, we will allocate an empty array (the API does not like nil).

Because we MAY need to know the current ASN and CC, this function MAY call MaybeLookupLocationContext.

The return value is either the check-in response or an error.

func (*Session) Close 

func (s *Session) Close() error

Close ensures that we close all the idle connections that the HTTP clients we are currently using may have created. It will also remove the temp dir that contains data from this session. Not calling this function may likely cause memory leaks in your application because of open idle connections, as well as excessive usage of disk space.

func (*Session) DefaultHTTPClient 

func (s *Session) DefaultHTTPClient() model.HTTPClient

DefaultHTTPClient returns the session's default HTTP client.

func (*Session) FetchPsiphonConfig added in v3.10.0 

func (s *Session) FetchPsiphonConfig(ctx context.Context) ([]byte, error)

FetchPsiphonConfig fetches psiphon config from the API.

func (*Session) FetchTorTargets added in v3.10.0 

func (s *Session) FetchTorTargets(
	ctx context.Context, cc string) (map[string]model.OOAPITorTarget, error)

FetchTorTargets fetches tor targets from the API.

func (*Session) GetTestHelpersByName 

func (s *Session) GetTestHelpersByName(name string) ([]model.OOAPIService, bool)

GetTestHelpersByName returns the available test helpers that use the specified name, or false if there's none.

func (*Session) KeyValueStore 

func (s *Session) KeyValueStore() model.KeyValueStore

KeyValueStore returns the configured key-value store.

func (*Session) KibiBytesReceived 

func (s *Session) KibiBytesReceived() float64

KibiBytesReceived accounts for the KibiBytes received by the HTTP clients managed by this session so far, including experiments.

func (*Session) KibiBytesSent 

func (s *Session) KibiBytesSent() float64

KibiBytesSent is like KibiBytesReceived but for the bytes sent.

func (*Session) Logger 

func (s *Session) Logger() model.Logger

Logger returns the logger used by the session.

func (*Session) LookupLocationContext 

func (s *Session) LookupLocationContext(ctx context.Context) (*enginelocate.Results, error)

LookupLocationContext performs a location lookup. If you want memoisation of the results, you should use MaybeLookupLocationContext.

func (*Session) MaybeLookupBackends 

func (s *Session) MaybeLookupBackends() error

MaybeLookupBackends is a caching OONI backends lookup call.

func (*Session) MaybeLookupBackendsContext 

func (s *Session) MaybeLookupBackendsContext(ctx context.Context) error

MaybeLookupBackendsContext is like MaybeLookupBackends but with context.

func (*Session) MaybeLookupLocation 

func (s *Session) MaybeLookupLocation() error

MaybeLookupLocation is a caching location lookup call.

func (*Session) MaybeLookupLocationContext 

func (s *Session) MaybeLookupLocationContext(ctx context.Context) error

MaybeLookupLocationContext is like MaybeLookupLocation but with a context that can be used to interrupt this long running operation. This function will fail IMMEDIATELY if given a cancelled context.

func (*Session) NewExperimentBuilder 

func (s *Session) NewExperimentBuilder(name string) (model.ExperimentBuilder, error)

NewExperimentBuilder returns a new experiment builder for the experiment with the given name, or an error if there's no such experiment with the given name

func (*Session) NewOrchestraClient 

func (s *Session) NewOrchestraClient(ctx context.Context) (*probeservices.Client, error)

NewOrchestraClient creates a new orchestra client. This client is registered and logged in with the OONI orchestra. An error is returned on failure.

This function is DEPRECATED. New code SHOULD NOT use it. It will eventually be made private or entirely removed from the codebase.

func (*Session) NewProbeServicesClient 

func (s *Session) NewProbeServicesClient(ctx context.Context) (*probeservices.Client, error)

NewProbeServicesClient creates a new client for talking with the OONI probe services. This function will benchmark the available probe services, and select the fastest. In case all probe services seem to be down, we try again applying circumvention tactics. This function will fail IMMEDIATELY if given a cancelled context.

func (*Session) NewSubmitter 

func (s *Session) NewSubmitter(ctx context.Context) (Submitter, error)

NewSubmitter creates a new submitter instance.

func (*Session) Platform 

func (s *Session) Platform() string

Platform returns the current platform. The platform is one of:

- android - ios - linux - macos - windows - unknown

When running on the iOS simulator, the returned platform is macos rather than ios if CGO is disabled. This is a known issue, that however should have a very limited impact.

func (*Session) ProbeASN 

func (s *Session) ProbeASN() uint

ProbeASN returns the probe ASN as an integer.

func (*Session) ProbeASNString 

func (s *Session) ProbeASNString() string

ProbeASNString returns the probe ASN as a string.

func (*Session) ProbeCC 

func (s *Session) ProbeCC() string

ProbeCC returns the probe CC.

func (*Session) ProbeIP 

func (s *Session) ProbeIP() string

ProbeIP returns the probe IP.

func (*Session) ProbeNetworkName 

func (s *Session) ProbeNetworkName() string

ProbeNetworkName returns the probe network name.

func (*Session) ProxyURL 

func (s *Session) ProxyURL() *url.URL

ProxyURL returns the Proxy URL, or nil if not set

func (*Session) ResolverASN 

func (s *Session) ResolverASN() uint

ResolverASN returns the resolver ASN

func (*Session) ResolverASNString 

func (s *Session) ResolverASNString() string

ResolverASNString returns the resolver ASN as a string

func (*Session) ResolverIP 

func (s *Session) ResolverIP() string

ResolverIP returns the resolver IP

func (*Session) ResolverNetworkName 

func (s *Session) ResolverNetworkName() string

ResolverNetworkName returns the resolver network name.

func (*Session) SoftwareName 

func (s *Session) SoftwareName() string

SoftwareName returns the application name.

func (*Session) SoftwareVersion 

func (s *Session) SoftwareVersion() string

SoftwareVersion returns the application version.

func (*Session) TempDir 

func (s *Session) TempDir() string

TempDir returns the temporary directory.

func (*Session) TorArgs 

func (s *Session) TorArgs() []string

TorArgs returns the configured extra args for the tor binary. If not set we will not pass in any extra arg. Applies to `-OTunnel=tor` mainly.

func (*Session) TorBinary 

func (s *Session) TorBinary() string

TorBinary returns the configured path to the tor binary. If not set we will attempt to use "tor". Applies to `-OTunnel=tor` mainly.

func (*Session) TunnelDir added in v3.14.0 

func (s *Session) TunnelDir() string

TunnelDir returns the persistent directory used by tunnels.

func (*Session) UserAgent 

func (s *Session) UserAgent() (useragent string)

UserAgent constructs the user agent to be used in this session.

type SessionConfig 

type SessionConfig struct {
	AvailableProbeServices []model.OOAPIService
	KVStore                model.KeyValueStore
	Logger                 model.Logger
	ProxyURL               *url.URL
	SoftwareName           string
	SoftwareVersion        string
	TempDir                string
	TorArgs                []string
	TorBinary              string

	// SnowflakeRendezvous is the rendezvous method
	// to be used by the torsf tunnel
	SnowflakeRendezvous string

	// TunnelDir is the directory where we should store
	// the state of persistent tunnels. This field is
	// optional _unless_ you want to use tunnels. In such
	// case, starting a tunnel will fail because there
	// is no directory where to store state.
	TunnelDir string
}

SessionConfig contains the Session config

type Submitter 

type Submitter = model.Submitter

Submitter is an alias for model.Submitter

func NewSubmitter 

func NewSubmitter(ctx context.Context, config SubmitterConfig) (Submitter, error)

NewSubmitter creates a new submitter instance. Depending on whether submission is enabled or not, the returned submitter instance migh just be a stub implementation.

type SubmitterConfig 

type SubmitterConfig struct {
	// Enabled is true if measurement submission is enabled.
	Enabled bool

	// Session is the current session.
	Session SubmitterSession

	// Logger is the logger to be used.
	Logger model.Logger
}

SubmitterConfig contains settings for NewSubmitter.

type SubmitterSession 

type SubmitterSession interface {
	// NewSubmitter creates a new probeservices Submitter.
	NewSubmitter(ctx context.Context) (Submitter, error)
}

SubmitterSession is the Submitter's view of the Session.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.