Documentation ¶
Overview ¶
Package probeservices contains code to contact OONI probe services.
The probe services are HTTPS endpoints distributed across a bunch of data centres implementing a bunch of OONI APIs. When started, OONI will benchmark the available probe services and select the fastest one. Eventually all the possible OONI APIs will run as probe services.
This package implements the following APIs:
1. v2.0.0 of the OONI bouncer specification defined in https://github.com/ooni/spec/blob/master/backends/bk-004-bouncer;
2. v2.0.0 of the OONI collector specification defined in https://github.com/ooni/spec/blob/master/backends/bk-003-collector.md;
3. most of the OONI orchestra API: login, register, fetch URLs for the Web Connectivity experiment, input for Tor and Psiphon.
Orchestra is a set of OONI APIs for probe orchestration. We currently mainly using it for fetching inputs for the tor, psiphon, and web experiments.
In addition, this package also contains code to benchmark the available probe services, discard non working ones, select the fastest.
Index ¶
- Constants
- Variables
- func Default() []model.OOAPIService
- func OnlyFallbacks(in []model.OOAPIService) (out []model.OOAPIService)
- func OnlyHTTPS(in []model.OOAPIService) (out []model.OOAPIService)
- func SortEndpoints(in []model.OOAPIService) (out []model.OOAPIService)
- type Candidate
- type Client
- func (c Client) CheckIn(ctx context.Context, config model.OOAPICheckInConfig) (*model.OOAPICheckInInfo, error)
- func (c Client) CheckReportID(ctx context.Context, reportID string) (bool, error)
- func (c Client) FetchPsiphonConfig(ctx context.Context) ([]byte, error)
- func (c Client) FetchTorTargets(ctx context.Context, cc string) (result map[string]model.OOAPITorTarget, err error)
- func (c Client) FetchURLList(ctx context.Context, config model.OOAPIURLListConfig) ([]model.OOAPIURLInfo, error)
- func (c Client) GetCredsAndAuth() (*LoginCredentials, *LoginAuth, error)
- func (c Client) GetMeasurementMeta(ctx context.Context, config MeasurementMetaConfig) (*MeasurementMeta, error)
- func (c Client) GetTestHelpers(ctx context.Context) (output map[string][]model.OOAPIService, err error)
- func (c Client) MaybeLogin(ctx context.Context) error
- func (c Client) MaybeRegister(ctx context.Context, metadata Metadata) error
- func (c Client) OpenReport(ctx context.Context, rt ReportTemplate) (ReportChannel, error)
- type LoginAuth
- type LoginCredentials
- type MeasurementMeta
- type MeasurementMetaConfig
- type Metadata
- type ReportChannel
- type ReportOpener
- type ReportTemplate
- type Session
- type State
- type StateFile
- type Submitter
Constants ¶
const ( // DefaultDataFormatVersion is the default data format version. // // See https://github.com/ooni/spec/tree/master/data-formats#history. DefaultDataFormatVersion = "0.2.0" // DefaultFormat is the default format DefaultFormat = "json" )
Variables ¶
var ( // ErrUnsupportedDataFormatVersion indicates that the user provided // in input a data format version that we do not support. ErrUnsupportedDataFormatVersion = errors.New("Unsupported data format version") // ErrUnsupportedFormat indicates that the format is not supported. ErrUnsupportedFormat = errors.New("Unsupported format") // ErrJSONFormatNotSupported indicates that the collector we're using // does not support the JSON report format. ErrJSONFormatNotSupported = errors.New("JSON format not supported") )
var ( // ErrUnsupportedEndpoint indicates that we don't support this endpoint type. ErrUnsupportedEndpoint = errors.New("probe services: unsupported endpoint type") // ErrUnsupportedCloudFrontAddress indicates that we don't support this // cloudfront address (e.g. wrong scheme, presence of port). ErrUnsupportedCloudFrontAddress = errors.New( "probe services: unsupported cloud front address", ) // ErrNotRegistered indicates that the probe is not registered // with the OONI orchestra backend. ErrNotRegistered = errors.New("not registered") // ErrNotLoggedIn indicates that we are not logged in ErrNotLoggedIn = errors.New("not logged in") // ErrInvalidMetadata indicates that the metadata is not valid ErrInvalidMetadata = errors.New("invalid metadata") )
Functions ¶
func OnlyFallbacks ¶
func OnlyFallbacks(in []model.OOAPIService) (out []model.OOAPIService)
OnlyFallbacks returns the fallback endpoints only.
func OnlyHTTPS ¶
func OnlyHTTPS(in []model.OOAPIService) (out []model.OOAPIService)
OnlyHTTPS returns the HTTPS endpoints only.
func SortEndpoints ¶
func SortEndpoints(in []model.OOAPIService) (out []model.OOAPIService)
SortEndpoints gives priority to https, then cloudfronted, then onion.
Types ¶
type Candidate ¶
type Candidate struct { // Duration is the time it took to access the service. Duration time.Duration // Err indicates whether the service works. Err error // Endpoint is the service endpoint. Endpoint model.OOAPIService // TestHelpers contains the data returned by the endpoint. TestHelpers map[string][]model.OOAPIService }
Candidate is a candidate probe service.
func SelectBest ¶
SelectBest selects the best among the candidates. If there is no suitable candidate, then this function returns nil.
func TryAll ¶
TryAll tries all the input services using the provided context and session. It returns a list containing information on each candidate that was tried. We will try all the HTTPS candidates first. So, the beginning of the list will contain all of them, and for each of them you will know whether it worked (by checking the Err field) and how fast it was (by checking the Duration field). You should pick the fastest one that worked. If none of them works, then TryAll will subsequently attempt with all the available fallbacks, and return at the first success. In such case, you will see a list of N failing HTTPS candidates, followed by a single successful fallback candidate (e.g. cloudfronted). If all candidates fail, you see in output a list containing all entries where Err is not nil.
type Client ¶
type Client struct { httpx.APIClientTemplate LoginCalls *atomicx.Int64 RegisterCalls *atomicx.Int64 StateFile StateFile }
Client is a client for the OONI probe services API.
func NewClient ¶
func NewClient(sess Session, endpoint model.OOAPIService) (*Client, error)
NewClient creates a new client for the specified probe services endpoint. This function fails, e.g., we don't support the specified endpoint.
func (Client) CheckIn ¶
func (c Client) CheckIn(ctx context.Context, config model.OOAPICheckInConfig) (*model.OOAPICheckInInfo, error)
CheckIn function is called by probes asking if there are tests to be run The config argument contains the mandatory settings. Returns the list of tests to run and the URLs, on success, or an explanatory error, in case of failure.
func (Client) CheckReportID ¶
CheckReportID checks whether the given ReportID exists.
func (Client) FetchPsiphonConfig ¶
FetchPsiphonConfig fetches psiphon config from authenticated OONI orchestra.
func (Client) FetchTorTargets ¶
func (c Client) FetchTorTargets(ctx context.Context, cc string) (result map[string]model.OOAPITorTarget, err error)
FetchTorTargets returns the targets for the tor experiment.
func (Client) FetchURLList ¶
func (c Client) FetchURLList(ctx context.Context, config model.OOAPIURLListConfig) ([]model.OOAPIURLInfo, error)
FetchURLList fetches the list of URLs used by WebConnectivity. The config argument contains the optional settings. Returns the list of URLs, on success, or an explanatory error, in case of failure.
func (Client) GetCredsAndAuth ¶
func (c Client) GetCredsAndAuth() (*LoginCredentials, *LoginAuth, error)
GetCredsAndAuth is an utility function that returns the credentials with which we are registered and the token with which we're logged in. If we're not registered or not logged in, an error is returned instead.
func (Client) GetMeasurementMeta ¶
func (c Client) GetMeasurementMeta( ctx context.Context, config MeasurementMetaConfig) (*MeasurementMeta, error)
GetMeasurementMeta returns meta information about a measurement.
func (Client) GetTestHelpers ¶
func (c Client) GetTestHelpers( ctx context.Context) (output map[string][]model.OOAPIService, err error)
GetTestHelpers is like GetCollectors but for test helpers.
func (Client) MaybeLogin ¶
MaybeLogin performs login if necessary
func (Client) MaybeRegister ¶
MaybeRegister registers this client if not already registered
func (Client) OpenReport ¶
func (c Client) OpenReport(ctx context.Context, rt ReportTemplate) (ReportChannel, error)
OpenReport opens a new report.
type LoginCredentials ¶
type LoginCredentials struct { ClientID string `json:"username"` Password string `json:"password"` }
LoginCredentials contains the login credentials
type MeasurementMeta ¶
type MeasurementMeta struct { // Fields returned by the API server whenever we are // calling /api/v1/measurement_meta. Anomaly bool `json:"anomaly"` CategoryCode string `json:"category_code"` Confirmed bool `json:"confirmed"` Failure bool `json:"failure"` Input *string `json:"input"` MeasurementStartTime time.Time `json:"measurement_start_time"` ProbeASN int64 `json:"probe_asn"` ProbeCC string `json:"probe_cc"` ReportID string `json:"report_id"` Scores string `json:"scores"` TestName string `json:"test_name"` TestStartTime time.Time `json:"test_start_time"` // This field is only included if the user has specified // the config.Full option, otherwise it's empty. RawMeasurement string `json:"raw_measurement"` }
MeasurementMeta contains measurement metadata.
type MeasurementMetaConfig ¶
type MeasurementMetaConfig struct { // ReportID is the mandatory report ID. ReportID string // Full indicates whether we also want the full measurement body. Full bool // Input is the optional input. Input string }
MeasurementMetaConfig contains configuration for GetMeasurementMeta.
type Metadata ¶
type Metadata struct { AvailableBandwidth string `json:"available_bandwidth,omitempty"` DeviceToken string `json:"device_token,omitempty"` Language string `json:"language,omitempty"` NetworkType string `json:"network_type,omitempty"` Platform string `json:"platform"` ProbeASN string `json:"probe_asn"` ProbeCC string `json:"probe_cc"` ProbeFamily string `json:"probe_family,omitempty"` ProbeTimezone string `json:"probe_timezone,omitempty"` SoftwareName string `json:"software_name"` SoftwareVersion string `json:"software_version"` SupportedTests []string `json:"supported_tests"` }
Metadata contains metadata about a probe. This message is included into a bunch of messages sent to orchestra.
func (Metadata) Valid ¶
Valid returns true if metadata is valid, false otherwise. Metadata is considered valid if all the mandatory fields are not empty. If a field is marked `json:",omitempty"` in the structure definition, then it's for sure mandatory. The "device_token" field is mandatory only if the platform is "ios" or "android", because there's currently no device token that we know of for desktop devices.
type ReportChannel ¶
type ReportChannel interface { CanSubmit(m *model.Measurement) bool ReportID() string SubmitMeasurement(ctx context.Context, m *model.Measurement) error }
ReportChannel is a channel through which one could submit measurements belonging to the same report. The Report struct belongs to this interface.
type ReportOpener ¶
type ReportOpener interface {
OpenReport(ctx context.Context, rt ReportTemplate) (ReportChannel, error)
}
ReportOpener is any struct that is able to open a new ReportChannel. The Client struct belongs to this interface.
type ReportTemplate ¶
type ReportTemplate struct { // DataFormatVersion is unconditionally set to DefaultDataFormatVersion // and you don't need to be concerned about it. DataFormatVersion string `json:"data_format_version"` // Format is unconditionally set to `json` and you don't need // to be concerned about it. Format string `json:"format"` // ProbeASN is the probe's autonomous system number (e.g. `AS1234`) ProbeASN string `json:"probe_asn"` // ProbeCC is the probe's country code (e.g. `IT`) ProbeCC string `json:"probe_cc"` // SoftwareName is the app name (e.g. `measurement-kit`) SoftwareName string `json:"software_name"` // SoftwareVersion is the app version (e.g. `0.9.1`) SoftwareVersion string `json:"software_version"` // TestName is the test name (e.g. `ndt`) TestName string `json:"test_name"` // TestStartTime contains the test start time TestStartTime string `json:"test_start_time"` // TestVersion is the test version (e.g. `1.0.1`) TestVersion string `json:"test_version"` }
ReportTemplate is the template for opening a report
func NewReportTemplate ¶
func NewReportTemplate(m *model.Measurement) ReportTemplate
NewReportTemplate creates a new ReportTemplate from a Measurement.
type Session ¶
type Session interface { DefaultHTTPClient() model.HTTPClient KeyValueStore() model.KeyValueStore Logger() model.Logger ProxyURL() *url.URL UserAgent() string }
Session is how this package sees a Session.
type State ¶
State is the state stored inside the state file
func (State) Auth ¶
Auth returns an authentication structure, if possible, otherwise it returns nil, meaning that you should login again.
func (State) Credentials ¶
func (s State) Credentials() *LoginCredentials
Credentials returns login credentials, if possible, otherwise it returns nil, meaning that you should create an account.
type StateFile ¶
type StateFile struct { Store model.KeyValueStore // contains filtered or unexported fields }
StateFile is the orchestra state file. It is backed by a generic key-value store configured by the user.
func NewStateFile ¶
func NewStateFile(kvstore model.KeyValueStore) StateFile
NewStateFile creates a new state file backed by a key-value store
func (StateFile) Get ¶
Get returns the current state. In case of any error with the underlying key-value store, we return an empty state.
func (StateFile) GetMockable ¶
func (sf StateFile) GetMockable(sfget func(string) ([]byte, error), unmarshal func([]byte, interface{}) error) (State, error)
GetMockable is a mockable version of Get
type Submitter ¶
type Submitter struct {
// contains filtered or unexported fields
}
Submitter is an abstraction allowing you to submit arbitrary measurements to a given OONI backend. This implementation will take care of opening reports when needed as well as of closing reports when needed. Nonetheless you need to remember to call its Close method when done, because there is likely an open report that has not been closed yet.
func NewSubmitter ¶
func NewSubmitter(opener ReportOpener, logger model.Logger) *Submitter
NewSubmitter creates a new Submitter instance.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package testorchestra helps with testing the OONI orchestra API.
|
Package testorchestra helps with testing the OONI orchestra API. |