flashlight

README

Lantern

Maintainers

@oxtoacart, @hwh33, @myleshorton

This repo contains the core Lantern library as well as the Android and iOS bindings.

The Lantern desktop application can be found at getlantern/lantern-desktop.

Process for making changes to config

flashlight is configured with per-proxy configuration loaded from the config-server, and global configuration loaded from S3 at runtime.

The global configuration is generated by genconfig running as a CRON job on cm-donyc3021etc. That job uses the latest version of genconfig that is pushed to releases in this repo via CI.

genconfig merges embeddedconfig/global.yaml.tmpl with dynamically verified masquerade hosts to produce the final global config. embeddedconfig/download_global.sh pulls in that change and runs anytime we run go generate. A CI Job runs go generate and automatically commits the results to this repo.

All clients, including old versions of flashlight, fetch the same global config from S3, so global.yaml.tmpl must remain backwards compatible with old clients.

If you're simply changing the contents of global.yaml.tmpl without any structural changes to the Go files in config, you can just change global.yaml.tmpl directly and once it's committed to main, it will fairly soon be reflected in S3.

If you're making changes to structure of the configuration, you need to ensure that this stays backwards compatible with old clients. To this end, we keep copies of old versions of config, for example config_v1. When making a structural change to the config, follow these steps:

1. Back up the current version of config, for example cp -R config config_v2
2. Update the code and tests in config as appropriate
3. Make sure the tests in config_v2, config_v1 etc. still work

Adding new domain fronting mappings

In addition to creating the domain front mappings in Cloudfront and Akamai, you also have to add the appropriate lines to provider_map.yaml with format: : .

[!IMPORTANT] Domain mappings must be added for both Cloudfront and Akamai!

Adding mapping through lantern-cloud

Mappings on Cloudfront and Akamai can be added using the terraform config in Lantern Cloud.

Adding mappings using the web consoles

• Cloudfront

  Open up the Cloudfront Distributions Console. Click 'Create Distribution' and enter the information as follows:

  • Origin Domain: Domain Cloudfront will send the request to.
  • Name: Use origin domain.
  • Allowed HTTP methods: Leave as default if only GET requests are needed, otherwise select the 3rd option to allow POST requests.
  • Cache key and origin request: Select the appropiate policies for each.
    • Cache: select optimized or disable if requests should not be cached.
    • Origin request: select policy based on what user information should be left in the request when sent to the Origin Domain. Can be left at 'None' if user info doesn't need to be forwarded. Everything else can be left at the default values unless neccessary. It will take several minutes to deploy after saving. The front-facing URL can be found in the 'Domain name' column from the distributions table. Add this to provider_map.yaml.

• Akamai

  Open up the Akamai Property Manager. Click 'New Property' and select 'Dynamic Site Accelerator'. Select 'Property Manager' as the method to set it up. Enter a meaningful property name, select 'latest' as the rule format, and click next. Configuring

  • Property Version Information:

    [!IMPORTANT] Security Options must be set to Standard TLS ready

  • Property Hostnames: Click '+Hostnames' -> 'Add Hostname'. Set to <name>.dsa.akamai.getiantem.org, where name is a meaningful descriptor (usually the same as the property name). This will be the front-facing URL that is added to provider_map.yaml. Click next and it will generate an Edge Hostname, then submit.

    [!NOTE] It should now show the hostname in the list and certificate value should be No certificate (HTTP Only).

  • Property Configuration Settings:

    • Origin Server Hostname: Domain request will be sent to.
    • Send True Client IP Header: Set to no. Leave everything else as the default values.
    • Set the Origin Server Forward Host Header and Cache Key Hostname to Origin Hostname if the property is masquerading as the origin.

    Click '+Behavior' -> 'Standard property behavor'.

    • Add content provider code and select 'Site Accelerator - 742552'.
    • Add caching and set appropiate caching option.

    From the tabs on the left go to 'Augment insights' -> 'Traffic reporting' and ensure the content provider code is the same as mentioned above. Click 'Save'.

    Go to the activate tab on the top of the page and activate on staging and production. It will take several minutes to activate.

Testing domain mappings

Domain mappings can be tested using the ddftool. You'll want to first use the ddftool to find valid IPs for each provider, then test for the expected response using one of the respective IPs and https://<provider mapping front-facing domain name>/<some origin domain path for testing> for the URL.

Building

In CI, flashlight used GH_TOKEN for access to private repositories.

You can build an SDK for use by external applications either for Android or for iOS.

Prerequisites

• Go 1.19 is the minimum supported version of Go
• GNU Make if you want to use the Makefile
• Dependencies are managed with Go Modules.
• Force git to use ssh instead of https by running git config --global url."git@github.com:".insteadOf "https://github.com/"

A note on iOS and memory usage

The iOS application needs to run as a background process on iOS, meaning that it's severely memory restricted. Because of this, we disable a lot of protocols and extra features using // go:build !ios in order to conserve memory.

Why not use // +build !ios

go-mobile automatically sets the ios build tag when building for iOS. In our case, we don't use this because in addition to the iOS app, we also distribute an iOS SDK that's intended for embedding inside of user-interactice apps. This SDK does not have to run in the background and is thus not memory constrained in the same way as our iOS app. Consequently, the sdk can and does include all of the standard lantern protocols and features.

Architecture

Features

We use "features" to enable/disable different characteristics/techniques in Flashlight, usually through the global config.

See ./config/features.go for a list of features. Below is a non-extensive description of each feature.

p2pcensoredpeer and p2pfreepeer

Allows the client to act either as a FreePeer or a CensoredPeer.

See overview of the p2p-proxying story here: https://docs.google.com/document/d/1JUjZHgpnunmwG3wUwlSmCKFwOGOXkkwyGd7cgrOJzbs/edit

A Note on Versioning

Until recently, flashlight and the applications that use it used only a single versions number, whatever the application itself was built with. This version number was used for various things:

• Displaying a version number in the UI
• Checking which application features are enabled based on the global configuration
• Telling our server infrastructure (especially config-server) what version of Lantern we're running so that it can assign appropriate proxies based on what that version supports
• Checking whether there's an update available via autoupdate

Because the various applications that use flashlight are in their own repos and built independently of each other, this created a coordination problem. Specifically, since pluggable transport support depends on the code level of flashlight, not of the application itself, we had to either synchronize the version numbering between the different apps, or configure the server-side infrastructure to recognize that depending on the application, different version numbers might actually mean the same flashlight code level.

To rectify this, we now uses two different version numbers.

Library Version - this is the version of the flashlight library and is based on the flashlight version tag.

Application Version - this is like the original Version that's baked in at application build time. It is still used for displaying the version in the UI, checking enabled features, and auto-updates. This version is NOT compiled into the flashlight library but is handled by the applications themselves and passed to flashlight when necessary (for example when checking enabled features).

When and how to update Library Version

Whenever we release a new version of the flashlight library, we tag it using standard [Go module version numbering], for example git tag v7.5.3. Then, we update lantern-desktop and android-lantern to use that version of the flashlight library. That's it.

What about major version changes

When changing major versions, for example v7 to v8, we need to udpate the package name as usual with Go. That means:

1. Update the module directive in go.mod
2. Find all imports of github.com/getlantern/flashlight and replace with github.com/getlantern/flashlight/v8
3. In dependent projects, perform the same search and replace as above
4. Also, dependent projects set embedded flashlight variables in their Makefiles, so make sure to update those paths per the above search and replace too

Documentation

Index

• type Flags
  • func ParseFlags() Flags
  • func (f Flags) AsMap() map[string]interface{}
• type Flashlight
  • func New(appName string, appVersion string, revisionDate string, configDir string, ...) (*Flashlight, error)
  • func (f *Flashlight) DisableNamedDomainRules(names ...string)
  • func (f *Flashlight) EnableNamedDomainRules(names ...string)
  • func (f *Flashlight) EnabledFeatures() map[string]bool
  • func (f *Flashlight) FeatureEnabled(feature, applicationVersion string) bool
  • func (f *Flashlight) FeatureOptions(feature string, opts config.FeatureOptions) error
  • func (f *Flashlight) Run(httpProxyAddr, socksProxyAddr string, afterStart func(cl *client.Client), ...)
  • func (f *Flashlight) RunClientListeners(httpProxyAddr, socksProxyAddr string, afterStart func(cl *client.Client), ...)
  • func (f *Flashlight) SetErrorHandler(handler func(t HandledErrorType, err error))
  • func (f *Flashlight) StartBackgroundServices() func()
  • func (f *Flashlight) Stop() error
• type HandledErrorType
  • func (t HandledErrorType) String() string
• type ProxyListener

Types

type Flags

type Flags struct {
	Addr                    string        `` /* 170-byte string literal not displayed */
	SocksAddr               string        `flag:"socksaddr" help:"ip:port on which to listen for SOCKS5 proxy requests."`
	ConfigDir               string        `flag:"configdir" help:"directory in which to store configuration. Defaults to platform-specific directories."`
	VPN                     bool          `help:"specify this flag to enable vpn mode"`
	CloudConfig             string        `flag:"cloudconfig" help:"optional http(s) URL to a cloud-based source for configuration updates"`
	CloudConfigCA           string        `flag:"cloudconfigca" help:"optional PEM encoded certificate used to verify TLS connections to fetch cloudconfig"`
	RegisterAt              string        `flag:"registerat" help:"base URL for peer DNS registry at which to register (e.g. https://peerscanner.getiantem.org)"`
	Country                 string        `help:"2 digit country code under which to report stats"`
	ClearProxySettings      bool          `flag:"clear-proxy-settings" help:"if true, Lantern removes proxy settings from the system."`
	CpuProfile              string        `flag:"cpuprofile" help:"write cpu profile to given file"`
	MemProfile              string        `flag:"memprofile" help:"write heap profile to given file"`
	UIAddr                  string        `flag:"uiaddr"  help:"if specified, indicates host:port the UI HTTP server should be started on"`
	ProxyAll                bool          `flag:"proxyall"  help:"set to true to proxy all traffic through Lantern network"`
	StickyConfig            bool          `flag:"stickyconfig" help:"set to true to only use the local config file"`
	AuthAddr                string        `flag:"authaddr" help:"if specified, indicates the address to use for the Lantern auth server"`
	YinbiAddr               string        `flag:"yinbiaddr" help:"if specified, indicates the address to use for the Yinbi server"`
	Headless                bool          `help:"if true, lantern will run with no ui"`
	Startup                 bool          `help:"if true, Lantern was automatically run on system startup"`
	Pprof                   bool          `flag:"pprof" help:"if true, run a pprof server on port 6060"`
	ForceProxyAddr          string        `` /* 141-byte string literal not displayed */
	ForceAuthToken          string        `flag:"force-auth-token" help:"if specified, force chained proxying to use this auth token instead of the configured one"`
	ForceConfigCountry      string        `` /* 127-byte string literal not displayed */
	ReadableConfig          bool          `flag:"readableconfig" help:"if specified, disables obfuscation of the config yaml so that it remains human readable"`
	Help                    bool          `flag:"help" help:"Get usage help"`
	NoUiHttpToken           bool          `flag:"no-ui-http-token" help:"don't require a HTTP token from the UI"`
	Standalone              bool          `flag:"standalone" help:"run Lantern in its own browser window (doesn't rely on system browser)"`
	Initialize              bool          `` /* 128-byte string literal not displayed */
	Timeout                 time.Duration `flag:"timeout" help:"force stop Lantern with an exit status of -1 after the timeout."`
	ReplicaRustUrl          string        `flag:"replica-rust-url" help:"use the replica-rust service at the provided endpoint"`
	Staging                 bool          `flag:"-"`
	Experiments             []string      `flag:"enabled-experiments" help:"comma separated list of experiments to enable"`
	P2PBep44TargetsAndSalts []string      `` /* 225-byte string literal not displayed */
	P2PRegistrarEndpoint    string        `flag:"p2p-registrar-endpoint" help:"Endpoint to use when p2pfreepeer feature is enabled. This overrides other sources"`
	P2PDomainWhitelist      []string      `` /* 223-byte string literal not displayed */
}

func ParseFlags

func ParseFlags() Flags

func (Flags) AsMap

func (f Flags) AsMap() map[string]interface{}

type Flashlight

type Flashlight struct {
	// contains filtered or unexported fields
}

func New

func New(
	appName string,
	appVersion string,
	revisionDate string,
	configDir string,
	enableVPN bool,
	disconnected func() bool,
	_proxyAll func() bool,
	_googleAds func() bool,
	allowPrivateHosts func() bool,
	autoReport func() bool,
	flagsAsMap map[string]interface{},
	onConfigUpdate func(*config.Global, config.Source),
	onProxiesUpdate func([]balancer.Dialer, config.Source),
	userConfig common.UserConfig,
	statsTracker stats.Tracker,
	isPro func() bool,
	lang func() string,
	adSwapTargetURL func() string,
	reverseDNS func(host string) (string, error),
	adTrackUrl func() string,
	eventWithLabel func(category, action, label string),
	dhtupContext *dhtup.Context,
) (*Flashlight, error)

New creates a client proxy.

func (*Flashlight) DisableNamedDomainRules

func (f *Flashlight) DisableNamedDomainRules(names ...string)

DisableNamedDomainRules removes named domain rules specified as arguments from the domainrouting rules table

func (*Flashlight) EnableNamedDomainRules

func (f *Flashlight) EnableNamedDomainRules(names ...string)

EnableNamedDomainRules adds named domain rules specified as arguments to the domainrouting rules table

func (*Flashlight) EnabledFeatures

func (f *Flashlight) EnabledFeatures() map[string]bool

EnabledFeatures gets all features enabled based on current conditions

func (*Flashlight) FeatureEnabled

func (f *Flashlight) FeatureEnabled(feature, applicationVersion string) bool

func (*Flashlight) FeatureOptions

func (f *Flashlight) FeatureOptions(feature string, opts config.FeatureOptions) error

FeatureOptions unmarshals options for the input feature. Feature names are tracked in the config package.

func (*Flashlight) Run

func (f *Flashlight) Run(httpProxyAddr, socksProxyAddr string,
	afterStart func(cl *client.Client),
	onError func(err error),
)

Run starts background services and runs the client proxy. It blocks as long as the proxy is running.

func (*Flashlight) RunClientListeners

func (f *Flashlight) RunClientListeners(httpProxyAddr, socksProxyAddr string,
	afterStart func(cl *client.Client),
	onError func(err error),
)

Runs client listeners, blocking as long as the proxy is running.

func (*Flashlight) SetErrorHandler

func (f *Flashlight) SetErrorHandler(handler func(t HandledErrorType, err error))

SetErrorHandler configures error handling. All errors provided to the handler are significant, but not enough to stop operation of the Flashlight instance. This method must be called before calling Run. All errors provided to the handler will be of a HandledErrorType defined in this package. The handler may be called multiple times concurrently.

If no handler is configured, these errors will be logged on the ERROR level.

func (*Flashlight) StartBackgroundServices

func (f *Flashlight) StartBackgroundServices() func()

Starts background services like config fetching

func (*Flashlight) Stop

func (f *Flashlight) Stop() error

Stops the local proxy

type HandledErrorType

type HandledErrorType int

HandledErrorType is used to differentiate error types to handlers configured via Flashlight.SetErrorHandler.

const (
	ErrorTypeProxySaveFailure  HandledErrorType = iota
	ErrorTypeConfigSaveFailure HandledErrorType = iota
)

func (HandledErrorType) String

func (t HandledErrorType) String() string

type ProxyListener

type ProxyListener interface {
	OnProxies(map[string]*commonconfig.ProxyConfig)
}