flashlight

README

Lantern

Maintainers

@hwh33, @myleshorton, @reflog

This repo contains the core Lantern library as well as the Android and iOS bindings.

The Lantern client application can be found at lantern-client.

Process for making changes to config

flashlight is configured with per-proxy configuration loaded from the config-server, and global configuration loaded from S3 at runtime.

The global configuration is generated by genconfig running as a CRON job on lantern-cloud. That job uses the latest version of genconfig that is pushed to releases in this repo via CI.

genconfig merges embeddedconfig/global.yaml.tmpl with dynamically verified masquerade hosts to produce the final global config. embeddedconfig/download_global.sh pulls in that change and runs anytime we run go generate. A CI Job runs go generate and automatically commits the results to this repo.

All clients, including old versions of flashlight, fetch the same global config from S3, so global.yaml.tmpl must remain backwards compatible with old clients.

If you're simply changing the contents of global.yaml.tmpl without any structural changes to the Go files in config, you can just change global.yaml.tmpl directly and once it's committed to main, it will fairly soon be reflected in S3.

If you're making changes to structure of the configuration, you need to ensure that this stays backwards compatible with old clients. To this end, we keep copies of old versions of config, for example config_v1. When making a structural change to the config, follow these steps:

1. Back up the current version of config, for example cp -R config config_v2
2. Update the code and tests in config as appropriate
3. Make sure the tests in config_v2, config_v1 etc. still work

Adding new domain fronting mappings

In addition to creating the domain front mappings in Cloudfront and Akamai, you also have to add the appropriate lines to provider_map.yaml with format: : .

[!IMPORTANT] Domain mappings must be added for both Cloudfront and Akamai!

Adding mapping through lantern-cloud

Mappings on Cloudfront and Akamai can be added using the terraform config in Lantern Cloud.

Adding mappings using the web consoles

• Cloudfront

  Open up the Cloudfront Distributions Console. Click 'Create Distribution' and enter the information as follows:

  • Origin Domain: Domain Cloudfront will send the request to.
  • Name: Use origin domain.
  • Allowed HTTP methods: Leave as default if only GET requests are needed, otherwise select the 3rd option to allow POST requests.
  • Cache key and origin request: Select the appropiate policies for each.
    • Cache: select optimized or disable if requests should not be cached.
    • Origin request: select policy based on what user information should be left in the request when sent to the Origin Domain. Can be left at 'None' if user info doesn't need to be forwarded. Everything else can be left at the default values unless neccessary. It will take several minutes to deploy after saving. The front-facing URL can be found in the 'Domain name' column from the distributions table. Add this to provider_map.yaml.

• Akamai

  Open up the Akamai Property Manager. Click 'New Property' and select 'Dynamic Site Accelerator'. Select 'Property Manager' as the method to set it up. Enter a meaningful property name, select 'latest' as the rule format, and click next. Configuring

  • Property Version Information:

    [!IMPORTANT] Security Options must be set to Standard TLS ready

  • Property Hostnames: Click '+Hostnames' -> 'Add Hostname'. Set to <name>.dsa.akamai.getiantem.org, where name is a meaningful descriptor (usually the same as the property name). This will be the front-facing URL that is added to provider_map.yaml. Click next and it will generate an Edge Hostname, then submit.

    [!NOTE] It should now show the hostname in the list and certificate value should be No certificate (HTTP Only).

  • Property Configuration Settings:

    • Origin Server Hostname: Domain request will be sent to.
    • Send True Client IP Header: Set to no. Leave everything else as the default values.
    • Set the Origin Server Forward Host Header and Cache Key Hostname to Origin Hostname if the property is masquerading as the origin.

    Click '+Behavior' -> 'Standard property behavor'.

    • Add content provider code and select 'Site Accelerator - 742552'.
    • Add caching and set appropiate caching option.

    From the tabs on the left go to 'Augment insights' -> 'Traffic reporting' and ensure the content provider code is the same as mentioned above. Click 'Save'.

    Go to the activate tab on the top of the page and activate on staging and production. It will take several minutes to activate.

Testing domain mappings

Domain mappings can be tested using the ddftool. You'll want to first use the ddftool to find valid IPs for each provider, then test for the expected response using one of the respective IPs and https://<provider mapping front-facing domain name>/<some origin domain path for testing> for the URL.

Building

In CI, flashlight used GH_TOKEN for access to private repositories.

You can build an SDK for use by external applications either for Android or for iOS.

Prerequisites

• Go 1.19 is the minimum supported version of Go
• GNU Make if you want to use the Makefile
• Dependencies are managed with Go Modules.
• Force git to use ssh instead of https by running git config --global url."git@github.com:".insteadOf "https://github.com/"

A note on iOS and memory usage

The iOS application needs to run as a background process on iOS, meaning that it's severely memory restricted. Because of this, we disable a lot of protocols and extra features using // go:build !ios in order to conserve memory.

Why not use // +build !ios

go-mobile automatically sets the ios build tag when building for iOS. In our case, we don't use this because in addition to the iOS app, we also distribute an iOS SDK that's intended for embedding inside of user-interactice apps. This SDK does not have to run in the background and is thus not memory constrained in the same way as our iOS app. Consequently, the sdk can and does include all of the standard lantern protocols and features.

Architecture

Features

We use "features" to enable/disable different characteristics/techniques in Flashlight, usually through the global config.

See ./config/features.go for a list of features. Below is a non-extensive description of each feature.

A Note on Versioning

Until recently, flashlight and the applications that use it used only a single versions number, whatever the application itself was built with. This version number was used for various things:

• Displaying a version number in the UI
• Checking which application features are enabled based on the global configuration
• Telling our server infrastructure (especially config-server) what version of Lantern we're running so that it can assign appropriate proxies based on what that version supports
• Checking whether there's an update available via autoupdate

Because the various applications that use flashlight are in their own repos and built independently of each other, this created a coordination problem. Specifically, since pluggable transport support depends on the code level of flashlight, not of the application itself, we had to either synchronize the version numbering between the different apps, or configure the server-side infrastructure to recognize that depending on the application, different version numbers might actually mean the same flashlight code level.

To rectify this, we now uses two different version numbers.

Library Version - this is the version of the flashlight library and is based on the flashlight version tag.

Application Version - this is like the original Version that's baked in at application build time. It is still used for displaying the version in the UI, checking enabled features, and auto-updates. This version is NOT compiled into the flashlight library but is handled by the applications themselves and passed to flashlight when necessary (for example when checking enabled features).

When and how to update Library Version

Whenever we release a new version of the flashlight library, we tag it using standard [Go module version numbering], for example git tag v7.5.3. Then, we update lantern-desktop and android-lantern to use that version of the flashlight library. That's it.

What about major version changes

When changing major versions, for example v7 to v8, we need to udpate the package name as usual with Go. That means:

1. Update the module directive in go.mod
2. Find all imports of github.com/getlantern/flashlight and replace with github.com/getlantern/flashlight/v8
3. In dependent projects, perform the same search and replace as above
4. Also, dependent projects set embedded flashlight variables in their Makefiles, so make sure to update those paths per the above search and replace too

Documentation

Index

• type Flags
  • func ParseFlags() Flags
  • func (f Flags) AsMap() map[string]interface{}
• type Flashlight
  • func New(appName string, appVersion string, revisionDate string, configDir string, ...) (*Flashlight, error)
  • func (f *Flashlight) DisableNamedDomainRules(names ...string)
  • func (f *Flashlight) EnableNamedDomainRules(names ...string)
  • func (f *Flashlight) EnabledFeatures() map[string]bool
  • func (f *Flashlight) FeatureEnabled(feature, applicationVersion string) bool
  • func (f *Flashlight) FeatureOptions(feature string, opts config.FeatureOptions) error
  • func (f *Flashlight) Run(httpProxyAddr, socksProxyAddr string, afterStart func(cl *client.Client), ...)
  • func (f *Flashlight) RunClientListeners(httpProxyAddr, socksProxyAddr string, afterStart func(cl *client.Client), ...)
  • func (f *Flashlight) SetErrorHandler(handler func(t HandledErrorType, err error))
  • func (f *Flashlight) StartBackgroundServices() func()
  • func (f *Flashlight) Stop() error
• type HandledErrorType
  • func (t HandledErrorType) String() string
• type Option
  • func WithInit(onInit func()) Option
  • func WithOnConfig(onConfigUpdate func(*config.Global, config.Source)) Option
  • func WithOnDialError(onDialError func(error, bool)) Option
  • func WithOnProxies(onProxiesUpdate func([]bandit.Dialer, config.Source)) Option
  • func WithOnSucceedingProxy(onSucceedingProxy func()) Option
• type ProxyListener

Types

type Flags

type Flags struct {
	Addr               string        `` /* 170-byte string literal not displayed */
	SocksAddr          string        `flag:"socksaddr" help:"ip:port on which to listen for SOCKS5 proxy requests."`
	ConfigDir          string        `flag:"configdir" help:"directory in which to store configuration. Defaults to platform-specific directories."`
	VPN                bool          `help:"specify this flag to enable vpn mode"`
	CloudConfig        string        `flag:"cloudconfig" help:"optional http(s) URL to a cloud-based source for configuration updates"`
	CloudConfigCA      string        `flag:"cloudconfigca" help:"optional PEM encoded certificate used to verify TLS connections to fetch cloudconfig"`
	RegisterAt         string        `flag:"registerat" help:"base URL for peer DNS registry at which to register (e.g. https://peerscanner.getiantem.org)"`
	Country            string        `help:"2 digit country code under which to report stats"`
	ClearProxySettings bool          `flag:"clear-proxy-settings" help:"if true, Lantern removes proxy settings from the system."`
	CpuProfile         string        `flag:"cpuprofile" help:"write cpu profile to given file"`
	MemProfile         string        `flag:"memprofile" help:"write heap profile to given file"`
	UIAddr             string        `flag:"uiaddr"  help:"if specified, indicates host:port the UI HTTP server should be started on"`
	ProxyAll           bool          `flag:"proxyall"  help:"set to true to proxy all traffic through Lantern network"`
	StickyConfig       bool          `flag:"stickyconfig" help:"set to true to only use the local config file"`
	AuthAddr           string        `flag:"authaddr" help:"if specified, indicates the address to use for the Lantern auth server"`
	YinbiAddr          string        `flag:"yinbiaddr" help:"if specified, indicates the address to use for the Yinbi server"`
	Headless           bool          `help:"if true, lantern will run with no ui"`
	Startup            bool          `help:"if true, Lantern was automatically run on system startup"`
	Pprof              bool          `flag:"pprof" help:"if true, run a pprof server on port 6060"`
	ForceProxyAddr     string        `` /* 141-byte string literal not displayed */
	ForceAuthToken     string        `flag:"force-auth-token" help:"if specified, force chained proxying to use this auth token instead of the configured one"`
	ForceConfigCountry string        `` /* 127-byte string literal not displayed */
	ReadableConfig     bool          `flag:"readableconfig" help:"if specified, disables obfuscation of the config yaml so that it remains human readable"`
	Help               bool          `flag:"help" help:"Get usage help"`
	NoUiHttpToken      bool          `flag:"no-ui-http-token" help:"don't require a HTTP token from the UI"`
	Standalone         bool          `flag:"standalone" help:"run Lantern in its own browser window (doesn't rely on system browser)"`
	Initialize         bool          `` /* 128-byte string literal not displayed */
	Timeout            time.Duration `flag:"timeout" help:"force stop Lantern with an exit status of -1 after the timeout."`
	ReplicaRustUrl     string        `flag:"replica-rust-url" help:"use the replica-rust service at the provided endpoint"`
	Staging            bool          `flag:"-"`
	Experiments        []string      `flag:"enabled-experiments" help:"comma separated list of experiments to enable"`
}

func ParseFlags

func ParseFlags() Flags

func (Flags) AsMap

func (f Flags) AsMap() map[string]interface{}

type Flashlight

type Flashlight struct {
	// contains filtered or unexported fields
}

func New

func New(
	appName string,
	appVersion string,
	revisionDate string,
	configDir string,
	enableVPN bool,
	disconnected func() bool,
	_proxyAll func() bool,
	allowPrivateHosts func() bool,
	autoReport func() bool,
	flagsAsMap map[string]interface{},
	userConfig common.UserConfig,
	statsTracker stats.Tracker,
	isPro func() bool,
	lang func() string,
	reverseDNS func(host string) (string, error),
	eventWithLabel func(category, action, label string),
	options ...Option,
) (*Flashlight, error)

New creates a client proxy.

func (*Flashlight) DisableNamedDomainRules

func (f *Flashlight) DisableNamedDomainRules(names ...string)

DisableNamedDomainRules removes named domain rules specified as arguments from the domainrouting rules table

func (*Flashlight) EnableNamedDomainRules

func (f *Flashlight) EnableNamedDomainRules(names ...string)

EnableNamedDomainRules adds named domain rules specified as arguments to the domainrouting rules table

func (*Flashlight) EnabledFeatures

func (f *Flashlight) EnabledFeatures() map[string]bool

EnabledFeatures gets all features enabled based on current conditions

func (*Flashlight) FeatureEnabled

func (f *Flashlight) FeatureEnabled(feature, applicationVersion string) bool

func (*Flashlight) FeatureOptions

func (f *Flashlight) FeatureOptions(feature string, opts config.FeatureOptions) error

FeatureOptions unmarshals options for the input feature. Feature names are tracked in the config package.

func (*Flashlight) Run

func (f *Flashlight) Run(httpProxyAddr, socksProxyAddr string,
	afterStart func(cl *client.Client),
	onError func(err error),
)

Run starts background services and runs the client proxy. It blocks as long as the proxy is running.

func (*Flashlight) RunClientListeners

func (f *Flashlight) RunClientListeners(httpProxyAddr, socksProxyAddr string,
	afterStart func(cl *client.Client),
	onError func(err error),
)

Runs client listeners, blocking as long as the proxy is running.

func (*Flashlight) SetErrorHandler

func (f *Flashlight) SetErrorHandler(handler func(t HandledErrorType, err error))

SetErrorHandler configures error handling. All errors provided to the handler are significant, but not enough to stop operation of the Flashlight instance. This method must be called before calling Run. All errors provided to the handler will be of a HandledErrorType defined in this package. The handler may be called multiple times concurrently.

If no handler is configured, these errors will be logged on the ERROR level.

func (*Flashlight) StartBackgroundServices

func (f *Flashlight) StartBackgroundServices() func()

Starts background services like config fetching

func (*Flashlight) Stop

func (f *Flashlight) Stop() error

Stops the local proxy

type HandledErrorType

type HandledErrorType int

HandledErrorType is used to differentiate error types to handlers configured via Flashlight.SetErrorHandler.

const (
	ErrorTypeProxySaveFailure  HandledErrorType = iota
	ErrorTypeConfigSaveFailure HandledErrorType = iota
)

func (HandledErrorType) String

func (t HandledErrorType) String() string

type Option

type Option func(*Flashlight)

func WithInit

func WithInit(onInit func()) Option

WithInit sets the callback that is called when flashlight is ready and has a config or needs to be initialized

func WithOnConfig

func WithOnConfig(onConfigUpdate func(*config.Global, config.Source)) Option

WithOnConfig sets the callback that is called when a config is successfully fetched

func WithOnDialError

func WithOnDialError(onDialError func(error, bool)) Option

WithOnDialError sets the callback that is called when an error occurs dialing a proxy. It includes the error itself and whether or not we have any successful dialers

func WithOnProxies

func WithOnProxies(onProxiesUpdate func([]bandit.Dialer, config.Source)) Option

WithOnProxies sets the callback when new proxies are received

func WithOnSucceedingProxy

func WithOnSucceedingProxy(onSucceedingProxy func()) Option

WithOnSucceedingProxy sets the onSucceedingProxy callback

type ProxyListener

type ProxyListener interface {
	OnProxies(map[string]*commonconfig.ProxyConfig)
}