server

package
v1.0.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 21, 2018 License: GPL-3.0 Imports: 58 Imported by: 0

Documentation

Overview

Package server implements the core tunnel functionality of a Psiphon server. The main function is RunServices, which runs one or all of a Psiphon API web server, a tunneling SSH server, and an Obfuscated SSH protocol server. The server configuration is created by the GenerateConfig function.

Index

Constants

View Source 
const (
	MAX_API_PARAMS_SIZE = 256 * 1024 // 256KB
	PADDING_MAX_BYTES   = 16 * 1024

	CLIENT_PLATFORM_ANDROID = "Android"
	CLIENT_PLATFORM_WINDOWS = "Windows"
	CLIENT_PLATFORM_IOS     = "iOS"
)
View Source 
const (
	SERVER_CONFIG_FILENAME               = "psiphond.config"
	SERVER_TRAFFIC_RULES_CONFIG_FILENAME = "psiphond-traffic-rules.config"
	SERVER_OSL_CONFIG_FILENAME           = "psiphond-osl.config"
	SERVER_TACTICS_CONFIG_FILENAME       = "psiphond-tactics.config"
	SERVER_ENTRY_FILENAME                = "server-entry.dat"
	DEFAULT_SERVER_IP_ADDRESS            = "127.0.0.1"
	WEB_SERVER_SECRET_BYTE_LENGTH        = 32
	DISCOVERY_VALUE_KEY_BYTE_LENGTH      = 32
	SSH_USERNAME_SUFFIX_BYTE_LENGTH      = 8
	SSH_PASSWORD_BYTE_LENGTH             = 32
	SSH_RSA_HOST_KEY_BITS                = 2048
	SSH_OBFUSCATED_KEY_BYTE_LENGTH       = 32
)
View Source 
const (
	DNS_SYSTEM_CONFIG_FILENAME      = "/etc/resolv.conf"
	DNS_SYSTEM_CONFIG_RELOAD_PERIOD = 5 * time.Second
	DNS_RESOLVER_PORT               = 53
)
View Source 
const (
	GEOIP_SESSION_CACHE_TTL = 60 * time.Minute
	GEOIP_UNKNOWN_VALUE     = "None"
)
View Source 
const (

	// Protocol version 1 clients can handle arbitrary length response bodies. Older clients
	// report no version number and expect at most 64K response bodies.
	MEEK_PROTOCOL_VERSION_1 = 1

	// Protocol version 2 clients initiate a session by sending an encrypted and obfuscated meek
	// cookie with their initial HTTP request. Connection information is contained within the
	// encrypted cookie payload. The server inspects the cookie and establishes a new session and
	// returns a new random session ID back to client via Set-Cookie header. The client uses this
	// session ID on all subsequent requests for the remainder of the session.
	MEEK_PROTOCOL_VERSION_2 = 2

	// Protocol version 3 clients include resiliency enhancements and will add a Range header
	// when retrying a request for a partially downloaded response payload.
	MEEK_PROTOCOL_VERSION_3 = 3

	MEEK_MAX_REQUEST_PAYLOAD_LENGTH     = 65536
	MEEK_TURN_AROUND_TIMEOUT            = 20 * time.Millisecond
	MEEK_EXTENDED_TURN_AROUND_TIMEOUT   = 100 * time.Millisecond
	MEEK_MAX_SESSION_STALENESS          = 45 * time.Second
	MEEK_HTTP_CLIENT_IO_TIMEOUT         = 45 * time.Second
	MEEK_MIN_SESSION_ID_LENGTH          = 8
	MEEK_MAX_SESSION_ID_LENGTH          = 20
	MEEK_DEFAULT_RESPONSE_BUFFER_LENGTH = 65536
	MEEK_DEFAULT_POOL_BUFFER_LENGTH     = 65536
	MEEK_DEFAULT_POOL_BUFFER_COUNT      = 2048
)
View Source 
const (
	DEFAULT_IDLE_TCP_PORT_FORWARD_TIMEOUT_MILLISECONDS        = 30000
	DEFAULT_IDLE_UDP_PORT_FORWARD_TIMEOUT_MILLISECONDS        = 30000
	DEFAULT_DIAL_TCP_PORT_FORWARD_TIMEOUT_MILLISECONDS        = 10000
	DEFAULT_MAX_TCP_DIALING_PORT_FORWARD_COUNT                = 64
	DEFAULT_MAX_TCP_PORT_FORWARD_COUNT                        = 512
	DEFAULT_MAX_UDP_PORT_FORWARD_COUNT                        = 32
	DEFAULT_MEEK_RATE_LIMITER_GARBAGE_COLLECTOR_TRIGGER_COUNT = 5000
	DEFAULT_MEEK_RATE_LIMITER_REAP_HISTORY_FREQUENCY_SECONDS  = 600
)
View Source 
const (
	SSH_AUTH_LOG_PERIOD                   = 30 * time.Minute
	SSH_HANDSHAKE_TIMEOUT                 = 30 * time.Second
	SSH_BEGIN_HANDSHAKE_TIMEOUT           = 1 * time.Second
	SSH_CONNECTION_READ_DEADLINE          = 5 * time.Minute
	SSH_TCP_PORT_FORWARD_COPY_BUFFER_SIZE = 8192
	SSH_TCP_PORT_FORWARD_QUEUE_SIZE       = 1024
	SSH_KEEP_ALIVE_PAYLOAD_MIN_BYTES      = 0
	SSH_KEEP_ALIVE_PAYLOAD_MAX_BYTES      = 256
	SSH_SEND_OSL_INITIAL_RETRY_DELAY      = 30 * time.Second
	SSH_SEND_OSL_RETRY_FACTOR             = 2
	OSL_SESSION_CACHE_TTL                 = 5 * time.Minute
	MAX_AUTHORIZATIONS                    = 16
	PRE_HANDSHAKE_RANDOM_STREAM_MAX_COUNT = 1
	RANDOM_STREAM_MAX_BYTES               = 10485760
)
View Source 
const WEB_SERVER_IO_TIMEOUT = 10 * time.Second

Variables

This section is empty.

Functions

func CommonLogger added in v1.0.5 

func CommonLogger(contextLogger *ContextLogger) *commonLogger

CommonLogger wraps a ContextLogger instance with an interface that conforms to common.Logger. This is used to make the ContextLogger available to other packages that don't import the "server" package.

func GenerateConfig 

func GenerateConfig(params *GenerateConfigParams) ([]byte, []byte, []byte, []byte, []byte, error)

GenerateConfig creates a new Psiphon server config. It returns JSON encoded configs and a client-compatible "server entry" for the server. It generates all necessary secrets and key material, which are emitted in the config file and server entry as necessary.

GenerateConfig uses sample values for many fields. The intention is for generated configs to be used for testing or as examples for production setup, not to generate production-ready configurations.

When tactics key material is provided in GenerateConfigParams, tactics capabilities are added for all meek protocols in TunnelProtocolPorts.

func InitLogging 

func InitLogging(config *Config) (retErr error)

InitLogging configures a logger according to the specified config params. If not called, the default logger set by the package init() is used. Concurrency notes: this should only be called from the main goroutine; InitLogging only has effect on the first call, as the logging facilities it initializes may be in use by other goroutines after that point.

func NewIntentionalPanicError 

func NewIntentionalPanicError(errorMessage string) error

NewIntentionalPanicError creates a new IntentionalPanicError.

func NewLogWriter 

func NewLogWriter() *io.PipeWriter

NewLogWriter returns an io.PipeWriter that can be used to write to the global logger. Caller must Close() the writer.

func RunServices 

func RunServices(configJSON []byte) error

RunServices initializes support functions including logging and GeoIP services; and then starts the server components and runs them until os.Interrupt or os.Kill signals are received. The config determines which components are run.

func RunWebServer 

func RunWebServer(
	support *SupportServices,
	shutdownBroadcast <-chan struct{}) error

RunWebServer runs a web server which supports tunneled and untunneled Psiphon API requests.

The HTTP request handlers are light wrappers around the base Psiphon API request handlers from the SSH API transport. The SSH API transport is preferred by new clients. The web API transport provides support for older clients.

The API is compatible with all tunnel-core clients but not backwards compatible with all legacy clients.

Note: new features, including authorizations, are not supported in the web API transport.

Types

type CachedResponse 

type CachedResponse struct {
	// contains filtered or unexported fields
}

CachedResponse is a data structure that supports meek protocol connection interruption resiliency: it stores payload data from the most recent response so that it may be resent if the client fails to receive it.

The meek server maintains one CachedResponse for each meek client. Psiphon's variant of meek streams response data, so responses are not fixed size. To limit the memory overhead of response caching, each CachedResponse has a fixed-size buffer that operates as a ring buffer, discarding older response bytes when the buffer fills. A CachedResponse that has discarded data may still satisfy a client retry where the client has already received part of the response payload.

A CachedResponse will also extend its capacity by borrowing buffers from a CachedResponseBufferPool, if available. When Reset is called, borrowed buffers are released back to the pool.

func NewCachedResponse 

func NewCachedResponse(
	bufferSize int,
	extendedBufferPool *CachedResponseBufferPool) *CachedResponse

NewCachedResponse creates a CachedResponse with a fixed buffer of size bufferSize and borrowing buffers from extendedBufferPool.

func (*CachedResponse) Available 

func (response *CachedResponse) Available() int

Available returns the size of the buffered response data.

func (*CachedResponse) CopyFromPosition 

func (response *CachedResponse) CopyFromPosition(
	position int, writer io.Writer) (int, error)

CopyFromPosition writes the response data, starting at the specified position, to writer. Any data before the position is skipped. CopyFromPosition will return an error if the specified position is not available. CopyFromPosition will copy no data and return no error if the position is at the end of its available data. CopyFromPosition can be called repeatedly to read the same data -- it does not advance or modify the CachedResponse.

func (*CachedResponse) HasPosition 

func (response *CachedResponse) HasPosition(position int) bool

HasPosition checks if the CachedResponse has buffered response data starting at or before the specified position.

func (*CachedResponse) Reset 

func (response *CachedResponse) Reset()

Reset reinitializes the CachedResponse state to have no buffered response and releases all extended buffers back to the pool. Reset _must_ be called before discarding a CachedResponse or extended buffers will not be released.

func (*CachedResponse) Write 

func (response *CachedResponse) Write(data []byte) (int, error)

Write appends data to the CachedResponse. All writes will succeed, but only the most recent bytes will be retained once the fixed buffer is full and no extended buffers are available.

Write may be called multiple times to record a single response; Reset should be called between responses.

Write conforms to the io.Writer interface.

type CachedResponseBufferPool 

type CachedResponseBufferPool struct {
	// contains filtered or unexported fields
}

CachedResponseBufferPool is a fixed-size pool of fixed-size buffers that are used to temporarily extend the capacity of CachedResponses.

func NewCachedResponseBufferPool 

func NewCachedResponseBufferPool(
	bufferSize, bufferCount int) *CachedResponseBufferPool

NewCachedResponseBufferPool creates a new CachedResponseBufferPool with the specified number of buffers. Buffers are allocated on demand and once allocated remain allocated.

func (*CachedResponseBufferPool) Get 

func (pool *CachedResponseBufferPool) Get() []byte

Get returns a buffer, if one is available, or returns nil when no buffer is available. Get does not block. Call Put to release the buffer back to the pool.

Note: currently, Buffers are not zeroed between use by different CachedResponses owned by different clients. A bug resulting in cross-client data transfer exposes only OSSH ciphertext in the case of meek's use of CachedResponses.

func (*CachedResponseBufferPool) Put 

func (pool *CachedResponseBufferPool) Put(buffer []byte)

Put releases a buffer back to the pool. The buffer must have been obtained from Get.

type Config 

type Config struct {

	// LogLevel specifies the log level. Valid values are:
	// panic, fatal, error, warn, info, debug
	LogLevel string

	// LogFilename specifies the path of the file to log
	// to. When blank, logs are written to stderr.
	LogFilename string

	// SkipPanickingLogWriter disables panicking when
	// unable to write any logs.
	SkipPanickingLogWriter bool

	// DiscoveryValueHMACKey is the network-wide secret value
	// used to determine a unique discovery strategy.
	DiscoveryValueHMACKey string

	// GeoIPDatabaseFilenames are paths of GeoIP2/GeoLite2
	// MaxMind database files. When empty, no GeoIP lookups are
	// performed. Each file is queried, in order, for the
	// logged fields: country code, city, and ISP. Multiple
	// file support accommodates the MaxMind distribution where
	// ISP data in a separate file.
	GeoIPDatabaseFilenames []string

	// PsinetDatabaseFilename is the path of the Psiphon automation
	// jsonpickle format Psiphon API data file.
	PsinetDatabaseFilename string

	// HostID is the ID of the server host; this is used for API
	// event logging.
	HostID string

	// ServerIPAddress is the public IP address of the server.
	ServerIPAddress string

	// WebServerPort is the listening port of the web server.
	// When <= 0, no web server component is run.
	WebServerPort int

	// WebServerSecret is the unique secret value that the client
	// must supply to make requests to the web server.
	WebServerSecret string

	// WebServerCertificate is the certificate the client uses to
	// authenticate the web server.
	WebServerCertificate string

	// WebServerPrivateKey is the private key the web server uses to
	// authenticate itself to clients.
	WebServerPrivateKey string

	// WebServerPortForwardAddress specifies the expected network
	// address ("<host>:<port>") specified in a client's port forward
	// HostToConnect and PortToConnect when the client is making a
	// tunneled connection to the web server. This address is always
	// exempted from validation against SSH_DISALLOWED_PORT_FORWARD_HOSTS
	// and AllowTCPPorts.
	WebServerPortForwardAddress string

	// WebServerPortForwardRedirectAddress specifies an alternate
	// destination address to be substituted and dialed instead of
	// the original destination when the port forward destination is
	// WebServerPortForwardAddress.
	WebServerPortForwardRedirectAddress string

	// TunnelProtocolPorts specifies which tunnel protocols to run
	// and which ports to listen on for each protocol. Valid tunnel
	// protocols include:
	// "SSH", "OSSH", "UNFRONTED-MEEK-OSSH", "UNFRONTED-MEEK-HTTPS-OSSH",
	// "UNFRONTED-MEEK-SESSION-TICKET-OSSH", "FRONTED-MEEK-OSSH",
	// "FRONTED-MEEK-HTTP-OSSH", "QUIC-OSSH", "MARIONETTE-OSSH", and
	// "TAPDANCE-OSSH".
	//
	// In the case of "MARIONETTE-OSSH" the port value is ignored and must be
	// set to 0. The port value specified in the Marionette format is used.
	TunnelProtocolPorts map[string]int

	// SSHPrivateKey is the SSH host key. The same key is used for
	// all protocols, run by this server instance, which use SSH.
	SSHPrivateKey string

	// SSHServerVersion is the server version presented in the
	// identification string. The same value is used for all
	// protocols, run by this server instance, which use SSH.
	SSHServerVersion string

	// SSHUserName is the SSH user name to be presented by the
	// the tunnel-core client. The same value is used for all
	// protocols, run by this server instance, which use SSH.
	SSHUserName string

	// SSHPassword is the SSH password to be presented by the
	// the tunnel-core client. The same value is used for all
	// protocols, run by this server instance, which use SSH.
	SSHPassword string

	// ObfuscatedSSHKey is the secret key for use in the Obfuscated
	// SSH protocol. The same secret key is used for all protocols,
	// run by this server instance, which use Obfuscated SSH.
	ObfuscatedSSHKey string

	// MeekCookieEncryptionPrivateKey is the NaCl private key used
	// to decrypt meek cookie payload sent from clients. The same
	// key is used for all meek protocols run by this server instance.
	MeekCookieEncryptionPrivateKey string

	// MeekObfuscatedKey is the secret key used for obfuscating
	// meek cookies sent from clients. The same key is used for all
	// meek protocols run by this server instance.
	MeekObfuscatedKey string

	// MeekProhibitedHeaders is a list of HTTP headers to check for
	// in client requests. If one of these headers is found, the
	// request fails. This is used to defend against abuse.
	MeekProhibitedHeaders []string

	// MeekProxyForwardedForHeaders is a list of HTTP headers which
	// may be added by downstream HTTP proxies or CDNs in front
	// of clients. These headers supply the original client IP
	// address, which is geolocated for stats purposes. Headers
	// include, for example, X-Forwarded-For. The header's value
	// is assumed to be a comma delimted list of IP addresses where
	// the client IP is the first IP address in the list. Meek protocols
	// look for these headers and use the client IP address from
	// the header if any one is present and the value is a valid
	// IP address; otherwise the direct connection remote address is
	// used as the client IP.
	MeekProxyForwardedForHeaders []string

	// MeekCachedResponseBufferSize is the size of a private,
	// fixed-size buffer allocated for every meek client. The buffer
	// is used to cache response payload, allowing the client to retry
	// fetching when a network connection is interrupted. This retry
	// makes the OSSH tunnel within meek resilient to interruptions
	// at the HTTP TCP layer.
	// Larger buffers increase resiliency to interruption, but consume
	// more memory as buffers as never freed. The maximum size of a
	// response payload is a function of client activity, network
	// throughput and throttling.
	// A default of 64K is used when MeekCachedResponseBufferSize is 0.
	MeekCachedResponseBufferSize int

	// MeekCachedResponsePoolBufferSize is the size of a fixed-size,
	// shared buffer used to temporarily extend a private buffer when
	// MeekCachedResponseBufferSize is insufficient. Shared buffers
	// allow some clients to successfully retry longer response payloads
	// without allocating large buffers for all clients.
	// A default of 64K is used when MeekCachedResponsePoolBufferSize
	// is 0.
	MeekCachedResponsePoolBufferSize int

	// MeekCachedResponsePoolBufferCount is the number of shared
	// buffers. Shared buffers are allocated on first use and remain
	// allocated, so shared buffer count * size is roughly the memory
	// overhead of this facility.
	// A default of 2048 is used when MeekCachedResponsePoolBufferCount
	// is 0.
	MeekCachedResponsePoolBufferCount int

	// UDPInterceptUdpgwServerAddress specifies the network address of
	// a udpgw server which clients may be port forwarding to. When
	// specified, these TCP port forwards are intercepted and handled
	// directly by this server, which parses the SSH channel using the
	// udpgw protocol. Handling includes udpgw transparent DNS: tunneled
	// UDP DNS packets are rerouted to the host's DNS server.
	//
	// The intercept is applied before the port forward destination is
	// validated against SSH_DISALLOWED_PORT_FORWARD_HOSTS and
	// AllowTCPPorts. So the intercept address may be any otherwise
	// prohibited destination.
	UDPInterceptUdpgwServerAddress string

	// DNSResolverIPAddress specifies the IP address of a DNS server
	// to be used when "/etc/resolv.conf" doesn't exist or fails to
	// parse. When blank, "/etc/resolv.conf" must contain a usable
	// "nameserver" entry.
	DNSResolverIPAddress string

	// LoadMonitorPeriodSeconds indicates how frequently to log server
	// load information (number of connected clients per tunnel protocol,
	// number of running goroutines, amount of memory allocated, etc.)
	// The default, 0, disables load logging.
	LoadMonitorPeriodSeconds int

	// ProcessProfileOutputDirectory is the path of a directory to which
	// process profiles will be written when signaled with SIGUSR2. The
	// files are overwritten on each invocation. When set to the default
	// value, blank, no profiles are written on SIGUSR2. Profiles include
	// the default profiles here: https://golang.org/pkg/runtime/pprof/#Profile.
	ProcessProfileOutputDirectory string

	// ProcessBlockProfileDurationSeconds specifies the sample duration for
	// "block" profiling. For the default, 0, no "block" profile is taken.
	ProcessBlockProfileDurationSeconds int

	// ProcessCPUProfileDurationSeconds specifies the sample duration for
	// CPU profiling. For the default, 0, no CPU profile is taken.
	ProcessCPUProfileDurationSeconds int

	// TrafficRulesFilename is the path of a file containing a JSON-encoded
	// TrafficRulesSet, the traffic rules to apply to Psiphon client tunnels.
	TrafficRulesFilename string

	// OSLConfigFilename is the path of a file containing a JSON-encoded
	// OSL Config, the OSL schemes to apply to Psiphon client tunnels.
	OSLConfigFilename string

	// RunPacketTunnel specifies whether to run a packet tunnel.
	RunPacketTunnel bool

	// PacketTunnelEgressInterface specifies tun.ServerConfig.EgressInterface.
	PacketTunnelEgressInterface string

	// PacketTunnelDownstreamPacketQueueSize specifies
	// tun.ServerConfig.DownStreamPacketQueueSize.
	PacketTunnelDownstreamPacketQueueSize int

	// PacketTunnelSessionIdleExpirySeconds specifies
	// tun.ServerConfig.SessionIdleExpirySeconds.
	PacketTunnelSessionIdleExpirySeconds int

	// PacketTunnelSudoNetworkConfigCommands sets
	// tun.ServerConfig.SudoNetworkConfigCommands.
	PacketTunnelSudoNetworkConfigCommands bool

	// MaxConcurrentSSHHandshakes specifies a limit on the number of concurrent
	// SSH handshake negotiations. This is set to mitigate spikes in memory
	// allocations and CPU usage associated with SSH handshakes when many clients
	// attempt to connect concurrently. When a maximum limit is specified and
	// reached, additional clients that establish TCP or meek connections will
	// be disconnected after a short wait for the number of concurrent handshakes
	// to drop below the limit.
	// The default, 0 is no limit.
	MaxConcurrentSSHHandshakes int

	// PeriodicGarbageCollectionSeconds turns on periodic calls to runtime.GC,
	// every specified number of seconds, to force garbage collection.
	// The default, 0 is off.
	PeriodicGarbageCollectionSeconds int

	// AccessControlVerificationKeyRing is the access control authorization
	// verification key ring used to verify signed authorizations presented
	// by clients. Verified, active (unexpired) access control types will be
	// available for matching in the TrafficRulesFilter for the client via
	// AuthorizedAccessTypes. All other authorizations are ignored.
	AccessControlVerificationKeyRing accesscontrol.VerificationKeyRing

	// TacticsConfigFilename is the path of a file containing a JSON-encoded
	// tactics server configuration.
	TacticsConfigFilename string

	// MarionetteFormat specifies a Marionette format to use with the
	// MARIONETTE-OSSH tunnel protocol. The format specifies the network
	// protocol port to listen on.
	MarionetteFormat string
}

Config specifies the configuration and behavior of a Psiphon server.

func LoadConfig 

func LoadConfig(configJSON []byte) (*Config, error)

LoadConfig loads and validates a JSON encoded server config.

func (*Config) RunLoadMonitor 

func (config *Config) RunLoadMonitor() bool

RunLoadMonitor indicates whether to monitor and log server load.

func (*Config) RunPeriodicGarbageCollection added in v1.0.5 

func (config *Config) RunPeriodicGarbageCollection() bool

RunPeriodicGarbageCollection indicates whether to run periodic garbage collection.

func (*Config) RunWebServer 

func (config *Config) RunWebServer() bool

RunWebServer indicates whether to run a web server component.

type ContextLogger 

type ContextLogger struct {
	*logrus.Logger
}

ContextLogger adds context logging functionality to the underlying logging packages.

func (*ContextLogger) LogPanicRecover 

func (logger *ContextLogger) LogPanicRecover(recoverValue interface{}, stack []byte)

LogPanicRecover calls LogRawFieldsWithTimestamp with standard fields for logging recovered panics.

func (*ContextLogger) LogRawFieldsWithTimestamp 

func (logger *ContextLogger) LogRawFieldsWithTimestamp(fields LogFields)

LogRawFieldsWithTimestamp directly logs the supplied fields adding only an additional "timestamp" field; and "host_id" and "build_rev" fields identifying this server and build. The stock "msg" and "level" fields are omitted. This log is emitted at the Error level. This function exists to support API logs which have neither a natural message nor severity; and omitting these values here makes it easier to ship these logs to existing API log consumers. Note that any existing "context"/"host_id"/"build_rev" field will be renamed to "field.<name>".

func (*ContextLogger) WithContext 

func (logger *ContextLogger) WithContext() *logrus.Entry

WithContext adds a "context" field containing the caller's function name and source file line number; and "host_id" and "build_rev" fields identifying this server and build. Use this function when the log has no fields.

func (*ContextLogger) WithContextFields 

func (logger *ContextLogger) WithContextFields(fields LogFields) *logrus.Entry

WithContextFields adds a "context" field containing the caller's function name and source file line number; and "host_id" and "build_rev" fields identifying this server and build. Use this function when the log has fields. Note that any existing "context"/"host_id"/"build_rev" field will be renamed to "field.<name>".

type CustomJSONFormatter 

type CustomJSONFormatter struct {
}

CustomJSONFormatter is a customized version of logrus.JSONFormatter

func (*CustomJSONFormatter) Format 

func (f *CustomJSONFormatter) Format(entry *logrus.Entry) ([]byte, error)

Format implements logrus.Formatter. This is a customized version of the standard logrus.JSONFormatter adapted from: https://github.com/Sirupsen/logrus/blob/f1addc29722ba9f7651bc42b4198d0944b66e7c4/json_formatter.go

The changes are: - "time" is renamed to "timestamp" - there's an option to omit the standard "msg" and "level" fields

type DNSResolver 

type DNSResolver struct {
	common.ReloadableFile
	// contains filtered or unexported fields
}

DNSResolver maintains fresh DNS resolver values, monitoring "/etc/resolv.conf" on platforms where it is available; and otherwise using a default value.

func NewDNSResolver 

func NewDNSResolver(defaultResolver string) (*DNSResolver, error)

NewDNSResolver initializes a new DNSResolver, loading it with fresh resolver values. The load must succeed, so either "/etc/resolv.conf" must contain valid "nameserver" lines with a DNS server IP address, or a valid "defaultResolver" default value must be provided. On systems without "/etc/resolv.conf", "defaultResolver" is required.

The resolver is considered stale and reloaded if last checked more than 5 seconds before the last Get(), which is similar to frequencies in other implementations:

func (*DNSResolver) Get 

func (dns *DNSResolver) Get() net.IP

Get returns one of the cached resolvers, selected at random, after first updating the cached values if they're stale. If reloading fails, the previous values are used.

Randomly selecting any one of the configured resolvers is expected to be more resiliant to failure; e.g., if one of the resolvers becomes unavailable.

func (*DNSResolver) GetAllIPv4 added in v1.0.5 

func (dns *DNSResolver) GetAllIPv4() []net.IP

GetAllIPv4 returns a list of all IPv4 DNS resolver addresses. Cached values are updated if they're stale. If reloading fails, the previous values are used.

func (*DNSResolver) GetAllIPv6 added in v1.0.5 

func (dns *DNSResolver) GetAllIPv6() []net.IP

GetAllIPv6 returns a list of all IPv6 DNS resolver addresses. Cached values are updated if they're stale. If reloading fails, the previous values are used.

type GenerateConfigParams 

type GenerateConfigParams struct {
	LogFilename                 string
	SkipPanickingLogWriter      bool
	LogLevel                    string
	ServerIPAddress             string
	WebServerPort               int
	EnableSSHAPIRequests        bool
	TunnelProtocolPorts         map[string]int
	MarionetteFormat            string
	TrafficRulesConfigFilename  string
	OSLConfigFilename           string
	TacticsConfigFilename       string
	TacticsRequestPublicKey     string
	TacticsRequestObfuscatedKey string
}

GenerateConfigParams specifies customizations to be applied to a generated server config.

type GeoIPData 

type GeoIPData struct {
	Country        string
	City           string
	ISP            string
	DiscoveryValue int
}

GeoIPData is GeoIP data for a client session. Individual client IP addresses are neither logged nor explicitly referenced during a session. The GeoIP country, city, and ISP corresponding to a client IP address are resolved and then logged along with usage stats. The DiscoveryValue is a special value derived from the client IP that's used to compartmentalize discoverable servers (see calculateDiscoveryValue for details).

func NewGeoIPData 

func NewGeoIPData() GeoIPData

NewGeoIPData returns a GeoIPData initialized with the expected GEOIP_UNKNOWN_VALUE values to be used when GeoIP lookup fails.

type GeoIPService 

type GeoIPService struct {
	// contains filtered or unexported fields
}

GeoIPService implements GeoIP lookup and session/GeoIP caching. Lookup is via a MaxMind database; the ReloadDatabase function supports hot reloading of MaxMind data while the server is running.

func NewGeoIPService 

func NewGeoIPService(
	databaseFilenames []string,
	discoveryValueHMACKey string) (*GeoIPService, error)

NewGeoIPService initializes a new GeoIPService.

func (*GeoIPService) GetSessionCache 

func (geoIP *GeoIPService) GetSessionCache(sessionID string) GeoIPData

GetSessionCache returns the cached GeoIPData for the specified session ID; a blank GeoIPData is returned if the session ID is not found in the cache.

func (*GeoIPService) InSessionCache added in v1.0.5 

func (geoIP *GeoIPService) InSessionCache(sessionID string) bool

InSessionCache returns whether the session ID is present in the session cache.

func (*GeoIPService) Lookup 

func (geoIP *GeoIPService) Lookup(ipAddress string) GeoIPData

Lookup determines a GeoIPData for a given client IP address.

func (*GeoIPService) MarkSessionCacheToExpire 

func (geoIP *GeoIPService) MarkSessionCacheToExpire(sessionID string)

MarkSessionCacheToExpire initiates expiry for an existing session cache entry, if the session ID is found in the cache. Concurrency note: SetSessionCache and MarkSessionCacheToExpire should not be called concurrently for a single session ID.

func (*GeoIPService) Reloaders 

func (geoIP *GeoIPService) Reloaders() []common.Reloader

Reloaders gets the list of reloadable databases in use by the GeoIPService. This list is used to hot reload these databases.

func (*GeoIPService) SetSessionCache 

func (geoIP *GeoIPService) SetSessionCache(sessionID string, geoIPData GeoIPData)

SetSessionCache adds the sessionID/geoIPData pair to the session cache. This value will not expire; the caller must call MarkSessionCacheToExpire to initiate expiry. Calling SetSessionCache for an existing sessionID will replace the previous value and reset any expiry.

type HTTPSServer 

type HTTPSServer struct {
	*http.Server
}

HTTPSServer is a wrapper around http.Server which adds the ServeTLS function.

func (*HTTPSServer) ServeTLS 

func (server *HTTPSServer) ServeTLS(listener net.Listener, config *tris.Config) error

ServeTLS is similar to http.Serve, but uses TLS.

The http package has both ListenAndServe and ListenAndServeTLS higher- level interfaces, but only Serve (not TLS) offers a lower-level interface that allows the caller to keep a refererence to the Listener, allowing for external shutdown. ListenAndServeTLS also requires the TLS cert and key to be in files and we avoid that here.

Note that the http.Server.TLSConfig field is ignored and the tris.Config parameter is used intead.

type IntentionalPanicError 

type IntentionalPanicError struct {
	// contains filtered or unexported fields
}

IntentionalPanicError is an error type that is used when calling panic() in a situation where recovers should propagate the panic.

func (IntentionalPanicError) Error 

func (err IntentionalPanicError) Error() string

Error implements the error interface.

type LogFields 

type LogFields logrus.Fields

LogFields is an alias for the field struct in the underlying logging package.

type MeekServer 

type MeekServer struct {
	// contains filtered or unexported fields
}

MeekServer implements the meek protocol, which tunnels TCP traffic (in the case of Psiphon, Obfuscated SSH traffic) over HTTP. Meek may be fronted (through a CDN) or direct and may be HTTP or HTTPS.

Upstream traffic arrives in HTTP request bodies and downstream traffic is sent in response bodies. The sequence of traffic for a given flow is associated using a session ID that's set as a HTTP cookie for the client to submit with each request.

MeekServer hooks into TunnelServer via the net.Conn interface by transforming the HTTP payload traffic for a given session into net.Conn conforming Read()s and Write()s via the meekConn struct.

func NewMeekServer 

func NewMeekServer(
	support *SupportServices,
	listener net.Listener,
	useTLS, isFronted, useObfuscatedSessionTickets bool,
	clientHandler func(clientTunnelProtocol string, clientConn net.Conn),
	stopBroadcast <-chan struct{}) (*MeekServer, error)

NewMeekServer initializes a new meek server.

func (*MeekServer) Run 

func (server *MeekServer) Run() error

Run runs the meek server; this function blocks while serving HTTP or HTTPS connections on the specified listener. This function also runs a goroutine which cleans up expired meek client sessions.

To stop the meek server, both Close() the listener and set the stopBroadcast signal specified in NewMeekServer.

func (*MeekServer) ServeHTTP 

func (server *MeekServer) ServeHTTP(responseWriter http.ResponseWriter, request *http.Request)

ServeHTTP handles meek client HTTP requests, where the request body contains upstream traffic and the response will contain downstream traffic.

type PanickingLogWriter 

type PanickingLogWriter struct {
	// contains filtered or unexported fields
}

PanickingLogWriter wraps an io.Writer and intentionally panics when a Write() fails.

func NewPanickingLogWriter 

func NewPanickingLogWriter(
	name string, writer io.Writer) *PanickingLogWriter

NewPanickingLogWriter creates a new PanickingLogWriter.

func (*PanickingLogWriter) Write 

func (w *PanickingLogWriter) Write(p []byte) (n int, err error)

Write implements the io.Writer interface.

type ProtocolStats 

type ProtocolStats map[string]map[string]int64

type RateLimits 

type RateLimits struct {
	ReadUnthrottledBytes  *int64
	ReadBytesPerSecond    *int64
	WriteUnthrottledBytes *int64
	WriteBytesPerSecond   *int64
	CloseAfterExhausted   *bool

	// UnthrottleFirstTunnelOnly specifies whether any
	// ReadUnthrottledBytes/WriteUnthrottledBytes apply
	// only to the first tunnel in a session.
	UnthrottleFirstTunnelOnly *bool
}

RateLimits is a clone of common.RateLimits with pointers to fields to enable distinguishing between zero values and omitted values in JSON serialized traffic rules. See common.RateLimits for field descriptions.

func (*RateLimits) CommonRateLimits 

func (rateLimits *RateLimits) CommonRateLimits() common.RateLimits

CommonRateLimits converts a RateLimits to a common.RateLimits.

type RegionStats 

type RegionStats map[string]map[string]map[string]int64

type SupportServices 

type SupportServices struct {
	Config             *Config
	TrafficRulesSet    *TrafficRulesSet
	OSLConfig          *osl.Config
	PsinetDatabase     *psinet.Database
	GeoIPService       *GeoIPService
	DNSResolver        *DNSResolver
	TunnelServer       *TunnelServer
	PacketTunnelServer *tun.Server
	TacticsServer      *tactics.Server
}

SupportServices carries common and shared data components across different server components. SupportServices implements a hot reload of traffic rules, psinet database, and geo IP database components, which allows these data components to be refreshed without restarting the server process.

func NewSupportServices 

func NewSupportServices(config *Config) (*SupportServices, error)

NewSupportServices initializes a new SupportServices.

func (*SupportServices) Reload 

func (support *SupportServices) Reload()

Reload reinitializes traffic rules, psinet database, and geo IP database components. If any component fails to reload, an error is logged and Reload proceeds, using the previous state of the component.

type TrafficRules 

type TrafficRules struct {

	// RateLimits specifies data transfer rate limits for the
	// client traffic.
	RateLimits RateLimits

	// DialTCPPortForwardTimeoutMilliseconds is the timeout period
	// for dialing TCP port forwards. A value of 0 specifies no timeout.
	// When omitted in DefaultRules,
	// DEFAULT_TCP_PORT_FORWARD_DIAL_TIMEOUT_MILLISECONDS is used.
	DialTCPPortForwardTimeoutMilliseconds *int

	// IdleTCPPortForwardTimeoutMilliseconds is the timeout period
	// after which idle (no bytes flowing in either direction)
	// client TCP port forwards are preemptively closed.
	// A value of 0 specifies no idle timeout. When omitted in
	// DefaultRules, DEFAULT_IDLE_TCP_PORT_FORWARD_TIMEOUT_MILLISECONDS
	// is used.
	IdleTCPPortForwardTimeoutMilliseconds *int

	// IdleUDPPortForwardTimeoutMilliseconds is the timeout period
	// after which idle (no bytes flowing in either direction)
	// client UDP port forwards are preemptively closed.
	// A value of 0 specifies no idle timeout. When omitted in
	// DefaultRules, DEFAULT_IDLE_UDP_PORT_FORWARD_TIMEOUT_MILLISECONDS
	// is used.
	IdleUDPPortForwardTimeoutMilliseconds *int

	// MaxTCPDialingPortForwardCount is the maximum number of dialing
	// TCP port forwards each client may have open concurrently. When
	// persistently at the limit, new TCP port forwards are rejected.
	// A value of 0 specifies no maximum. When omitted in
	// DefaultRules, DEFAULT_MAX_TCP_DIALING_PORT_FORWARD_COUNT is used.
	MaxTCPDialingPortForwardCount *int

	// MaxTCPPortForwardCount is the maximum number of established TCP
	// port forwards each client may have open concurrently. If at the
	// limit when a new TCP port forward is established, the LRU
	// established TCP port forward is closed.
	// A value of 0 specifies no maximum. When omitted in
	// DefaultRules, DEFAULT_MAX_TCP_PORT_FORWARD_COUNT is used.
	MaxTCPPortForwardCount *int

	// MaxUDPPortForwardCount is the maximum number of UDP port
	// forwards each client may have open concurrently. If at the
	// limit when a new UDP port forward is created, the LRU
	// UDP port forward is closed.
	// A value of 0 specifies no maximum. When omitted in
	// DefaultRules, DEFAULT_MAX_UDP_PORT_FORWARD_COUNT is used.
	MaxUDPPortForwardCount *int

	// AllowTCPPorts specifies a whitelist of TCP ports that
	// are permitted for port forwarding. When set, only ports
	// in the list are accessible to clients.
	AllowTCPPorts []int

	// AllowUDPPorts specifies a whitelist of UDP ports that
	// are permitted for port forwarding. When set, only ports
	// in the list are accessible to clients.
	AllowUDPPorts []int

	// AllowSubnets specifies a list of IP address subnets for
	// which all TCP and UDP ports are allowed. This list is
	// consulted if a port is disallowed by the AllowTCPPorts
	// or AllowUDPPorts configuration. Each entry is a IP subnet
	// in CIDR notation.
	// Limitation: currently, AllowSubnets only matches port
	// forwards where the client sends an IP address. Domain
	// names aren not resolved before checking AllowSubnets.
	AllowSubnets []string
}

TrafficRules specify the limits placed on client traffic.

type TrafficRulesFilter 

type TrafficRulesFilter struct {

	// TunnelProtocols is a list of client tunnel protocols that must be
	// in use to match this filter. When omitted or empty, any protocol
	// matches.
	TunnelProtocols []string

	// Regions is a list of countries that the client must geolocate to in
	// order to match this filter. When omitted or empty, any client country
	// matches.
	Regions []string

	// ISPs is a list of ISPs that the client must geolocate to in order to
	// match this filter. When omitted or empty, any client ISP matches.
	ISPs []string

	// APIProtocol specifies whether the client must use the SSH
	// API protocol (when "ssh") or the web API protocol (when "web").
	// When omitted or blank, any API protocol matches.
	APIProtocol string

	// HandshakeParameters specifies handshake API parameter names and
	// a list of values, one of which must be specified to match this
	// filter. Only scalar string API parameters may be filtered.
	// Values may be patterns containing the '*' wildcard.
	HandshakeParameters map[string][]string

	// AuthorizedAccessTypes specifies a list of access types, at least
	// one of which the client must have presented an active authorization
	// for and which must not be revoked.
	// AuthorizedAccessTypes is ignored when AuthorizationsRevoked is true.
	AuthorizedAccessTypes []string

	// AuthorizationsRevoked indicates whether the client's authorizations
	// must have been revoked. When true, authorizations must have been
	// revoked. When omitted or false, this field is ignored.
	AuthorizationsRevoked bool
}

TrafficRulesFilter defines a filter to match against client attributes.

type TrafficRulesSet 

type TrafficRulesSet struct {
	common.ReloadableFile

	// DefaultRules are the base values to use as defaults for all
	// clients.
	DefaultRules TrafficRules

	// FilteredTrafficRules is an ordered list of filter/rules pairs.
	// For each client, the first matching Filter in FilteredTrafficRules
	// determines the additional Rules that are selected and applied
	// on top of DefaultRules.
	FilteredRules []struct {
		Filter TrafficRulesFilter
		Rules  TrafficRules
	}

	// MeekRateLimiterHistorySize enables the late-stage meek rate limiter and
	// sets its history size. The late-stage meek rate limiter acts on client
	// IPs relayed in MeekProxyForwardedForHeaders, and so it must wait for
	// the HTTP headers to be read. This rate limiter immediately terminates
	// any client endpoint request or any request to create a new session, but
	// not any meek request for an existing session, if the
	// MeekRateLimiterHistorySize requests occur in
	// MeekRateLimiterThresholdSeconds. The scope of rate limiting may be
	// limited using LimitMeekRateLimiterRegions and LimitMeekRateLimiterISPs.
	//
	// Hot reloading a new history size will result in existing history being
	// truncated.
	MeekRateLimiterHistorySize int

	// MeekRateLimiterThresholdSeconds is part of the meek rate limiter
	// specification and must be set when MeekRateLimiterHistorySize is set.
	MeekRateLimiterThresholdSeconds int

	// MeekRateLimiterRegions, if set, limits application of the meek
	// late-stage rate limiter to clients in the specified list of GeoIP
	// countries. When omitted or empty, meek rate limiting, if configured,
	// is applied to all client countries.
	MeekRateLimiterRegions []string

	// MeekRateLimiterISPs, if set, limits application of the meek
	// late-stage rate limiter to clients in the specified list of GeoIP
	// ISPs. When omitted or empty, meek rate limiting, if configured,
	// is applied to all client ISPs.
	MeekRateLimiterISPs []string

	// MeekRateLimiterGarbageCollectionTriggerCount specifies the number of
	// rate limit events after which garbage collection is manually triggered
	// in order to reclaim memory used by rate limited and other rejected
	// requests.
	// A default of 5000 is used when
	// MeekRateLimiterGarbageCollectionTriggerCount is 0.
	MeekRateLimiterGarbageCollectionTriggerCount int

	// MeekRateLimiterReapHistoryFrequencySeconds specifies a schedule for
	// reaping old records from the rate limit history.
	// A default of 600 is used when
	// MeekRateLimiterReapHistoryFrequencySeconds is 0.
	MeekRateLimiterReapHistoryFrequencySeconds int
}

TrafficRulesSet represents the various traffic rules to apply to Psiphon client tunnels. The Reload function supports hot reloading of rules data while the server is running.

For a given client, the traffic rules are determined by starting with DefaultRules, then finding the first (if any) FilteredTrafficRules match and overriding the defaults with fields set in the selected FilteredTrafficRules.

func NewTrafficRulesSet 

func NewTrafficRulesSet(filename string) (*TrafficRulesSet, error)

NewTrafficRulesSet initializes a TrafficRulesSet with the rules data in the specified config file.

func (*TrafficRulesSet) GetMeekRateLimiterConfig added in v1.0.6 

func (set *TrafficRulesSet) GetMeekRateLimiterConfig() (int, int, []string, []string, int, int)

GetMeekRateLimiterConfig gets a snapshot of the meek rate limiter configuration values.

func (*TrafficRulesSet) GetTrafficRules 

func (set *TrafficRulesSet) GetTrafficRules(
	isFirstTunnelInSession bool,
	tunnelProtocol string,
	geoIPData GeoIPData,
	state handshakeState) TrafficRules

GetTrafficRules determines the traffic rules for a client based on its attributes. For the return value TrafficRules, all pointer and slice fields are initialized, so nil checks are not required. The caller must not modify the returned TrafficRules.

func (*TrafficRulesSet) Validate 

func (set *TrafficRulesSet) Validate() error

Validate checks for correct input formats in a TrafficRulesSet.

type TunnelServer 

type TunnelServer struct {
	// contains filtered or unexported fields
}

TunnelServer is the main server that accepts Psiphon client connections, via various obfuscation protocols, and provides port forwarding (TCP and UDP) services to the Psiphon client. At its core, TunnelServer is an SSH server. SSH is the base protocol that provides port forward multiplexing, and transport security. Layered on top of SSH, optionally, is Obfuscated SSH and meek protocols, which provide further circumvention capabilities.

func NewTunnelServer 

func NewTunnelServer(
	support *SupportServices,
	shutdownBroadcast <-chan struct{}) (*TunnelServer, error)

NewTunnelServer initializes a new tunnel server.

func (*TunnelServer) ExpectClientDomainBytes added in v1.0.5 

func (server *TunnelServer) ExpectClientDomainBytes(
	sessionID string) (bool, error)

ExpectClientDomainBytes indicates whether the client was configured to report domain bytes in its handshake response.

func (*TunnelServer) GetClientHandshaked 

func (server *TunnelServer) GetClientHandshaked(
	sessionID string) (bool, bool, error)

GetClientHandshaked indicates whether the client has completed a handshake and whether its traffic rules are immediately exhausted.

func (*TunnelServer) GetEstablishTunnels 

func (server *TunnelServer) GetEstablishTunnels() bool

GetEstablishTunnels returns whether new tunnels may be established or not.

func (*TunnelServer) GetLoadStats 

func (server *TunnelServer) GetLoadStats() (ProtocolStats, RegionStats)

GetLoadStats returns load stats for the tunnel server. The stats are broken down by protocol ("SSH", "OSSH", etc.) and type. Types of stats include current connected client count, total number of current port forwards.

func (*TunnelServer) ResetAllClientOSLConfigs 

func (server *TunnelServer) ResetAllClientOSLConfigs()

ResetAllClientOSLConfigs resets all established client OSL state to use the latest OSL config. Any existing OSL state is lost, including partial progress towards SLOKs.

func (*TunnelServer) ResetAllClientTrafficRules 

func (server *TunnelServer) ResetAllClientTrafficRules()

ResetAllClientTrafficRules resets all established client traffic rules to use the latest config and client properties. Any existing traffic rule state is lost, including throttling state.

func (*TunnelServer) Run 

func (server *TunnelServer) Run() error

Run runs the tunnel server; this function blocks while running a selection of listeners that handle connection using various obfuscation protocols.

Run listens on each designated tunnel port and spawns new goroutines to handle each client connection. It halts when shutdownBroadcast is signaled. A list of active clients is maintained, and when halting all clients are cleanly shutdown.

Each client goroutine handles its own obfuscation (optional), SSH handshake, SSH authentication, and then looping on client new channel requests. "direct-tcpip" channels, dynamic port fowards, are supported. When the UDPInterceptUdpgwServerAddress config parameter is configured, UDP port forwards over a TCP stream, following the udpgw protocol, are handled.

A new goroutine is spawned to handle each port forward for each client. Each port forward tracks its bytes transferred. Overall per-client stats for connection duration, GeoIP, number of port forwards, and bytes transferred are tracked and logged when the client shuts down.

Note: client handler goroutines may still be shutting down after Run() returns. See comment in sshClient.stop(). TODO: fully synchronized shutdown.

func (*TunnelServer) SetClientHandshakeState 

func (server *TunnelServer) SetClientHandshakeState(
	sessionID string,
	state handshakeState,
	authorizations []string) ([]string, []string, error)

SetClientHandshakeState sets the handshake state -- that it completed and what parameters were passed -- in sshClient. This state is used for allowing port forwards and for future traffic rule selection. SetClientHandshakeState also triggers an immediate traffic rule re-selection, as the rules selected upon tunnel establishment may no longer apply now that handshake values are set.

The authorizations received from the client handshake are verified and the resulting list of authorized access types are applied to the client's tunnel and traffic rules. A list of active authorization IDs and authorized access types is returned for responding to the client and logging.

func (*TunnelServer) SetEstablishTunnels 

func (server *TunnelServer) SetEstablishTunnels(establish bool)

SetEstablishTunnels sets whether new tunnels may be established or not. When not establishing, incoming connections are immediately closed.

func (*TunnelServer) UpdateClientAPIParameters added in v1.0.9 

func (server *TunnelServer) UpdateClientAPIParameters(
	sessionID string,
	apiParams common.APIParameters) error

UpdateClientAPIParameters updates the recorded handhake API parameters for the client corresponding to sessionID.

Source Files

View all Source files

Directories

Path Synopsis
Package psinet implements psinet database services.
 Package psinet implements psinet database services.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.