Documentation ¶
Overview ¶
Package osl implements the Obfuscated Server List (OSL) mechanism. This mechanism is a method of distributing server lists only to clients that demonstrate certain behavioral traits. Clients are seeded with Server List Obfuscation Keys (SLOKs) as they meet the configured criteria. These keys are stored and later combined to assemble keys to decrypt out-of-band distributed OSL files that contain server lists.
This package contains the core routines used in psiphond (to track client traits and issue SLOKs), clients (to manage SLOKs and decrypt OSLs), and automation (to create OSLs for distribution).
Index ¶
- Constants
- func GetOSLFileURL(baseURL string, oslID []byte) string
- func GetOSLFilename(baseDirectory string, oslID []byte) string
- func GetOSLRegistryFilename(baseDirectory string) string
- func GetOSLRegistryURL(baseURL string) string
- func NewOSLReader(oslFileContent io.ReadSeeker, fileSpec *OSLFileSpec, lookup SLOKLookup, ...) (io.Reader, error)
- type ClientSeedPortForward
- type ClientSeedProgress
- type ClientSeedState
- func (state *ClientSeedState) ClearSeedPayload()
- func (state *ClientSeedState) GetSeedPayload() *SeedPayload
- func (state *ClientSeedState) Hibernate()
- func (state *ClientSeedState) NewClientSeedPortForward(upstreamIPAddress net.IP) *ClientSeedPortForward
- func (state *ClientSeedState) Resume(signalIssueSLOKs chan struct{})
- type Config
- func (config *Config) CurrentOSLIDs(schemeIndex int) (map[string]string, error)
- func (config *Config) NewClientSeedState(clientRegion, propagationChannelID string, signalIssueSLOKs chan struct{}) *ClientSeedState
- func (config *Config) Pave(startTime time.Time, endTime time.Time, propagationChannelID string, ...) ([]*PaveFile, error)
- type KeyShares
- type KeySplit
- type OSLFileSpec
- type PaveFile
- type PaveLogInfo
- type Registry
- type RegistryStreamer
- type SLOK
- type SLOKLookup
- type Scheme
- type SeedPayload
- type SeedSpec
- type TrafficValues
Constants ¶
const ( KEY_LENGTH_BYTES = 32 REGISTRY_FILENAME = "osl-registry" OSL_FILENAME_FORMAT = "osl-%s" )
Variables ¶
This section is empty.
Functions ¶
func GetOSLFileURL ¶
GetOSLFileURL returns the URL for an OSL file. Once the client has determined, from GetSeededOSLIDs, which OSLs it has sufficiently seeded, it calls this to fetch the OSLs for download and decryption. Clients are responsible for tracking whether the remote file has changed or not before downloading.
func GetOSLFilename ¶
GetOSLFilename returns an appropriate filename for the resumable download destination for the OSL file.
func GetOSLRegistryFilename ¶
GetOSLRegistryFilename returns an appropriate filename for the resumable download destination for the OSL registry.
func GetOSLRegistryURL ¶
GetOSLRegistryURL returns the URL for an OSL registry. Clients call this when fetching the registry from out-of-band distribution sites. Clients are responsible for tracking whether the remote file has changed or not before downloading.
func NewOSLReader ¶ added in v1.0.5
func NewOSLReader( oslFileContent io.ReadSeeker, fileSpec *OSLFileSpec, lookup SLOKLookup, signingPublicKey string) (io.Reader, error)
NewOSLReader decrypts, authenticates and streams an OSL payload.
Types ¶
type ClientSeedPortForward ¶
type ClientSeedPortForward struct {
// contains filtered or unexported fields
}
ClientSeedPortForward map a client port forward, which is relaying traffic to a specific upstream address, to all seed state progress counters for SeedSpecs with subnets containing the upstream address. As traffic is relayed through the port forwards, the bytes transferred and duration count towards the progress of these SeedSpecs and associated SLOKs.
func (*ClientSeedPortForward) UpdateProgress ¶
func (portForward *ClientSeedPortForward) UpdateProgress( bytesRead, bytesWritten, durationNanoseconds int64)
UpdateProgress adds port forward bytes transferred and duration to all seed spec progresses associated with the port forward. If UpdateProgress is invoked after the SLOK time period has rolled over, any pending seeded SLOKs are issued and all progress is reset. UpdateProgress may be invoked concurrently by many psiphond port relay goroutines. The implementation of UpdateProgress prioritizes not blocking port forward relaying; a consequence of this lock-free design is that progress reported at the exact time of SLOK time period rollover may be dropped.
type ClientSeedProgress ¶
type ClientSeedProgress struct {
// contains filtered or unexported fields
}
ClientSeedProgress tracks client progress towards seeding SLOKs for a particular scheme.
type ClientSeedState ¶
type ClientSeedState struct {
// contains filtered or unexported fields
}
ClientSeedState tracks the progress of a client towards seeding SLOKs across all schemes the client qualifies for.
func (*ClientSeedState) ClearSeedPayload ¶
func (state *ClientSeedState) ClearSeedPayload()
ClearSeedPayload resets the accumulated SLOK payload (but not SLOK progress). psiphond calls this after the client has acknowledged receipt of a payload.
func (*ClientSeedState) GetSeedPayload ¶
func (state *ClientSeedState) GetSeedPayload() *SeedPayload
GetSeedPayload issues any pending SLOKs and returns the accumulated SLOKs for a given client. psiphond will calls this when it receives signalIssueSLOKs which is the trigger to check for new SLOKs. Note: caller must not modify the SLOKs in SeedPayload.SLOKs as these are shared data.
func (*ClientSeedState) Hibernate ¶
func (state *ClientSeedState) Hibernate()
Hibernate clears references to short-lived objects (currently, signalIssueSLOKs) so that a ClientSeedState can be stored for later resumption without blocking garbage collection of the short-lived objects.
The ClientSeedState will still hold references to its Config; the caller is responsible for discarding hibernated seed states when the config changes.
The caller should ensure that all ClientSeedPortForwards associated with this ClientSeedState are closed before hibernation.
func (*ClientSeedState) NewClientSeedPortForward ¶
func (state *ClientSeedState) NewClientSeedPortForward( upstreamIPAddress net.IP) *ClientSeedPortForward
NewClientSeedPortForward creates a new client port forward traffic progress tracker. Port forward progress reported to the ClientSeedPortForward is added to seed state progress for all seed specs containing upstreamIPAddress in their subnets. The return value will be nil when activity for upstreamIPAddress does not count towards any progress. NewClientSeedPortForward may be invoked concurrently by many psiphond port forward establishment goroutines.
func (*ClientSeedState) Resume ¶
func (state *ClientSeedState) Resume( signalIssueSLOKs chan struct{})
Resume resumes a hibernated ClientSeedState by resetting the required objects (currently, signalIssueSLOKs) cleared by Hibernate.
type Config ¶
type Config struct { common.ReloadableFile Schemes []*Scheme }
Config is an OSL configuration, which consists of a list of schemes. The Reload function supports hot reloading of rules data while the process is running.
func LoadConfig ¶
LoadConfig loads, validates, and initializes a JSON encoded OSL configuration.
func (*Config) CurrentOSLIDs ¶ added in v1.0.5
CurrentOSLIDs returns a mapping from each propagation channel ID in the specified scheme to the corresponding current time period, hex-encoded OSL ID.
func (*Config) NewClientSeedState ¶
func (config *Config) NewClientSeedState( clientRegion, propagationChannelID string, signalIssueSLOKs chan struct{}) *ClientSeedState
NewClientSeedState creates a new client seed state to track client progress towards seeding SLOKs. psiphond maintains one ClientSeedState for each connected client.
A signal is sent on signalIssueSLOKs when sufficient progress has been made that a new SLOK *may* be issued. psiphond will receive the signal and then call GetClientSeedPayload/IssueSLOKs to issue SLOKs, generate payload, and send to the client. The sender will not block sending to signalIssueSLOKs; the channel should be appropriately buffered.
func (*Config) Pave ¶
func (config *Config) Pave( startTime time.Time, endTime time.Time, propagationChannelID string, signingPublicKey string, signingPrivateKey string, paveServerEntries map[string][]string, omitMD5SumsSchemes []int, omitEmptyOSLsSchemes []int, logCallback func(*PaveLogInfo)) ([]*PaveFile, error)
Pave creates the full set of OSL files, for all schemes in the configuration, to be dropped in an out-of-band distribution site. Only OSLs for the propagation channel ID associated with the distribution site are paved. This function is used by automation.
The Name component of each file relates to the values returned by the client functions GetRegistryURL and GetOSLFileURL.
Pave returns a pave file for the entire registry of all OSLs from epoch to endTime, and a pave file for each OSL. paveServerEntries is a map from hex-encoded OSL IDs to server entries to pave into that OSL. When entries are found, OSL will contain those entries, newline separated. Otherwise the OSL will still be issued, but be empty (unless the scheme is in omitEmptyOSLsSchemes). The server entries are paved in string value sort order, ensuring that the OSL content remains constant as long as the same _set_ of server entries is input.
If startTime is specified and is after epoch, the pave file will contain OSLs for the first period at or after startTime.
As OSLs outside the epoch-endTime range will no longer appear in the registry, Pave is intended to be used to create the full set of OSLs for a distribution site; i.e., not incrementally.
Automation is responsible for consistently distributing server entries to OSLs in the case where OSLs are repaved in subsequent calls.
type KeyShares ¶
type KeyShares struct {}
KeyShares is a tree data structure which describes the key splits used to divide a secret key. BoxedShares are encrypted shares of the key, and #Threshold amount of decrypted BoxedShares are required to reconstruct the secret key. The keys for BoxedShares are either SLOKs (referenced by SLOK ID) or random keys that are themselves split as described in child KeyShares.
type KeySplit ¶
KeySplit defines a secret key splitting scheme where the secret is split into n (total) shares and any K (threshold) of N shares must be known to recostruct the split secret.
type OSLFileSpec ¶
An OSLFileSpec includes an ID which is used to reference the OSL file and describes the key splits used to divide the OSL file key along with the SLOKs required to reassemble those keys.
The MD5Sum field is a checksum of the contents of the OSL file to be used to skip redownloading previously downloaded files. MD5 is not cryptographically secure and this checksum is not relied upon for OSL verification. MD5 is used for compatibility with out-of-band distribution hosts.
type PaveFile ¶
PaveFile describes an OSL data file to be paved to an out-of-band distribution drop site. There are two types of files: a registry, which describes how to assemble keys for OSLs, and the encrypted OSL files.
type PaveLogInfo ¶
type Registry ¶
type Registry struct {
FileSpecs []*OSLFileSpec
}
Registry describes a set of OSL files.
type RegistryStreamer ¶ added in v1.0.5
type RegistryStreamer struct {
// contains filtered or unexported fields
}
RegistryStreamer authenticates and processes a JSON encoded OSL registry. The streamer processes the registry without loading the entire file into memory, parsing each OSL file spec in turn and returning those OSL file specs for which the client has sufficient SLOKs to reassemble the OSL key and decrypt.
At this stage, SLOK reassembly simply does SLOK ID lookups and threshold counting and does not derive keys for every OSL. This allows the client to defer key derivation until NewOSLReader for cases where it has not already imported the OSL.
The client's propagation channel ID is used implicitly: it determines the base URL used to download the registry and OSL files. If the client has seeded SLOKs from a propagation channel ID different than the one associated with its present base URL, they will not appear in the registry and not be used.
func NewRegistryStreamer ¶ added in v1.0.5
func NewRegistryStreamer( registryFileContent io.ReadSeeker, signingPublicKey string, lookup SLOKLookup) (*RegistryStreamer, error)
NewRegistryStreamer creates a new RegistryStreamer.
func (*RegistryStreamer) Next ¶ added in v1.0.5
func (s *RegistryStreamer) Next() (*OSLFileSpec, error)
Next returns the next OSL file spec that the client has sufficient SLOKs to decrypt. The client calls NewOSLReader with the file spec to process that OSL. Next returns nil at EOF.
type SLOK ¶
SLOK is a seeded SLOK issued to a client. The client will store the SLOK in its local database; look it up by ID when checking which OSLs it can reassemble keys for; and use the key material to reassemble OSL file keys.
type SLOKLookup ¶
SLOKLookup is a callback to lookup SLOK keys by ID.
type Scheme ¶
type Scheme struct { // Epoch is the start time of the scheme, the start time of the // first OSL and when SLOKs will first be issued. It must be // specified in UTC and must be a multiple of SeedPeriodNanoseconds. Epoch string // Regions is a list of client country codes this scheme applies to. // If empty, the scheme applies to all regions. Regions []string // PropagationChannelIDs is a list of client propagtion channel IDs // this scheme applies to. Propagation channel IDs are an input // to SLOK key derivation. PropagationChannelIDs []string // MasterKey is the base random key used for SLOK key derivation. It // must be unique for each scheme. It must be 32 random bytes, base64 // encoded. MasterKey []byte // SeedSpecs is the set of different client network activity patterns // that will result in issuing SLOKs. For a given time period, a distinct // SLOK is issued for each SeedSpec. // Duplicate subnets may appear in multiple SeedSpecs. SeedSpecs []*SeedSpec // SeedSpecThreshold is the threshold scheme for combining SLOKs to // decrypt an OSL. For any fixed time period, at least K (threshold) of // N (total) SLOKs from the N SeedSpecs must be seeded for a client to be // able to reassemble the OSL key. // Limitation: thresholds must be at least 2. SeedSpecThreshold int // SeedPeriodNanoseconds is the time period granularity of SLOKs. // New SLOKs are issued every SeedPeriodNanoseconds. Client progress // towards activity levels is reset at the end of each period. SeedPeriodNanoseconds int64 // KeySplits is the time period threshold scheme layered on top of the // SeedSpecThreshold scheme for combining SLOKs to decrypt an OSL. // There must be at least one level. For one level, any K (threshold) of // N (total) SeedSpec SLOK groups must be sufficiently seeded for a client // to be able to reassemble the OSL key. When an additional level is // specified, then K' of N' groups of N of K SeedSpec SLOK groups must be // sufficiently seeded. And so on. The first level in the list is the // lowest level. The time period for OSLs is determined by the totals in // the KeySplits. // // Example: // // SeedSpecs = <3 specs> // SeedSpecThreshold = 2 // SeedPeriodNanoseconds = 100,000,000 = 100 milliseconds // SeedPeriodKeySplits = [{10, 7}, {60, 5}] // // In these scheme, up to 3 distinct SLOKs, one per spec, are issued // every 100 milliseconds. // // Distinct OSLs are paved for every minute (60 seconds). Each OSL // key is split such that, for those 60 seconds, a client must seed // 2/3 spec SLOKs for 7 of 10 consecutive 100 ms. time periods within // a second, for any 5 of 60 seconds within the minute. // SeedPeriodKeySplits []KeySplit // contains filtered or unexported fields }
Scheme defines a OSL seeding and distribution strategy. SLOKs to decrypt OSLs are issued based on client network activity -- defined in the SeedSpecs -- and time. OSLs are created for periods of time and can be decrypted by clients that are seeded with a sufficient selection of SLOKs for that time period. Distribution of server entries to OSLs is delegated to automation.
func (*Scheme) GetOSLDuration ¶
GetOSLDuration returns the total time duration of an OSL, which is a function of the scheme's SeedPeriodNanoSeconds, the duration of a single SLOK, and the scheme's SeedPeriodKeySplits, the number of SLOKs associated with an OSL.
type SeedPayload ¶
type SeedPayload struct {
SLOKs []*SLOK
}
SeedPayload is the list of seeded SLOKs sent to a client.
type SeedSpec ¶
type SeedSpec struct { Description string ID []byte UpstreamSubnets []string Targets TrafficValues }
SeedSpec defines a client traffic pattern that results in a seeded SLOK. For each time period, a unique SLOK is issued to a client that meets the traffic levels specified in Targets. All upstream port forward traffic to UpstreamSubnets is counted towards the targets.
ID is a SLOK key derivation component and must be 32 random bytes, base64 encoded. UpstreamSubnets is a list of CIDRs. Description is not used; it's for JSON config file comments.
type TrafficValues ¶
type TrafficValues struct { BytesRead int64 BytesWritten int64 PortForwardDurationNanoseconds int64 }
TrafficValues defines a client traffic level that seeds a SLOK. BytesRead and BytesWritten are the minimum bytes transferred counts to seed a SLOK. Both UDP and TCP data will be counted towards these totals. PortForwardDurationNanoseconds is the duration that a TCP or UDP port forward is active (not connected, in the UDP case). All threshold settings must be met to seed a SLOK; any threshold may be set to 0 to be trivially satisfied.