lib

package
v0.7.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 13, 2023 License: Apache-2.0 Imports: 32 Imported by: 1

Documentation

Index

Constants

View Source 
const AES_GCM_TAG_SIZE = 16

AES_GCM_TAG_SIZE the size of the aesgcm tag used when generating the client to station message.

View Source 
const DETECTOR_REG_CHANNEL string = "dark_decoy_map"

DETECTOR_REG_CHANNEL is a constant that defines the name of the redis map that we send validated registrations over in order to notify all detector cores.

View Source 
const PrivateKeyLength = 32

PrivateKeyLength is the expected length of the station (ed25519) private key in bytes.

Variables

This section is empty.

Functions

func Proxy 

func Proxy(reg *DecoyRegistration, clientConn net.Conn, logger *log.Logger)

Proxy take a registration and a net.Conn and forwards client traffic to the clients covert destination.

Types

type Config 

type Config struct {
	*ZMQConfig
	*RegConfig

	// Log verbosity level
	LogLevel string `toml:"log_level"`

	// Path to private key file
	PrivateKeyPath string `toml:"privkey_path"`

	// PrefixFilePath provides a path to a file containing supported prefix specifications for the
	// prefix transport.
	// [TODO] refactor into a more general transport config object
	PrefixFilePath         string `toml:"supplemental_prefix_path"`
	DisableDefaultPrefixes bool   `toml:"disable_default_prefixes"`
}

Config - Station golang configuration struct

func ParseConfig 

func ParseConfig() (*Config, error)

ParseConfig parses the config from the CJ_STATION_CONFIG environment variable.

func (*Config) ParsePrivateKey 

func (c *Config) ParsePrivateKey() ([32]byte, error)

ParsePrivateKey tries to use either the PrivateKeyPath (`privkey_path`) config variable or the CJ_PRIVKEY environment variable to locate the file from which it can parse the station private key

type ConnectingTpStats added in v0.6.0 

type ConnectingTpStats interface {
	AddCreatedConnecting(asn uint, cc string, tp string)
	AddCreatedToSuccessfulConnecting(asn uint, cc string, tp string)
	AddCreatedToTimeoutConnecting(asn uint, cc string, tp string)
	AddSuccessfulToDiscardedConnecting(asn uint, cc string, tp string)
	AddOtherFailConnecting(asn uint, cc string, tp string)
}

type ConnectingTransport 

type ConnectingTransport interface {
	Transport

	// Connect attempts to connect to the client from the phantom address derived in the
	// registration.
	Connect(context.Context, transports.Registration) (net.Conn, error)
}

ConnectingTransport describes transports that actively form an outgoing connection to clients to initiate the conversation.

type DecoyRegistration 

type DecoyRegistration struct {
	PhantomIp    net.IP
	PhantomPort  uint16
	PhantomProto pb.IPProto

	Keys         *core.ConjureSharedKeys
	Covert, Mask string
	Flags        *pb.RegistrationFlags
	Transport    pb.TransportType
	TransportPtr *Transport

	RegistrationTime   time.Time
	RegistrationSource *pb.RegistrationSource
	DecoyListVersion   uint32

	// validity marks whether the registration has been validated through liveness and other checks.
	// This also denotes whether the registration has been shared with the detector.
	Valid bool
	// contains filtered or unexported fields
}

DecoyRegistration is a struct for tracking individual sessions that are expecting or tracking connections.

func (*DecoyRegistration) GenerateC2SWrapper 

func (reg *DecoyRegistration) GenerateC2SWrapper() *pb.C2SWrapper

GenerateC2SWrapper creates a C2SWrapper struct. This is used in registration sharing between stations where the station notifies other stations of a registration.

func (*DecoyRegistration) GetDstPort 

func (reg *DecoyRegistration) GetDstPort() uint16

GetDstPort returns the destination port of the phantom flow selected when the registration was created. For now there is no extra fancy-ness needed here because every valid registration will have selected a uint16 destination port on creation.

func (*DecoyRegistration) GetRegistrationAddress 

func (reg *DecoyRegistration) GetRegistrationAddress() string

GetRegistrationAddress returns the address that was used to create this registration. This should almost never be used - it exists to get the address for debugging and for logging misbehaving client IPs.

func (*DecoyRegistration) GetSrcPort 

func (reg *DecoyRegistration) GetSrcPort() uint16

GetSrcPort returns a source port if one was registered. Currently this is not supported -- for now this is intended as plumbing for potentially supporting seeded source port selection for the client.

func (*DecoyRegistration) IDString 

func (reg *DecoyRegistration) IDString() string

IDString - return a short version of the id (HMAC-ID) of a registration for logging

func (*DecoyRegistration) PhantomIP added in v0.6.3 

func (reg *DecoyRegistration) PhantomIP() *net.IP

PhantomIP returns the phantom IP

func (*DecoyRegistration) PreScanned 

func (reg *DecoyRegistration) PreScanned() bool

PreScanned returns true if a regisration has been pre-scanned - i.e scanned by another station before being shared

func (*DecoyRegistration) SetTransportKeys added in v0.7.0 

func (reg *DecoyRegistration) SetTransportKeys(k interface{}) error

SetTransportKeys returns the obfs4 public key

func (*DecoyRegistration) SharedSecret added in v0.6.3 

func (reg *DecoyRegistration) SharedSecret() []byte

SharedSecret returns the shared secret of the registration

func (*DecoyRegistration) String 

func (reg *DecoyRegistration) String() string

String -- Print a digest of the important identifying information for this registration. [TODO]{priority:soon} Find a way to add the client IP to this logging for now it is logged in the detector associating registrant IP with shared secret.

func (*DecoyRegistration) TransportKeys added in v0.7.0 

func (reg *DecoyRegistration) TransportKeys() interface{}

TransportKeys returns the obfs4 public key

func (*DecoyRegistration) TransportParams 

func (reg *DecoyRegistration) TransportParams() any

TransportParams returns the transport params associated with the registration

func (*DecoyRegistration) TransportReader added in v0.7.0 

func (reg *DecoyRegistration) TransportReader() io.Reader

TransportReader returns the obfs4 private key

func (*DecoyRegistration) TransportType added in v0.6.3 

func (reg *DecoyRegistration) TransportType() pb.TransportType

TransportType returns the protobuf transport type

type DecoyTimeout 

type DecoyTimeout struct {
	// contains filtered or unexported fields
}

DecoyTimeout contains all fields required to track registration validity / expiration.

type ProxyStats 

type ProxyStats struct {
	time.Time // epoch start time
	// contains filtered or unexported fields
}

ProxyStats track metrics about byte transfer.

func GetProxyStats 

func GetProxyStats() *ProxyStats

GetProxyStats returns our singleton for proxy stats

func (*ProxyStats) PrintAndReset 

func (s *ProxyStats) PrintAndReset(logger *log.Logger)

PrintAndReset implements the stats interface

func (*ProxyStats) Reset 

func (s *ProxyStats) Reset()

Reset implements the stats interface

type RegConfig 

type RegConfig struct {
	*liveness.Config
	*geoip.DBConfig

	// number of worker threads for ingesting incoming registrations.
	IngestWorkerCount int `toml:"ingest_worker_count"`

	// Bool to enable or disable sharing of registrations over API when received over decoy registrar
	EnableShareOverAPI bool `toml:"enable_share_over_api"`

	// REST endpoint to share decoy registrations.
	PreshareEndpoint string `toml:"preshare_endpoint"`

	// isthe station capable of handling v4 / v6 with independent toggles.
	EnableIPv4 bool `toml:"enable_v4"`
	EnableIPv6 bool `toml:"enable_v6"`

	// Local list of disallowed subnets for covert addresses.
	CovertBlocklistSubnets []string `toml:"covert_blocklist_subnets"`

	// At launch add all public addresses from machine to blocklist.
	CovertBlocklistPublicAddrs bool `toml:"covert_blocklist_public_addrs"`
	// Local list of allowed subnets for covert addresses.
	CovertAllowlistSubnets []string `toml:"covert_allowlist_subnets"`

	// Local list of disallowed domain patterns for covert addresses.
	CovertBlocklistDomains []string `toml:"covert_blocklist_domains"`

	// Local list of disallowed subnets patterns for phantom addresses.
	PhantomBlocklist []string `toml:"phantom_blocklist"`

	// ConnectingStats records stats related to connecting transports
	ConnectingStats ConnectingTpStats
	// contains filtered or unexported fields
}

RegConfig contains all configuration options directly related to processing registrations lifecycle (ingest, validity, and eviction).

func (*RegConfig) IsBlocklistedPhantom 

func (c *RegConfig) IsBlocklistedPhantom(addr net.IP) bool

IsBlocklistedPhantom checks if the provided address should be denied by on of the blocklisted Phantom subnets.

func (*RegConfig) ParseBlocklists 

func (c *RegConfig) ParseBlocklists()

ParseBlocklists converts string arrays of blocklisted domains, addresses and subnets and parses them into a usable format

func (*RegConfig) ParseOrResolveBlocklisted 

func (c *RegConfig) ParseOrResolveBlocklisted(provided string) (string, bool)

ParseOrResolveBlocklisted attempts to return an IP:port string whenever possible either by parsing the IP to ensure correct format or resolving domain names. It also checks the configuration blocklists for both domain name and IP address. The intention of this function is that it be used to prevent SSRF DNS rebinding by doing resolution to final address to be used by net.Dial and checking blocklists in the same step.

If a bad address / domain is given and empty string will be returned The bool return indicates whether the station resolved the domain name or not

type RegisteredDecoys 

type RegisteredDecoys struct {
	// contains filtered or unexported fields
}

RegisteredDecoys provides a container struct for tracking all registrations and their expiration.

func NewRegisteredDecoys 

func NewRegisteredDecoys() *RegisteredDecoys

NewRegisteredDecoys returns a new struct with which to track registrations.

func (*RegisteredDecoys) RegistrationExists 

func (r *RegisteredDecoys) RegistrationExists(d *DecoyRegistration) *DecoyRegistration

RegistrationExists - For use outside of this struct only (so there are no data races.)

func (*RegisteredDecoys) TotalRegistrations 

func (r *RegisteredDecoys) TotalRegistrations() int

TotalRegistrations return the total number of current registrations

func (*RegisteredDecoys) Track 

func (r *RegisteredDecoys) Track(d *DecoyRegistration) error

Track informs the registered decoys struct of a new registration to track.

For use outside of this struct (so there are no data races.)

func (*RegisteredDecoys) TrackIfNotExists added in v0.7.8 

func (r *RegisteredDecoys) TrackIfNotExists(d *DecoyRegistration) (bool, error)

TrackIfNotExists tracks a registration if it is not already tracked and returns a bool indicating whether the registration was already tracked. These are combined to prevent a Time-of-Check-Time-of-Use bug that allows more than one message to be sent to the detector per registration.

For use outside of this struct (so there are no data races.)

type RegistrationManager 

type RegistrationManager struct {
	*RegConfig
	*RegistrationStats

	Logger          *log.Logger
	PhantomSelector *phantoms.PhantomIPSelector
	LivenessTester  liveness.Tester
	GeoIP           geoip.Database
	// contains filtered or unexported fields
}

RegistrationManager manages registration tracking for the station.

func NewRegistrationManager 

func NewRegistrationManager(conf *RegConfig) *RegistrationManager

NewRegistrationManager returns a newly initialized registration Manager

func (*RegistrationManager) AddRegistration 

func (regManager *RegistrationManager) AddRegistration(d *DecoyRegistration)

AddRegistration officially adds the registration to usage by marking it as valid.

func (*RegistrationManager) AddTransport 

func (regManager *RegistrationManager) AddTransport(index pb.TransportType, t Transport) error

AddTransport initializes a transport so that it can be tracked by the manager when clients register.

func (*RegistrationManager) Cleanup 

func (regManager *RegistrationManager) Cleanup()

Cleanup sends a signal to the detector to empty cached sessions. This ensures that the detector does not forward traffic for sessions that it knows about for a previous launch of the station that the current session doesn't know about.

func (*RegistrationManager) CountRegistrations 

func (regManager *RegistrationManager) CountRegistrations(phantomAddr net.IP) int

CountRegistrations counts the number of registrations tracked that are using a specific phantom address.

func (*RegistrationManager) GetConnectingTransports added in v0.6.0 

func (regManager *RegistrationManager) GetConnectingTransports() map[pb.TransportType]ConnectingTransport

GetConnectingTransports Returns a map of the connecting transport types to their transports. This return value can be mutated freely.

func (*RegistrationManager) GetRegistrations 

func (regManager *RegistrationManager) GetRegistrations(phantomAddr net.IP) map[string]transports.Registration

GetRegistrations returns registrations associated with a specific phantom address.

func (*RegistrationManager) GetWrappingTransports 

func (regManager *RegistrationManager) GetWrappingTransports() map[pb.TransportType]WrappingTransport

GetWrappingTransports Returns a map of the wrapping transport types to their transports. This return value can be mutated freely.

func (*RegistrationManager) HandleRegUpdates 

func (rm *RegistrationManager) HandleRegUpdates(ctx context.Context, regChan <-chan interface{}, parentWG *sync.WaitGroup)

HandleRegUpdates is responsible for launching and managing registration ingest from the perspective of the RegistrationManager. The keys to success in this job are:

  1. Launch a fixed number of workers to process registration messages
  2. read from the registration channel as though it is blocking
  3. write jobs to works non-blocking. any time a registration is received, but a worker is not available the registration is simple dropped and counted for metrics
  4. Keep a shallow buffer to drop as few registrations as possible. The buffer must be shallow so that registrations that end up buffered are still relevant when they make it out of the buffer.

func (*RegistrationManager) IsEnabledTransport 

func (regManager *RegistrationManager) IsEnabledTransport(index pb.TransportType) bool

IsEnabledTransport checks if the provided transport ID is enabled in the regisrtar

func (*RegistrationManager) MarkActive 

func (regManager *RegistrationManager) MarkActive(reg *DecoyRegistration)

MarkActive indicates that an incoming connection has successfully been make with the registration provided in the argument.

func (*RegistrationManager) NewRegistration 

func (rm *RegistrationManager) NewRegistration(c2s *pb.ClientToStation, conjureKeys *core.ConjureSharedKeys, includeV6 bool, registrationSource *pb.RegistrationSource) (*DecoyRegistration, error)

NewRegistration creates a new registration from details provided. Adds the registration to tracking map, But marks it as not valid. This is a utility function, it its not used in the ingest pipeline

func (*RegistrationManager) NewRegistrationC2SWrapper 

func (rm *RegistrationManager) NewRegistrationC2SWrapper(c2sw *pb.C2SWrapper, includeV6 bool) (*DecoyRegistration, error)

NewRegistrationC2SWrapper creates a new registration from details provided. Adds the registration to tracking map, But marks it as not valid.

func (*RegistrationManager) OnReload 

func (regManager *RegistrationManager) OnReload(conf *RegConfig)

OnReload is meant to be used when Reloading Configuration while things are already running. Only reloads phantom selector and blocklists. Does not (yet) modify ingest worker pipeline or liveness testing configuration.

func (*RegistrationManager) PhantomIsLive 

func (regManager *RegistrationManager) PhantomIsLive(addr string, port uint16) (bool, error)

PhantomIsLive - Test whether the phantom is live using 8 syns which returns syn-acks from 99% of sites within 1 second. see ZMap: Fast Internet-wide Scanning and Its Security Applications https://www.usenix.org/system/files/conference/usenixsecurity13/sec13-paper_durumeric.pdf

return: bool true - host is live 

		false - host is not liev
error	reason decision was made

func (*RegistrationManager) PrintAndReset 

func (s *RegistrationManager) PrintAndReset(logger *log.Logger)

PrintAndReset implements the stats interface. Overrides the Registration stats implementation of PrintAndReset so we have access to the current state of the registration manager.

func (*RegistrationManager) RegistrationExists 

func (regManager *RegistrationManager) RegistrationExists(reg *DecoyRegistration) bool

RegistrationExists checks if the registration is already tracked by the manager, this is independent of the validity tag, this just checks to see if the registration exists.

func (*RegistrationManager) RemoveOldRegistrations 

func (regManager *RegistrationManager) RemoveOldRegistrations()

RemoveOldRegistrations garbage collects old registrations

func (*RegistrationManager) TrackRegIfNotExists added in v0.7.8 

func (regManager *RegistrationManager) TrackRegIfNotExists(reg *DecoyRegistration) (bool, error)

TrackRegIfNotExists tracks a registration if it is not already tracked and returns a bool indicating whether the registration was already tracked. These are combined to prevent a Time-of-Check-Time-of-Use bug that allows more than one message to be sent to the detector per registration.

func (*RegistrationManager) TrackRegistration 

func (regManager *RegistrationManager) TrackRegistration(d *DecoyRegistration) error

TrackRegistration adds the registration to the map WITHOUT marking it valid.

func (*RegistrationManager) ValidateRegistration 

func (regManager *RegistrationManager) ValidateRegistration(reg *DecoyRegistration) (bool, error)

ValidateRegistration checks expected fields and combinations for common errors to prevent wasted time on registration ingest.

type RegistrationStats 

type RegistrationStats struct {
	// contains filtered or unexported fields
}

RegistrationStats track metrics relating to registration management and lifecycle

func (*RegistrationStats) AddBlocklistedPhantomReg 

func (s *RegistrationStats) AddBlocklistedPhantomReg()

AddBlocklistedPhantomReg adds one to the count of registrations that errored this epoch

func (*RegistrationStats) AddDupReg 

func (s *RegistrationStats) AddDupReg()

AddDupReg adds one to the count of registrations that saw duplicated this epoch

func (*RegistrationStats) AddErrReg 

func (s *RegistrationStats) AddErrReg()

AddErrReg adds one to the count of registrations that errored this epoch

func (*RegistrationStats) AddExpiredRegs 

func (s *RegistrationStats) AddExpiredRegs(total, valid int64)

AddExpiredRegs updates registration stats count.

func (*RegistrationStats) AddRegStats 

func (s *RegistrationStats) AddRegStats(reg *DecoyRegistration)

AddRegStats updates registration stats. Will only be called for registrations marked valid

func (*RegistrationStats) PrintAndReset 

func (s *RegistrationStats) PrintAndReset(logger *log.Logger)

PrintAndReset implements the stats interface

func (*RegistrationStats) Reset 

func (s *RegistrationStats) Reset()

Reset implements the stats interface

type Stats 

type Stats struct {
	// contains filtered or unexported fields
}

Stats contains counts of many things we want to keep track of in any given epoch as well as reference to modular metrics interfaces from related modules. These are used to print usage in a regulated and consumable way.

fields are int64 because we occasionally need to atomically subtract, which is not supported for uint64

func Stat 

func Stat() *Stats

Stat returns our singleton for stats

func (*Stats) AddBytes 

func (s *Stats) AddBytes(n int64, dir string)

func (*Stats) AddBytesDown 

func (s *Stats) AddBytesDown(n int64)

func (*Stats) AddBytesUp 

func (s *Stats) AddBytesUp(n int64)

func (*Stats) AddConn 

func (s *Stats) AddConn()

func (*Stats) AddDupReg 

func (s *Stats) AddDupReg()

func (*Stats) AddErrReg 

func (s *Stats) AddErrReg()

func (*Stats) AddLivenessCached 

func (s *Stats) AddLivenessCached()

func (*Stats) AddLivenessFail 

func (s *Stats) AddLivenessFail()

func (*Stats) AddLivenessPass 

func (s *Stats) AddLivenessPass()

func (*Stats) AddMissedReg 

func (s *Stats) AddMissedReg()

func (*Stats) AddReg 

func (s *Stats) AddReg(generation uint32, source *pb.RegistrationSource)

will only be called for registrations marked valid

func (*Stats) AddStatsModule 

func (s *Stats) AddStatsModule(sm stats, isVerbose bool)

func (*Stats) CloseConn 

func (s *Stats) CloseConn()

func (*Stats) ConnErr 

func (s *Stats) ConnErr()

func (*Stats) ExpireReg 

func (s *Stats) ExpireReg(generation uint32, source *pb.RegistrationSource)

should only be called for registrations marked valid

func (*Stats) PrintStats 

func (s *Stats) PrintStats(isVerbose bool)

func (*Stats) Reset 

func (s *Stats) Reset()

func (*Stats) ResetAll 

func (s *Stats) ResetAll()

type Transport 

type Transport interface {
	// The human-friendly name of the transport.
	Name() string

	// The prefix used when including this transport in logs.
	LogPrefix() string

	// GetIdentifier takes in a registration and returns an identifier for it. This identifier
	// should be unique for each registration on a given phantom; registrations on different
	// phantoms can have the same identifier.
	GetIdentifier(transports.Registration) string

	// GetProto returns the IP protocol used by the transport. Typical transports will use TCP or
	// UDP, if something beyond these is required you will need to update the enum in the protobuf
	// file and change the packet processing in the detector.
	GetProto() pb.IPProto

	// GetDstPort Given the library version, a seed, and a generic object containing parameters the
	// transport should be able to return the destination port that a clients phantom connection
	// will attempt to reach. The libVersion is provided incase of version dependent changes in the
	// transports port selection algorithm.
	GetDstPort(libVersion uint, seed []byte, parameters any) (uint16, error)

	// ParseParams gives the specific transport an option to parse a generic object into parameters
	// provided by the client during registration. The libVersion is provided incase of version
	// dependent changes in the transport params or param parsing.
	ParseParams(libVersion uint, data *anypb.Any) (any, error)

	// ParamStrings returns an array of tag string that will be added to tunStats when a proxy
	// session is closed.
	ParamStrings(p any) []string
}

Transport defines the interface for the manager to interface with variable transports that wrap the traffic sent by clients.

type WrappingTransport 

type WrappingTransport interface {
	Transport

	// WrapConnection attempts to wrap the given connection in the transport. It takes the
	// information gathered so far on the connection in data, attempts to identify itself, and if it
	// positively identifies itself wraps the connection in the transport, returning a connection
	// that's ready to be used by others.
	//
	// If the returned error is nil or non-nil and non-{ transports.ErrTryAgain,
	// transports.ErrNotTransport }, the caller may no longer use data or conn.
	//
	// Implementations should not Read from conn unless they have positively identified that the
	// transport exists and are in the process of wrapping the connection.
	//
	// Implementations should not Read from data unless they are are attempting to wrap the
	// connection. Use data.Bytes() to get all of the data that has been seen on the connection.
	//
	// If implementations cannot tell if the transport exists on the connection (e.g. there hasn't
	// yet been enough data sent to be conclusive), they should return transports.ErrTryAgain. If
	// the transport can be conclusively determined to not exist on the connection, implementations
	// should return transports.ErrNotTransport.
	WrapConnection(data *bytes.Buffer, conn net.Conn, phantom net.IP, rm transports.RegManager) (reg transports.Registration, wrapped net.Conn, err error)
}

WrappingTransport describes any transport that is able to passively listen to incoming network connections and identify itself, then actively wrap the connection.

type ZMQConfig 

type ZMQConfig struct {
	SocketName        string         `toml:"socket_name"`
	ConnectSockets    []socketConfig `toml:"connect_sockets"`
	HeartbeatInterval int            `toml:"heartbeat_interval"`
	HeartbeatTimeout  int            `toml:"heartbeat_timeout"`
}

ZMQConfig - Configuration options relevant to the ZMQ Proxy utility

type ZMQIngester 

type ZMQIngester struct {
	*ZMQConfig
	// contains filtered or unexported fields
}

ZMQIngester manages registration ingest over ZMQ.

func NewZMQIngest 

func NewZMQIngest(connectAddr string, regchan chan<- interface{}, privkey [32]byte, conf *ZMQConfig) (*ZMQIngester, error)

NewZMQIngest returns a struct that manages registration ingest over ZMQ.

func (*ZMQIngester) PrintAndReset 

func (zi *ZMQIngester) PrintAndReset(logger *log.Logger)

PrintAndReset implements the Stats interface

func (*ZMQIngester) Reset 

func (zi *ZMQIngester) Reset()

Reset implements the Stats interface

func (*ZMQIngester) RunZMQ 

func (zi *ZMQIngester) RunZMQ(ctx context.Context)

RunZMQ start the receive loop that writes into the provided message receive channel

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.