mtglib

package
v2.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 5, 2021 License: MIT Imports: 19 Imported by: 1

Documentation

Overview

mtglib defines a package with MTPROTO proxy.

Since mtg itself is build as an example of how to work with mtglib, it worth to telling a couple of words about a project organization.

A core object of the project is mtglib.Proxy. This is a proxy you expect: that one which you configure, set to serve on a listener and/or shutdown on application termination.

But it also has a core logic unrelated to Telegram per se: anti replay cache, network connectivity (who knows, maybe you want to have a native VMESS integration) and so on.

You can supply such parts to a proxy with interfaces. The rest of the packages in mtg define some default implementations of these interfaces. But if you want to integrate it with, let say, influxdb, you can do it easily.

Index

Constants

View Source 
const (
	// DefaultConcurrency is a default max count of simultaneously
	// connected clients.
	DefaultConcurrency = 4096

	// DefaultBufferSize is a default size of a copy buffer.
	DefaultBufferSize = 16 * 1024 // 16 kib

	// DefaultDomainFrontingPort is a default port (HTTPS) to connect to in
	// case of probe-resistance activity.
	DefaultDomainFrontingPort = 443

	// DefaultIdleTimeout is a default timeout for closing a connection
	// in case of idling.
	//
	// Deprecated: no longer in use because of changed TCP relay
	// algorithm.
	DefaultIdleTimeout = time.Minute

	// DefaultTolerateTimeSkewness is a default timeout for time
	// skewness on a faketls timeout verification.
	DefaultTolerateTimeSkewness = 3 * time.Second

	// DefaultPreferIP is a default value for Telegram IP connectivity
	// preference.
	DefaultPreferIP = "prefer-ipv6"

	// SecretKeyLength defines a length of the secret bytes used
	// by Telegram and a proxy.
	SecretKeyLength = 16

	// ConnectionIDBytesLength defines a count of random bytes used to generate
	// a stream/connection ids.
	ConnectionIDBytesLength = 16

	// TCPRelayReadTimeout defines a max time period between two consecuitive
	// reads from Telegram after which connection will be terminated. This is
	// required to abort stale connections.
	TCPRelayReadTimeout = 20 * time.Second
)

Variables

View Source 
var (
	// ErrSecretEmpty is returned if you are trying to create a proxy
	// but do not provide a secret.
	ErrSecretEmpty = errors.New("secret is empty")

	// ErrSecretInvalid is returned if you are trying to create a proxy
	// but secret value is invalid (no host or payload are zeroes).
	ErrSecretInvalid = errors.New("secret is invalid")

	// ErrNetworkIsNotDefined is returned if you are trying to create a
	// proxy but network value is undefined.
	ErrNetworkIsNotDefined = errors.New("network is not defined")

	// ErrAntiReplayCacheIsNotDefined is returned if you are trying to
	// create a proxy but anti replay cache value is undefined.
	ErrAntiReplayCacheIsNotDefined = errors.New("anti-replay cache is not defined")

	// ErrIPBlocklistIsNotDefined is returned if you are trying to
	// create a proxy but ip blocklist instance is not defined.
	ErrIPBlocklistIsNotDefined = errors.New("ip blocklist is not defined")

	// ErrEventStreamIsNotDefined is returned if you are trying to create a
	// proxy but event stream instance is not defined.
	ErrEventStreamIsNotDefined = errors.New("event stream is not defined")

	// ErrLoggerIsNotDefined is returned if you are trying to
	// create a proxy but logger is not defined.
	ErrLoggerIsNotDefined = errors.New("logger is not defined")
)

Functions

This section is empty.

Types

type AntiReplayCache 

type AntiReplayCache interface {
	// Seen before checks if this set of bytes was observed before or
	// not. If it is required to store this information somewhere else,
	// then it has to do that.
	SeenBefore(data []byte) bool
}

AntiReplayCache is an interface that is used to detect replay attacks based on some traffic fingerprints.

Replay attacks are probe attacks whose main goal is to identify if server software can be classified in some way. For example, if you send some HTTP request to a web server, then you can expect that this server will respond with HTTP response back.

There is a problem though. Let's imagine, that connection is encrypted. Let's imagine, that it is encrypted with some static key like ShadowSocks (https://shadowsocks.org/assets/whitepaper.pdf). In that case, in theory, if you repeat the same bytes, you can get the same responses. Let's imagine, that you've cracked the key. then if you send the same bytes, you can decrypt a response and see its structure. Based on its structure you can identify if this server is SOCKS5, MTPROTO proxy etc.

This is just one example, maybe not the best or not the most relevant. In real life, different organizations use such replay attacks to perform some reverse engineering of the proxy, do some statical analysis to identify server software.

There are many ways how to protect your proxy against them. One is domain fronting which is a core part of mtg. Another one is to collect some 'handshake fingerprints' and forbid duplication.

So, it one is sending the same byte flow right after you (or a couple of hours after), mtg should detect that and reject this connection (or redirect to fronting domain).

type Event 

type Event interface {
	// StreamID returns an identifier of the stream, connection,
	// request, you name it. All events within the same stream returns
	// the same stream id.
	StreamID() string

	// Timestamp returns a timestamp when this event was generated.
	Timestamp() time.Time
}

Event is a data structure which is populated during mtg request processing lifecycle. Each request popluates many events:

1. Client connected

2. Request is finished

3. Connection to Telegram server is established

and so on. All these events are data structures but all of them must conform the same interface.

type EventConcurrencyLimited 

type EventConcurrencyLimited struct {
	// contains filtered or unexported fields
}

EventConcurrencyLimited is emitted when connection was declined because of the concurrency limit of the worker pool.

func NewEventConcurrencyLimited 

func NewEventConcurrencyLimited() EventConcurrencyLimited

NewEventConcurrencyLimited creates a new EventConcurrencyLimited event.

func (EventConcurrencyLimited) StreamID 

func (e EventConcurrencyLimited) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventConcurrencyLimited) Timestamp 

func (e EventConcurrencyLimited) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventConnectedToDC 

type EventConnectedToDC struct {

	// RemoteIP is an IP address of the Telegram server proxy has been
	// connected to.
	RemoteIP net.IP

	// DC is an index of the datacenter proxy has been connected to.
	DC int
	// contains filtered or unexported fields
}

EventConnectedToDC is emitted when mtg proxy has connected to a Telegram server.

func NewEventConnectedToDC 

func NewEventConnectedToDC(streamID string, remoteIP net.IP, dc int) EventConnectedToDC

NewEventConnectedToDC creates a new EventConnectedToDC event.

func (EventConnectedToDC) StreamID 

func (e EventConnectedToDC) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventConnectedToDC) Timestamp 

func (e EventConnectedToDC) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventDomainFronting 

type EventDomainFronting struct {
	// contains filtered or unexported fields
}

EventDomainFronting is emitted when we connect to a front domain instead of Telegram server.

func NewEventDomainFronting 

func NewEventDomainFronting(streamID string) EventDomainFronting

NewEventDomainFronting creates a new EventDomainFronting event.

func (EventDomainFronting) StreamID 

func (e EventDomainFronting) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventDomainFronting) Timestamp 

func (e EventDomainFronting) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventFinish 

type EventFinish struct {
	// contains filtered or unexported fields
}

EventFinish is emitted when we stop to manage a connection.

func NewEventFinish 

func NewEventFinish(streamID string) EventFinish

NewEventFinish creates a new EventFinish event.

func (EventFinish) StreamID 

func (e EventFinish) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventFinish) Timestamp 

func (e EventFinish) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventIPBlocklisted 

type EventIPBlocklisted struct {
	RemoteIP net.IP
	// contains filtered or unexported fields
}

EventIPBlocklisted is emitted when connection was declined because IP address was found in IP blocklist.

func NewEventIPBlocklisted 

func NewEventIPBlocklisted(remoteIP net.IP) EventIPBlocklisted

NewEventIPBlocklisted creates a new EventIPBlocklisted event.

func (EventIPBlocklisted) StreamID 

func (e EventIPBlocklisted) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventIPBlocklisted) Timestamp 

func (e EventIPBlocklisted) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventReplayAttack 

type EventReplayAttack struct {
	// contains filtered or unexported fields
}

EventReplayAttack is emitted when mtg detects a replay attack on a connection.

func NewEventReplayAttack 

func NewEventReplayAttack(streamID string) EventReplayAttack

NewEventReplayAttack creates a new EventReplayAttack event.

func (EventReplayAttack) StreamID 

func (e EventReplayAttack) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventReplayAttack) Timestamp 

func (e EventReplayAttack) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventStart 

type EventStart struct {

	// RemoteIP is an IP address of the client.
	RemoteIP net.IP
	// contains filtered or unexported fields
}

EventStart is emitted when mtg proxy starts to process a new connection.

func NewEventStart 

func NewEventStart(streamID string, remoteIP net.IP) EventStart

NewEventStart creates a new EventStart event.

func (EventStart) StreamID 

func (e EventStart) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventStart) Timestamp 

func (e EventStart) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventStream 

type EventStream interface {
	// Send delivers an event to observers. Given context has to be
	// respected. If the context is closed, all blocking operations should
	// be released ASAP.
	//
	// It is possible that context is closed but the message is delivered.
	// EventStream implementations should solve this issue somehow.
	Send(context.Context, Event)
}

EventStream is an abstraction that accepts a set of events produced by mtg. Its main goal is to inject your logging or monitoring system.

The idea is simple. When mtg works, it emits a set of events during a lifecycle of the requestor: EventStart, EventFinish etc. mtg is a producer which puts these events into a stream. Responsibility of the stream is to deliver this event to consumers/observers. There might be many different observers (for example, you want to have both statsd and prometheus), mtg should know nothing about them.

type EventTraffic 

type EventTraffic struct {

	// Traffic is a count of bytes which were transmitted.
	Traffic uint

	// IsRead defines if we _read_ or _write_ to connection. A rule of
	// thumb is simple: EventTraffic is bound to a remote connection. Not
	// to a client one, but either to Telegram or front domain one.
	//
	// In the case of Telegram, isRead means that we've fetched some bytes
	// from Telegram to send it to a client.
	//
	// In the case of the front domain, it means that we've fetched some
	// bytes from this domain to send it to a client.
	IsRead bool
	// contains filtered or unexported fields
}

EventTraffic is emitted when we read/write some bytes on a connection.

func NewEventTraffic 

func NewEventTraffic(streamID string, traffic uint, isRead bool) EventTraffic

NewEventTraffic creates a new EventTraffic event.

func (EventTraffic) StreamID 

func (e EventTraffic) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventTraffic) Timestamp 

func (e EventTraffic) Timestamp() time.Time

Timestamp return a time when this event was generated.

type IPBlocklist 

type IPBlocklist interface {
	// Contains checks if given IP address belongs to this blocklist If.
	// it is, a connection is terminated                               .
	Contains(net.IP) bool
}

IPBlocklist filters requests based on IP address.

If this filter has an IP address, then mtg closes a request without reading anything from a socket. It also does not give such request to a worker pool, so in worst cases you can expect that you invoke this object more frequent than defined proxy concurrency.

type Logger 

type Logger interface {
	// Named returns a new logger with a bound name. Name chaining is
	// allowed and appreciated.
	Named(name string) Logger

	// BindInt binds new integer parameter to a new logger instance.
	BindInt(name string, value int) Logger

	// BindStr binds new string parameter to a new logger instance.
	BindStr(name, value string) Logger

	// BindJSON binds a new JSON-encoded string to a new logger instance.
	BindJSON(name, value string) Logger

	// Printf is to support log.Logger behavior.
	Printf(format string, args ...interface{})

	// Info puts a message about some normal situation.
	Info(msg string)

	// InfoError puts a message about some normal situation but this
	// situation is related to a given error.
	InfoError(msg string, err error)

	// Warning puts a message about some extraordinary situation
	// worth to look at.
	Warning(msg string)

	// WarningError puts a message about some extraordinary situation
	// worth to look at. This situation is related to a given error.
	WarningError(msg string, err error)

	// Debug puts a message useful for debugging only.
	Debug(msg string)

	// Debug puts a message useful for debugging only. This message is
	// related to a given error.
	DebugError(msg string, err error)
}

Logger defines an interface of the logger used by mtglib.

Each logger has a name. It is possible to stack names to organize poor-man namespaces. Also, each logger must be able to bind parameters to avoid pushing them all the time.

Example 

logger := SomeLogger{}
logger = logger.BindStr("ip", net.IP{127, 0, 0, 1})
logger.Info("Hello")

In that case, ip is bound as a parameter. It is a great idea to put this parameter somewhere in a log message. 

logger1 = logger.BindStr("param1", "11")
logger2 = logger.BindInt("param2", 11)

logger1 should see no param2 and vice versa, logger2 should not see param1 If you attach a parameter to a logger, parents should not know about that.

type Network 

type Network interface {
	// Dial establishes context-free TCP connections.
	Dial(network, address string) (net.Conn, error)

	// DialContext dials using a context. This is a preferrable
	// way of establishing TCP connections.
	DialContext(ctx context.Context, network, address string) (net.Conn, error)

	// MakeHTTPClient build an HTTP client with given dial function. If
	// nothing is provided, then DialContext of this interface is going
	// to be used.
	MakeHTTPClient(func(ctx context.Context, network, address string) (net.Conn, error)) *http.Client
}

Network defines a knowledge how to work with a network. It may sound fun but it encapsulates all the knowledge how to properly establish connections to remote hosts and configure HTTP clients.

For example, if you want to use SOCKS5 proxy, you probably want to have all traffic routed to this proxy: telegram connections, http requests and so on. This knowledge is encapsulated into instances of such interface.

mtglib uses Network for:

1. Dialing to Telegram

2. Dialing to front domain

3. Doing HTTP requests (for example, for FireHOL ipblocklist).

type Proxy 

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy is an MTPROTO proxy structure.

func NewProxy 

func NewProxy(opts ProxyOpts) (*Proxy, error)

NewProxy makes a new proxy instance.

func (*Proxy) DomainFrontingAddress 

func (p *Proxy) DomainFrontingAddress() string

DomainFrontingAddress returns a host:port pair for a fronting domain.

func (*Proxy) Serve 

func (p *Proxy) Serve(listener net.Listener) error

Serve starts a proxy on a given listener.

func (*Proxy) ServeConn 

func (p *Proxy) ServeConn(conn net.Conn)

ServeConn serves a connection. We do not check IP blocklist and concurrency limit here.

func (*Proxy) Shutdown 

func (p *Proxy) Shutdown()

Shutdown 'gracefully' shutdowns all connections. Please remember that it does not close an underlying listener.

type ProxyOpts 

type ProxyOpts struct {
	// Secret defines a secret which should be used by a proxy.
	//
	// This is a mandatory setting.
	Secret Secret

	// Network defines a network instance which should be used for all
	// network communications made by proxies.
	//
	// This is a mandatory setting.
	Network Network

	// AntiReplayCache defines an instance of antireplay cache.
	//
	// This is a mandatory setting.
	AntiReplayCache AntiReplayCache

	// IPBlocklist defines an instance of IP blocklist.
	//
	// This is a mandatory setting.
	IPBlocklist IPBlocklist

	// EventStream defines an instance of event stream.
	//
	// This ia a mandatory setting.
	EventStream EventStream

	// Logger defines an instance of the logger.
	//
	// This is a mandatory setting.
	Logger Logger

	// BufferSize is a size of the copy buffer in bytes.
	//
	// Please remember that we multiply this number in 2, because when
	// we relay between proxies, we have to create 2 intermediate
	// buffers: to and from.
	//
	// This is an optional setting.
	BufferSize uint

	// Concurrency is a size of the worker pool for connection management.
	//
	// If we have more connections than this number, they are going to be
	// rejected.
	//
	// This is an optional setting.
	Concurrency uint

	// IdleTimeout is a timeout for relay when we have to break a
	// stream.
	//
	// This is a timeout for any activity. So, if we have any message
	// which will pass to either direction, a timer is reset. If we have
	// no any reads or writes for this timeout, a connection will be
	// aborted.
	//
	// This is an optional setting.
	IdleTimeout time.Duration

	// TolerateTimeSkewness is a time boundary that defines a time
	// range where faketls timestamp is acceptable.
	//
	// This means that if if you got a timestamp X, now is Y, then
	// if |X-Y| < TolerateTimeSkewness, then you accept a packet.
	//
	// This is an optional setting.
	TolerateTimeSkewness time.Duration

	// PreferIP defines an IP connectivity preference. Valid values are:
	// 'prefer-ipv4', 'prefer-ipv6', 'only-ipv4', 'only-ipv6'.
	//
	// This is an optional setting.
	PreferIP string

	// DomainFrontingPort is a port we use to connect to a fronting
	// domain.
	//
	// This is required because secret does not specify a port. It
	// specifies a hostname only.
	//
	// This is an optional setting.
	DomainFrontingPort uint

	// AllowFallbackOnUnknownDC defines how proxy behaves if unknown DC was
	// requested. If this setting is set to false, then such connection
	// will be rejected. Otherwise, proxy will chose any DC.
	//
	// Telegram is designed in a way that any DC can serve any request,
	// the problem is a latency.
	//
	// This is an optional setting.
	AllowFallbackOnUnknownDC bool

	// UseTestDCs defines if we have to connect to production or to staging
	// DCs of Telegram.
	//
	// This is required if you use mtglib as an integration library for
	// your Telegram-related projects.
	//
	// This is an optional setting.
	UseTestDCs bool
}

ProxyOpts is a structure with settings to mtg proxy.

This is not required per se, but this is to shorten function signature and give an ability to conveniently provide default values.

type Secret 

type Secret struct {
	// Key is a set of bytes used for traffic authentication.
	Key [SecretKeyLength]byte

	// Host is a domain fronting hostname.
	Host string
}

Secret is a data structure that presents a secret.

Telegram secret is not a simple string like "ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d". Actually, this is a serialized datastructure of 2 parts: key and host. 

ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d
|-|-------------------------------|-------------------------------------------
p key                             hostname

Serialized secret starts with 'ee'. Actually, in the past we also had 'dd' secrets and prefixless ones. But this is history. Currently, we do have only 'ee' secrets which mean faketls + protection from statistical attacks on a length. 'ee' is a byte 238 (0xee).

After that, we have 16 bytes of the key. This is a random generated secret data of the proxy and this data is used to derive authentication schemas. These secrets are mixed into hmacs and sha256 checksums which are used to build AEAD ciphers for obfuscated2 protocol and ensure faketls handshake.

Host is a domain fronting hostname in latin1 (ASCII) encoding. This hostname should be used for SNI in faketls and MTG verifies it. Also, this is when mtg gets about a domain fronting hostname.

Secrets can be serialized into 2 forms: hex and base64. If you decode both forms into bytes, you'll get the same byte array. Telegram clients nowadays accept all forms.

func GenerateSecret 

func GenerateSecret(hostname string) Secret

GenerateSecret makes a new secret with a given hostname.

func ParseSecret 

func ParseSecret(secret string) (Secret, error)

ParseSecret parses a secret (both hex and base64 forms).

func (Secret) Base64 

func (s Secret) Base64() string

Base64 returns a base64-encoded form of this secret.

func (Secret) Hex 

func (s Secret) Hex() string

Hex returns a hex-encoded form of this secret (ee-secret).

func (Secret) MarshalText 

func (s Secret) MarshalText() ([]byte, error)

MarshalText is to support text.Marshaller interface.

func (*Secret) Set added in v2.1.0 

func (s *Secret) Set(text string) error

func (Secret) String 

func (s Secret) String() string

String is to support fmt.Stringer interface.

func (*Secret) UnmarshalText 

func (s *Secret) UnmarshalText(data []byte) error

UnmarshalText is to support text.Unmarshaller interface.

func (Secret) Valid 

func (s Secret) Valid() bool

Valid checks if this secret is valid and can be used in proxy.

Source Files

View all Source files

Directories

Path Synopsis
internal
faketls
faketls/record
obfuscated2
relay
telegram

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.