clienthellod

package module
v0.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 11, 2023 License: Apache-2.0 Imports: 17 Imported by: 2

README

clienthellod

Go Build Status Go Report Card DeepSource FOSSA Status

ClientHello Parser/Resolver as a Service from tlsfingerprint.io.

What does it do

clienthellod, read as "client hello DEE", is a service that parses and resolves the ClientHello message sent by the client to the server. It is a part of the TLS fingerprintability research project which spans tlsfingerprint.io and quic.tlsfingerprint.io. It parses the ClientHello messages sent by TLS clients and QUIC Client Initial Packets sent by QUIC clients and display the parsed information in a human-readable format with high programmability.

See tlsfingerprint.io and quic.tlsfingerprint.io for more details about the project.

How to use

clienthellod is provided as a Go library in the root directory of this repository.

Quick Start
TLS ClientHello
    tcpLis, err := net.Listen("tcp", ":443")
    defer tcpLis.Close()

    conn, err := tcpLis.Accept()
	if err != nil {
        panic(err)
	}
    defer conn.Close()

	ch, err := clienthellod.ReadClientHello(conn) // saves ClientHello
    if err != nil {
        panic(err)
    }

    err := ch.ParseClientHello() // parses ClientHello's fields
    if err != nil {
        panic(err)
    }

    jsonB, err = json.MarshalIndent(ch, "", "  ")
    if err != nil {
        panic(err)
    }

    fmt.Println(string(jsonB))
    fmt.Println("ClientHello ID: " + ch.FingerprintID(false)) // prints ClientHello's original fingerprint ID, as TLS extension IDs in their provided order
    fmt.Println("ClientHello NormID: " + ch.FingerprintID(true)) // prints ClientHello's normalized fingerprint ID, as TLS extension IDs in a sorted order
QUIC Client Initial Packet
    udpConn, err := net.ListenUDP("udp", ":443")
    defer udpConn.Close()

    buf := make([]byte, 65535)
    n, addr, err := udpConn.ReadFromUDP(buf)
    if err != nil {
        panic(err)
    }

    cip, err := clienthellod.ParseQUICCIP(buf[:n]) // reads in and parses QUIC Client Initial Packet
    if err != nil {
        panic(err)    
    }

    jsonB, err = json.MarshalIndent(cip, "", "  ")
    if err != nil {
        panic(err)
    }

    fmt.Println(string(jsonB)) // including fingerprint IDs of: ClientInitialPacket, QUIC Header, QUIC ClientHello, QUIC Transport Parameters' combination
Use with Caddy

clienthellod is also provided as a Caddy plugin, modcaddy, which can be used to capture ClientHello messages and QUIC Client Initial Packets. See Section modcaddy for more details.

modcaddy

modcaddy is a Caddy plugin that provides:

  • An caddy app that can be used to temporarily store captured ClientHello messages and QUIC Client Initial Packets.
  • A caddy handler that can be used to serve the ClientHello messages and QUIC Client Initial Packets to the client sending the request.
  • A caddy listener that can be used to capture ClientHello messages and QUIC Client Initial Packets.

You will need to use xcaddy to rebuild Caddy with modcaddy included.

It is worth noting that some web browsers may not choose to switch to QUIC protocol in localhost environment, which may result in the QUIC Client Initial Packet not being sent and therefore not being captured/analyzed.

Build
xcaddy build --with github.com/gaukas/clienthellod/modcaddy
When build locally with changes
xcaddy build --with github.com/gaukas/clienthellod/modcaddy --with github.com/gaukas/clienthellod/=./
Caddyfile

A sample Caddyfile is provided below.

{      
    # debug # for debugging purpose
    # https_port   443 # currently, QUIC listener works only on port 443, otherwise you need to make changes to the code
    order clienthellod before file_server # make sure it hits handler before file_server
    clienthellod { # app (reservoir)
        validfor 120s 30s # params: validFor [cleanEvery] # increased for QUIC
    }
    servers {
        listener_wrappers {
            clienthellod { # listener
                tcp # listens for TCP and saves TLS ClientHello 
                udp # listens for UDP and saves QUIC Client Initial Packet
            }
            tls
        }
        # protocols h3
    }
}

1.mydomain.com {
    # tls internal
    clienthellod { # handler
        # quic # mutually exclusive with tls
        tls # listener_wrappers.clienthellod.tcp must be set
    }
    file_server {
        root /var/www/html
    }
}

2.mydomain.com {
    # tls internal
    clienthellod { # handler
        quic # listener_wrappers.clienthellod.udp must be set
        # tls # mutually exclusive with quic
    }
    file_server {
        root /var/www/html
    }
}

License

This project is developed and distributed under Apache-2.0 license.

FOSSA Status

Documentation

Index

Constants

View Source 
const (
	QUICFrame_PADDING uint64 = 0 // 0
	QUICFrame_PING    uint64 = 1 // 1
	QUICFrame_CRYPTO  uint64 = 6 // 6
)
View Source 
const (
	TOKEN_ABSENT  uint32 = 0x00000000
	TOKEN_PRESENT uint32 = 0x00000001
)
View Source 
const (
	QTP_GREASE = 27

	UNSET_VLI_BITS = true // if false, unsetVLIBits() will be nop
)

Variables

View Source 
var (
	ErrNotQUICLongHeaderFormat = errors.New("packet is not in QUIC Long Header Format")
	ErrNotQUICInitialPacket    = errors.New("packet is not a QUIC Initial Packet")
)
View Source 
var ErrNoQUICClientHello = errors.New("no QUIC ClientHello found in the packet")

Functions

func ClientInitialKeysCalc added in v0.3.0 

func ClientInitialKeysCalc(initialRandom []byte) (clientKey, clientIV, clientHpKey []byte, err error)

func ComputeHeaderProtection added in v0.3.0 

func ComputeHeaderProtection(clientHpKey, sample []byte) ([]byte, error)

func DecodeVLI added in v0.3.0 

func DecodeVLI(vli []byte) (val uint64, err error)

func DecryptAES128GCM added in v0.3.0 

func DecryptAES128GCM(iv []byte, recordNum uint64, key, ciphertext, recdata, authtag []byte) (plaintext []byte, err error)

func IsGREASETransportParameter added in v0.3.0 

func IsGREASETransportParameter(paramType uint64) bool

func ReadNextVLI added in v0.3.0 

func ReadNextVLI(r io.Reader) (val uint64, n int, err error)

ReadNextVLI unpacks the next variable-length integer from the given io.Reader. It returns the decoded value and the number of bytes read. For example: 

0x0a -> 0xa, 1
0x80 0x10 0x00 0x00 -> 0x100000, 4

func ReassembleCRYPTOFrames added in v0.3.0 

func ReassembleCRYPTOFrames(frames []Frame) ([]byte, error)

Types

type CRYPTO added in v0.3.0 

type CRYPTO struct {
	Offset uint64 `json:"offset,omitempty"` // offset of crypto data, from VLI
	Length uint64 `json:"length,omitempty"` // length of crypto data, from VLI
	Data   []byte `json:"data,omitempty"`   // crypto data
}

CRYPTO frame

func (*CRYPTO) FrameType added in v0.3.0 

func (*CRYPTO) FrameType() uint64

FrameType implements Frame interface.

func (*CRYPTO) ReadReader added in v0.3.0 

func (f *CRYPTO) ReadReader(r io.Reader) (rr io.Reader, err error)

ReadFrom implements Frame interface. It reads the offset, length and crypto data from the input reader.

type ClientHello 

type ClientHello struct {
	TLSRecordVersion    uint16 `json:"tls_record_version"`    // TLS record version (major, minor)
	TLSHandshakeVersion uint16 `json:"tls_handshake_version"` // TLS handshake version (major, minor)

	CipherSuites         []uint16       `json:"cipher_suites"`
	CompressionMethods   utils.Uint8Arr `json:"compression_methods"`
	Extensions           []uint16       `json:"extensions"`            // extension IDs in original order
	ExtensionsNormalized []uint16       `json:"extensions_normalized"` // sorted extension IDs

	ServerName          string         `json:"server_name"`            // server_name(0)
	NamedGroupList      []uint16       `json:"supported_groups"`       // supported_groups(10) a.k.a elliptic_curves
	ECPointFormatList   utils.Uint8Arr `json:"ec_point_formats"`       // ec_point_formats(11)
	SignatureSchemeList []uint16       `json:"signature_algorithms"`   // signature_algorithms(13)
	ALPN                []string       `json:"alpn"`                   // alpn(16)
	CertCompressAlgo    []uint16       `json:"compress_certificate"`   // compress_certificate(27)
	RecordSizeLimit     utils.Uint8Arr `json:"record_size_limit"`      // record_size_limit(28)
	SupportedVersions   []uint16       `json:"supported_versions"`     // supported_versions(43)
	PSKKeyExchangeModes utils.Uint8Arr `json:"psk_key_exchange_modes"` // psk_key_exchange_modes(45)
	KeyShare            []uint16       `json:"key_share"`              // key_share(51)
	ApplicationSettings []string       `json:"application_settings"`   // application_settings(17513) a.k.a ALPS

	UserAgent string `json:"user_agent,omitempty"` // User-Agent header, set by the caller

	NID     int64  `json:"nid,omitempty"`      // NID of the fingerprint
	NormNID int64  `json:"norm_nid,omitempty"` // Normalized NID of the fingerprint
	ID      string `json:"id,omitempty"`       // ID of the fingerprint (hex string)
	NormID  string `json:"norm_id,omitempty"`  // Normalized ID of the fingerprint (hex string)
	// contains filtered or unexported fields
}

func ReadClientHello 

func ReadClientHello(r io.Reader) (ch *ClientHello, err error)

ReadClientHello reads a ClientHello from a connection (io.Reader) and returns a ClientHello struct.

It will return an error if the reader does not give a stream of bytes representing a valid ClientHello. But all bytes read from the reader will be stored in the ClientHello struct to be rewinded by the caller.

func (*ClientHello) FingerprintID 

func (ch *ClientHello) FingerprintID(normalized bool) string

FingerprintID calculates fingerprint ID of ClientHello and represents it as hexadecimal string.

func (*ClientHello) FingerprintNID 

func (ch *ClientHello) FingerprintNID(normalized bool) int64

FingerprintNID calculates fingerprint Numerical ID of ClientHello. Fingerprint is defined by

func (*ClientHello) ParseClientHello 

func (ch *ClientHello) ParseClientHello() error

ParseClientHello parses the raw bytes of a ClientHello into a ClientHello struct.

func (*ClientHello) Raw 

func (ch *ClientHello) Raw() []byte

type ClientInitialPacket added in v0.3.0 

type ClientInitialPacket struct {
	QHdr *QUICHeader              `json:"quic_header,omitempty"`               // QUIC header, set by the caller
	QCH  *QUICClientHello         `json:"quic_client_hello,omitempty"`         // TLS ClientHello, set by the caller
	QTP  *QUICTransportParameters `json:"quic_transport_parameters,omitempty"` // QUIC Transport Parameters, set by the caller

	HexID     string `json:"cip_fp_id,omitempty"`  // normalized
	NumericID uint64 `json:"cip_fp_nid,omitempty"` // original

	UserAgent string `json:"user_agent,omitempty"` // User-Agent header, set by the caller
	// contains filtered or unexported fields
}

func ParseQUICCIP added in v0.3.0 

func ParseQUICCIP(p []byte) (*ClientInitialPacket, error)

type Frame added in v0.3.0 

type Frame interface {
	// FrameType returns the type of the frame.
	FrameType() uint64

	// ReadReader takes a Reader and reads the rest of the frame from it,
	// starting from the first byte after the frame type.
	//
	// The returned io.Reader contains the rest of the frame, it could be
	// the input Reader itself (if no extra bytes are read) or a rewinded
	// Reader (if extra bytes are read and rewinding is needed).
	ReadReader(io.Reader) (io.Reader, error)
}

func ReadAllFrames added in v0.3.0 

func ReadAllFrames(r io.Reader) ([]Frame, error)

type PADDING added in v0.3.0 

type PADDING struct {
	Length uint64 `json:"length,omitempty"` // count 0x00 bytes until not 0x00
}

PADDING frame

func (*PADDING) FrameType added in v0.3.0 

func (*PADDING) FrameType() uint64

FrameType implements Frame interface.

func (*PADDING) ReadReader added in v0.3.0 

func (f *PADDING) ReadReader(r io.Reader) (rr io.Reader, err error)

ReadFrom implements Frame interface. It keeps reading until it finds a non-zero byte, then the non-zero byte is rewinded back to the reader and the reader is returned.

type PING added in v0.3.0 

type PING struct{}

PING frame

func (*PING) FrameType added in v0.3.0 

func (*PING) FrameType() uint64

FrameType implements Frame interface.

func (*PING) ReadReader added in v0.3.0 

func (*PING) ReadReader(r io.Reader) (rr io.Reader, err error)

ReadFrom implements Frame interface. It does nothing and returns the input reader.

type QUICClientHello added in v0.3.0 

type QUICClientHello struct {
	ClientHello
}

func ParseQUICClientHello added in v0.3.0 

func ParseQUICClientHello(p []byte) (*QUICClientHello, error)

func (*QUICClientHello) Raw added in v0.3.0 

func (qch *QUICClientHello) Raw() []byte

type QUICHeader added in v0.3.0 

type QUICHeader struct {
	Version      utils.Uint8Arr `json:"version,omitempty"` // 4-byte version
	DCIDLength   uint32         `json:"dcid_len,omitempty"`
	SCIDLength   uint32         `json:"scid_len,omitempty"`
	PacketNumber utils.Uint8Arr `json:"pn,omitempty"` // VLI

	// These two fields are not strictly part of QUIC header, but we need them before parsing QUIC ClientHello
	FrameIDs utils.Uint8Arr `json:"frame_id,omitempty"` // sorted

	Token bool `json:"token,omitempty"`

	HexID     string `json:"hdrid,omitempty"`
	NumericID uint64 `json:"hdrnid,omitempty"`
	// contains filtered or unexported fields
}

QUICHeader includes header fields of a QUIC packet and the following frames. It is used to calculate the fingerprint of a QUIC Header.

func DecodeQUICHeaderAndFrames added in v0.3.0 

func DecodeQUICHeaderAndFrames(p []byte) (*QUICHeader, error)

DecodeQUICHeaderAndFrames decodes a QUIC initial packet and returns a QUICHeader.

func (*QUICHeader) Frames added in v0.3.2 

func (qHdr *QUICHeader) Frames() []Frame

Frames returns all recognized frames in the QUIC header.

func (*QUICHeader) HID added in v0.3.2 

func (qHdr *QUICHeader) HID() string

HID returns a hex fingerprint ID for the QUIC header.

func (*QUICHeader) NID added in v0.3.2 

func (qHdr *QUICHeader) NID() uint64

NID returns a numeric fingerprint ID for the QUIC header.

type QUICTransportParameters added in v0.3.0 

type QUICTransportParameters struct {
	MaxIdleTimeout                 utils.Uint8Arr `json:"max_idle_timeout,omitempty"`
	MaxUDPPayloadSize              utils.Uint8Arr `json:"max_udp_payload_size,omitempty"`
	InitialMaxData                 utils.Uint8Arr `json:"initial_max_data,omitempty"`
	InitialMaxStreamDataBidiLocal  utils.Uint8Arr `json:"initial_max_stream_data_bidi_local,omitempty"`
	InitialMaxStreamDataBidiRemote utils.Uint8Arr `json:"initial_max_stream_data_bidi_remote,omitempty"`
	InitialMaxStreamDataUni        utils.Uint8Arr `json:"initial_max_stream_data_uni,omitempty"`
	InitialMaxStreamsBidi          utils.Uint8Arr `json:"initial_max_streams_bidi,omitempty"`
	InitialMaxStreamsUni           utils.Uint8Arr `json:"initial_max_streams_uni,omitempty"`
	AckDelayExponent               utils.Uint8Arr `json:"ack_delay_exponent,omitempty"`
	MaxAckDelay                    utils.Uint8Arr `json:"max_ack_delay,omitempty"`

	ActiveConnectionIDLimit utils.Uint8Arr `json:"active_connection_id_limit,omitempty"`
	QTPIDs                  []uint64       `json:"qtpid,omitempty"` // sorted

	HexID     string `json:"tpfpid,omitempty"`
	NumericID uint64 `json:"tpfnid,omitempty"`
	// contains filtered or unexported fields
}

QUICTransportParameters is a struct to hold the parsed QUIC transport parameters as a combination.

func ParseQUICTransportParameters added in v0.3.0 

func ParseQUICTransportParameters(extData []byte) *QUICTransportParameters

ParseQUICTransportParameters parses the transport parameters from the extension data of TLS Extension "QUIC Transport Parameters" (57)

If any error occurs, the returned struct will have parseError set to the error.

func (*QUICTransportParameters) HID added in v0.3.2 

func (qtp *QUICTransportParameters) HID() string

HID returns the hex ID of this transport parameters combination.

func (*QUICTransportParameters) NID added in v0.3.2 

func (qtp *QUICTransportParameters) NID() uint64

NID returns the numeric ID of this transport parameters combination.

func (*QUICTransportParameters) ParseError added in v0.3.2 

func (qtp *QUICTransportParameters) ParseError() error

ParseError returns the error that occurred during parsing, if any.

Source Files

View all Source files

Directories

Path Synopsis
internal
utils
app
handler
listener

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.