kcp

package module
v5.0.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 28, 2018 License: MIT Imports: 27 Imported by: 0

README

kcp-go

GoDoc Powered MIT licensed Build Status Go Report Card Coverage Statusd

Introduction

kcp-go is a Production-Grade Reliable-UDP library for golang.

This library intents to provide a smooth, resilient, ordered, error-checked and anonymous delivery of streams over UDP packets, it has been battle-tested with opensource project kcptun. Millions of devices(from low-end MIPS routers to high-end servers) have deployed kcp-go powered program in a variety of forms like online games, live broadcasting, file synchronization and network acceleration.

Lastest Release

Features

  1. Designed for Latency-sensitive scenarios.
  2. Cache friendly and Memory optimized design, offers extremely High Performance core.
  3. Handles >5K concurrent connections on a single commodity server.
  4. Compatible with net.Conn and net.Listener, a drop-in replacement for net.TCPConn.
  5. FEC(Forward Error Correction) Support with Reed-Solomon Codes
  6. Packet level encryption support with AES, TEA, 3DES, Blowfish, Cast5, Salsa20, etc. in CFB mode, which generates completely anonymous packet.
  7. Only A fixed number of goroutines will be created for the entire server application, costs in context switch between goroutines have been taken into consideration.
  8. Compatible with skywind3000's C version with various improvements.

Documentation

For complete documentation, see the associated Godoc.

Specification

Frame Format 
+-----------------+
| SESSION         |
+-----------------+
| KCP(ARQ)        |
+-----------------+
| FEC(OPTIONAL)   |
+-----------------+
| CRYPTO(OPTIONAL)|
+-----------------+
| UDP(PACKET)     |
+-----------------+
| IP              |
+-----------------+
| LINK            |
+-----------------+
| PHY             |
+-----------------+
(LAYER MODEL OF KCP-GO)

Usage

Client: full demo

kcpconn, err := kcp.DialWithOptions("192.168.0.1:10000", nil, 10, 3)

Server: full demo

lis, err := kcp.ListenWithOptions(":10000", nil, 10, 3)

Benchmark

  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro14,1
  Processor Name:	Intel Core i5
  Processor Speed:	3.1 GHz
  Number of Processors:	1
  Total Number of Cores:	2
  L2 Cache (per Core):	256 KB
  L3 Cache:	4 MB
  Memory:	8 GB

$ go test -v -run=^$ -bench .
beginning tests, encryption:salsa20, fec:10/3
goos: darwin
goarch: amd64
pkg: github.com/xtaci/kcp-go
BenchmarkSM4-4                 	   50000	     32180 ns/op	  93.23 MB/s	       0 B/op	       0 allocs/op
BenchmarkAES128-4              	  500000	      3285 ns/op	 913.21 MB/s	       0 B/op	       0 allocs/op
BenchmarkAES192-4              	  300000	      3623 ns/op	 827.85 MB/s	       0 B/op	       0 allocs/op
BenchmarkAES256-4              	  300000	      3874 ns/op	 774.20 MB/s	       0 B/op	       0 allocs/op
BenchmarkTEA-4                 	  100000	     15384 ns/op	 195.00 MB/s	       0 B/op	       0 allocs/op
BenchmarkXOR-4                 	20000000	        89.9 ns/op	33372.00 MB/s	       0 B/op	       0 allocs/op
BenchmarkBlowfish-4            	   50000	     26927 ns/op	 111.41 MB/s	       0 B/op	       0 allocs/op
BenchmarkNone-4                	30000000	        45.7 ns/op	65597.94 MB/s	       0 B/op	       0 allocs/op
BenchmarkCast5-4               	   50000	     34258 ns/op	  87.57 MB/s	       0 B/op	       0 allocs/op
Benchmark3DES-4                	   10000	    117149 ns/op	  25.61 MB/s	       0 B/op	       0 allocs/op
BenchmarkTwofish-4             	   50000	     33538 ns/op	  89.45 MB/s	       0 B/op	       0 allocs/op
BenchmarkXTEA-4                	   30000	     45666 ns/op	  65.69 MB/s	       0 B/op	       0 allocs/op
BenchmarkSalsa20-4             	  500000	      3308 ns/op	 906.76 MB/s	       0 B/op	       0 allocs/op
BenchmarkCRC32-4               	20000000	        65.2 ns/op	15712.43 MB/s
BenchmarkCsprngSystem-4        	 1000000	      1150 ns/op	  13.91 MB/s
BenchmarkCsprngMD5-4           	10000000	       145 ns/op	 110.26 MB/s
BenchmarkCsprngSHA1-4          	10000000	       158 ns/op	 126.54 MB/s
BenchmarkCsprngNonceMD5-4      	10000000	       153 ns/op	 104.22 MB/s
BenchmarkCsprngNonceAES128-4   	100000000	        19.1 ns/op	 837.81 MB/s
BenchmarkFECDecode-4           	 1000000	      1119 ns/op	1339.61 MB/s	    1606 B/op	       2 allocs/op
BenchmarkFECEncode-4           	 2000000	       832 ns/op	1801.83 MB/s	      17 B/op	       0 allocs/op
BenchmarkFlush-4               	 5000000	       272 ns/op	       0 B/op	       0 allocs/op
BenchmarkEchoSpeed4K-4         	    5000	    259617 ns/op	  15.78 MB/s	    5451 B/op	     149 allocs/op
BenchmarkEchoSpeed64K-4        	    1000	   1706084 ns/op	  38.41 MB/s	   56002 B/op	    1604 allocs/op
BenchmarkEchoSpeed512K-4       	     100	  14345505 ns/op	  36.55 MB/s	  482597 B/op	   13045 allocs/op
BenchmarkEchoSpeed1M-4         	      30	  34859104 ns/op	  30.08 MB/s	 1143773 B/op	   27186 allocs/op
BenchmarkSinkSpeed4K-4         	   50000	     31369 ns/op	 130.57 MB/s	    1566 B/op	      30 allocs/op
BenchmarkSinkSpeed64K-4        	    5000	    329065 ns/op	 199.16 MB/s	   21529 B/op	     453 allocs/op
BenchmarkSinkSpeed256K-4       	     500	   2373354 ns/op	 220.91 MB/s	  166332 B/op	    3554 allocs/op
BenchmarkSinkSpeed1M-4         	     300	   5117927 ns/op	 204.88 MB/s	  310378 B/op	    6988 allocs/op
PASS
ok  	github.com/xtaci/kcp-go	50.349s

Key Design Considerations

  1. slice vs. container/list

kcp.flush() loops through the send queue for retransmission checking for every 20ms(interval).

I've wrote a benchmark for comparing sequential loop through slice and container/list here:

https://github.com/xtaci/notes/blob/master/golang/benchmark2/cachemiss_test.go

BenchmarkLoopSlice-4   	2000000000	         0.39 ns/op
BenchmarkLoopList-4    	100000000	        54.6 ns/op

List structure introduces heavy cache misses compared to slice which owns better locality, 5000 connections with 32 window size and 20ms interval will cost 6us/0.03%(cpu) using slice, and 8.7ms/43.5%(cpu) for list for each kcp.flush().

  1. Timing accuracy vs. syscall clock_gettime

Timing is critical to RTT estimator, inaccurate timing leads to false retransmissions in KCP, but calling time.Now() costs 42 cycles(10.5ns on 4GHz CPU, 15.6ns on my MacBook Pro 2.7GHz).

The benchmark for time.Now() lies here:

https://github.com/xtaci/notes/blob/master/golang/benchmark2/syscall_test.go

BenchmarkNow-4         	100000000	        15.6 ns/op

In kcp-go, after each kcp.output() function call, current clock time will be updated upon return, and for a single kcp.flush() operation, current time will be queried from system once. For most of the time, 5000 connections costs 5000 * 15.6ns = 78us(a fixed cost while no packet needs to be sent), as for 10MB/s data transfering with 1400 MTU, kcp.output() will be called around 7500 times and costs 117us for time.Now() in every second.

Connection Termination

Control messages like SYN/FIN/RST in TCP are not defined in KCP, you need some keepalive/heartbeat mechanism in the application-level. A real world example is to use some multiplexing protocol over session, such as smux(with embedded keepalive mechanism), see kcptun for example.

FAQ

Q: I'm handling >5K connections on my server, the CPU utilization is so high.

A: A standalone agent or gate server for running kcp-go is suggested, not only for CPU utilization, but also important to the precision of RTT measurements(timing) which indirectly affects retransmission. By increasing update interval with SetNoDelay like conn.SetNoDelay(1, 40, 1, 1) will dramatically reduce system load, but lower the performance.

Who is using this?

  1. https://github.com/xtaci/kcptun -- A Secure Tunnel Based On KCP over UDP.
  2. https://github.com/getlantern/lantern -- Lantern delivers fast access to the open Internet.
  3. https://github.com/smallnest/rpcx -- A RPC service framework based on net/rpc like alibaba Dubbo and weibo Motan.
  4. https://github.com/gonet2/agent -- A gateway for games with stream multiplexing.
  5. https://github.com/syncthing/syncthing -- Open Source Continuous File Synchronization.
  6. https://play.google.com/store/apps/details?id=com.k17game.k3 -- Battle Zone - Earth 2048, a world-wide strategy game.
  1. https://github.com/xtaci/libkcp -- FEC enhanced KCP session library for iOS/Android in C++
  2. https://github.com/skywind3000/kcp -- A Fast and Reliable ARQ Protocol
  3. https://github.com/klauspost/reedsolomon -- Reed-Solomon Erasure Coding in Go

Documentation

Overview

Package kcp - A Fast and Reliable ARQ Protocol

Index

Constants

View Source 
const (
	IKCP_RTO_NDL     = 30  // no delay min rto
	IKCP_RTO_MIN     = 100 // normal min rto
	IKCP_RTO_DEF     = 200
	IKCP_RTO_MAX     = 60000
	IKCP_CMD_PUSH    = 81 // cmd: push data
	IKCP_CMD_ACK     = 82 // cmd: ack
	IKCP_CMD_WASK    = 83 // cmd: window probe (ask)
	IKCP_CMD_WINS    = 84 // cmd: window size (tell)
	IKCP_ASK_SEND    = 1  // need to send IKCP_CMD_WASK
	IKCP_ASK_TELL    = 2  // need to send IKCP_CMD_WINS
	IKCP_WND_SND     = 32
	IKCP_WND_RCV     = 32
	IKCP_MTU_DEF     = 1400
	IKCP_ACK_FAST    = 3
	IKCP_INTERVAL    = 100
	IKCP_OVERHEAD    = 24
	IKCP_DEADLINK    = 20
	IKCP_THRESH_INIT = 2
	IKCP_THRESH_MIN  = 2
	IKCP_PROBE_INIT  = 7000   // 7 secs to probe window size
	IKCP_PROBE_LIMIT = 120000 // up to 120 secs to probe window
)

Variables

This section is empty.

Functions

func Dial 

func Dial(raddr string) (net.Conn, error)

Dial connects to the remote address "raddr" on the network "udp"

func Listen 

func Listen(laddr string) (net.Listener, error)

Listen listens for incoming KCP packets addressed to the local address laddr on the network "udp",

Types

type BlockCrypt 

type BlockCrypt interface {
	// Encrypt encrypts the whole block in src into dst.
	// Dst and src may point at the same memory.
	Encrypt(dst, src []byte)

	// Decrypt decrypts the whole block in src into dst.
	// Dst and src may point at the same memory.
	Decrypt(dst, src []byte)
}

BlockCrypt defines encryption/decryption methods for a given byte slice. Notes on implementing: the data to be encrypted contains a builtin nonce at the first 16 bytes

func NewAESBlockCrypt 

func NewAESBlockCrypt(key []byte) (BlockCrypt, error)

NewAESBlockCrypt https://en.wikipedia.org/wiki/Advanced_Encryption_Standard

func NewBlowfishBlockCrypt 

func NewBlowfishBlockCrypt(key []byte) (BlockCrypt, error)

NewBlowfishBlockCrypt https://en.wikipedia.org/wiki/Blowfish_(cipher)

func NewCast5BlockCrypt 

func NewCast5BlockCrypt(key []byte) (BlockCrypt, error)

NewCast5BlockCrypt https://en.wikipedia.org/wiki/CAST-128

func NewNoneBlockCrypt 

func NewNoneBlockCrypt(key []byte) (BlockCrypt, error)

NewNoneBlockCrypt does nothing but copying

func NewSM4BlockCrypt 

func NewSM4BlockCrypt(key []byte) (BlockCrypt, error)

NewSM4BlockCrypt https://github.com/tjfoc/gmsm/tree/master/sm4

func NewSalsa20BlockCrypt 

func NewSalsa20BlockCrypt(key []byte) (BlockCrypt, error)

NewSalsa20BlockCrypt https://en.wikipedia.org/wiki/Salsa20

func NewSimpleXORBlockCrypt 

func NewSimpleXORBlockCrypt(key []byte) (BlockCrypt, error)

NewSimpleXORBlockCrypt simple xor with key expanding

func NewTEABlockCrypt 

func NewTEABlockCrypt(key []byte) (BlockCrypt, error)

NewTEABlockCrypt https://en.wikipedia.org/wiki/Tiny_Encryption_Algorithm

func NewTripleDESBlockCrypt 

func NewTripleDESBlockCrypt(key []byte) (BlockCrypt, error)

NewTripleDESBlockCrypt https://en.wikipedia.org/wiki/Triple_DES

func NewTwofishBlockCrypt 

func NewTwofishBlockCrypt(key []byte) (BlockCrypt, error)

NewTwofishBlockCrypt https://en.wikipedia.org/wiki/Twofish

func NewXTEABlockCrypt 

func NewXTEABlockCrypt(key []byte) (BlockCrypt, error)

NewXTEABlockCrypt https://en.wikipedia.org/wiki/XTEA

type Entropy 

type Entropy interface {
	Init()
	Fill(nonce []byte)
}

Entropy defines a entropy source

type KCP 

type KCP struct {
	// contains filtered or unexported fields
}

KCP defines a single KCP connection

func NewKCP 

func NewKCP(conv uint32, output output_callback) *KCP

NewKCP create a new kcp control object, 'conv' must equal in two endpoint from the same connection.

func (*KCP) Check 

func (kcp *KCP) Check() uint32

Check determines when should you invoke ikcp_update: returns when you should invoke ikcp_update in millisec, if there is no ikcp_input/_send calling. you can call ikcp_update in that time, instead of call update repeatly. Important to reduce unnacessary ikcp_update invoking. use it to schedule ikcp_update (eg. implementing an epoll-like mechanism, or optimize ikcp_update when handling massive kcp connections)

func (*KCP) Input 

func (kcp *KCP) Input(data []byte, regular, ackNoDelay bool) int

Input when you received a low level packet (eg. UDP packet), call it regular indicates a regular packet has received(not from FEC)

func (*KCP) NoDelay 

func (kcp *KCP) NoDelay(nodelay, interval, resend, nc int) int

NoDelay options fastest: ikcp_nodelay(kcp, 1, 20, 2, 1) nodelay: 0:disable(default), 1:enable interval: internal update timer interval in millisec, default is 100ms resend: 0:disable fast resend(default), 1:enable fast resend nc: 0:normal congestion control(default), 1:disable congestion control

func (*KCP) PeekSize 

func (kcp *KCP) PeekSize() (length int)

PeekSize checks the size of next message in the recv queue

func (*KCP) Recv 

func (kcp *KCP) Recv(buffer []byte) (n int)

Recv is user/upper level recv: returns size, returns below zero for EAGAIN

func (*KCP) Send 

func (kcp *KCP) Send(buffer []byte) int

Send is user/upper level send, returns below zero for error

func (*KCP) SetMtu 

func (kcp *KCP) SetMtu(mtu int) int

SetMtu changes MTU size, default is 1400

func (*KCP) Update 

func (kcp *KCP) Update()

Update updates state (call it repeatedly, every 10ms-100ms), or you can ask ikcp_check when to call it again (without ikcp_input/_send calling). 'current' - current timestamp in millisec.

func (*KCP) WaitSnd 

func (kcp *KCP) WaitSnd() int

WaitSnd gets how many packet is waiting to be sent

func (*KCP) WndSize 

func (kcp *KCP) WndSize(sndwnd, rcvwnd int) int

WndSize sets maximum window size: sndwnd=32, rcvwnd=32 by default

type Listener 

type Listener struct {
	// contains filtered or unexported fields
}

Listener defines a server which will be waiting to accept incoming connections

func ListenWithOptions 

func ListenWithOptions(laddr string, block BlockCrypt, dataShards, parityShards int) (*Listener, error)

ListenWithOptions listens for incoming KCP packets addressed to the local address laddr on the network "udp" with packet encryption, dataShards, parityShards defines Reed-Solomon Erasure Coding parameters

func ServeConn 

func ServeConn(block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*Listener, error)

ServeConn serves KCP protocol for a single packet connection.

func (*Listener) Accept 

func (l *Listener) Accept() (net.Conn, error)

Accept implements the Accept method in the Listener interface; it waits for the next call and returns a generic Conn.

func (*Listener) AcceptKCP 

func (l *Listener) AcceptKCP() (*UDPSession, error)

AcceptKCP accepts a KCP connection

func (*Listener) Addr 

func (l *Listener) Addr() net.Addr

Addr returns the listener's network address, The Addr returned is shared by all invocations of Addr, so do not modify it.

func (*Listener) Close 

func (l *Listener) Close() error

Close stops listening on the UDP address. Already Accepted connections are not closed.

func (*Listener) SetDSCP 

func (l *Listener) SetDSCP(dscp int) error

SetDSCP sets the 6bit DSCP field of IP header

func (*Listener) SetDeadline 

func (l *Listener) SetDeadline(t time.Time) error

SetDeadline sets the deadline associated with the listener. A zero time value disables the deadline.

func (*Listener) SetReadBuffer 

func (l *Listener) SetReadBuffer(bytes int) error

SetReadBuffer sets the socket read buffer for the Listener

func (*Listener) SetReadDeadline 

func (l *Listener) SetReadDeadline(t time.Time) error

SetReadDeadline implements the Conn SetReadDeadline method.

func (*Listener) SetWriteBuffer 

func (l *Listener) SetWriteBuffer(bytes int) error

SetWriteBuffer sets the socket write buffer for the Listener

func (*Listener) SetWriteDeadline 

func (l *Listener) SetWriteDeadline(t time.Time) error

SetWriteDeadline implements the Conn SetWriteDeadline method.

type Snmp 

type Snmp struct {
	BytesSent        uint64 // bytes sent from upper level
	BytesReceived    uint64 // bytes received to upper level
	MaxConn          uint64 // max number of connections ever reached
	ActiveOpens      uint64 // accumulated active open connections
	PassiveOpens     uint64 // accumulated passive open connections
	CurrEstab        uint64 // current number of established connections
	InErrs           uint64 // UDP read errors reported from net.PacketConn
	InCsumErrors     uint64 // checksum errors from CRC32
	KCPInErrors      uint64 // packet iput errors reported from KCP
	InPkts           uint64 // incoming packets count
	OutPkts          uint64 // outgoing packets count
	InSegs           uint64 // incoming KCP segments
	OutSegs          uint64 // outgoing KCP segments
	InBytes          uint64 // UDP bytes received
	OutBytes         uint64 // UDP bytes sent
	RetransSegs      uint64 // accmulated retransmited segments
	FastRetransSegs  uint64 // accmulated fast retransmitted segments
	EarlyRetransSegs uint64 // accmulated early retransmitted segments
	LostSegs         uint64 // number of segs infered as lost
	RepeatSegs       uint64 // number of segs duplicated
	FECRecovered     uint64 // correct packets recovered from FEC
	FECErrs          uint64 // incorrect packets recovered from FEC
	FECParityShards  uint64 // FEC segments received
	FECShortShards   uint64 // number of data shards that's not enough for recovery
}

Snmp defines network statistics indicator 

var DefaultSnmp *Snmp

DefaultSnmp is the global KCP connection statistics collector

func (*Snmp) Copy 

func (s *Snmp) Copy() *Snmp

Copy make a copy of current snmp snapshot

func (*Snmp) Header 

func (s *Snmp) Header() []string

Header returns all field names

func (*Snmp) Reset 

func (s *Snmp) Reset()

Reset values to zero

func (*Snmp) ToSlice 

func (s *Snmp) ToSlice() []string

ToSlice returns current snmp info as slice

type UDPSession 

type UDPSession struct {
	// contains filtered or unexported fields
}

UDPSession defines a KCP session implemented by UDP

func DialWithOptions 

func DialWithOptions(raddr string, block BlockCrypt, dataShards, parityShards int) (*UDPSession, error)

DialWithOptions connects to the remote address "raddr" on the network "udp" with packet encryption

func NewConn 

func NewConn(raddr string, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error)

NewConn establishes a session and talks KCP protocol over a packet connection.

func (*UDPSession) Close 

func (s *UDPSession) Close() error

Close closes the connection.

func (*UDPSession) GetConv 

func (s *UDPSession) GetConv() uint32

GetConv gets conversation id of a session

func (*UDPSession) LocalAddr 

func (s *UDPSession) LocalAddr() net.Addr

LocalAddr returns the local network address. The Addr returned is shared by all invocations of LocalAddr, so do not modify it.

func (*UDPSession) Read 

func (s *UDPSession) Read(b []byte) (n int, err error)

Read implements net.Conn

func (*UDPSession) RemoteAddr 

func (s *UDPSession) RemoteAddr() net.Addr

RemoteAddr returns the remote network address. The Addr returned is shared by all invocations of RemoteAddr, so do not modify it.

func (*UDPSession) SetACKNoDelay 

func (s *UDPSession) SetACKNoDelay(nodelay bool)

SetACKNoDelay changes ack flush option, set true to flush ack immediately,

func (*UDPSession) SetDSCP 

func (s *UDPSession) SetDSCP(dscp int) error

SetDSCP sets the 6bit DSCP field of IP header, no effect if it's accepted from Listener

func (*UDPSession) SetDUP 

func (s *UDPSession) SetDUP(dup int)

SetDUP duplicates udp packets for kcp output, for testing purpose only

func (*UDPSession) SetDeadline 

func (s *UDPSession) SetDeadline(t time.Time) error

SetDeadline sets the deadline associated with the listener. A zero time value disables the deadline.

func (*UDPSession) SetMtu 

func (s *UDPSession) SetMtu(mtu int) bool

SetMtu sets the maximum transmission unit(not including UDP header)

func (*UDPSession) SetNoDelay 

func (s *UDPSession) SetNoDelay(nodelay, interval, resend, nc int)

SetNoDelay calls nodelay() of kcp https://github.com/skywind3000/kcp/blob/master/README.en.md#protocol-configuration

func (*UDPSession) SetReadBuffer 

func (s *UDPSession) SetReadBuffer(bytes int) error

SetReadBuffer sets the socket read buffer, no effect if it's accepted from Listener

func (*UDPSession) SetReadDeadline 

func (s *UDPSession) SetReadDeadline(t time.Time) error

SetReadDeadline implements the Conn SetReadDeadline method.

func (*UDPSession) SetStreamMode 

func (s *UDPSession) SetStreamMode(enable bool)

SetStreamMode toggles the stream mode on/off

func (*UDPSession) SetWindowSize 

func (s *UDPSession) SetWindowSize(sndwnd, rcvwnd int)

SetWindowSize set maximum window size

func (*UDPSession) SetWriteBuffer 

func (s *UDPSession) SetWriteBuffer(bytes int) error

SetWriteBuffer sets the socket write buffer, no effect if it's accepted from Listener

func (*UDPSession) SetWriteDeadline 

func (s *UDPSession) SetWriteDeadline(t time.Time) error

SetWriteDeadline implements the Conn SetWriteDeadline method.

func (*UDPSession) SetWriteDelay 

func (s *UDPSession) SetWriteDelay(delay bool)

SetWriteDelay delays write for bulk transfer until the next update interval

func (*UDPSession) Write 

func (s *UDPSession) Write(b []byte) (n int, err error)

Write implements net.Conn

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.