prefork

README

Prefork

Server prefork implementation.

Preforks master process between several child processes increases performance, because Go doesn't have to share and manage memory between cores.

WARNING: using prefork prevents the use of any global state!. Things like in-memory caches won't work.

• How it works:

import (
    "github.com/valyala/fasthttp"
    "github.com/valyala/fasthttp/prefork"
)

server := &fasthttp.Server{
    // Your configuration
}

// Wraps the server with prefork
preforkServer := prefork.New(server)

if err := preforkServer.ListenAndServe(":8080"); err != nil {
    panic(err)
}

Benchmarks

Environment:

• Machine: MacBook Pro 13-inch, 2017
• OS: MacOS 10.15.3
• Go: go1.13.6 darwin/amd64

Handler code:

func requestHandler(ctx *fasthttp.RequestCtx) {
    // Simulates some hard work
    time.Sleep(100 * time.Millisecond)
}

Test command:

$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 512 --timeout 8 -t 4 http://localhost:8080

Results:

• prefork

Running 15s test @ http://localhost:8080
  4 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.75ms    4.27ms 126.24ms   97.45%
    Req/Sec    26.46k     4.16k   71.18k    88.72%
  Latency Distribution
     50%    4.55ms
     75%    4.82ms
     90%    5.46ms
     99%   15.49ms
  1581916 requests in 15.09s, 140.30MB read
  Socket errors: connect 0, read 318, write 0, timeout 0
Requests/sec: 104861.58
Transfer/sec:      9.30MB

• non-prefork

Running 15s test @ http://localhost:8080
  4 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.42ms   11.83ms 177.19ms   96.42%
    Req/Sec    24.96k     5.83k   56.83k    82.93%
  Latency Distribution
     50%    4.53ms
     75%    4.93ms
     90%    6.94ms
     99%   74.54ms
  1472441 requests in 15.09s, 130.59MB read
  Socket errors: connect 0, read 265, write 0, timeout 0
Requests/sec:  97553.34
Transfer/sec:      8.65MB

Documentation

Index

• Variables
• func IsChild() bool
• type Logger
• type Prefork
  • func New(s *fasthttp.Server) *Prefork
  • func (p *Prefork) ListenAndServe(addr string) error
  • func (p *Prefork) ListenAndServeTLS(addr, certKey, certFile string) error
  • func (p *Prefork) ListenAndServeTLSEmbed(addr string, certData, keyData []byte) error

Variables

var (

	// ErrOverRecovery is returned when the times of starting over child prefork processes exceed
	// the threshold.
	ErrOverRecovery = errors.New("exceeding the value of RecoverThreshold")

	// ErrOnlyReuseportOnWindows is returned when Reuseport is false.
	ErrOnlyReuseportOnWindows = errors.New("windows only supports Reuseport = true")
)

Functions

func IsChild

func IsChild() bool

IsChild checks if the current thread/process is a child.

Types

type Logger

type Logger interface {
	// Printf must have the same semantics as log.Printf.
	Printf(format string, args ...any)
}

Logger is used for logging formatted messages.

type Prefork

type Prefork struct {
	// The network must be "tcp", "tcp4" or "tcp6".
	//
	// By default is "tcp4"
	Network string

	// Flag to use a listener with reuseport, if not a file Listener will be used
	// See: https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/
	//
	// It's disabled by default
	Reuseport bool

	// Child prefork processes may exit with failure and will be started over until the times reach
	// the value of RecoverThreshold, then it will return and terminate the server.
	RecoverThreshold int

	// By default standard logger from log package is used.
	Logger Logger

	ServeFunc         func(ln net.Listener) error
	ServeTLSFunc      func(ln net.Listener, certFile, keyFile string) error
	ServeTLSEmbedFunc func(ln net.Listener, certData, keyData []byte) error
	// contains filtered or unexported fields
}

Prefork implements fasthttp server prefork.

Preforks master process (with all cores) between several child processes increases performance significantly, because Go doesn't have to share and manage memory between cores.

WARNING: using prefork prevents the use of any global state! Things like in-memory caches won't work.

func New

func New(s *fasthttp.Server) *Prefork

New wraps the fasthttp server to run with preforked processes.

func (*Prefork) ListenAndServe

func (p *Prefork) ListenAndServe(addr string) error

ListenAndServe serves HTTP requests from the given TCP addr.

func (*Prefork) ListenAndServeTLS

func (p *Prefork) ListenAndServeTLS(addr, certKey, certFile string) error

ListenAndServeTLS serves HTTPS requests from the given TCP addr.

certFile and keyFile are paths to TLS certificate and key files.

func (*Prefork) ListenAndServeTLSEmbed

func (p *Prefork) ListenAndServeTLSEmbed(addr string, certData, keyData []byte) error

ListenAndServeTLSEmbed serves HTTPS requests from the given TCP addr.

certData and keyData must contain valid TLS certificate and key data.