gzip

README

English | 中文

Gzip Middleware for Go

An out-of-the-box, also customizable gzip middleware for Gin and net/http.

Examples

Use DefaultHandler() to create a ready-to-go gzip middleware.

Gin

import github.com/nanmu42/gzip

func main() {
	g := gin.Default()

    // use default settings
	g.Use(gzip.DefaultHandler().Gin)

	g.GET("/", func(c *gin.Context) {
		c.JSON(http.StatusOK, map[string]interface{}{
			"code": 0,
			"msg":  "hello",
			"data": fmt.Sprintf("l%sng!", strings.Repeat("o", 1000)),
		})
	})

	log.Println(g.Run(fmt.Sprintf(":%d", 3000)))
}

net/http

import github.com/nanmu42/gzip

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		writeString(w, fmt.Sprintf("This content is compressed: l%sng!", strings.Repeat("o", 1000)))
	})

    // wrap http.Handler using default settings
	log.Println(http.ListenAndServe(fmt.Sprintf(":%d", 3001), gzip.DefaultHandler().WrapHandler(mux)))
}

func writeString(w http.ResponseWriter, payload string) {
	w.Header().Set("Content-Type", "text/plain; charset=utf8")
	_, _ = io.WriteString(w, payload+"\n")
}

Customize Handler

Handler can be customized using NewHandler and Config:

import github.com/nanmu42/gzip

handler := gzip.NewHandler(gzip.Config{
    // gzip compression level to use
	CompressionLevel: 6, 
    // minimum content length to trigger gzip, the unit is in byte.
	MinContentLength: 1024,
    // RequestFilter decide whether or not to compress response judging by request.
    // Filters are applied in the sequence here.
	RequestFilter: []RequestFilter{
	    NewCommonRequestFilter(),
	    DefaultExtensionFilter(),
	},
    // ResponseHeaderFilter decide whether or not to compress response
    // judging by response header
	ResponseHeaderFilter: []ResponseHeaderFilter{
		NewSkipCompressedFilter(),
		DefaultContentTypeFilter(),
	},
})

RequestFilter and ResponseHeaderFilter are interfaces. You may define one that specially suits your need.

Performance

• When response payload is small, the handler is smart enough to skip compression automatically, which takes neglectable overhead.
• At the time when the payload is big enough, gzip kicks in and there is a reasonable price.

$ go test -benchmem -bench .
goos: linux
goarch: amd64
pkg: github.com/nanmu42/gzip
BenchmarkSoleGin_SmallPayload-4                          4104684               276 ns/op              64 B/op          2 allocs/op
BenchmarkGinWithDefaultHandler_SmallPayload-4            1683307               707 ns/op              96 B/op          3 allocs/op
BenchmarkSoleGin_BigPayload-4                            4198786               274 ns/op              64 B/op          2 allocs/op
BenchmarkGinWithDefaultHandler_BigPayload-4                44780             27636 ns/op             190 B/op          5 allocs/op
PASS
ok      github.com/nanmu42/gzip 6.373s

Note: due to an awkward man-mistake, benchmark of and before v1.0.0 are not accurate.

Limitation

• You should always provide a Content-Type in http response's header, though handler guesses by http.DetectContentType()as makeshift;
• When Content-Length is not available, handler may buffer your writes to decide if its big enough to do a meaningful compression. A high MinContentLength may bring memory overhead, although the handler tries to be smart by reusing buffers and testing if len(data) of the first http.ResponseWriter.Write(data []byte) calling suffices or not.

Status: Stable

All APIs are finalized, and no breaking changes will be made in the 1.x series of releases.

Acknowledgement

During the development of this work, the author took following works/materials as reference:

• https://github.com/caddyserver/caddy/tree/master/caddyhttp/gzip (Apache License 2.0)
• https://github.com/gin-contrib/gzip (MIT License)
• https://blog.cloudflare.com/results-experimenting-brotli/
• https://support.cloudflare.com/hc/en-us/articles/200168396-What-will-Cloudflare-compress-

This package uses klauspost's compress package to handle gzip compression.

Logo generated at Gopherize.me.

License

MIT License
Copyright (c) 2019 LI Zhennan

Caddy is licensed under the Apache License
Copyright 2015 Light Code Labs, LLC

Documentation

Overview

Package gzip implements gzip middleware for Gin and net/http.

* introduction: https://github.com/nanmu42/gzip

* examples: https://github.com/nanmu42/gzip/tree/master/examples

Index

• Constants
• type CommonRequestFilter
  • func NewCommonRequestFilter() *CommonRequestFilter
  • func (c *CommonRequestFilter) ShouldCompress(req *http.Request) bool
• type Config
• type ContentTypeFilter
  • func DefaultContentTypeFilter() *ContentTypeFilter
  • func NewContentTypeFilter(types []string) *ContentTypeFilter
  • func (e *ContentTypeFilter) ShouldCompress(header http.Header) bool
• type ExtensionFilter
  • func DefaultExtensionFilter() *ExtensionFilter
  • func NewExtensionFilter(extensions []string) *ExtensionFilter
  • func (e *ExtensionFilter) ShouldCompress(req *http.Request) bool
• type Handler
  • func DefaultHandler() *Handler
  • func NewHandler(config Config) *Handler
  • func (h *Handler) Gin(c *gin.Context)
  • func (h *Handler) WrapHandler(next http.Handler) http.Handler
• type RequestFilter
• type ResponseHeaderFilter
• type SkipCompressedFilter
  • func NewSkipCompressedFilter() *SkipCompressedFilter
  • func (s *SkipCompressedFilter) ShouldCompress(header http.Header) bool

Constants

const (
	NoCompression      = gzip.NoCompression
	BestSpeed          = gzip.BestSpeed
	BestCompression    = gzip.BestCompression
	DefaultCompression = gzip.DefaultCompression
	HuffmanOnly        = gzip.HuffmanOnly
	// Stateless will do compression but without maintaining any state
	// between Write calls, so long running responses will not take memory.
	// There will be no memory kept between Write calls,
	// but compression and speed will be suboptimal.
	// Because of this, the size of actual Write calls will affect output size.
	Stateless = gzip.StatelessCompression
)

These constants are copied from the gzip package

Types

type CommonRequestFilter

type CommonRequestFilter struct{}

CommonRequestFilter judge via common easy criteria like http method, accept-encoding header, etc.

func NewCommonRequestFilter

func NewCommonRequestFilter() *CommonRequestFilter

NewCommonRequestFilter ...

func (*CommonRequestFilter) ShouldCompress

func (c *CommonRequestFilter) ShouldCompress(req *http.Request) bool

ShouldCompress implements RequestFilter interface

type Config

type Config struct {
	// gzip compression level to use,
	// valid value: -3 => 9.
	//
	// see https://golang.org/pkg/compress/gzip/#NewWriterLevel
	CompressionLevel int
	// Minimum content length to trigger gzip,
	// the unit is in byte.
	//
	// When `Content-Length` is not available, handler may buffer your writes to
	// decide if its big enough to do a meaningful compression.
	// A high `MinContentLength` may bring memory overhead,
	// although the handler tries to be smart by reusing buffers
	// and testing if `len(data)` of the first
	// `http.ResponseWriter.Write(data []byte)` calling suffices or not.
	MinContentLength int64
	// Filters are applied in the sequence here
	RequestFilter []RequestFilter
	// Filters are applied in the sequence here
	ResponseHeaderFilter []ResponseHeaderFilter
}

Config is used in Handler initialization

type ContentTypeFilter

type ContentTypeFilter struct {
	Types      *acascii.Matcher
	AllowEmpty bool
}

ContentTypeFilter judge via the response content type

Omit this filter if you want to compress all content type.

func DefaultContentTypeFilter

func DefaultContentTypeFilter() *ContentTypeFilter

DefaultContentTypeFilter permits

func NewContentTypeFilter

func NewContentTypeFilter(types []string) *ContentTypeFilter

NewContentTypeFilter ...

func (*ContentTypeFilter) ShouldCompress

func (e *ContentTypeFilter) ShouldCompress(header http.Header) bool

ShouldCompress implements RequestFilter interface

type ExtensionFilter

type ExtensionFilter struct {
	Exts       *acascii.Matcher
	AllowEmpty bool
}

ExtensionFilter judge via the extension in path

Omit this filter if you want to compress all extension.

func DefaultExtensionFilter

func DefaultExtensionFilter() *ExtensionFilter

DefaultExtensionFilter permits

func NewExtensionFilter

func NewExtensionFilter(extensions []string) *ExtensionFilter

NewExtensionFilter returns a extension or panics

func (*ExtensionFilter) ShouldCompress

func (e *ExtensionFilter) ShouldCompress(req *http.Request) bool

ShouldCompress implements RequestFilter interface

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

Handler implement gzip compression for gin and net/http

func DefaultHandler

func DefaultHandler() *Handler

DefaultHandler creates a gzip handler to take care of response compression, with meaningful preset.

func NewHandler

func NewHandler(config Config) *Handler

NewHandler initialized a costumed gzip handler to take care of response compression.

config must not be modified after calling on NewHandler()

func (*Handler) Gin

func (h *Handler) Gin(c *gin.Context)

Gin implement gin's middleware

func (*Handler) WrapHandler

func (h *Handler) WrapHandler(next http.Handler) http.Handler

WrapHandler wraps a http.Handler, returning its gzip-enabled version

type RequestFilter

type RequestFilter interface {
	// ShouldCompress decide whether or not to compress response,
	// judging by request
	ShouldCompress(req *http.Request) bool
}

RequestFilter decide whether or not to compress response judging by request

type ResponseHeaderFilter

type ResponseHeaderFilter interface {
	// ShouldCompress decide whether or not to compress response,
	// judging by response header
	ShouldCompress(header http.Header) bool
}

ResponseHeaderFilter decide whether or not to compress response judging by response header

type SkipCompressedFilter

type SkipCompressedFilter struct{}

SkipCompressedFilter judges whether content has been already compressed

func NewSkipCompressedFilter

func NewSkipCompressedFilter() *SkipCompressedFilter

NewSkipCompressedFilter ...

func (*SkipCompressedFilter) ShouldCompress

func (s *SkipCompressedFilter) ShouldCompress(header http.Header) bool

ShouldCompress implements ResponseHeaderFilter interface

Content-Encoding: https://tools.ietf.org/html/rfc2616#section-3.5