statigz

README

statigz

statigz serves pre-compressed embedded files with http in Go 1.16 and later.

Why?

Since version 1.16 Go provides standard way to embed static assets. This API has advantages over previous solutions:

• assets are processed during build, so there is no need for manual generation step,
• embedded data does not need to be kept in residential memory (as opposed to previous solutions that kept data in regular byte slices).

A common case for embedding is to serve static assets of a web application. In order to save bandwidth and improve latency, those assets are often served compressed. Compression concerns are out of embed responsibilities, yet they are quite important. Previous solutions (for example vfsgen with httpgzip) can optimize performance by storing compressed assets and serving them directly to capable user agents. This library implements such functionality for embedded file systems.

Read more in a blog post.

Example

package main

import (
	"embed"
	"log"
	"net/http"

	"github.com/vearutop/statigz"
	"github.com/vearutop/statigz/brotli"
)

// Declare your embedded assets.

//go:embed static/*
var st embed.FS

func main() {
	// Plug static assets handler to your server or router.
	err := http.ListenAndServe(":80", statigz.FileServer(st, brotli.AddEncoding))
	if err != nil {
		log.Fatal(err)
	}
}

Usage

Behavior is based on nginx gzip static module and github.com/lpar/gzipped.

Static assets have to be manually compressed with additional file extension, e.g. bundle.js would become bundle.js.gz (compressed with gzip) or index.html would become index.html.br (compressed with brotli).

NOTE: zopfli provides better compression than gzip while being backwards compatible with it.

Upon request server checks if there is a compressed file matching Accept-Encoding and serves it directly.

If user agent does not support available compressed data, server uses an uncompressed file if it is available ( e.g. bundle.js). If uncompressed file is not available, then server would decompress a compressed file into response.

Responses have ETag headers (64-bit FNV-1 hash of file contents) to enable caching. Responses that are not dynamically decompressed are served with http.ServeContent for ranges support.

Brotli support

Support for brotli is optional. Using brotli adds about 260 KB to binary size, that's why it is moved to a separate package.

NOTE: Although brotli has better compression than gzip and already has wide support in browsers, it has limitations for non-https servers, see this and this.

Runtime encoding

Recommended way of embedding assets is to compress assets before the build, so that binary includes *.gz or *.br files. This can be inconvenient in some cases, there is EncodeOnInit option to compress assets in runtime when creating file server. Once compressed, assets will be served directly without additional dynamic compression.

Files with extensions ".gz", ".br", ".gif", ".jpg", ".png", ".webp" are excluded from runtime encoding by default.

NOTE: Compressing assets in runtime can degrade startup performance and increase memory usage to prepare and store compressed data.

Mounting a subdirectory

It may be convenient to strip leading directory from an embedded file system, you can do that with statigz.FSPrefix.

package main

import (
	"embed"
	"log"
	"net/http"

	"github.com/vearutop/statigz"
	"github.com/vearutop/statigz/brotli"
)

// Declare your embedded assets.

//go:embed static/*
var st embed.FS

func main() {
	// Plug static assets handler to your server or router.
	err := http.ListenAndServe(":80", statigz.FileServer(st, brotli.AddEncoding, statigz.FSPrefix("static")))
	if err != nil {
		log.Fatal(err)
	}
}

Custom error handling

Error states can be handled with the staticgz.OnError and staticgz.OnNotFound options. These allow you to customize the response sent to the client when an error occurs or when no resource is found.

fileServer := statigz.FileServer(
	st,
	staticgz.OnError(func(w http.ResponseWriter, r *http.Request, err error) {
		// Handle error.
		http.Error(w, err.Error(), http.StatusInternalServerError)
	}),
	staticgz.OnNotFound(func(w http.ResponseWriter, r *http.Request) {
		// Handle not found.
		http.Error(w, "Not found", http.StatusNotFound)
		
		// Or to serve a alternative path instead you could;
        //r.URL.Path = "/alternative/path"
        //fileServer.ServeHTTP(w, r)
	}),
)

if err := http.ListenAndServe("localhost:80", fileServer); err != nil {
    log.Fatal(err)
}

Documentation

Overview

Package statigz serves pre-compressed embedded files with http.

Index

• Variables
• func EncodeOnInit(server *Server)
• func FSPrefix(prefix string) func(server *Server)
• func OnError(onErr func(rw http.ResponseWriter, r *http.Request, err error)) func(server *Server)
• func OnNotFound(onErr func(rw http.ResponseWriter, r *http.Request)) func(server *Server)
• type Encoding
  • func GzipEncoding() Encoding
• type Server
  • func FileServer(fs fs.ReadDirFS, options ...func(server *Server)) *Server
  • func (s *Server) Found(req *http.Request) bool
  • func (s *Server) ServeHTTP(rw http.ResponseWriter, req *http.Request)

Examples

• FileServer
• Server.Found

Variables

var SkipCompressionExt = []string{".gz", ".br", ".gif", ".jpg", ".png", ".webp"}

SkipCompressionExt lists file extensions of data that is already compressed.

Functions

func EncodeOnInit

func EncodeOnInit(server *Server)

EncodeOnInit enables runtime encoding for unencoded files to allow compression for uncompressed embedded files.

Enabling this option can degrade startup performance and memory usage in case of large embeddings, use with caution.

func FSPrefix

func FSPrefix(prefix string) func(server *Server)

FSPrefix declares file system path prefix that should be ignored.

func OnError

func OnError(onErr func(rw http.ResponseWriter, r *http.Request, err error)) func(server *Server)

OnError is an option to customize error handling in Server.

func OnNotFound

func OnNotFound(onErr func(rw http.ResponseWriter, r *http.Request)) func(server *Server)

OnNotFound is an option to customize not found (404) handling in Server.

Types

type Encoding

type Encoding struct {
	// FileExt is an extension of file with compressed content, for example ".gz".
	FileExt string

	// ContentEncoding is encoding name that is used in Accept-Encoding and Content-Encoding
	// headers, for example "gzip".
	ContentEncoding string

	// Decoder is a function that can decode data for an agent that does not accept encoding,
	// can be nil to disable dynamic decompression.
	Decoder func(r io.Reader) (io.Reader, error)

	// Encoder is a function that can encode data
	Encoder func(r io.Reader) ([]byte, error)
}

Encoding describes content encoding.

func GzipEncoding

func GzipEncoding() Encoding

GzipEncoding provides gzip Encoding.

type Server

type Server struct {
	// OnError controls error handling during Serve.
	OnError func(rw http.ResponseWriter, r *http.Request, err error)

	// OnNotFound controls handling of not found files.
	OnNotFound func(rw http.ResponseWriter, r *http.Request)

	// Encodings contains supported encodings, default GzipEncoding.
	Encodings []Encoding

	// EncodeOnInit encodes files that does not have encoded version on Server init.
	// This allows embedding uncompressed files and still leverage one time compression
	// for multiple requests.
	// Enabling this option can degrade startup performance and memory usage in case
	// of large embeddings, use with caution.
	EncodeOnInit bool

	// FSPrefix is a path prefix shat should be ignored.
	// It is prepended to the incoming HTTP path.
	// This can help to keep static assets in a subdirectory, e.g.
	//   //go:embed static/*
	// But access files from HTTP without "/static/" prefix in the path.
	FSPrefix string
	// contains filtered or unexported fields
}

Server is a http.Handler that directly serves compressed files from file system to capable agents.

Please use FileServer to create an instance of Server.

If agent does not accept encoding and uncompressed file is not available in file system, it would decompress the file before serving.

Compressed files should have an additional extension to indicate their encoding, for example "style.css.gz" or "bundle.js.br".

Caching is implemented with ETag and If-None-Match headers. Range requests are supported with help of http.ServeContent.

Behavior is similar to http://nginx.org/en/docs/http/ngx_http_gzip_static_module.html and https://github.com/lpar/gzipped, except compressed data can be decompressed for an incapable agent.

func FileServer

func FileServer(fs fs.ReadDirFS, options ...func(server *Server)) *Server

FileServer creates an instance of Server from file system.

This function indexes provided file system to optimize further serving, so it is not recommended running it in the loop (for example for each request).

Typically, file system would be an embed.FS.

//go:embed *.png *.br
var FS embed.FS

Brotli support is optionally available with brotli.AddEncoding.

Example

package main

import (
	"embed"
	"io/fs"
	"log"
	"net/http"

	"github.com/vearutop/statigz"
	"github.com/vearutop/statigz/brotli"
)

// Declare your embedded assets.

//go:embed testdata/*
var st embed.FS

func main() {
	s, err := fs.Sub(st, "testdata")
	if err != nil {
		log.Fatal(err)
	}

	// Plug static assets handler to your server or router.
	err = http.ListenAndServe(":80", statigz.FileServer(s.(fs.ReadDirFS), brotli.AddEncoding))
	if err != nil {
		log.Fatal(err)
	}
}

func (*Server) Found

func (s *Server) Found(req *http.Request) bool

Found returns true if http.Request would be fulfilled by Server.

This can be useful for custom handling of requests to non-existent resources.

Example

fileServer := statigz.FileServer(st)
customHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	// Serve existing static resource.
	if fileServer.Found(r) {
		fileServer.ServeHTTP(w, r)

		return
	}

	// Do something custom for non-existing resource, for example serve index page.
	// (This is an example, serving index instead of 404 might not be the best idea in real life 😅).
	r.URL.Path = "/"
	fileServer.ServeHTTP(w, r)
})

// Plug static assets handler to your server or router.
if err := http.ListenAndServe("localhost:80", customHandler); err != nil {
	log.Fatal(err)
}

func (*Server) ServeHTTP

func (s *Server) ServeHTTP(rw http.ResponseWriter, req *http.Request)

ServeHTTP serves static files.

For compatibility with std http.FileServer: if request path ends with /index.html, it is redirected to base directory; if request path points to a directory without trailing "/", it is redirected to a path with trailing "/".