imageproxy

README

imageproxy

imageproxy is a caching image proxy server written in go. It features:

• basic image adjustments like resizing, cropping, and rotation
• access control using allowed hosts list or request signing (HMAC-SHA256)
• support for jpeg, png, webp (decode only), tiff, and gif image formats (including animated gifs)
• caching in-memory, on disk, or with Amazon S3, Google Cloud Storage, Azure Storage, or Redis
• easy deployment, since it's pure go

Personally, I use it primarily to dynamically resize images hosted on my own site (read more in this post). But you can also enable request signing and use it as an SSL proxy for remote images, similar to atmos/camo but with additional image adjustment options.

I aim to keep imageproxy compatible with the two most recent major go releases. I also keep track of the minimum go version that still works (currently go1.11 with modules enabled), but that might change at any time. You can see the go versions that are tested against in .github/workflows/tests.yml.

URL Structure

imageproxy URLs are of the form http://localhost/{options}/{remote_url}.

Options

Options are available for cropping, resizing, rotation, flipping, and digital signatures among a few others. Options for are specified as a comma delimited list of parameters, which can be supplied in any order. Duplicate parameters overwrite previous values.

See the full list of available options at https://godoc.org/willnorris.com/go/imageproxy#ParseOptions.

Remote URL

The URL of the original image to load is specified as the remainder of the path, without any encoding. For example, http://localhost/200/https://willnorris.com/logo.jpg.

In order to optimize caching, it is recommended that URLs not contain query strings.

Examples

The following live examples demonstrate setting different options on this source image, which measures 1024 by 678 pixels.

Options	Meaning	Image
200x	200px wide, proportional height
x0.15	15% original height, proportional width
100x150	100 by 150 pixels, cropping as needed
100	100px square, cropping as needed
150,fit	scale to fit 150px square, no cropping
100,r90	100px square, rotated 90 degrees
100,fv,fh	100px square, flipped horizontal and vertical
200x,q60	200px wide, proportional height, 60% quality
200x,png	200px wide, converted to PNG format
cx175,cw400,ch300,100x	crop to 400x300px starting at (175,0), scale to 100px wide

The smart crop feature can best be seen by comparing crops of this source image, with and without smart crop enabled.

Options	Meaning	Image
150x300	150x300px, standard crop
150x300,sc	150x300px, smart crop

Transformation also works on animated gifs. Here is this source image resized to 200px square and rotated 270 degrees:

Getting Started

Install the package using:

go get willnorris.com/go/imageproxy/cmd/imageproxy

Once installed, ensure $GOPATH/bin is in your $PATH, then run the proxy using:

imageproxy

This will start the proxy on port 8080, without any caching and with no allowed host list (meaning any remote URL can be proxied). Test this by navigating to http://localhost:8080/500/https://octodex.github.com/images/codercat.jpg and you should see a 500px square coder octocat.

Cache

By default, the imageproxy command does not cache responses, but caching can be enabled using the -cache flag. It supports the following values:

• memory - uses an in-memory LRU cache. By default, this is limited to 100mb. To customize the size of the cache or the max age for cached items, use the format memory:size:age where size is measured in mb and age is a duration. For example, memory:200:4h will create a 200mb cache that will cache items no longer than 4 hours.

• directory on local disk (e.g. /tmp/imageproxy) - will cache images on disk

• s3 URL (e.g. s3://region/bucket-name/optional-path-prefix) - will cache images on Amazon S3. This requires either an IAM role and instance profile with access to your your bucket or AWS_ACCESS_KEY_ID and AWS_SECRET_KEY environmental variables be set. (Additional methods of loading credentials are documented in the aws-sdk-go session package).

  Additional configuration options (further documented here) may be specified as URL query string parameters, which are mostly useful when working with s3-compatible services:

  • "endpoint" - specify an alternate API endpoint
  • "disableSSL" - set to "1" to disable SSL when calling the API
  • "s3ForcePathStyle" - set to "1" to force the request to use path-style addressing

  For example, when working with minio, which doesn't use regions, provide a dummy region value and custom endpoint value:

  s3://fake-region/bucket/folder?endpoint=minio:9000&disableSSL=1&s3ForcePathStyle=1
  

  Similarly, for Digital Ocean Spaces, provide a dummy region value and the appropriate endpoint for your space:

  s3://fake-region/bucket/folder?endpoint=sfo2.digitaloceanspaces.com
  

• gcs URL (e.g. gcs://bucket-name/optional-path-prefix) - will cache images on Google Cloud Storage. Authentication is documented in Google's Application Default Credentials docs.

• azure URL (e.g. azure://container-name/) - will cache images on Azure Storage. This requires AZURESTORAGE_ACCOUNT_NAME and AZURESTORAGE_ACCESS_KEY environment variables to bet set.

• redis URL (e.g. redis://hostname/) - will cache images on the specified redis host. The full URL syntax is defined by the redis URI registration. Rather than specify password in the URI, use the REDIS_PASSWORD environment variable.

For example, to cache files on disk in the /tmp/imageproxy directory:

imageproxy -cache /tmp/imageproxy

Reload the codercat URL, and then inspect the contents of /tmp/imageproxy. Within the subdirectories, there should be two files, one for the original full-size codercat image, and one for the resized 500px version.

Multiple caches can be specified by separating them by spaces or by repeating the -cache flag multiple times. The caches will be created in a tiered fashion. Typically this is used to put a smaller and faster in-memory cache in front of a larger but slower on-disk cache. For example, the following will first check an in-memory cache for an image, followed by a gcs bucket:

imageproxy -cache memory -cache gcs://my-bucket/

Allowed Referrer List

You can limit images to only be accessible for certain hosts in the HTTP referrer header, which can help prevent others from hotlinking to images. It can be enabled by running:

imageproxy  -referrers example.com

Reload the codercat URL, and you should now get an error message. You can specify multiple hosts as a comma separated list, or prefix a host value with *. to allow all sub-domains as well.

Allowed and Denied Hosts List

You can limit the remote hosts that the proxy will fetch images from using the allowHosts and denyHosts flags. This is useful, for example, for locking the proxy down to your own hosts to prevent others from abusing it. Of course if you want to support fetching from any host, leave off these flags.

Try it out by running:

imageproxy -allowHosts example.com

Reload the codercat URL, and you should now get an error message. Alternately, try running:

imageproxy -denyHosts octodex.github.com

Reloading the codercat URL will still return an error message.

You can specify multiple hosts as a comma separated list to either flag, or prefix a host value with *. to allow or deny all sub-domains. You can also specify a netblock in CIDR notation (127.0.0.0/8) -- this is useful for blocking reserved ranges like 127.0.0.0/8, 192.168.0.0/16, etc.

If a host matches both an allowed and denied host, the request will be denied.

Allowed Content-Type List

You can limit what content types can be proxied by using the contentTypes flag. By default, this is set to image/*, meaning that imageproxy will process any image types. You can specify multiple content types as a comma separated list, and suffix values with * to perform a wildcard match. Set the flag to an empty string to proxy all requests, regardless of content type.

Signed Requests

Instead of an allowed host list, you can require that requests be signed. This is useful in preventing abuse when you don't have just a static list of hosts you want to allow. Signatures are generated using HMAC-SHA256 against the remote URL, and url-safe base64 encoding the result:

base64urlencode(hmac.New(sha256, <key>).digest(<remote_url>))

The HMAC key is specified using the signatureKey flag. If this flag begins with an "@", the remainder of the value is interpreted as a file on disk which contains the HMAC key.

Try it out by running:

imageproxy -signatureKey "secretkey"

Reload the codercat URL, and you should see an error message. Now load a signed codercat URL (which contains the signature option) and verify that it loads properly.

Some simple code samples for generating signatures in various languages can be found in docs/url-signing.md. Multiple valid signature keys may be provided to support key rotation by repeating the signatureKey flag multiple times, or by providing a space-separated list of keys. To use a key with a literal space character, load the key from a file using the "@" prefix documented above.

If both a whiltelist and signatureKey are specified, requests can match either. In other words, requests that match one of the allowed hosts don't necessarily need to be signed, though they can be.

Default Base URL

Typically, remote images to be proxied are specified as absolute URLs. However, if you commonly proxy images from a single source, you can provide a base URL and then specify remote images relative to that base. Try it out by running:

imageproxy -baseURL https://octodex.github.com/

Then load the codercat image, specified as a URL relative to that base: http://localhost:8080/500/images/codercat.jpg. Note that this is not an effective method to mask the true source of the images being proxied; it is trivial to discover the base URL being used. Even when a base URL is specified, you can always provide the absolute URL of the image to be proxied.

Scaling beyond original size

By default, the imageproxy won't scale images beyond their original size. However, you can use the scaleUp command-line flag to allow this to happen:

imageproxy -scaleUp true

WebP and TIFF support

Imageproxy can proxy remote webp images, but they will be served in either jpeg or png format (this is because the golang webp library only supports webp decoding) if any transformation is requested. If no format is specified, imageproxy will use jpeg by default. If no transformation is requested (for example, if you are just using imageproxy as an SSL proxy) then the original webp image will be served as-is without any format conversion.

Because so few browsers support tiff images, they will be converted to jpeg by default if any transformation is requested. To force encoding as tiff, pass the "tiff" option. Like webp, tiff images will be served as-is without any format conversion if no transformation is requested.

Run imageproxy -help for a complete list of flags the command accepts. If you want to use a different caching implementation, it's probably easiest to just make a copy of cmd/imageproxy/main.go and customize it to fit your needs... it's a very simple command.

Environment Variables

All configuration flags have equivalent environment variables of the form IMAGEPROXY_$NAME. For example, an on-disk cache could be configured by calling

IMAGEPROXY_CACHE="/tmp/imageproxy" imageproxy

Deploying

In most cases, you can follow the normal procedure for building a deploying any go application. For example:

• go build willnorris.com/go/imageproxy/cmd/imageproxy
• copy resulting binary to /usr/local/bin
• copy etc/imageproxy.service to /lib/systemd/system and enable using systemctl.

Instructions have been contributed below for running on other platforms, but I don't have much experience with them personally.

Heroku

It's easy to vendorize the dependencies with Godep and deploy to Heroku. Take a look at this GitHub repo (make sure you use the heroku branch).

AWS Elastic Beanstalk

O’Reilly Media set up a repository with everything you need to deploy imageproxy to Elastic Beanstalk. Just follow the instructions in the README.

Docker

A docker image is available at willnorris/imageproxy.

You can run it by

docker run -p 8080:8080 willnorris/imageproxy -addr 0.0.0.0:8080

Or in your Dockerfile:

ENTRYPOINT ["/app/imageproxy", "-addr 0.0.0.0:8080"]

If running imageproxy inside docker with a bind-mounted on-disk cache, make sure the container is running as a user that has write permission to the mounted host directory. See more details in #198.

nginx

Use the proxy_pass directive to send requests to your imageproxy instance. For example, to run imageproxy at the path "/api/imageproxy/", set:

  location /api/imageproxy/ {
    proxy_pass http://localhost:4593/;
  }

Depending on other directives you may have in your nginx config, you might need to alter the precedence order by setting:

  location ^~ /api/imageproxy/ {
    proxy_pass http://localhost:4593/;
  }

License

imageproxy is copyright its respective authors. All of my personal work on imageproxy through 2020 (which accounts for the majority of the code) is copyright Google, my employer at the time. It is available under the Apache 2.0 License.

Documentation

Overview

Package imageproxy provides an image proxy server. For typical use of creating and using a Proxy, see cmd/imageproxy/main.go.

Index

• Variables
• func Transform(img []byte, opt Options) ([]byte, error)
• type Cache
• type Options
  • func ParseOptions(str string) Options
  • func (o Options) String() string
• type Proxy
  • func NewProxy(transport http.RoundTripper, cache Cache, overrideHeader map[string]string) *Proxy
  • func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request)
• type Request
  • func NewRequest(r *http.Request, baseURL *url.URL) (*Request, error)
  • func (r Request) String() string
• type TransformingTransport
  • func (t *TransformingTransport) RoundTrip(req *http.Request) (*http.Response, error)
• type URLError
  • func (e URLError) Error() string

Variables

var NopCache = new(nopCache)

NopCache provides a no-op cache implementation that doesn't actually cache anything.

Functions

func Transform

func Transform(img []byte, opt Options) ([]byte, error)

Transform the provided image. img should contain the raw bytes of an encoded image in one of the supported formats (gif, jpeg, or png). The bytes of a similarly encoded image is returned.

Types

type Cache

type Cache interface {
	// Get retrieves the cached data for the provided key.
	Get(key string) (data []byte, ok bool)

	// Set caches the provided data.
	Set(key string, data []byte)

	// Delete deletes the cached data at the specified key.
	Delete(key string)
}

The Cache interface defines a cache for storing arbitrary data. The interface is designed to align with httpcache.Cache.

type Options

type Options struct {
	// See ParseOptions for interpretation of Width and Height values
	Width  float64
	Height float64

	// If true, resize the image to fit in the specified dimensions.  Image
	// will not be cropped, and aspect ratio will be maintained.
	Fit bool

	// Rotate image the specified degrees counter-clockwise.  Valid values
	// are 90, 180, 270.
	Rotate int

	FlipVertical   bool
	FlipHorizontal bool

	// Quality of output image
	Quality int

	// HMAC Signature for signed requests.
	Signature string

	// Allow image to scale beyond its original dimensions.  This value
	// will always be overwritten by the value of Proxy.ScaleUp.
	ScaleUp bool

	// Desired image format. Valid values are "jpeg", "png", "tiff".
	Format string

	// Crop rectangle params
	CropX      float64
	CropY      float64
	CropWidth  float64
	CropHeight float64

	// Automatically find good crop points based on image content.
	SmartCrop bool
}

Options specifies transformations to be performed on the requested image.

func ParseOptions

func ParseOptions(str string) Options

ParseOptions parses str as a list of comma separated transformation options. The options can be specified in in order, with duplicate options overwriting previous values.

Rectangle Crop

There are four options controlling rectangle crop:

cx{x}      - X coordinate of top left rectangle corner (default: 0)
cy{y}      - Y coordinate of top left rectangle corner (default: 0)
cw{width}  - rectangle width (default: image width)
ch{height} - rectangle height (default: image height)

For all options, integer values are interpreted as exact pixel values and floats between 0 and 1 are interpreted as percentages of the original image size. Negative values for cx and cy are measured from the right and bottom edges of the image, respectively.

If the crop width or height exceed the width or height of the image, the crop width or height will be adjusted, preserving the specified cx and cy values. Rectangular crop is applied before any other transformations.

Smart Crop

The "sc" option will perform a content-aware smart crop to fit the requested image width and height dimensions (see Size and Cropping below). The smart crop option will override any requested rectangular crop.

Size and Cropping

The size option takes the general form "{width}x{height}", where width and height are numbers. Integer values greater than 1 are interpreted as exact pixel values. Floats between 0 and 1 are interpreted as percentages of the original image size. If either value is omitted or set to 0, it will be automatically set to preserve the aspect ratio based on the other dimension. If a single number is provided (with no "x" separator), it will be used for both height and width.

Depending on the size options specified, an image may be cropped to fit the requested size. In all cases, the original aspect ratio of the image will be preserved; imageproxy will never stretch the original image.

When no explicit crop mode is specified, the following rules are followed:

- If both width and height values are specified, the image will be scaled to fill the space, cropping if necessary to fit the exact dimension.

- If only one of the width or height values is specified, the image will be resized to fit the specified dimension, scaling the other dimension as needed to maintain the aspect ratio.

If the "fit" option is specified together with a width and height value, the image will be resized to fit within a containing box of the specified size. As always, the original aspect ratio will be preserved. Specifying the "fit" option with only one of either width or height does the same thing as if "fit" had not been specified.

Rotation and Flips

The "r{degrees}" option will rotate the image the specified number of degrees, counter-clockwise. Valid degrees values are 90, 180, and 270.

The "fv" option will flip the image vertically. The "fh" option will flip the image horizontally. Images are flipped after being rotated.

Quality

The "q{qualityPercentage}" option can be used to specify the quality of the output file (JPEG only). If not specified, the default value of "95" is used.

Format

The "jpeg", "png", and "tiff" options can be used to specify the desired image format of the proxied image.

Signature

The "s{signature}" option specifies an optional base64 encoded HMAC used to sign the remote URL in the request. The HMAC key used to verify signatures is provided to the imageproxy server on startup.

See https://github.com/willnorris/imageproxy/blob/master/docs/url-signing.md for examples of generating signatures.

Examples

0x0         - no resizing
200x        - 200 pixels wide, proportional height
x0.15       - 15% original height, proportional width
100x150     - 100 by 150 pixels, cropping as needed
100         - 100 pixels square, cropping as needed
150,fit     - scale to fit 150 pixels square, no cropping
100,r90     - 100 pixels square, rotated 90 degrees
100,fv,fh   - 100 pixels square, flipped horizontal and vertical
200x,q60    - 200 pixels wide, proportional height, 60% quality
200x,png    - 200 pixels wide, converted to PNG format
cw100,ch100 - crop image to 100px square, starting at (0,0)
cx10,cy20,cw100,ch200 - crop image starting at (10,20) is 100px wide and 200px tall

func (Options) String

func (o Options) String() string

type Proxy

type Proxy struct {
	Client *http.Client // client used to fetch remote URLs
	Cache  Cache        // cache used to cache responses

	// AllowHosts specifies a list of remote hosts that images can be
	// proxied from.  An empty list means all hosts are allowed.
	AllowHosts []string

	// DenyHosts specifies a list of remote hosts that images cannot be
	// proxied from.
	DenyHosts []string

	// Referrers, when given, requires that requests to the image
	// proxy come from a referring host. An empty list means all
	// hosts are allowed.
	Referrers []string

	// IncludeReferer controls whether the original Referer request header
	// is included in remote requests.
	IncludeReferer bool

	// FollowRedirects controls whether imageproxy will follow redirects or not.
	FollowRedirects bool

	// DefaultBaseURL is the URL that relative remote URLs are resolved in
	// reference to.  If nil, all remote URLs specified in requests must be
	// absolute.
	DefaultBaseURL *url.URL

	// The Logger used by the image proxy
	Logger *log.Logger

	// SignatureKeys is a list of HMAC keys used to verify signed requests.
	// Any of them can be used to verify signed requests.
	SignatureKeys [][]byte

	// Allow images to scale beyond their original dimensions.
	ScaleUp bool

	// Timeout specifies a time limit for requests served by this Proxy.
	// If a call runs for longer than its time limit, a 504 Gateway Timeout
	// response is returned.  A Timeout of zero means no timeout.
	Timeout time.Duration

	// If true, log additional debug messages
	Verbose bool

	// ContentTypes specifies a list of content types to allow. An empty
	// list means all content types are allowed.
	ContentTypes []string

	// The User-Agent used by imageproxy when requesting origin image
	UserAgent       string
	HeaderOverrides map[string]string
}

Proxy serves image requests.

func NewProxy

func NewProxy(transport http.RoundTripper, cache Cache, overrideHeader map[string]string) *Proxy

NewProxy constructs a new proxy. The provided http RoundTripper will be used to fetch remote URLs. If nil is provided, http.DefaultTransport will be used.

func (*Proxy) ServeHTTP

func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP handles incoming requests.

type Request

type Request struct {
	URL      *url.URL      // URL of the image to proxy
	Options  Options       // Image transformation to perform
	Original *http.Request // The original HTTP request
}

Request is an imageproxy request which includes a remote URL of an image to proxy, and an optional set of transformations to perform.

func NewRequest

func NewRequest(r *http.Request, baseURL *url.URL) (*Request, error)

NewRequest parses an http.Request into an imageproxy Request. Options and the remote image URL are specified in the request path, formatted as: /{options}/{remote_url}. Options may be omitted, so a request path may simply contain /{remote_url}. The remote URL must be an absolute "http" or "https" URL, should not be URL encoded, and may contain a query string.

Assuming an imageproxy server running on localhost, the following are all valid imageproxy requests:

http://localhost/100x200/http://example.com/image.jpg
http://localhost/100x200,r90/http://example.com/image.jpg?foo=bar
http://localhost//http://example.com/image.jpg
http://localhost/http://example.com/image.jpg

func (Request) String

func (r Request) String() string

String returns the request URL as a string, with r.Options encoded in the URL fragment.

type TransformingTransport

type TransformingTransport struct {
	// Transport is the underlying http.RoundTripper used to satisfy
	// non-transform requests (those that do not include a URL fragment).
	Transport http.RoundTripper

	// CachingClient is used to fetch images to be resized.  This client is
	// used rather than Transport directly in order to ensure that
	// responses are properly cached.
	CachingClient *http.Client
	// contains filtered or unexported fields
}

TransformingTransport is an implementation of http.RoundTripper that optionally transforms images using the options specified in the request URL fragment.

func (*TransformingTransport) RoundTrip

func (t *TransformingTransport) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements the http.RoundTripper interface.

type URLError

type URLError struct {
	Message string
	URL     *url.URL
}

URLError reports a malformed URL error.

func (URLError) Error

func (e URLError) Error() string