gulter

README

Gulter

Gulter is a Go HTTP middleware designed to simplify the process of uploading files for your web apps. It follows the standard http.Handler and http.HandlerFunc interfaces so you can always use with any of framework or the standard library router.

Multiple files per form field are already supported

Name and idea was gotten from the insanely popular multer package in NodeJS that does the same.

Installation

go get -u -v github.com/adelowo/gulter

Usage

Assuming you have a HTML form like this:

<form action="/" method="post" enctype="multipart/form-data">
  <input type="file" name="form-field-1" />
  <input type="file" name="form-field-2" />
</form>

To create a new Gulter instance, you can do something like this:

 handler, _ := gulter.New(
  gulter.WithMaxFileSize(10<<20),
  gulter.WithValidationFunc(
   gulter.ChainValidators(gulter.MimeTypeValidator("image/jpeg", "image/png"),
    func(f gulter.File) error {
     // Your own custom validation function on the file here
     // Else you can really just drop the ChainValidators and use only the MimeTypeValidator or just
     // one custom validator alone
     return nil
    })),
  gulter.WithStorage(s3Store),
 )

The handler is really just a HTTP middleware with the following signature Upload(keys ...string) func(next http.Handler) http.Handler. keys here are the input names from the HTML form, so you can chain this into almost any HTTP router.

Standard HTTP router

package main

import (
 "fmt"
 "net/http"

 "github.com/adelowo/gulter"
 "github.com/adelowo/gulter/storage"
)

func main() {
 s3Store, err := storage.NewS3FromEnvironment(storage.S3Options{
  Bucket: "std-router",
 })
 if err != nil {
  panic(err.Error())
 }

 handler, err := gulter.New(
  gulter.WithMaxFileSize(10<<20),
  gulter.WithStorage(s3Store),
 )

 mux := http.NewServeMux()

 // upload all files with the "name" and "lanre" fields on this route
 mux.Handle("/", handler.Upload("name", "lanre")(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
  fmt.Println("Uploaded file")

  // return all uploaded files
  f, err := gulter.FilesFromContext(r)
  if err != nil {
   fmt.Println(err)
   return
  }

  // return uploaded files with the form field "lanre"
  ff, err := gulter.FilesFromContextWithKey(r, "lanre")
  if err != nil {
   fmt.Println(err)
   return
  }

  fmt.Printf("%+v", ff)

  for _, v := range f {
   fmt.Printf("%+v", v)
   fmt.Println()
  }
 })))

 http.ListenAndServe(":3300", mux)

Chi router and other compatible HTTP handlers

 s3Store, err := storage.NewS3FromEnvironment(storage.S3Options{
  Bucket: "chi-router",
 })
 if err != nil {
  panic(err.Error())
 }

 handler := gulter.New(
  gulter.WithMaxFileSize(10<<20),
  gulter.WithValidationFunc(gulter.ChainValidators(gulter.MimeTypeValidator("image/jpeg", "image/png"))),
  gulter.WithStorage(s3Store),
 )

 router := chi.NewMux()

  // upload all files in the form fields called "form-field-1" and "form-field-2"
 router.With(handler.Upload("form-field-1", "form-field-2")).Post("/", func(w http.ResponseWriter, r *http.Request) {
  fmt.Println("Uploaded file")

  f, err := gulter.FilesFromContext(r)
  if err != nil {
   fmt.Println(err)
   return
  }

  ff, err := gulter.FilesFromContextWithKey(r, "form-field-1") // or form-field-2
  if err != nil {
   fmt.Println(err)
   return
  }

  fmt.Printf("%+v", ff)

  for _, v := range f {
   fmt.Printf("%+v", v)
   fmt.Println()
  }
 })

API

While this middleware automatically uploads your files, sometimes you need details about the uploaded file to show to the user, this could be making up the image url or path to the image. To get that in your HTTP handler, you can use either:

• FilesFromContextWithKey: retrieve a named input uploaded file.
• FilesFromContext: retrieve all uploaded files

Gulter also ships with two storage implementations at the moment:

• S3Store : supports S3 or any compatible service like Minio,R2 & others
• DiskStore: uses a local filesystem backed store to upload files
• CloudinaryStore: uploads file to cloudinary

Ignoring non existent keys in the multipart Request

Sometimes, the keys you have configured the middleware might get dropped from the frontend for some reason, ideally the middleware fails if it cannot find a configured key in the request. To disable this behavior and ignore the missing key, you can make use of the WithIgnoreNonExistentKey(true) option to prevent the middleware from causing an error when such keys do not exists

Customizing the error response

Since Gulter is a middleware that runs, it returns an error to the client if found, this might not match your existing structure, so to configure the response, use the WithErrorResponseHandler. The default is shown below and can be used as a template to define yours.

 errHandler ErrResponseHandler = func(err error) http.HandlerFunc {
  return func(w http.ResponseWriter, _ *http.Request) {
   w.Header().Set("Content-Type", "application/json")
   w.WriteHeader(http.StatusInternalServerError)
   fmt.Fprintf(w, `{"message" : "could not upload file", "error" : %s}`, err.Error())
  }
 }

Writing your custom validator logic

Sometimes, you could have some custom logic to validate uploads, in this example below, we limit the size of the upload based on the mimeypes of the uploaded files

var customValidator gulter.ValidationFunc = func(f gulter.File) error {
 switch f.MimeType {
 case "image/png":
  if f.Size > 4096 {
   return errors.New("file size too large")
  }

  return nil

 case "application/pdf":
  if f.Size > (1024 * 10) {
   return errors.New("file size too large")
  }

  return nil
 default:
  return nil
 }
}

Documentation

Index

• Constants
• type ErrResponseHandler
• type File
  • func FilesFromContextWithKey(r *http.Request, key string) ([]File, error)
• type Files
  • func FilesFromContext(r *http.Request) (Files, error)
• type Gulter
  • func New(opts ...Option) (*Gulter, error)
  • func (h *Gulter) Upload(keys ...string) func(next http.Handler) http.Handler
• type NameGeneratorFunc
• type Option
  • func WithErrorResponseHandler(errHandler ErrResponseHandler) Option
  • func WithIgnoreNonExistentKey(ignore bool) Option
  • func WithMaxFileSize(i int64) Option
  • func WithNameFuncGenerator(nameFunc NameGeneratorFunc) Option
  • func WithStorage(store Storage) Option
  • func WithValidationFunc(validationFunc ValidationFunc) Option
• type Storage
• type UploadFileOptions
• type UploadedFileMetadata
• type ValidationFunc
  • func ChainValidators(validators ...ValidationFunc) ValidationFunc
  • func MimeTypeValidator(validMimeTypes ...string) ValidationFunc

Constants

const (
	ErrNoFilesUploaded = errorMsg("gulter: no uploadable files found in request")
)

Types

type ErrResponseHandler

type ErrResponseHandler func(error) http.HandlerFunc

ErrResponseHandler is a custom error that should be used to handle errors when an upload fails

type File

type File struct {
	// FieldName denotes the field from the multipart form
	FieldName string `json:"field_name,omitempty"`

	// The name of the file from the client side
	OriginalName string `json:"original_name,omitempty"`
	// UploadedFileName denotes the name of the file when it was ultimately
	// uploaded to the storage layer. The distinction is important because of
	// potential changes to the file name that may be done
	UploadedFileName string `json:"uploaded_file_name,omitempty"`
	// FolderDestination is the folder that holds the uploaded file
	FolderDestination string `json:"folder_destination,omitempty"`

	// StorageKey can be used to retrieve the file from the storage backend
	StorageKey string `json:"storage_key,omitempty"`

	// MimeType of the uploaded file
	MimeType string `json:"mime_type,omitempty"`

	// Size in bytes of the uploaded file
	Size int64 `json:"size,omitempty"`
}

func FilesFromContextWithKey

func FilesFromContextWithKey(r *http.Request, key string) ([]File, error)

FilesFromContextWithKey returns all files that have been uploaded during the request and sorts by the provided form field

type Files

type Files map[string][]File

func FilesFromContext

func FilesFromContext(r *http.Request) (Files, error)

FilesFromContext returns all files that have been uploaded during the request

type Gulter

type Gulter struct {
	// contains filtered or unexported fields
}

func New

func New(opts ...Option) (*Gulter, error)

func (*Gulter) Upload

func (h *Gulter) Upload(keys ...string) func(next http.Handler) http.Handler

Upload is a HTTP middleware that takes in a list of form fields and the next HTTP handler to run after the upload prodcess is completed

type NameGeneratorFunc

type NameGeneratorFunc func(s string) string

NameGeneratorFunc allows you alter the name of the file before it is ultimately uplaoded and stored. This is necessarily if you have to adhere to specific formats as an example

type Option

type Option func(*Gulter)

func WithErrorResponseHandler

func WithErrorResponseHandler(errHandler ErrResponseHandler) Option

func WithIgnoreNonExistentKey

func WithIgnoreNonExistentKey(ignore bool) Option

func WithMaxFileSize

func WithMaxFileSize(i int64) Option

WithMaxFileSize allows you limit the size of file uploads to accept

func WithNameFuncGenerator

func WithNameFuncGenerator(nameFunc NameGeneratorFunc) Option

WithNameFuncGenerator allows you configure how you'd like to rename your uploaded files

func WithStorage

func WithStorage(store Storage) Option

func WithValidationFunc

func WithValidationFunc(validationFunc ValidationFunc) Option

type Storage

type Storage interface {
	// Upload copies the reader to the backend file storage
	// The name of the file is also provided.
	Upload(context.Context, io.Reader, *UploadFileOptions) (*UploadedFileMetadata, error)
	io.Closer
}

type UploadFileOptions

type UploadFileOptions struct {
	FileName string
	Metadata map[string]string
}

type UploadedFileMetadata

type UploadedFileMetadata struct {
	FolderDestination string `json:"folder_destination,omitempty"`
	Key               string `json:"key,omitempty"`
	Size              int64  `json:"size,omitempty"`
}

type ValidationFunc

type ValidationFunc func(f File) error

ValidationFunc is a type that can be used to dynamically validate a file

func ChainValidators

func ChainValidators(validators ...ValidationFunc) ValidationFunc

ChainValidators returns a validator that accepts multiple validating criteriacriteria

func MimeTypeValidator

func MimeTypeValidator(validMimeTypes ...string) ValidationFunc

MimeTypeValidator makes sure we only accept a valid mimetype. It takes in an array of supported mimes