composer

README

go-multipart-composer

Prepares bodies of HTTP requests with MIME multipart messages according to RFC7578 without reading entire file contents to memory. Instead of writing files to a multipart writer right away, it collects readers for each part of the form and lets them stream to the network once the request has been sent. Avoids buffering of the request body simpler than with goroutines and pipes. See the documentation for more information.

Installation

Add this package to go.mod and go.sub in your Go project:

go get github.com/prantlf/go-multipart-composer

Usage

Upload a file with comment:

import (
  "net/http"
  composer "github.com/prantlf/go-multipart-composer"
)
// compose a multipart form-data content
comp := composer.NewComposer()
comp.AddField("comment", "a comment")
err := comp.AddFile("file", "test.txt")
// post a request with the generated content type and body
resp, err := http.DefaultClient.Post("http://host.com/upload",
  comp.FormDataContentType(), comp.DetachReader())

If the server does not support chunked encoding and requires Content-=Length in the header:

comp := composer.NewComposer()
comp.AddField("comment", "a comment")
err := comp.AddFile("file", "test.txt")
reqBody, contentLength, err := comp.DetachReaderWithSize()
if err != nil {
  comp.Close() // DetachReaderWithSize does not close the composer on failure
  log.Fatal(err)
}
// post a request with the generated body, content type and content length
req, err := http.NewRequest("POST", "http://host.com/upload", reqBody)
req.Header.Add("Content-Type", comp.FormDataContentType())
req.ContentLength = contentLength
resp, err := http.DefaultClient.Do(request)

See the documentation for the full interface.

Documentation

Overview

Package composer prepares bodies of HTTP requests with MIME multipart messages without reading entire file contents to memory. Instead of writing files to multipart Writer right away, it collects Readers for each part of the form and lets them stream to the network once the request has been sent. Avoids buffering of the request body simpler than with goroutines and pipes.

Text fields and files can be appended by convenience methods:

comp := composer.NewComposer()
comp.AddField("comment", "a comment")
err := comp.AddFile("file", "test.txt")

The multipart form-data content type and a reader for the full request body can be passed directly the HTTP request methods. They close a closable writer even in case of failure:

resp, err := http.DefaultClient.Post("http://host.com/upload",
  comp.FormDataContentType(), comp.DetachReader())

Example

package main

import (
	"log"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	// Create a new multipart message composer with a random boundary.
	comp := composer.NewComposer()
	// Close added files or readers if a failure before DetachReader occurred.
	// Not needed if you add no file, or if you add or just one file and then
	// do not abandon the composer before you succeed to return the result of
	// DetachReader or DetachReaderWithSize.
	defer comp.Close()

	// Add a textual field.
	comp.AddField("comment", "a comment")
	// Add a file content. Fails if the file cannot be opened.
	if err := comp.AddFile("file", "demo/test.txt"); err != nil {
		log.Fatal(err)
	}

	// Get the content type of the composed multipart message.
	contentType := comp.FormDataContentType()
	// Collect the readers for added fields and files to a single compound
	// reader including the total size and empty the composer by detaching
	// the original readers from it.
	reqBody, contentLength, err := comp.DetachReaderWithSize()
	if err != nil {
		log.Fatal(err)
	}
	// Close added files or readers after the request body reader was used.
	// Not needed if the consumer of reqBody is called right away and will
	// guarantee to close the reader even in case of failure. Because this
	// is the case here, here it is for demonstration purposes only.
	defer reqBody.Close()

	// Make a network request with the composed content type and request body.
	demo.PrintRequestWithLength(contentLength, contentType, reqBody)
}

Output:

Content-Length: 383
Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="comment"

a comment
--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

Index

• type Composer
  • func NewComposer() *Composer
  • func (c *Composer) AddField(name, value string)
  • func (c *Composer) AddFieldReader(name string, reader io.Reader)
  • func (c *Composer) AddFile(fieldName, filePath string) error
  • func (c *Composer) AddFileObject(fieldName string, file *os.File) error
  • func (c *Composer) AddFileReader(fieldName, fileName string, reader io.Reader)
  • func (c *Composer) AddPart(header textproto.MIMEHeader, reader io.Reader)
  • func (c *Composer) Boundary() string
  • func (c *Composer) Clear()
  • func (c *Composer) Close() error
  • func (c *Composer) CreateFieldPart(name string) textproto.MIMEHeader
  • func (c *Composer) CreateFilePart(fieldName, fileName string) textproto.MIMEHeader
  • func (c *Composer) CreatePart(disposition map[string]string) textproto.MIMEHeader
  • func (c *Composer) DetachReader() io.ReadCloser
  • func (c *Composer) DetachReaderWithSize() (io.ReadCloser, int64, error)
  • func (c *Composer) FormDataContentType() string
  • func (c *Composer) ResetBoundary() error
  • func (c *Composer) SetBoundary(boundary string) error

Examples

• Package
• Composer
• Composer.AddField
• Composer.AddFieldReader
• Composer.AddFile
• Composer.AddFileReader
• Composer.Boundary
• Composer.Clear
• Composer.Close
• Composer.DetachReader
• Composer.DetachReaderWithSize
• Composer.FormDataContentType
• Composer.ResetBoundary
• Composer.SetBoundary
• NewComposer

Types

type Composer

type Composer struct {
	// CloseReaders, if set to false, prevents closing of added files
	// or readers when Close is called, or when the reader returned by
	// DetachReader is closed. The initial value set by NewComposer is true.
	CloseReaders bool
	// contains filtered or unexported fields
}

A Composer generates multipart messages with delayed content supplied by readers.

Example

package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	// Create an invalid composer for results returned in case of error.
	comp := composer.Composer{}

	fmt.Printf("Empty composer: %v", comp.Boundary() == "")
}

Output:

Empty composer: true

func NewComposer

func NewComposer() *Composer

NewComposer returns a new multipart message Composer with a random boundary.

If you are going to add parts with readers that needs closing (files), defer a call to Close in case an error occurs, the best right after calling this method.

Example

package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	// Create a new multipart message composer with a random boundary.
	comp := composer.NewComposer()

	fmt.Printf("Close added files or readers: %v", comp.CloseReaders)
}

Output:

Close added files or readers: true

func (*Composer) AddField

func (c *Composer) AddField(name, value string)

AddField creates a new multipart section with a field value. It inserts a header with the provided field name and value.

Example

package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a textual field.
	comp.AddField("foo", "bar")

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="foo"

bar
--1879bcd06ac39a4d8fa5--

func (*Composer) AddFieldReader

func (c *Composer) AddFieldReader(name string, reader io.Reader)

AddFieldReader creates a new multipart section with a field value. It inserts a header using the given field name and then appends the value reader.

Example

package main

import (
	"strings"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a textual field with a value supplied by a reader.
	comp.AddFieldReader("foo", strings.NewReader("bar"))

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="foo"

bar
--1879bcd06ac39a4d8fa5--

func (*Composer) AddFile

func (c *Composer) AddFile(fieldName, filePath string) error

AddFile is a convenience wrapper around AddFileReader. It opens the given file and uses its name, stats and content to create the new part.

The opened file wil be owned by the Composer. Do not forget to close the composer, once you do not need it, or defer the closure to perform it automatically in case of a failure.

Example

package main

import (
	"log"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a file content. Fails if the file cannot be opened.
	if err := comp.AddFile("file", "demo/test.txt"); err != nil {
		log.Fatal(err)
	}

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

func (*Composer) AddFileObject

func (c *Composer) AddFileObject(fieldName string, file *os.File) error

AddFileObject is a convenience wrapper around AddFileReader. It uses the name, stats and content of the opened file to create the new part.

The opened file wil be owned by the Composer. Do not forget to close the composer, once you do not need it, or defer the closure to perform it automatically in case of a failure. However, do not close the source file. The reader taking part in the request body creation would fail.

func (*Composer) AddFileReader

func (c *Composer) AddFileReader(fieldName, fileName string, reader io.Reader)

AddFileReader creates a new multipart section with a file content. It inserts a header using the given field name, file name and the content type inferred from the file extension, then appends the reader's content.

If the reader passed in is a ReaderCloser, it will be owned and eventually freed by the Composer. Do not forget to close the composer, once you do not need it, or defer the closure to perform it automatically in case of a failure. However, do not close the source file. The reader taking part in the request body creation would fail.

Example

package main

import (
	"log"
	"os"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a file content supplied as a separate reader.
	file, err := os.Open("demo/test.txt")
	if err != nil {
		log.Fatal(err)
	}
	comp.AddFileReader("file", "test.txt", file)

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

func (*Composer) AddPart

func (c *Composer) AddPart(header textproto.MIMEHeader, reader io.Reader)

AddPart creates a new multipart section prepared earlier with CreatePart, CreateFieldPart or CreateFilePart. It inserts all headers prepared earlier and then appends the value reader.

func (*Composer) Boundary

func (c *Composer) Boundary() string

Boundary returns the Composer's boundary.

Example

package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	comp := composer.NewComposer()

	// Get the initial randomly-genenrated boundary.
	boundary := comp.Boundary()

	fmt.Printf("Boundary set: %v", len(boundary) > 0)
}

Output:

Boundary set: true

func (*Composer) Clear

func (c *Composer) Clear()

Clear closes all closable readers added by AddFileReader or AddFile and clears their collection, making the composer ready to start empty again.

Example

package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()
	comp.AddField("foo", "bar")

	// Abandon the composed content and clear the added fields.
	comp.Clear()

	comp.AddField("foo", "bar")

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="foo"

bar
--1879bcd06ac39a4d8fa5--

func (*Composer) Close

func (c *Composer) Close() error

Close closes all closable readers added by AddFileReader or AddFile. If some of them fail, the first error will be returned.

Example

package main

import (
	"log"
	"os"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a file reader which will be closed automatically.
	file, err := os.Open("demo/test.txt")
	if err != nil {
		log.Fatal(err)
	}
	comp.AddFileReader("file", "test.txt", file)

	// Close the added files and readers.
	comp.Close()
	if _, err := file.Stat(); err == nil {
		log.Fatal("open")
	}

	// Start again with disabled closing of files and readers.
	comp.Clear()
	comp.CloseReaders = false

	// Add a file reader which will not be closed automatically.
	file, err = os.Open("demo/test.txt")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()
	comp.AddFileReader("file", "test.txt", file)

	// Adding a file by path is impossible if automatic closing is disabled.
	if err := comp.AddFile("file", "demo/test.txt"); err == nil {
		log.Fatal("added")
	}

	// Getting the final reader or closing the composer will not close the file.
	reqBody := comp.DetachReader()
	comp.Close()
	if _, err := file.Stat(); err != nil {
		log.Fatal(err)
	}

	demo.PrintRequest(comp.FormDataContentType(), reqBody)
}

Output:

Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

func (*Composer) CreateFieldPart

func (c *Composer) CreateFieldPart(name string) textproto.MIMEHeader

CreateFilePart creates a new multipart section for a field, but does not add it to the composer yet. Passing the returned header to AddPart will add it to the composer.

func (*Composer) CreateFilePart

func (c *Composer) CreateFilePart(fieldName, fileName string) textproto.MIMEHeader

CreateFilePart creates a new multipart section for a file, but does not add it to the composer yet. Passing the returned header to AddPart will add it to the composer.

func (*Composer) CreatePart

func (c *Composer) CreatePart(disposition map[string]string) textproto.MIMEHeader

CreateFilePart creates a new general multipart section, but does not add it to the composer yet. Passing the returned header to AddPart will add it to the composer.

func (*Composer) DetachReader

func (c *Composer) DetachReader() io.ReadCloser

DetachReader finishes the multipart message by adding the trailing boundary end line to the output and moves the closable readers to be closed with the returned compound reader.

Example

package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Get a multipart message with no parts.
	reqBody := comp.DetachReader()

	demo.PrintRequestBody(reqBody)
}

Output:

--1879bcd06ac39a4d8fa5--

func (*Composer) DetachReaderWithSize

func (c *Composer) DetachReaderWithSize() (io.ReadCloser, int64, error)

DetachReaderWithSize finishes the multipart message by adding the trailing boundary end line to the output and moves the closable readers to be closed with the returned compound reader. It tries computing the total request body size, which will work if size was available for all readers.

If it fails, the composer instance will not be closed.

Example

package main

import (
	"log"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Get a multipart message with no parts including its length.
	reqBody, contentLength, err := comp.DetachReaderWithSize()
	if err != nil {
		log.Fatal(err)
	}

	demo.PrintContentLength(contentLength)
	demo.PrintContentType(comp.FormDataContentType())
	demo.PrintRequestBody(reqBody)
}

Output:

Content-Length: 68
Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

--1879bcd06ac39a4d8fa5--

func (*Composer) FormDataContentType

func (c *Composer) FormDataContentType() string

FormDataContentType returns the value of Content-Type for an HTTP request with the body prepared by this Composer. It will include the constant "multipart/form-data" and this Composers's Boundary.

Example

package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Get the content type for the composed multipart message.
	contentType := comp.FormDataContentType()

	demo.PrintContentType(contentType)
}

Output:

Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

func (*Composer) ResetBoundary

func (c *Composer) ResetBoundary() error

ResetBoundary overrides the Composer's current boundary separator with a randomly generared one.

ResetBoundary must be called before any parts are added, or after all parts were detached by one of the DetachReader methods.

Example

package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	comp := composer.NewComposer()
	comp.SetBoundary("1")

	// Generate a new random boundary to separate the message parts.
	comp.ResetBoundary()

	fmt.Printf("Boundary reset: %v", len(comp.Boundary()) > 1)
}

Output:

Boundary reset: true

func (*Composer) SetBoundary

func (c *Composer) SetBoundary(boundary string) error

SetBoundary overrides the Composer's initial boundary separator with an explicit value.

SetBoundary must be called before any parts are added, or after all parts were detached by one of the DetachReader methods. may only contain certain ASCII characters, and must be non-empty and at most 70 bytes long. (See RFC 2046, section 5.1.1.)

Example

package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	comp := composer.NewComposer()

	// Set an explicit boundary to separate the message parts.
	comp.SetBoundary("3a494cd3b73de6555202")

	fmt.Print(comp.Boundary())
}

Output:

3a494cd3b73de6555202