Documentation
¶
Overview ¶
Prepares bodies of HTTP requests with MIME multipart messages without reading entire file contents to memory. Instead of writing files to multipart Writer right away, it collects Readers for each part of the form and lets them stream to the network once the request has been sent. Avoids buffering of the request body simpler than with goroutines and pipes.
Text fields and files can be appended by convenience methods:
comp := composer.NewComposer() comp.AddField("comment", "a comment") err := comp.AddFile("file", "test.txt")
The multipart form-data content type and a reader for the full request body can be passed directly the HTTP request methods. They close a closable writer even in case of failure:
resp, err := http.DefaultClient.Post("http://host.com/upload", comp.FormDataContentType(), comp.DetachReader())
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { // Create a new multipart message composer with a random boundary. comp := composer.NewComposer() // Close added files or readers if a failure before DetachReader occurred. // Not needed if you add no file, or if you add or just one file and then // do not abandon the composer before you succeed to return the result of // DetachReader or DetachReaderWithSize. defer comp.Close() // Add a textual field. comp.AddField("comment", "a comment") // Add a file content. Fails if the file cannot be opened. if err := comp.AddFile("file", "test.txt"); err != nil { log.Fatal(err) } // Get the content type of the composed multipart message. contentType := comp.FormDataContentType() // Collect the readers for added fields and files to a single compound // reader including the total size and empty the composer by detaching // the original readers from it. reqBody, contentLength, err := comp.DetachReaderWithSize() if err != nil { log.Fatal(err) } // Close added files or readers after the request body reader was used. // Not needed if the consumer of reqBody is called right away and will // guarantee to close the reader even in case of failure. Because this // is the case here, here it is for demonstration purposes only. defer reqBody.Close() // Make a network request with the composed content type and request body. printRequestWithLength(contentLength, contentType, reqBody) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var contentTypeBoundary = regexp.MustCompile("boundary=.+") var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestWithLength(contentLength int64, contentType string, reqBody io.ReadCloser) { printContentLength(contentLength) printRequest(contentType, reqBody) } func printRequest(contentType string, reqBody io.ReadCloser) { printContentType(contentType) fmt.Println() printRequestBody(reqBody) } func printContentType(contentType string) { if contentType[len(contentType)-20:] != fixedBoundary { contentType = contentTypeBoundary.ReplaceAllLiteralString(contentType, "boundary="+commonBoundary) } fmt.Printf("Content-Type: %s\n", contentType) } func printContentLength(contentLength int64) { fmt.Printf("Content-Length: %d\n", contentLength) } func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: Content-Length: 383 Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5 --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="comment" a comment --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain; charset=utf-8 text file content --1879bcd06ac39a4d8fa5--
Index ¶
- type Composer
- func (c *Composer) AddField(name, value string)
- func (c *Composer) AddFieldReader(name string, reader io.Reader)
- func (c *Composer) AddFile(fieldName, filePath string) error
- func (c *Composer) AddFileReader(fieldName, fileName string, reader io.Reader)
- func (c *Composer) Boundary() string
- func (c *Composer) Clear()
- func (c *Composer) Close() error
- func (c *Composer) DetachReader() io.ReadCloser
- func (c *Composer) DetachReaderWithSize() (io.ReadCloser, int64, error)
- func (c *Composer) FormDataContentType() string
- func (c *Composer) ResetBoundary() error
- func (c *Composer) SetBoundary(boundary string) error
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Composer ¶
type Composer struct { // CloseReaders, if set to false, prevents closing of added files // or readers when Close is called, or when the reader returned by // DetachReader is closed. The initial value set by NewComposer is true. CloseReaders bool // contains filtered or unexported fields }
A Composer generates multipart messages with delayed content supplied by readers.
Example ¶
package main import ( "fmt" composer "github.com/prantlf/go-multipart-composer" ) func main() { // Create an invalid composer for results returned in case of error. comp := composer.Composer{} fmt.Printf("Empty composer: %v", comp.Boundary() == "") }
Output: Empty composer: true
func NewComposer ¶
func NewComposer() *Composer
NewComposer returns a new multipart message Composer with a random boundary.
Example ¶
package main import ( "fmt" composer "github.com/prantlf/go-multipart-composer" ) func main() { // Create a new multipart message composer with a random boundary. comp := composer.NewComposer() fmt.Printf("Close added files or readers: %v", comp.CloseReaders) }
Output: Close added files or readers: true
func (*Composer) AddField ¶
AddField is a convenience wrapper around AddFileReader. It creates a new multipart section with the provided field name and value.
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Add a textual field. comp.AddField("foo", "bar") printRequestBody(comp.DetachReader()) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="foo" bar --1879bcd06ac39a4d8fa5--
func (*Composer) AddFieldReader ¶
AddFieldReader creates a new multipart section with a field value. It inserts a header using the given field name and then appends the value reader.
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Add a textual field with a value supplied by a reader. comp.AddFieldReader("foo", strings.NewReader("bar")) printRequestBody(comp.DetachReader()) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="foo" bar --1879bcd06ac39a4d8fa5--
func (*Composer) AddFile ¶
AddFile is a convenience wrapper around AddFileReader. It creates a new multipart section with the provided field name and file content.
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Add a file content. Fails if the file cannot be opened. if err := comp.AddFile("file", "test.txt"); err != nil { log.Fatal(err) } printRequestBody(comp.DetachReader()) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain; charset=utf-8 text file content --1879bcd06ac39a4d8fa5--
func (*Composer) AddFileReader ¶
AddFileReader creates a new multipart section with a file content. It inserts a header using the given field name, file name and the content type inferred from the file extension, then appends the file reader.
Example ¶
package main import ( "fmt" "io" "log" "os" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Add a file content supplied as a separate reader. file, err := os.Open("test.txt") if err != nil { log.Fatal(err) } comp.AddFileReader("file", "test.txt", file) printRequestBody(comp.DetachReader()) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain; charset=utf-8 text file content --1879bcd06ac39a4d8fa5--
func (*Composer) Boundary ¶
Boundary returns the Composer's boundary.
Example ¶
package main import ( "fmt" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Get the initial randomly-genenrated boundary. boundary := comp.Boundary() fmt.Printf("Boundary set: %v", len(boundary) > 0) }
Output: Boundary set: true
func (*Composer) Clear ¶
func (c *Composer) Clear()
Clear closes all closable readers added by AddFileReader or AddFile and clears their collection, making the composer ready to start empty again.
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() comp.AddField("foo", "bar") // Abandon the composed content and clear the added fields. comp.Clear() comp.AddField("foo", "bar") printRequestBody(comp.DetachReader()) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="foo" bar --1879bcd06ac39a4d8fa5--
func (*Composer) Close ¶
Close closes all closable readers added by AddFileReader or AddFile. If some of them fail, the first error will be returned.
Example ¶
package main import ( "fmt" "io" "log" "os" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Add a file reader which will be closed automatically. file, err := os.Open("test.txt") if err != nil { log.Fatal(err) } comp.AddFileReader("file", "test.txt", file) // Close the added files and readers. comp.Close() if _, err := file.Stat(); err == nil { log.Fatal("open") } // Start again with disabled closing of files and readers. comp.Clear() comp.CloseReaders = false // Add a file reader which will not be closed automatically. file, err = os.Open("test.txt") if err != nil { log.Fatal(err) } defer file.Close() comp.AddFileReader("file", "test.txt", file) // Adding a file by path is impossible if automatic closing is disabled. if err := comp.AddFile("file", "test.txt"); err == nil { log.Fatal("added") } // Getting the final reader or closing the composer will not close the file. reqBody := comp.DetachReader() comp.Close() if _, err := file.Stat(); err != nil { log.Fatal(err) } printRequest(comp.FormDataContentType(), reqBody) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var contentTypeBoundary = regexp.MustCompile("boundary=.+") var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequest(contentType string, reqBody io.ReadCloser) { printContentType(contentType) fmt.Println() printRequestBody(reqBody) } func printContentType(contentType string) { if contentType[len(contentType)-20:] != fixedBoundary { contentType = contentTypeBoundary.ReplaceAllLiteralString(contentType, "boundary="+commonBoundary) } fmt.Printf("Content-Type: %s\n", contentType) } func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5 --1879bcd06ac39a4d8fa5 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain; charset=utf-8 text file content --1879bcd06ac39a4d8fa5--
func (*Composer) DetachReader ¶
func (c *Composer) DetachReader() io.ReadCloser
DetachReader finishes the multipart message by adding the trailing boundary end line to the output and moves the closable readers to be closed with the returned compound reader.
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Get a multipart message with no parts. reqBody := comp.DetachReader() printRequestBody(reqBody) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: --1879bcd06ac39a4d8fa5--
func (*Composer) DetachReaderWithSize ¶
func (c *Composer) DetachReaderWithSize() (io.ReadCloser, int64, error)
DetachReaderWithSize finishes the multipart message by adding the trailing boundary end line to the output and moves the closable readers to be closed with the returned compound reader. It tries computing the total request body size, which will work if size was available for all readers.
If it fails, the composer instance will not be closed.
Example ¶
package main import ( "fmt" "io" "log" "regexp" "strings" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Get a multipart message with no parts including its length. reqBody, contentLength, err := comp.DetachReaderWithSize() if err != nil { log.Fatal(err) } printContentLength(contentLength) printContentType(comp.FormDataContentType()) printRequestBody(reqBody) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var contentTypeBoundary = regexp.MustCompile("boundary=.+") var requestBodyBoundary = regexp.MustCompile("--[0-9a-z]+") func printContentType(contentType string) { if contentType[len(contentType)-20:] != fixedBoundary { contentType = contentTypeBoundary.ReplaceAllLiteralString(contentType, "boundary="+commonBoundary) } fmt.Printf("Content-Type: %s\n", contentType) } func printContentLength(contentLength int64) { fmt.Printf("Content-Length: %d\n", contentLength) } func printRequestBody(reqBody io.ReadCloser) { defer reqBody.Close() reqBuf := stringifyReader(reqBody) if err := reqBody.Close(); err != nil { log.Fatal(err) } if reqBuf[len(reqBuf)-24:len(reqBuf)-4] != fixedBoundary { reqBuf = requestBodyBoundary.ReplaceAllLiteralString(reqBuf, "--"+commonBoundary) } fmt.Println(reqBuf) } func stringifyReader(reqBody io.Reader) string { builder := new(strings.Builder) if _, err := io.Copy(builder, reqBody); err != nil { log.Fatal(err) } return strings.ReplaceAll(builder.String(), "\r\n", "\n") }
Output: Content-Length: 68 Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5 --1879bcd06ac39a4d8fa5--
func (*Composer) FormDataContentType ¶
FormDataContentType returns the Content-Type for an HTTP multipart/form-data with this Composers's Boundary.
Example ¶
package main import ( "fmt" "regexp" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Get the content type for the composed multipart message. contentType := comp.FormDataContentType() printContentType(contentType) } const fixedBoundary = "3a494cd3b73de6555202" const commonBoundary = "1879bcd06ac39a4d8fa5" var contentTypeBoundary = regexp.MustCompile("boundary=.+") func printContentType(contentType string) { if contentType[len(contentType)-20:] != fixedBoundary { contentType = contentTypeBoundary.ReplaceAllLiteralString(contentType, "boundary="+commonBoundary) } fmt.Printf("Content-Type: %s\n", contentType) }
Output: Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5
func (*Composer) ResetBoundary ¶
ResetBoundary overrides the Composer's current boundary separator with a randomly generared one.
ResetBoundary must be called before any parts are added.
Example ¶
package main import ( "fmt" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() comp.SetBoundary("1") // Generate a new random boundary to separate the message parts. comp.ResetBoundary() fmt.Printf("Boundary reset: %v", len(comp.Boundary()) > 1) }
Output: Boundary reset: true
func (*Composer) SetBoundary ¶
SetBoundary overrides the Composer's initial boundary separator with an explicit value.
SetBoundary must be called before any parts are added, may only contain certain ASCII characters, and must be non-empty and at most 70 bytes long.
Example ¶
package main import ( "fmt" composer "github.com/prantlf/go-multipart-composer" ) func main() { comp := composer.NewComposer() // Set an explicit boundary to separate the message parts. comp.SetBoundary("3a494cd3b73de6555202") fmt.Print(comp.Boundary()) }
Output: 3a494cd3b73de6555202