message

package
v0.0.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 6, 2024 License: MIT Imports: 17 Imported by: 1

Documentation

Overview

Package message provides functions for reading and writing email messages, ensuring they are correctly formatted.

Index

Examples

Constants

View Source 
const RFC5322Z = "2 Jan 2006 15:04:05 -0700"

Timestamp as used in internet mail messages.

Variables

View Source 
var (
	ErrMessageSize = errors.New("message too large")
	ErrCompose     = errors.New("compose")
)
View Source 
var (
	ErrBadContentType = errors.New("bad content-type")
	ErrHeader         = errors.New("bad message header")
)
View Source 
var ErrHeaderSeparator = errors.New("no header separator found")
View Source 
var Pedantic bool

Pedantic enables stricter parsing.

Functions

func DecodeReader added in v0.0.9 

func DecodeReader(charset string, r io.Reader) io.Reader

DecodeReader returns a reader that reads from r, decoding as charset. If charset is empty, us-ascii, utf-8 or unknown, the original reader is returned and no decoding takes place.

Example 
package main

import (
	"bytes"
	"fmt"
	"io"
	"log"

	"github.com/mjl-/mox/message"
)

func main() {
	// Convert from iso-8859-1 to utf-8.
	input := []byte{'t', 0xe9, 's', 't'}
	output, err := io.ReadAll(message.DecodeReader("iso-8859-1", bytes.NewReader(input)))
	if err != nil {
		log.Fatalf("read from decoder: %v", err)
	}
	fmt.Printf("%s\n", string(output))
}

Output:

tést

func HeaderCommentDomain added in v0.0.6 

func HeaderCommentDomain(domain dns.Domain, smtputf8 bool) string

HeaderCommentDomain returns domain name optionally followed by a message header comment with ascii-only name.

The comment is only present when smtputf8 is true and the domain name is unicode.

Caller should make sure the comment is allowed in the syntax. E.g. for Received, it is often allowed before the next field, so make sure such a next field is present.

func MessageIDCanonical added in v0.0.7 

func MessageIDCanonical(s string) (string, bool, error)

MessageIDCanonical parses the Message-ID, returning a canonical value that is lower-cased, without <>, and no unneeded quoting. For matching in threading, with References/In-Reply-To. If the message-id is invalid (e.g. no <>), an error is returned. If the message-id could not be parsed as address (localpart "@" domain), the raw value and the bool return parameter true is returned. It is quite common that message-id's don't adhere to the localpart @ domain syntax.

Example 
package main

import (
	"fmt"

	"github.com/mjl-/mox/message"
)

func main() {
	// Valid message-id.
	msgid, invalidAddress, err := message.MessageIDCanonical("<ok@localhost>")
	if err != nil {
		fmt.Printf("invalid message-id: %v\n", err)
	} else {
		fmt.Printf("canonical: %s %v\n", msgid, invalidAddress)
	}

	// Missing <>.
	msgid, invalidAddress, err = message.MessageIDCanonical("bogus@localhost")
	if err != nil {
		fmt.Printf("invalid message-id: %v\n", err)
	} else {
		fmt.Printf("canonical: %s %v\n", msgid, invalidAddress)
	}

	// Invalid address, but returned as not being in error.
	msgid, invalidAddress, err = message.MessageIDCanonical("<invalid>")
	if err != nil {
		fmt.Printf("invalid message-id: %v\n", err)
	} else {
		fmt.Printf("canonical: %s %v\n", msgid, invalidAddress)
	}

}

Output:

canonical: ok@localhost false
invalid message-id: not a message-id: missing <
canonical: invalid true

func NeedsQuotedPrintable added in v0.0.6 

func NeedsQuotedPrintable(text string) bool

NeedsQuotedPrintable returns whether text, with crlf-separated lines, should be encoded with quoted-printable, based on line lengths and any bare carriage return or bare newline. If not, it can be included as 7bit or 8bit encoding in a new message.

func ParseHeaderFields added in v0.0.7 

func ParseHeaderFields(header []byte, scratch []byte, fields [][]byte) (textproto.MIMEHeader, error)

ParseHeaderFields parses only the header fields in "fields" from the complete header buffer "header". It uses "scratch" as temporary space, which can be reused across calls, potentially saving lots of unneeded allocations when only a few headers are needed and/or many messages are parsed.

func ReadHeaders 

func ReadHeaders(msg *bufio.Reader) ([]byte, error)

ReadHeaders returns the headers of a message, ending with a single crlf. Returns ErrHeaderSeparator if no header separator is found.

func ReferencedIDs added in v0.0.7 

func ReferencedIDs(references []string, inReplyTo []string) ([]string, error)

ReferencedIDs returns the Message-IDs referenced from the References header(s), with a fallback to the In-Reply-To header(s). The ids are canonicalized for thread-matching, like with MessageIDCanonical. Empty message-id's are skipped.

func ThreadSubject added in v0.0.7 

func ThreadSubject(subject string, allowNull bool) (threadSubject string, isResponse bool)

ThreadSubject returns the base subject to use for matching against other messages, to see if they belong to the same thread. A matching subject is always required to match to an existing thread, both if References/In-Reply-To header(s) are present, and if not.

isResponse indicates if this message is a response, such as a reply or a forward.

Subject should already be q/b-word-decoded.

If allowNull is true, base subjects with a \0 can be returned. If not set, an empty string is returned if a base subject would have a \0.

Example 
package main

import (
	"fmt"

	"github.com/mjl-/mox/message"
)

func main() {
	// Basic subject.
	s, isResp := message.ThreadSubject("nothing special", false)
	fmt.Printf("%s, response: %v\n", s, isResp)

	// List tags and "re:" are stripped.
	s, isResp = message.ThreadSubject("[list1] [list2] Re: test", false)
	fmt.Printf("%s, response: %v\n", s, isResp)

	// "fwd:" is stripped.
	s, isResp = message.ThreadSubject("fwd: a forward", false)
	fmt.Printf("%s, response: %v\n", s, isResp)

	// Trailing "(fwd)" is also a forward.
	s, isResp = message.ThreadSubject("another forward (fwd)", false)
	fmt.Printf("%s, response: %v\n", s, isResp)

	// [fwd: ...] is stripped.
	s, isResp = message.ThreadSubject("[fwd: [list] fwd: re: it's complicated]", false)
	fmt.Printf("%s, response: %v\n", s, isResp)

}

Output:

nothing special, response: false
test, response: true
a forward, response: true
another forward, response: true
it's complicated, response: true

Types

type Address 

type Address struct {
	Name string // Free-form name for display in mail applications.
	User string // Localpart, encoded as string. Must be parsed before using as Localpart.
	Host string // Domain in ASCII.
}

Address as used in From and To headers.

func ParseAddressList added in v0.0.12 

func ParseAddressList(s string) ([]Address, error)

ParseAddressList parses a string as an address list header value (potentially multiple addresses, comma-separated, with optional display name).

type AuthMethod added in v0.0.6 

type AuthMethod struct {
	// E.g. "dkim", "spf", "iprev", "auth".
	Method  string
	Version string // For optional method version. "1" is implied when missing, which is common.
	Result  string // Each method has a set of known values, e.g. "pass", "temperror", etc.
	Comment string // Optional, message header comment.
	Reason  string // Optional.
	Props   []AuthProp
}

AuthMethod is a result for one authentication method.

Example encoding in the header: "spf=pass smtp.mailfrom=example.net".

type AuthProp added in v0.0.6 

type AuthProp struct {
	// Valid values maintained at https://www.iana.org/assignments/email-auth/email-auth.xhtml
	Type     string
	Property string
	Value    string
	// Whether value is address-like (localpart@domain, or domain). Or another value,
	// which is subject to escaping.
	IsAddrLike bool
	Comment    string // If not empty, header comment without "()", added after Value.
}

AuthProp describes properties for an authentication method. Each method has a set of known properties. Encoded in the header as "type.property=value", e.g. "smtp.mailfrom=example.net" for spf.

func MakeAuthProp added in v0.0.6 

func MakeAuthProp(typ, property, value string, isAddrLike bool, Comment string) AuthProp

MakeAuthProp is a convenient way to make an AuthProp.

type AuthResults added in v0.0.6 

type AuthResults struct {
	Hostname string
	// Optional version of Authentication-Results header, assumed "1" when absent,
	// which is common.
	Version string
	Comment string       // If not empty, header comment without "()", added after Hostname.
	Methods []AuthMethod // Can be empty, in case of "none".
}

Authentication-Results header, see RFC 8601.

func ParseAuthResults added in v0.0.11 

func ParseAuthResults(s string) (ar AuthResults, err error)

ParseAuthResults parses a Authentication-Results header value.

Comments are not populated in the returned AuthResults. Both crlf and lf line-endings are accepted. The input string must end with either crlf or lf.

func (AuthResults) Header added in v0.0.6 

func (h AuthResults) Header() string

Header returns an Authentication-Results header, possibly spanning multiple lines, always ending in crlf.

type Composer added in v0.0.8 

type Composer struct {
	Has8bit  bool  // Whether message contains 8bit data.
	SMTPUTF8 bool  // Whether message needs to be sent with SMTPUTF8 extension.
	Size     int64 // Total bytes written.
	// contains filtered or unexported fields
}

Composer helps compose a message. Operations that fail call panic, which should be caught with recover(), checking for ErrCompose and optionally ErrMessageSize. Writes are buffered.

Example 
package main

import (
	"bytes"
	"errors"
	"fmt"
	"log"
	"strings"
	"time"

	"github.com/mjl-/mox/dns"
	"github.com/mjl-/mox/message"
	"github.com/mjl-/mox/smtp"
)

func main() {
	// We store in a buffer. We could also write to a file.
	var b bytes.Buffer

	// NewComposer. Keep in mind that operations on a Composer will panic on error.
	const smtputf8 = false
	xc := message.NewComposer(&b, 10*1024*1024, smtputf8)

	// Catch and handle errors when composing.
	defer func() {
		x := recover()
		if x == nil {
			return
		}
		if err, ok := x.(error); ok && errors.Is(err, message.ErrCompose) {
			log.Printf("compose: %v", err)
		}
		panic(x)
	}()

	// Add an address header.
	xc.HeaderAddrs("From", []message.NameAddress{{DisplayName: "Charlie", Address: smtp.NewAddress("root", dns.Domain{ASCII: "localhost"})}})

	// Add subject header, with encoding
	xc.Subject("hi ☺")

	// Add Date and Message-ID headers, required.
	tm, _ := time.Parse(time.RFC3339, "2006-01-02T15:04:05+07:00")
	xc.Header("Date", tm.Format(message.RFC5322Z))
	xc.Header("Message-ID", "<unique@host>") // Should generate unique id for each message.

	xc.Header("MIME-Version", "1.0")

	// Write content-* headers for the text body.
	body, ct, cte := xc.TextPart("plain", "this is the body")
	xc.Header("Content-Type", ct)
	xc.Header("Content-Transfer-Encoding", cte)

	// Header/Body separator
	xc.Line()

	// The part body. Use mime/multipart to make messages with multiple parts.
	xc.Write(body)

	// Flush any buffered writes to the original writer.
	xc.Flush()

	fmt.Println(strings.ReplaceAll(b.String(), "\r\n", "\n"))
}

Output:

From: "Charlie" <root@localhost>
Subject: hi =?utf-8?q?=E2=98=BA?=
Date: 2 Jan 2006 15:04:05 +0700
Message-ID: <unique@host>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

this is the body

func NewComposer added in v0.0.8 

func NewComposer(w io.Writer, maxSize int64, smtputf8 bool) *Composer

NewComposer initializes a new composer with a buffered writer around w, and with a maximum message size if maxSize is greater than zero.

smtputf8 must be set when the message must be delivered with smtputf8: if any email address localpart has non-ascii (utf-8).

Operations on a Composer do not return an error. Caller must use recover() to catch ErrCompose and optionally ErrMessageSize errors.

func (*Composer) Checkf added in v0.0.8 

func (c *Composer) Checkf(err error, format string, args ...any)

Checkf checks err, panicing with sentinel error value.

func (*Composer) Flush added in v0.0.8 

func (c *Composer) Flush()

Flush writes any buffered output.

func (*Composer) Header added in v0.0.8 

func (c *Composer) Header(k, v string)

Header writes a message header.

func (*Composer) HeaderAddrs added in v0.0.8 

func (c *Composer) HeaderAddrs(k string, l []NameAddress)

HeaderAddrs writes a message header with addresses.

func (*Composer) Line added in v0.0.8 

func (c *Composer) Line()

Line writes an empty line.

func (*Composer) Subject added in v0.0.8 

func (c *Composer) Subject(subject string)

Subject writes a subject message header.

func (*Composer) TextPart added in v0.0.8 

func (c *Composer) TextPart(subtype, text string) (textBody []byte, ct, cte string)

TextPart prepares a text part to be added. Text should contain lines terminated with newlines (lf), which are replaced with crlf. The returned text may be quotedprintable, if needed. The returned ct and cte headers are for use with Content-Type and Content-Transfer-Encoding headers.

func (*Composer) Write added in v0.0.8 

func (c *Composer) Write(buf []byte) (int, error)

Write implements io.Writer, but calls panic (that is handled higher up) on i/o errors.

type Envelope 

type Envelope struct {
	Date      time.Time
	Subject   string // Q/B-word-decoded.
	From      []Address
	Sender    []Address
	ReplyTo   []Address
	To        []Address
	CC        []Address
	BCC       []Address
	InReplyTo string // From In-Reply-To header, includes <>.
	MessageID string // From Message-Id header, includes <>.
}

Envelope holds the basic/common message headers as used in IMAP4.

func From 

func From(elog *slog.Logger, strict bool, r io.ReaderAt, p *Part) (raddr smtp.Address, envelope *Envelope, header textproto.MIMEHeader, rerr error)

From extracts the address in the From-header.

An RFC5322 message must have a From header. In theory, multiple addresses may be present. In practice zero or multiple From headers may be present. From returns an error if there is not exactly one address. This address can be used for evaluating a DMARC policy against SPF and DKIM results.

type HeaderWriter 

type HeaderWriter struct {
	// contains filtered or unexported fields
}

HeaderWriter helps create headers, folding to the next line when it would become too large. Useful for creating Received and DKIM-Signature headers.

func (*HeaderWriter) Add 

func (w *HeaderWriter) Add(separator string, texts ...string)

Add adds texts, each separated by separator. Individual elements in text are not wrapped.

func (*HeaderWriter) AddWrap 

func (w *HeaderWriter) AddWrap(buf []byte, text bool)

AddWrap adds data. If text is set, wrapping happens at space/tab, otherwise anywhere in the buffer (e.g. for base64 data).

func (*HeaderWriter) Addf 

func (w *HeaderWriter) Addf(separator string, format string, args ...any)

Addf formats the string and calls Add.

func (*HeaderWriter) Newline added in v0.0.9 

func (w *HeaderWriter) Newline()

Newline starts a new line.

func (*HeaderWriter) String 

func (w *HeaderWriter) String() string

String returns the header in string form, ending with \r\n.

type NameAddress added in v0.0.8 

type NameAddress struct {
	DisplayName string
	Address     smtp.Address
}

NameAddress holds both an address display name, and an SMTP path address.

type Part 

type Part struct {
	BoundaryOffset int64 // Offset in message where bound starts. -1 for top-level message.
	HeaderOffset   int64 // Offset in message file where header starts.
	BodyOffset     int64 // Offset in message file where body starts.
	EndOffset      int64 // Where body of part ends. Set when part is fully read.
	RawLineCount   int64 // Number of lines in raw, undecoded, body of part. Set when part is fully read.
	DecodedSize    int64 // Number of octets when decoded. If this is a text mediatype, lines ending only in LF are changed end in CRLF and DecodedSize reflects that.

	MediaType               string            // From Content-Type, upper case. E.g. "TEXT". Can be empty because content-type may be absent. In this case, the part may be treated as TEXT/PLAIN.
	MediaSubType            string            // From Content-Type, upper case. E.g. "PLAIN".
	ContentTypeParams       map[string]string // E.g. holds "boundary" for multipart messages. Has lower-case keys, and original case values.
	ContentID               string
	ContentDescription      string
	ContentTransferEncoding string    // In upper case.
	Envelope                *Envelope // Email message headers. Not for non-message parts.

	Parts []Part // Parts if this is a multipart.

	// Only for message/rfc822 and message/global. This part may have a buffer as
	// backing io.ReaderAt, because a message/global can have a non-identity
	// content-transfer-encoding. This part has a nil parent.
	Message *Part
	// contains filtered or unexported fields
}

Part represents a whole mail message, or a part of a multipart message. It is designed to handle IMAP requirements efficiently.

Example 
package main

import (
	"log"
	"log/slog"
	"strings"

	"github.com/mjl-/mox/message"
)

func main() {
	// Parse a message from an io.ReaderAt, which could be a file.
	strict := false
	r := strings.NewReader("header: value\r\nanother: value\r\n\r\nbody ...\r\n")
	part, err := message.Parse(slog.Default(), strict, r)
	if err != nil {
		log.Fatalf("parsing message: %v", err)
	}

	// The headers of the first part have been parsed, i.e. the message headers.
	// A message can be multipart (e.g. alternative, related, mixed), and possibly
	// nested.

	// By walking the entire message, all part metadata (like offsets into the file
	// where a part starts) is recorded.
	err = part.Walk(slog.Default(), nil)
	if err != nil {
		log.Fatalf("walking message: %v", err)
	}

	// Messages can have a recursive multipart structure. Print the structure.
	var printPart func(indent string, p message.Part)
	printPart = func(indent string, p message.Part) {
		log.Printf("%s- part: %v", indent, part)
		for _, pp := range p.Parts {
			printPart("  "+indent, pp)
		}
	}
	printPart("", part)
}

Output:

func EnsurePart 

func EnsurePart(elog *slog.Logger, strict bool, r io.ReaderAt, size int64) (Part, error)

EnsurePart parses a part as with Parse, but ensures a usable part is always returned, even if error is non-nil. If a parse error occurs, the message is returned as application/octet-stream, and headers can still be read if they were valid.

If strict is set, fewer attempts are made to continue parsing when errors are encountered, such as with invalid content-type headers or bare carriage returns.

func Parse 

func Parse(elog *slog.Logger, strict bool, r io.ReaderAt) (Part, error)

Parse reads the headers of the mail message and returns a part. A part provides access to decoded and raw contents of a message and its multiple parts.

If strict is set, fewer attempts are made to continue parsing when errors are encountered, such as with invalid content-type headers or bare carriage returns.

func (*Part) Header 

func (p *Part) Header() (textproto.MIMEHeader, error)

Header returns the parsed header of this part.

Returns a ErrHeader for messages with invalid header syntax.

func (*Part) HeaderReader 

func (p *Part) HeaderReader() io.Reader

HeaderReader returns a reader for the header section of this part, including ending bare CRLF.

func (*Part) IsDSN added in v0.0.10 

func (p *Part) IsDSN() bool

IsDSN returns whether the MIME structure of the part is a DSN.

func (*Part) ParseNextPart 

func (p *Part) ParseNextPart(elog *slog.Logger) (*Part, error)

ParseNextPart parses the next (sub)part of this multipart message. ParseNextPart returns io.EOF and a nil part when there are no more parts. Only used for initial parsing of message. Once parsed, use p.Parts.

func (*Part) RawReader 

func (p *Part) RawReader() io.Reader

RawReader returns a reader for the raw, undecoded body content. E.g. with quoted-printable or base64 content intact. Fully reading a part helps its parent part find its next part efficiently.

func (*Part) Reader 

func (p *Part) Reader() io.Reader

Reader returns a reader for the decoded body content.

func (*Part) ReaderUTF8OrBinary added in v0.0.6 

func (p *Part) ReaderUTF8OrBinary() io.Reader

ReaderUTF8OrBinary returns a reader for the decoded body content, transformed to utf-8 for known mime/iana encodings (only if they aren't us-ascii or utf-8 already). For unknown or missing character sets/encodings, the original reader is returned.

func (*Part) SetMessageReaderAt 

func (p *Part) SetMessageReaderAt() error

SetMessageReaderAt sets a reader on p.Message, which must be non-nil.

func (*Part) SetReaderAt 

func (p *Part) SetReaderAt(r io.ReaderAt)

SetReaderAt sets r as reader for this part and all its sub parts, recursively. No reader is set for any Message subpart, see SetMessageReaderAt.

func (*Part) String 

func (p *Part) String() string

String returns a debugging representation of the part.

func (*Part) Walk 

func (p *Part) Walk(elog *slog.Logger, parent *Part) error

Walk through message, decoding along the way, and collecting mime part offsets and sizes, and line counts.

type Writer 

type Writer struct {
	HaveBody bool  // Body is optional in a message. ../rfc/5322:343
	Has8bit  bool  // Whether a byte with the high/8bit has been read. So whether this needs SMTP 8BITMIME instead of 7BIT.
	Size     int64 // Number of bytes written, may be different from bytes read due to LF to CRLF conversion.
	// contains filtered or unexported fields
}

Writer is a write-through helper, collecting properties about the written message and replacing bare \n line endings with \r\n.

Example 
package main

import (
	"fmt"
	"strings"

	"github.com/mjl-/mox/message"
)

func main() {
	// NewWriter on a string builder.
	var b strings.Builder
	w := message.NewWriter(&b)

	// Write some lines, some with proper CRLF line ending, others without.
	fmt.Fprint(w, "header: value\r\n")
	fmt.Fprint(w, "another: value\n") // missing \r
	fmt.Fprint(w, "\r\n")
	fmt.Fprint(w, "hi ☺\n") // missing \r

	fmt.Printf("%q\n", b.String())
	fmt.Printf("%v %v", w.HaveBody, w.Has8bit)
}

Output:

"header: value\r\nanother: value\r\n\r\nhi ☺\r\n"
true true

func NewWriter added in v0.0.6 

func NewWriter(w io.Writer) *Writer

func (*Writer) Write 

func (w *Writer) Write(buf []byte) (int, error)

Write implements io.Writer, and writes buf as message to the Writer's underlying io.Writer. It converts bare new lines (LF) to carriage returns with new lines (CRLF).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.