httphead

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 8, 2020 License: MIT Imports: 5 Imported by: 117

README

httphead.go

GoDoc

Tiny HTTP header value parsing library in go.

Overview

This library contains low-level functions for scanning HTTP RFC2616 compatible header value grammars.

Install

    go get github.com/gobwas/httphead

Example

The example below shows how multiple-choise HTTP header value could be parsed with this library:

	options, ok := httphead.ParseOptions([]byte(`foo;bar=1,baz`), nil)
	fmt.Println(options, ok)
	// Output: [{foo map[bar:1]} {baz map[]}] true

The low-level example below shows how to optimize keys skipping and selection of some key:

	// The right part of full header line like:
	// X-My-Header: key;foo=bar;baz,key;baz
	header := []byte(`foo;a=0,foo;a=1,foo;a=2,foo;a=3`)

	// We want to search key "foo" with an "a" parameter that equal to "2".
	var (
		foo = []byte(`foo`)
		a   = []byte(`a`)
		v   = []byte(`2`)
	)
	var found bool
	httphead.ScanOptions(header, func(i int, key, param, value []byte) Control {
		if !bytes.Equal(key, foo) {
			return ControlSkip
		}
		if !bytes.Equal(param, a) {
			if bytes.Equal(value, v) {
				// Found it!
				found = true
				return ControlBreak
			}
			return ControlSkip
		}
		return ControlContinue
	})

For more usage examples please see docs or package tests.

Documentation

Overview

Package httphead contains utils for parsing HTTP and HTTP-grammar compatible text protocols headers.

That is, this package first aim is to bring ability to easily parse constructions, described here https://tools.ietf.org/html/rfc2616#section-2

Index

Examples

Constants

This section is empty.

Variables

View Source 
var DefaultCookieScanner = CookieScanner{}

DefaultCookieScanner is a CookieScanner which is used by ScanCookie(). Note that it is intended to have the same behavior as http.Request.Cookies() has.

View Source 
var OctetTypes [256]OctetType

OctetTypes is a table of octets.

Functions

func CanonicalizeHeaderKey 

func CanonicalizeHeaderKey(k []byte)

CanonicalizeHeaderKey is like standard textproto/CanonicalMIMEHeaderKey, except that it operates with slice of bytes and modifies it inplace without copying.

func IntFromASCII 

func IntFromASCII(bts []byte) (ret int, ok bool)

IntFromASCII converts ascii encoded decimal numeric value from HTTP entities to an integer.

func ParseHeaderLine 

func ParseHeaderLine(line []byte) (k, v []byte, ok bool)

ParseHeaderLine parses HTTP header as key-value pair. It returns parsed values and true if parse is ok.

func ParseVersion 

func ParseVersion(bts []byte) (major, minor int, ok bool)

ParseVersion parses major and minor version of HTTP protocol. It returns parsed values and true if parse is ok.

func ReadLine 

func ReadLine(br *bufio.Reader) ([]byte, error)

ReadLine reads line from br. It reads until '\n' and returns bytes without '\n' or '\r\n' at the end. It returns err if and only if line does not end in '\n'. Note that read bytes returned in any case of error.

It is much like the textproto/Reader.ReadLine() except the thing that it returns raw bytes, instead of string. That is, it avoids copying bytes read from br.

textproto/Reader.ReadLineBytes() is also makes copy of resulting bytes to be safe with future I/O operations on br.

We could control I/O operations on br and do not need to make additional copy for safety.

func RemoveByte 

func RemoveByte(data []byte, c byte) []byte

RemoveByte returns data without c. If c is not present in data it returns the same slice. If not, it copies data without c.

func ScanCookie 

func ScanCookie(data []byte, it func(key, value []byte) bool) bool

ScanCookie scans cookie pairs from data using DefaultCookieScanner.Scan() method.

func ScanOptions 

func ScanOptions(data []byte, it func(index int, option, attribute, value []byte) Control) bool

ScanOptions parses data in this form:

values = 1#value value = token *( ";" param ) param = token [ "=" (token | quoted-string) ]

It calls given callback with the index of the option, option itself and its parameter (attribute and its value, both could be nil). Index is useful when header contains multiple choises for the same named option.

Given callback should return one of the defined Control* values. ControlSkip means that passed key is not in caller's interest. That is, all parameters of that key will be skipped. ControlBreak means that no more keys and parameters should be parsed. That is, it must break parsing immediately. ControlContinue means that caller want to receive next parameter and its value or the next key.

It returns false if data is malformed.

Example 
foo := map[string]string{}

ScanOptions([]byte(`foo;bar=1;baz`), func(index int, key, param, value []byte) Control {
	foo[string(param)] = string(value)
	return ControlContinue
})

fmt.Printf("bar:%s baz:%s", foo["bar"], foo["baz"])

Output:

bar:1 baz:

func ScanPairGreedy 

func ScanPairGreedy(data []byte, open, close byte) (n int)

ScanPairGreedy scans for complete pair of opening and closing chars in greedy manner. Note that first opening byte must not be present in data.

func ScanTokens 

func ScanTokens(data []byte, it func([]byte) bool) bool

ScanTokens parses data in this form:

list = 1#token

It returns false if data is malformed.

Example 
var values []string

ScanTokens([]byte(`a,b,c`), func(v []byte) bool {
	values = append(values, string(v))
	return v[0] != 'b'
})

fmt.Println(values)

Output:

[a b]

func ScanUntil 

func ScanUntil(data []byte, c byte) (n int)

ScanUntil scans for first non-escaped character c in given data. It returns index of matched c and -1 if c is not found.

func SkipSpace 

func SkipSpace(p []byte) (n int)

SkipSpace skips spaces and lws-sequences from p. It returns number ob bytes skipped.

func SplitRequestLine 

func SplitRequestLine(line []byte) (method, uri, version []byte)

SplitRequestLine splits given slice of bytes into three chunks without parsing.

func SplitResponseLine 

func SplitResponseLine(line []byte) (version, status, reason []byte)

SplitResponseLine splits given slice of bytes into three chunks without parsing.

func ValidCookieName 

func ValidCookieName(name []byte) bool

ValidCookieName reports wheter given bytes is a valid RFC2616 "token" bytes.

func ValidCookieValue 

func ValidCookieValue(value []byte, strict bool) bool

ValidCookieValue reports whether given value is a valid RFC6265 "cookie-octet" bytes.

cookie-octet = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E 

; US-ASCII characters excluding CTLs,
; whitespace DQUOTE, comma, semicolon,
; and backslash

Note that the false strict parameter disables errors on space 0x20 and comma 0x2c. This could be useful to bring some compatibility with non-compliant clients/servers in the real world. It acts the same as standard library cookie parser if strict is false.

func WriteOptions 

func WriteOptions(dest io.Writer, options []Option) (n int, err error)

WriteOptions write options list to the dest. It uses the same form as {Scan,Parse}Options functions: values = 1#value value = token *( ";" param ) param = token [ "=" (token | quoted-string) ]

It wraps valuse into the quoted-string sequence if it contains any non-token characters.

Example 
opts := []Option{
	NewOption("foo", map[string]string{
		"param": "hello, world!",
	}),
	NewOption("bar", nil),
	NewOption("b a z", nil),
}

buf := bytes.Buffer{}
bw := bufio.NewWriter(&buf)

WriteOptions(bw, opts)
bw.Flush()

Output:

foo;param="hello, world!",bar,"b a z"

Types

type Control 

type Control byte

Control represents operation that scanner should perform. 

const (
	// ControlContinue causes scanner to continue scan tokens.
	ControlContinue Control = iota
	// ControlBreak causes scanner to stop scan tokens.
	ControlBreak
	// ControlSkip causes scanner to skip current entity.
	ControlSkip
)

type CookieScanner 

type CookieScanner struct {
	// DisableNameValidation disables name validation of a cookie. If false,
	// only RFC2616 "tokens" are accepted.
	DisableNameValidation bool

	// DisableValueValidation disables value validation of a cookie. If false,
	// only RFC6265 "cookie-octet" characters are accepted.
	//
	// Note that Strict option also affects validation of a value.
	//
	// If Strict is false, then scanner begins to allow space and comma
	// characters inside the value for better compatibility with non standard
	// cookies implementations.
	DisableValueValidation bool

	// BreakOnPairError sets scanner to immediately return after first pair syntax
	// validation error.
	// If false, scanner will try to skip invalid pair bytes and go ahead.
	BreakOnPairError bool

	// Strict enables strict RFC6265 mode scanning. It affects name and value
	// validation, as also some other rules.
	// If false, it is intended to bring the same behavior as
	// http.Request.Cookies().
	Strict bool
}

CookieScanner contains options for scanning cookie pairs. See https://tools.ietf.org/html/rfc6265#section-4.1.1

func (CookieScanner) Scan 

func (c CookieScanner) Scan(data []byte, it func(name, value []byte) bool) bool

Scan maps data to name and value pairs. Usually data represents value of the Cookie header.

type ItemType 

type ItemType int

ItemType encodes type of the lexing token. 

const (
	// ItemUndef reports that token is undefined.
	ItemUndef ItemType = iota
	// ItemToken reports that token is RFC2616 token.
	ItemToken
	// ItemSeparator reports that token is RFC2616 separator.
	ItemSeparator
	// ItemString reports that token is RFC2616 quouted string.
	ItemString
	// ItemComment reports that token is RFC2616 comment.
	ItemComment
	// ItemOctet reports that token is octet slice.
	ItemOctet
)

func ScanToken 

func ScanToken(p []byte) (n int, t ItemType)

ScanToken scan for next token in p. It returns length of the token and its type. It do not trim p.

type OctetType 

type OctetType byte

OctetType desribes character type.

From the "Basic Rules" chapter of RFC2616 See https://tools.ietf.org/html/rfc2616#section-2.2

OCTET = <any 8-bit sequence of data> CHAR = <any US-ASCII character (octets 0 - 127)> UPALPHA = <any US-ASCII uppercase letter "A".."Z"> LOALPHA = <any US-ASCII lowercase letter "a".."z"> ALPHA = UPALPHA | LOALPHA DIGIT = <any US-ASCII digit "0".."9"> CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)> CR = <US-ASCII CR, carriage return (13)> LF = <US-ASCII LF, linefeed (10)> SP = <US-ASCII SP, space (32)> HT = <US-ASCII HT, horizontal-tab (9)> <"> = <US-ASCII double-quote mark (34)> CRLF = CR LF LWS = [CRLF] 1*( SP | HT )

Many HTTP/1.1 header field values consist of words separated by LWS or special characters. These special characters MUST be in a quoted string to be used within a parameter value (as defined in section 3.6).

token = 1*<any CHAR except CTLs or separators> separators = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <"> | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT

func (OctetType) IsChar 

func (t OctetType) IsChar() bool

IsChar reports whether octet is CHAR.

func (OctetType) IsControl 

func (t OctetType) IsControl() bool

IsControl reports whether octet is CTL.

func (OctetType) IsSeparator 

func (t OctetType) IsSeparator() bool

IsSeparator reports whether octet is separator.

func (OctetType) IsSpace 

func (t OctetType) IsSpace() bool

IsSpace reports whether octet is space (SP or HT).

func (OctetType) IsToken 

func (t OctetType) IsToken() bool

IsToken reports whether octet is token.

type Option 

type Option struct {
	Name       []byte
	Parameters Parameters
}

Option represents a header option.

func NewOption 

func NewOption(name string, params map[string]string) Option

NewOption creates named option with given parameters.

func ParseOptions 

func ParseOptions(data []byte, options []Option) ([]Option, bool)

ParseOptions parses all header options and appends it to given slice of Option. It returns flag of successful (wellformed input) parsing.

Note that appended options are all consist of subslices of data. That is, mutation of data will mutate appended options.

Example 
options, ok := ParseOptions([]byte(`foo;bar=1,baz`), nil)
fmt.Println(options, ok)

Output:

[{foo [bar:1]} {baz []}] true

func (Option) Clone 

func (opt Option) Clone() Option

Clone is a shorthand for making slice of opt.Size() sequenced with Copy() call.

func (Option) Copy 

func (opt Option) Copy(p []byte) Option

Copy copies all underlying []byte slices into p and returns new Option. Note that p must be at least of opt.Size() length.

func (Option) Equal 

func (opt Option) Equal(b Option) bool

Equal reports whether option is equal to b.

func (Option) Size 

func (opt Option) Size() int

Size returns number of bytes need to be allocated for use in opt.Copy.

func (Option) String 

func (opt Option) String() string

String represents option as a string.

type OptionSelector 

type OptionSelector struct {
	// Check is a filter function that applied to every Option that possibly
	// could be selected.
	// If Check is nil all options will be selected.
	Check func(Option) bool

	// Flags contains flags for options selection.
	Flags SelectFlag

	// Alloc used to allocate slice of bytes when selector is configured with
	// SelectCopy flag. It will be called with number of bytes needed for copy
	// of single Option.
	// If Alloc is nil make is used.
	Alloc func(n int) []byte
}

OptionSelector contains configuration for selecting Options from header value.

func (OptionSelector) Select 

func (s OptionSelector) Select(data []byte, options []Option) ([]Option, bool)

Select parses header data and appends it to given slice of Option. It also returns flag of successful (wellformed input) parsing.

type Parameters 

type Parameters struct {
	// contains filtered or unexported fields
}

Parameters represents option's parameters.

func (*Parameters) Copy 

func (p *Parameters) Copy(dst []byte) (Parameters, []byte)

Copy copies all underlying []byte slices into dst and returns new Parameters. Note that dst must be at least of p.Size() length.

func (Parameters) Equal 

func (p Parameters) Equal(b Parameters) bool

Equal reports whether a equal to b.

func (*Parameters) ForEach 

func (p *Parameters) ForEach(cb func(k, v []byte) bool)

ForEach iterates over parameters key-value pairs and calls cb for each one.

func (*Parameters) Get 

func (p *Parameters) Get(key string) (value []byte, ok bool)

Get returns value by key and flag about existence such value.

func (*Parameters) Set 

func (p *Parameters) Set(key, value []byte)

Set sets value by key.

func (*Parameters) Size 

func (p *Parameters) Size() int

Size returns number of bytes that needed to copy p.

func (*Parameters) String 

func (p *Parameters) String() (ret string)

String represents parameters as a string.

type RequestLine 

type RequestLine struct {
	Method  []byte
	URI     []byte
	Version Version
}

RequestLine contains parameters parsed from the first request line.

func ParseRequestLine 

func ParseRequestLine(line []byte) (r RequestLine, ok bool)

ParseRequestLine parses http request line like "GET / HTTP/1.0".

type ResponseLine 

type ResponseLine struct {
	Version Version
	Status  int
	Reason  []byte
}

ResponseLine contains parameters parsed from the first response line.

func ParseResponseLine 

func ParseResponseLine(line []byte) (r ResponseLine, ok bool)

ParseResponseLine parses first response line into ResponseLine struct.

type Scanner 

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner represents header tokens scanner. See https://tools.ietf.org/html/rfc2616#section-2

func NewScanner 

func NewScanner(data []byte) *Scanner

NewScanner creates new RFC2616 data scanner.

func (*Scanner) Advance 

func (l *Scanner) Advance(n int) bool

Advance moves current position index at n bytes. It returns true on successful move.

func (*Scanner) Buffered 

func (l *Scanner) Buffered() int

Buffered reporst how many bytes there are left to scan.

func (*Scanner) Bytes 

func (l *Scanner) Bytes() []byte

Bytes returns current token bytes.

func (*Scanner) FetchUntil 

func (l *Scanner) FetchUntil(c byte) bool

FetchUntil fetches ItemOctet from current scanner position to first occurence of the c or to the end of the underlying data.

func (*Scanner) Next 

func (l *Scanner) Next() bool

Next scans for next token. It returns true on successful scanning, and false on error or EOF.

func (*Scanner) Peek 

func (l *Scanner) Peek() byte

Peek reads byte at current position without advancing it. On end of data it returns 0.

func (*Scanner) Peek2 

func (l *Scanner) Peek2() (a, b byte)

Peek2 reads two first bytes at current position without advancing it. If there not enough data it returs 0.

func (*Scanner) Skip 

func (l *Scanner) Skip(c byte)

Skip skips all bytes until first occurence of c.

func (*Scanner) SkipEscaped 

func (l *Scanner) SkipEscaped(c byte)

SkipEscaped skips all bytes until first occurence of non-escaped c.

func (*Scanner) Type 

func (l *Scanner) Type() ItemType

Type reports current token type.

type SelectFlag 

type SelectFlag byte

SelectFlag encodes way of options selection. 

const (
	// SelectCopy causes selector to copy selected option before appending it
	// to resulting slice.
	// If SelectCopy flag is not passed to selector, then appended options will
	// contain sub-slices of the initial data.
	SelectCopy SelectFlag = 1 << iota

	// SelectUnique causes selector to append only not yet existing option to
	// resulting slice. Unique is checked by comparing option names.
	SelectUnique
)

func (SelectFlag) String 

func (f SelectFlag) String() string

String represetns flag as string.

type Version 

type Version struct {
	Major int
	Minor int
}

Version contains protocol major and minor version.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.