percent

README

percent

codeberg.org/piman/percent implements a transform.Transformer for percent encoding (aka URL encoding, query escaping, etc.) from RFC 3986 §2.1; and another for decoding.

This is similar to net/url.EscapePath, net/url.UnescapePath, and so on. If you need to handle short strings of regular URL components those may be preferable. The transformers here help handle unbounded streams of data, unusual escaping requirements, or percent encodings as one possible transformation alongside others.

License

Copyright 2023 Joe Wreschnig

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Documentation

Overview

Package percent implements a transform.Transformer for percent encoding (aka URL encoding, query escaping, etc.) from RFC 3986 §2.1; and another for decoding.

This is similar to net/url.EscapePath, net/url.UnescapePath, and so on. If you need to handle short strings of regular URL components those may be preferable. The transformers here help handle unbounded streams of data, unusual escaping requirements, or percent encodings as one possible transformation alongside others.

Example (Decoding)

package main

import (
	"os"

	"golang.org/x/text/transform"

	"codeberg.org/piman/percent"
)

func main() {
	w := transform.NewWriter(os.Stdout, percent.Decoder)
	for _, s := range []string{
		"%44%69%65%20%41%68%6E%75%6E%67%20%61%75%66%64%C3%A4%6D%6D%",
		"65%72%74%2C%20%64%61%C3%9F%20%64%69%65%20%6A%65%74%7A%69%6",
		"7%65%20%47%65%73%65%6C%6C%73%63%68%61%66%74%20%6B%65%69%6E",
		"%20%66%65%73%74%65%72%0A%4B%72%69%73%74%61%6C%6C%2C%20%73%",
		"6F%6E%64%65%72%6E%20%65%69%6E%20%75%6D%77%61%6E%64%6C%75%6",
		"E%67%73%66%C3%A4%68%69%67%65%72%20%75%6E%64%20%62%65%73%74",
		"%C3%A4%6E%64%69%67%20%69%6D%20%50%72%6F%7A%65%C3%9F%0A%64%",
		"65%72%20%55%6D%77%61%6E%64%6C%75%6E%67%20%62%65%67%72%69%6",
		"6%66%65%6E%65%72%20%4F%72%67%61%6E%69%73%6D%75%73%20%69%73",
		"%74%2E",
	} {
		_, _ = w.Write([]byte(s))
	}
}

Output:

Die Ahnung aufdämmert, daß die jetzige Gesellschaft kein fester
Kristall, sondern ein umwandlungsfähiger und beständig im Prozeß
der Umwandlung begriffener Organismus ist.

Example (Encoding)

package main

import (
	"os"

	"golang.org/x/text/transform"

	"codeberg.org/piman/percent"
)

func main() {
	w := transform.NewWriter(
		os.Stdout,
		percent.NewEncoder(
			percent.Bytes{}.Allow('.', '0', '\n'),
		),
	)
	_, _ = w.Write([]byte{10, 46, 46, 46, 48, 48, 48, 46, 46, 46,
		46, 46, 48, 48, 48, 46, 46, 46, 10, 46, 48, 48, 0, 0,
		48, 0, 0, 46, 10, 48, 48, 0, 46, 48, 48, 0, 46, 48, 48,
		0, 10, 48, 48, 0, 46, 46, 48, 48, 48, 46, 46, 48, 48, 0,
		10, 46, 48, 48, 0, 46, 46, 46, 46, 46, 48, 48, 0, 46,
		10, 46, 46, 48, 48, 0, 46, 46, 46, 48, 48, 0, 46, 46,
		10, 46, 46, 46, 48, 48, 0, 46, 48, 48, 0, 46, 46, 46,
		10, 46, 46, 46, 46, 48, 48, 0, 48, 0, 46, 46, 46, 46,
		10, 46, 46, 46, 46, 46, 46, 48, 48, 0, 46, 46, 46, 46,
		46, 46, 10, 46, 46, 46, 46, 46, 46, 46, 46, 48, 46, 46,
		46, 46, 46, 46, 46, 46})
}

Output:

...000.....000...
.00%00%000%00%00.
00%00.00%00.00%00
00%00..000..00%00
.00%00.....00%00.
..00%00...00%00..
...00%00.00%00...
....00%000%00....
......00%00......
........0........

Index

• Variables
• func NewEncoder(allowed Bytes) transform.Transformer
• type Bytes
  • func (b Bytes) Allow(xs ...byte) Bytes
  • func (b Bytes) AllowRange(lo, hi byte) Bytes
  • func (b Bytes) Allowed(x byte) bool
  • func (b Bytes) Reserve(xs ...byte) Bytes

Examples

• Package (Decoding)
• Package (Encoding)

Variables

var (
	// URIEncoder percent-escapes all bytes except ASCII
	// alphanumerics and ‘-’, ‘.’, and ‘_’, unreserved in RFC 3986
	// §2.3. ‘~’ is still escaped as it was historically also
	// reserved. (E.g. it is not present in RFC 1738 “safe”.)
	//
	// Percent encoding has been used in many contexts with slightly
	// differing sets of allowed bytes. This encoder should be safe
	// in most of them.
	URIEncoder = NewEncoder(Bytes{}.
				AllowRange('0', '9').
				AllowRange('A', 'Z').
				AllowRange('a', 'z').
				Allow('-', '.', '_'))

	// Decoder percent-decodes text per RFC 3986 §2.2. It is capable
	// of decoding any percent-encoded text regardless of the
	// encoder.
	//
	// Decoded text will never be longer than encoded text, so you
	// can safely use the same buffer for input and output of any
	// transformation.
	//
	// If the decoder encounters malformed input it returns a
	// [url.EscapeError].
	Decoder transform.Transformer = strictDecoder{}

	// LaxDecoder percent-decodes text per the WHATWG [URL Living
	// Standard], in which any bytes which would fail to decode are
	// decoded as themselves; it therefore never returns an error.
	//
	// [URL Living Standard]: https://url.spec.whatwg.org/#percent-encoded-bytes
	LaxDecoder transform.Transformer = laxDecoder{}
)

Functions

func NewEncoder

func NewEncoder(allowed Bytes) transform.Transformer

NewEncoder return a percent-encoding transformer per RFC 3986 §2.1. These encoders are stateless; they does not need to be reset and can safely be used concurrently. ‘%’ is always escaped (as ‘%25’) regardless of the allowed bytes.

Types

type Bytes

type Bytes struct {
	// contains filtered or unexported fields
}

Bytes indicate byte values which can represent themselves when encoded. Bytes are immutable; modifications return new Bytes. A zero Bytes permits no value to represent itself.

func (Bytes) Allow

func (b Bytes) Allow(xs ...byte) Bytes

Allow allows the given bytes to represent themselves when encoded.

func (Bytes) AllowRange

func (b Bytes) AllowRange(lo, hi byte) Bytes

AllowRange allows all bytes between the two values, inclusively.

func (Bytes) Allowed

func (b Bytes) Allowed(x byte) bool

Allowed checks if a byte value may represent itself.

func (Bytes) Reserve

func (b Bytes) Reserve(xs ...byte) Bytes

Reserve reserves the given bytes; they may not represent themselves when encoded and will be escaped.