go_percent_encoding

README

go-percent-encoding 🚀

go-percent-encoding is a Go package designed to handle percent encoding with various character encodings. Although the URI standard (RFC 3986) doesn't specify a required character encoding for percent-encoded data, UTF-8 is widely used and recommended. However, some systems, especially older ones or those in specific regions, may use different encodings like GBK. This package allows developers to convert percent-encoded URIs between UTF-8 and other character encodings, ensuring compatibility with various systems.

Features ✨

• Convert Non-UTF-8 Encoded URIs to UTF-8: Easily convert percent-encoded strings that were originally encoded using a different character encoding to a UTF-8 encoded string.

• Convert UTF-8 Encoded URIs to Other Encodings: Convert percent-encoded strings from UTF-8 to a different character encoding.

• Custom Encoding Support: Supports custom encoding standards via the Go text encoding package.

Installation 📦

To install go-percent-encoding, run:

go get github.com/leoleaf/go-percent-encoding

Usage 🛠️

Converting to UTF-8

import (
    "github.com/leoleaf/go-percent-encoding"
    "golang.org/x/text/encoding/simplifiedchinese"
)

func example() {
    input := "%D6%D0%CE%C4" // Chinese characters in GBK encoding
    decoder := simplifiedchinese.GBK.NewDecoder()
    result, err := go_percent_encoding.Other2Utf8(input, decoder)
    if err != nil {
        fmt.Println("Error:", err)
    }
    fmt.Println("Converted:", result)
}

Converting from UTF-8 to Another Encoding

import (
    "github.com/leoleaf/go-percent-encoding"
    "golang.org/x/text/encoding/simplifiedchinese"
)

func example() {
    input := "%E4%B8%AD%E6%96%87" // UTF-8 percent-encoded string for "中文"
    encoder := simplifiedchinese.GBK.NewEncoder()
    result, err := go_percent_encoding.Utf8ToOther(input, encoder)
    if err != nil {
        fmt.Println("Error:", err)
    }
    fmt.Println("Converted:", result)
}

Encoding a Byte Slice

import (
    "github.com/leoleaf/go-percent-encoding"
)

func example() {
    input := []byte("Go语言") // "Go语言" in UTF-8
    result := go_percent_encoding.Encode(input)
    fmt.Println("Encoded:", result)
}

License ⚖️

This project is licensed under the MIT License - see the LICENSE file for details.

Documentation

Index

• func Encode(bs []byte) string
• func Other2Utf8(raw string, from *encoding.Decoder) (string, error)
• func Utf8ToOther(raw string, to *encoding.Encoder) (string, error)
• type Error
  • func (e *Error) Error() string

Examples

• Other2Utf8
• Utf8ToOther

Functions

func Encode

func Encode(bs []byte) string

Encode encodes a byte slice into a percent-encoded string. Each byte is converted to its corresponding percent-encoded representation using uppercase hexadecimal digits. This method does not perform any character encoding, but simply represents the raw bytes as percent-encoded.

func Other2Utf8

func Other2Utf8(raw string, from *encoding.Decoder) (string, error)

Other2Utf8 converts a percent-encoded string from a specific encoding (e.g., GBK) to a UTF-8 encoded string. This function is useful when working with URIs that have been percent-encoded using non-UTF-8 encodings. It takes the original encoded string and a decoder for the source encoding, returning the UTF-8 encoded result or an error if the input is malformed.

Example

package main

import (
	"fmt"
	"net/url"

	go_percent_encoding "github.com/leoleaf/go-percent-encoding"
	"golang.org/x/text/encoding/simplifiedchinese"
)

func main() {
	// assume the web server encoding is GBK
	// get the url from the web server
	urlFromServer := "http://www.example.com/register?age=19&name=%D5%C5%C8%FD"
	u, err := url.Parse(urlFromServer)
	if err != nil {
		fmt.Println(err)
		return
	}

	dec := simplifiedchinese.GBK.NewDecoder()
	u.RawQuery, err = go_percent_encoding.Other2Utf8(u.RawQuery, dec)
	if err != nil {
		fmt.Println(err)
		return
	}

	query := u.Query()

	fmt.Println(query.Get("age"))
	fmt.Println(query.Get("name"))
}

Output:

19
张三

func Utf8ToOther

func Utf8ToOther(raw string, to *encoding.Encoder) (string, error)

Utf8ToOther converts a UTF-8 encoded, percent-encoded string to a string encoded with a different character encoding (e.g., GBK). This function is useful for generating URIs expected to be interpreted by systems that use non-UTF-8 encodings. It takes the UTF-8 encoded string and an encoder for the target encoding, returning the newly encoded result or an error if the input is malformed.

Example

package main

import (
	"fmt"
	"net/url"

	go_percent_encoding "github.com/leoleaf/go-percent-encoding"
	"golang.org/x/text/encoding/simplifiedchinese"
)

func main() {
	u, err := url.Parse("http://www.example.com/register")
	if err != nil {
		fmt.Println(err)
		return
	}

	// add query string
	query := u.Query()
	query.Add("age", "19")
	query.Add("name", "张三")

	u.RawQuery = query.Encode()

	fmt.Println(u.String())

	// if the web server encoding is not utf-8, you need to encode the query string.
	// assume the web server encoding is GBK
	enc := simplifiedchinese.GBK.NewEncoder()
	targetUrl, err := go_percent_encoding.Utf8ToOther(u.String(), enc)
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Println(targetUrl)
}

Output:

http://www.example.com/register?age=19&name=%E5%BC%A0%E4%B8%89
http://www.example.com/register?age=19&name=%D5%C5%C8%FD

Types

type Error

type Error struct {
	// contains filtered or unexported fields
}

func (*Error) Error

func (e *Error) Error() string