defang_uri_schemes

README

Defang URI Schemes

Define defanging algorithm for non-HTTP[S] URI schemes.

Description

To defang a URI is to remove its fangs: make it un-harmful. Much software supports URI hyperlinks, so that a URI can be easily clicked on. However, this poses a security risk.

There is a convention in security to defang a URI scheme such that we write hxxp[s] instead of http[s]. Unfortunately, many schemes exist that do not have a well-defined method of defanging.

The present library suggests a simple algorithm to defang any™ URI scheme, with the following goals:

1. Invalidate the URI: the "defanged" scheme must not be a valid scheme; and
2. One-to-one mapping: the defanged scheme should map to one and only one un-fanged scheme, so that it is unambiguous for the security researcher.

---

Quick Start

Get the module:

$ go get github.com/jakewilliami/defang-uri-schemes

Generating the library file and checking its validity:

$ go generate
[INFO] Generating library file
[INFO] Found base module path at /Users/jakeireland/projects/defang-uri-schemes
[INFO] found table [columns [Range Registration Procedures] count 3]
[INFO] found table [columns [URI Scheme Template Description Status Well-Known URI Support Reference Notes] count 384]
[INFO] found table [columns [Range Registration Procedures Note] count 5]
[INFO] found table [columns [Name Range (dec) Range (hex) Range Length (Bits) Reference Change Controller] count 2]
[INFO] found table [columns [Range Registration Procedures] count 3]
[INFO] found table [columns [Value Description Reference] count 28]
[INFO] found table [columns [Range Registration Procedures] count 6]
[INFO] found table [columns [Value Description Reference] count 2]
[INFO] found table [columns [ID Name Organization Contact URI Last Updated] count 113]
[INFO] Wrote 86552 bytes to "/Users/jakeireland/projects/defang-uri-schemes/uri_scheme_consts.go"
[INFO] Successfully ran `go fmt` on output file "/Users/jakeireland/projects/defang-uri-schemes/uri_scheme_consts.go"
[INFO] Checking library file meets defang safety requirements
[WARN] Only checking validity of permanent URI schemes
[INFO] Checking that the defang algorithm does not produce any valid schemes
[INFO] Checking that the defang algorithm is (kind of) invertible

$ go run tools/defangdump/defangdump.go
Dumping Python code for defining schemes

URI_SCHEMES = [
     "aaa", "aaas", "about", "acap", "acct", "acd", "acr", "adiumxtra", "adt",
     "afp", "afs", "aim", "amss", "android", "appdata", "apt", "ar", "ark",
     ...,
     "z39.50r", "z39.50s",
]

URI_SCHEMES_DEFANGED_MAP = {
    "aaa": "axa",
    "aaas": "aaxs",
    ...
    "z39.50s": "z39[.]50s",
}

Basic library usage:

package main

import (
	"fmt"

	"github.com/jakewilliami/defang-uri-schemes"
)

type Scheme = defang_uri_schemes.Scheme

var UriSchemeMap = defang_uri_schemes.UriSchemeMap

func main() {
	scheme := UriSchemeMap[".https"]
	defanged := scheme.DefangedUriScheme
	fmt.Printf("%v\n", defanged)  // "hxxps"
}

Types:

type Scheme struct {
	UriScheme           string
	DefangedUriScheme   string
	Template            string
	Description         string
	Status              SchemeStatus
	WellKnownUriSupport string
	Reference           string
	Notes               string
}

const (
	Permanent   SchemeStatus = "Permanent"
	Provisional SchemeStatus = "Provisional"
	Historical  SchemeStatus = "Historical"
)

Documentation

Index

• Variables
• func DefangScheme(scheme string) string
• type Scheme
  • func (s *Scheme) Validate() error
• type SchemeStatus

Variables

var ADDITIONAL_ALLOWED_SCHEME_CHARS = []rune{'-', '+', '.'}

As well as [a-z], these characters are allowed in URI schemes https://github.com/JuliaWeb/URIs.jl/blob/dce395c3/src/URIs.jl#L91-L108 TODO: handle user info and IPv6 hosts

var ADDITIONAL_ALLOWED_SCHEME_CHARS_PATTERN = additionalAllowedSchemeCharsPattern()

var SCHEME_PATTERN = schemePattern()

var UriSchemeMap = map[string]Scheme{}/* 384 elements not displayed */

Functions

func DefangScheme

func DefangScheme(scheme string) string

The goal of defanging is to malform the URI such that it does not open if clicked.

However, as there is a *[re]fang* option in the Tomtils library, we need an algorithm to map invertibly fanged and defanged schemes. Many libraries do not support schemes beyond http[s] [1, 2], as browsers do not support many different schemes. However, it may be the case that different schemes are supported on different non-browser applications, so we *should* support defanging.

There is also consideration to have enough information in a defanged stream such that it is invertible* to its original scheme. Actually, not invertible, as there will not always be enough information just from the defanged scheme to reconstruct the scheme without having the list of valid schemes. So what we need is for the defanged scheme to be one-to-one, so that given a defanged scheme, you know that there is a single valid scheme.

Types

type Scheme

type Scheme struct {
	UriScheme           string
	DefangedUriScheme   string
	Template            string
	Description         string
	Status              SchemeStatus `validate:"oneof=Permanent Provisional Historical"`
	WellKnownUriSupport string
	Reference           string
	Notes               string
}

func (*Scheme) Validate

func (s *Scheme) Validate() error

Validate Scheme struct https://stackoverflow.com/a/71934231

type SchemeStatus

type SchemeStatus string

Status types https://stackoverflow.com/a/71934535

const (
	Permanent   SchemeStatus = "Permanent"
	Provisional SchemeStatus = "Provisional"
	Historical  SchemeStatus = "Historical"
)