encoder

Documentation

Overview

Package encoder provides functionality to translate file names for usage on restrictive storage systems.

The restricted set of characters are mapped to a unicode equivalent version (most to their FULLWIDTH variant) to increase compatibility with other storage systems. See: http://unicode-search.net/unicode-namesearch.pl?term=FULLWIDTH

Encoders will also quote reserved characters to differentiate between the raw and encoded forms.

Index

• Constants
• func FromStandardName(e Encoder, s string) string
• func FromStandardPath(e Encoder, s string) string
• func ToStandardName(e Encoder, s string) string
• func ToStandardPath(e Encoder, s string) string
• type Encoder
  • func Identity() Encoder
• type MultiEncoder
  • func (mask MultiEncoder) Decode(in string) string
  • func (mask MultiEncoder) Encode(in string) string
  • func (mask MultiEncoder) FromStandardName(s string) string
  • func (mask MultiEncoder) FromStandardPath(s string) string
  • func (mask MultiEncoder) Has(flag MultiEncoder) bool
  • func (mask *MultiEncoder) Scan(s fmt.ScanState, ch rune) error
  • func (mask *MultiEncoder) Set(in string) error
  • func (mask MultiEncoder) String() string
  • func (mask MultiEncoder) ToStandardName(s string) string
  • func (mask MultiEncoder) ToStandardPath(s string) string
  • func (mask MultiEncoder) Type() string

Constants

const Base = (EncodeZero |
	EncodeSlash |
	EncodeDot)

Base only encodes the zero byte and slash

const Display = Standard

Display is the internal encoding for logging and output

const OS = Base

OS is the encoding used by the local backend for non windows platforms

const (

	// QuoteRune is the rune used for quoting reserved characters
	QuoteRune = '‛' // SINGLE HIGH-REVERSED-9 QUOTATION MARK
)

const Standard = (EncodeZero |
	EncodeSlash |
	EncodeCtl |
	EncodeDel |
	EncodeDot)

Standard defines the encoding that is used for paths in- and output by rclone.

List of replaced characters:

  (0x00)  -> '␀' // SYMBOL FOR NULL
/ (slash) -> '／' // FULLWIDTH SOLIDUS

Functions

func FromStandardName

func FromStandardName(e Encoder, s string) string

FromStandardName takes name in Standard encoding and converts it in the given encoding.

func FromStandardPath

func FromStandardPath(e Encoder, s string) string

FromStandardPath takes a / separated path in Standard encoding and converts it to a / separated path in the given encoding.

func ToStandardName

func ToStandardName(e Encoder, s string) string

ToStandardName takes name in the given encoding and converts it in Standard encoding.

func ToStandardPath

func ToStandardPath(e Encoder, s string) string

ToStandardPath takes a / separated path in the given encoding and converts it to a / separated path in Standard encoding.

Types

type Encoder

type Encoder interface {
	// Encode takes a raw name and substitutes any reserved characters and
	// patterns in it
	Encode(string) string
	// Decode takes a name and undoes any substitutions made by Encode
	Decode(string) string

	// FromStandardPath takes a / separated path in Standard encoding
	// and converts it to a / separated path in this encoding.
	FromStandardPath(string) string
	// FromStandardName takes name in Standard encoding and converts
	// it in this encoding.
	FromStandardName(string) string
	// ToStandardPath takes a / separated path in this encoding
	// and converts it to a / separated path in Standard encoding.
	ToStandardPath(string) string
	// ToStandardName takes name in this encoding and converts
	// it in Standard encoding.
	ToStandardName(string) string
}

Encoder can transform names to and from the original and translated version.

func Identity

func Identity() Encoder

Identity returns an Encoder that always returns the input value

type MultiEncoder

type MultiEncoder uint

MultiEncoder is a configurable Encoder. The Encode* constants in this package can be combined using bitwise or (|) to enable handling of multiple character classes

const (
	EncodeZero          MultiEncoder = 0 // NUL(0x00)
	EncodeRaw           MultiEncoder = 1 << (iota - 1)
	EncodeSlash                      // /
	EncodeLtGt                       // <>
	EncodeDoubleQuote                // "
	EncodeSingleQuote                // '
	EncodeBackQuote                  // `
	EncodeDollar                     // $
	EncodeColon                      // :
	EncodeQuestion                   // ?
	EncodeAsterisk                   // *
	EncodePipe                       // |
	EncodeHash                       // #
	EncodePercent                    // %
	EncodeBackSlash                  // \
	EncodeCrLf                       // CR(0x0D), LF(0x0A)
	EncodeDel                        // DEL(0x7F)
	EncodeCtl                        // CTRL(0x01-0x1F)
	EncodeLeftSpace                  // Leading SPACE
	EncodeLeftPeriod                 // Leading .
	EncodeLeftTilde                  // Leading ~
	EncodeLeftCrLfHtVt               // Leading CR LF HT VT
	EncodeRightSpace                 // Trailing SPACE
	EncodeRightPeriod                // Trailing .
	EncodeRightCrLfHtVt              // Trailing CR LF HT VT
	EncodeInvalidUtf8                // Invalid UTF-8 bytes
	EncodeDot                        // . and .. names
	EncodeSquareBracket              // []
	EncodeSemicolon                  // ;
	EncodeExclamation                // !

	// Synthetic
	EncodeWin         = EncodeColon | EncodeQuestion | EncodeDoubleQuote | EncodeAsterisk | EncodeLtGt | EncodePipe // :?"*<>|
	EncodeHashPercent = EncodeHash | EncodePercent                                                                  // #%
)

Possible flags for the MultiEncoder

func (MultiEncoder) Decode

func (mask MultiEncoder) Decode(in string) string

Decode takes a name and undoes any substitutions made by Encode

func (MultiEncoder) Encode

func (mask MultiEncoder) Encode(in string) string

Encode takes a raw name and substitutes any reserved characters and patterns in it

func (MultiEncoder) FromStandardName

func (mask MultiEncoder) FromStandardName(s string) string

FromStandardName takes name in Standard encoding and converts it in this encoding.

func (MultiEncoder) FromStandardPath

func (mask MultiEncoder) FromStandardPath(s string) string

FromStandardPath takes a / separated path in Standard encoding and converts it to a / separated path in this encoding.

func (MultiEncoder) Has

func (mask MultiEncoder) Has(flag MultiEncoder) bool

Has returns true if flag is contained in mask

func (*MultiEncoder) Scan

func (mask *MultiEncoder) Scan(s fmt.ScanState, ch rune) error

Scan implements the fmt.Scanner interface

func (*MultiEncoder) Set

func (mask *MultiEncoder) Set(in string) error

Set converts a string into a MultiEncoder

func (MultiEncoder) String

func (mask MultiEncoder) String() string

String converts the MultiEncoder into text

func (MultiEncoder) ToStandardName

func (mask MultiEncoder) ToStandardName(s string) string

ToStandardName takes name in this encoding and converts it in Standard encoding.

func (MultiEncoder) ToStandardPath

func (mask MultiEncoder) ToStandardPath(s string) string

ToStandardPath takes a / separated path in this encoding and converts it to a / separated path in Standard encoding.

func (MultiEncoder) Type

func (mask MultiEncoder) Type() string

Type returns a textual type of the MultiEncoder to satisfy the pflag.Value interface