utils

Documentation

Index

• Constants
• func DecryptWithAES256(data []byte, key []byte) (error, []byte)
• func EncryptWithAES256RandomKey(data []byte, key []byte) (error, []byte)
• func GetBackend(ctx context.Context, backedClient backend.BackendService, backendName string) (*backend.BackendDetail, error)
• func GetRandomNBitKey(numBits int) ([]byte, error)
• func Md5Content(data []byte) (base64Encoded, hexEncoded string)
• func SetRepresentTenant(ctx context.Context, requestTenant, sourceTenant string) context.Context
• func WrapAlignedEncryptionReader(reader io.Reader, startOffset int64, encryptionKey []byte, ...) (wrappedReader io.Reader, err error)
• func WrapEncryptionReader(reader io.Reader, encryptionKey []byte, initializationVector []byte) (wrappedReader io.Reader, err error)
• type Database
• type ListObjsAppendInfo
• type ObjsCountInfo

Constants

const (
	Tier1   = 1
	Tier99  = 99
	Tier999 = 999
)

Tier1, Tier99 and Tier999 just like the tiers of hot, warm, cold. In the future, we will provide the ability for users to add new storage tiers, if we use 1, 2 and 3, then no space for new storage tiers.

const (
	AWS_STANDARD    = "STANDARD"
	AWS_STANDARD_IA = "STANDARD_IA"
	AWS_GLACIER     = "GLACIER"
)

const (
	GCS_MULTI_REGIONAL = "MULTI_REGIONAL"
	GCS_REGIONAL       = "REGIONAL"
	GCS_NEARLINE       = "NEARLINE"
	GCS_COLDLINE       = "COLDLINE"
)

const (
	ALIBABA_STANDARD = "Standard"
	ALIBABA_IA       = "IA"
	ALIBABA_ARCHIVE  = "Archive"
)

const (
	OSTYPE_OPENSDS       = "OpenSDS"
	OSTYPE_AWS           = "aws-s3"
	OSTYPE_Azure         = "azure-blob"
	OSTYPE_OBS           = "hw-obs"
	OSTYPE_GCS           = "gcp-s3"
	OSTYPE_CEPH          = "ceph-s3"
	OSTYPE_FUSIONSTORAGE = "fusionstorage-object"
	OSTYPE_ALIBABA       = "alibaba-oss"
)

Object Storage Type

const (
	DBKEY_DELETEMARKER   = "isdeletemarker"
	DBKEY_INITFLAG       = "initflag"
	DBKEY_OBJECTKEY      = "objectkey"
	DBKEY_UPLOADID       = "uploadid"
	DBKEY_LASTMODIFIED   = "lastmodified"
	DBKEY_SUPPOSEDSTATUS = "supposedstatus"
	DBKEY_LOCKOBJ_OBJKEY = "objkey"
	DBKEY_BUCKET         = "bucket"
	DBKEY_INITTIME       = "inittime"
	DBKEY_NAME           = "name"
	DBKEY_LIFECYCLE      = "lifecycleconfiguration"
	DBKEY_ID             = "id"
)

const (
	MaxObjectList  = 1000 // Limit number of objects in a listObjectsResponse.
	MaxUploadsList = 1000 // Limit number of uploads in a listUploadsResponse.
	MaxPartsList   = 1000 // Limit number of parts in a listPartsResponse.
)

const (
	VersioningEnabled   = "Enabled"
	VersioningDisabled  = "Disabled"
	VersioningSuspended = "Suspended"
)

const (
	MoveType_Invalid = iota
	MoveType_MoveCrossBuckets
	MoveType_ChangeLocation
	MoveType_ChangeStorageTier
)

const (
	REQUEST_HEADER_SSE_KEY          = "x-amz-server-side-encryption"
	REQUEST_HEADER_SSE_VALUE_AES256 = "AES256"
)

const (
	CEPH_STANDARD = "STDANDARD"
)

const (
	RequestType_Lifecycle = "lifecycle"
)

Functions

func DecryptWithAES256

func DecryptWithAES256(data []byte, key []byte) (error, []byte)

func EncryptWithAES256RandomKey

func EncryptWithAES256RandomKey(data []byte, key []byte) (error, []byte)

func GetBackend

func GetBackend(ctx context.Context, backedClient backend.BackendService, backendName string) (*backend.BackendDetail,
	error)

func GetRandomNBitKey

func GetRandomNBitKey(numBits int) ([]byte, error)

func Md5Content

func Md5Content(data []byte) (base64Encoded, hexEncoded string)

func SetRepresentTenant

func SetRepresentTenant(ctx context.Context, requestTenant, sourceTenant string) context.Context

func WrapAlignedEncryptionReader

func WrapAlignedEncryptionReader(reader io.Reader, startOffset int64, encryptionKey []byte,
	initializationVector []byte) (wrappedReader io.Reader, err error)

AES is a block cipher with block size of 16 bytes, i.e. the basic unit of encryption/decryption is 16 bytes. As an HTTP range request could start from any byte, we need to read one more block if necessary. Also, our chosen mode of operation for YIG is CTR(counter), which features parallel encryption/decryption and random read access. We need all these three features, this leaves us only three choices: ECB, CTR, and GCM. ECB is best known for its insecurity, meanwhile the GCM implementation of golang(as in 1.7) discourage users to encrypt large files in one pass, which requires us to read the whole file into memory. So the implement complexity is similar between GCM and CTR, we choose CTR because it's faster(but more prone to man-in-the-middle modifications)

See https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation and http://stackoverflow.com/questions/39347206

func WrapEncryptionReader

func WrapEncryptionReader(reader io.Reader, encryptionKey []byte,
	initializationVector []byte) (wrappedReader io.Reader, err error)

Wraps reader with encryption if encryptionKey is not empty

Types

type Database

type Database struct {
	Credential string `conf:"credential,username:password@tcp(ip:port)/dbname"`
	Driver     string `conf:"driver,mongodb"`
	Endpoint   string `conf:"endpoint,localhost:27017"`
}

type ListObjsAppendInfo

type ListObjsAppendInfo struct {
	Prefixes   []string
	Truncated  bool
	NextMarker string
}

type ObjsCountInfo

type ObjsCountInfo struct {
	Size  int64
	Count int64
}