storage

README

storage

An application-oriented unified storage layer for Golang.

Goal

• Production ready
• High performance
• Vendor agnostic

Features

Widely services support

• azblob: Azure Blob storage
• cos: Tencent Cloud Object Storage
• dropbox: Dropbox
• fs: Local file system
• gcs: Google Cloud Storage
• kodo: qiniu kodo
• oss: Aliyun Object Storage
• qingstor: QingStor Object Storage
• s3: Amazon S3
• uss: UPYUN Storage Service

Servicer operation support

• List: list all Storager in service
• Get: get a Storager via name
• Create: create a Storager
• Delete: delete a Storager

Storager operation support

Basic operations

• Metadata: get storager's metadata
• Read: read file content
• Write: write content into file
• Stat: get file's metadata
• Delete: delete a file or directory

Extended operations

• Copy: copy a file inside storager
• Move: move a file inside storager
• Reach: generate a public accessible url

Multiple list style support

• ListDir: list files and directories under a directory
• ListPrefix: list files under a prefix

Segment/Multipart support

• ListPrefixSegment: list segments under a prefix
• InitIndexSegment: initiate an index type segment
• WriteIndexSegment: write content into an index type segment
• CompleteSegment: complete a segment to create a file
• AbortSegment: abort a segment

Object metadata support

Common metadata

• id: unique key in service
• name: relative path towards service's work dir
• type: object type cloud be file, dir, link or unknown

Optional metadata

• size: object's content size.
• updated-at: object's last updated time.
• content-md5: md5 digest as defined in rfc2616
• content-type: media type as defined in rfc2616
• etag: entity tag as defined in rfc2616
• storage-class: object's storage class as defined in storage proposal

Quick Start

import (
    "log"

    "github.com/aos-dev/go-storage/v2"
    "github.com/aos-dev/go-storage/v2/pairs"
    "github.com/aos-dev/go-services-fs"
)

// Init a service.
store, err := fs.NewStorager(pairs.WithWorkDir("/tmp"))
if err != nil {
    log.Fatalf("service init failed: %v", err)
}

// Use Storager API to maintain data.
var buf bytes.Buffer

n, err := store.Read("path/to/file", &buf)
if err != nil {
    log.Printf("storager read: %v", err)
}

Sponsor

Documentation

Overview

Package storage intends to provide a unified storage layer for Golang.

Goals

- Production ready: high test coverage, enterprise storage software adaptation, semantic versioning, well documented.

- High performance: more code generation, less runtime reflect.

- Vendor agnostic: more generic abstraction, less internal details.

Examples

The most common case to use a Storager service could be following:

1. Init a storager.

    store, err := fs.NewStorager(pairs.WithWorkDir("/tmp"))
	if err != nil {
		log.Fatalf("service init failed: %v", err)
	}

2. Use Storager API to maintain data.

var buf bytes.Buffer

n, err := store.Read("path/to/file", &buf)
if err != nil {
	log.Printf("storager read: %v", err)
}