gostorages

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 8, 2019 License: MIT Imports: 16 Imported by: 0

README 

gostorages
==========

A unified interface to manipulate storage engine for Go.

gostorages is used in `picfit <https://github.com/thoas/picfit>`_ to allow us
switching over storage engine.

Currently, it supports the following storages:

* Amazon S3
* File system

Installation
============

Just run:

::

    $ go get github.com/ulule/gostorages

Usage
=====

It offers you a single API to manipulate your files on multiple storages.

If you are migrating from a File system storage to an Amazon S3, you don't need
to migrate all your methods anymore!

Be lazy again!

File system
-----------

To use the ``FileSystemStorage`` you must have a location to save your files.

.. code-block:: go

    package main

    import (
        "fmt"
        "github.com/ulule/gostorages"
        "os"
    )

    func main() {
        tmp := os.TempDir()

        storage := gostorages.NewFileSystemStorage(tmp, "http://img.example.com")

        // Saving a file named test
        storage.Save("test", gostorages.NewContentFile([]byte("(╯°□°）╯︵ ┻━┻")))

        fmt.Println(storage.URL("test")) // => http://img.example.com/test

        // Deleting the new file on the storage
        storage.Delete("test")
    }


Amazon S3
---------

To use the ``S3Storage`` you must have:

* An access key id
* A secret access key
* A bucket name
* Give the region of your bucket
* Give the ACL you want to use

You can find your credentials in `Security credentials <https://console.aws.amazon.com/iam/home?nc2=h_m_sc#security_credential>`_.

In the following example, I'm assuming my bucket is located in european region.

.. code-block:: go

    package main

    import (
        "fmt"
        "github.com/ulule/gostorages"
        "github.com/mitchellh/goamz/aws"
        "github.com/mitchellh/goamz/s3"
        "os"
    )

    func main() {
        baseURL := "http://s3-eu-west-1.amazonaws.com/my-bucket"

        storage := gostorages.NewS3Storage(os.Getenv("ACCESS_KEY_ID"), os.Getenv("SECRET_ACCESS_KEY"), "my-bucket", "", aws.Regions["eu-west-1"], s3.PublicReadWrite, baseURL)

        // Saving a file named test
        storage.Save("test", gostorages.NewContentFile([]byte("(>_<)")))

        fmt.Println(storage.URL("test")) // => http://s3-eu-west-1.amazonaws.com/my-bucket/test

        // Deleting the new file on the storage
        storage.Delete("test")
    }

Roadmap
=======

see `issues <https://github.com/ulule/gostorages/issues>`_

Don't hesitate to send patch or improvements.

Documentation

Index

Constants

View Source 
const LastModifiedFormat = time.RFC1123

Variables

View Source 
var ACLs = map[string]s3.ACL{
	"private":                   s3.Private,
	"public-read":               s3.PublicRead,
	"public-read-write":         s3.PublicReadWrite,
	"authenticated-read":        s3.AuthenticatedRead,
	"bucket-owner-read":         s3.BucketOwnerRead,
	"bucket-owner-full-control": s3.BucketOwnerFull,
}
View Source 
var DefaultFilePermissions os.FileMode = 0755

Functions

This section is empty.

Types

type BaseStorage 

type BaseStorage struct {
	BaseURL  string
	Location string
}

func NewBaseStorage 

func NewBaseStorage(location string, baseURL string) *BaseStorage

func (*BaseStorage) HasBaseURL 

func (s *BaseStorage) HasBaseURL() bool

func (*BaseStorage) Path 

func (s *BaseStorage) Path(filepath string) string

Path joins the given file to the storage path

func (*BaseStorage) URL 

func (s *BaseStorage) URL(filename string) string

type ContentFile 

type ContentFile struct {
	*bytes.Reader
}

func NewContentFile 

func NewContentFile(content []byte) *ContentFile

func (*ContentFile) Close 

func (f *ContentFile) Close() error

func (*ContentFile) ReadAll 

func (f *ContentFile) ReadAll() ([]byte, error)

func (*ContentFile) Size 

func (f *ContentFile) Size() int64

type File 

type File interface {
	Size() int64
	Read(b []byte) (int, error)
	ReadAll() ([]byte, error)
	Close() error
}

type FileSystemFile 

type FileSystemFile struct {
	*os.File
	Storage  Storage
	FileInfo os.FileInfo
}

func NewFileSystemFile 

func NewFileSystemFile(storage Storage, file *os.File) (*FileSystemFile, error)

func (*FileSystemFile) ReadAll 

func (f *FileSystemFile) ReadAll() ([]byte, error)

func (*FileSystemFile) Size 

func (f *FileSystemFile) Size() int64

type FileSystemStorage 

type FileSystemStorage struct {
	*BaseStorage
}

Storage is a file system storage handler

func (*FileSystemStorage) AccessedTime 

func (s *FileSystemStorage) AccessedTime(filepath string) (time.Time, error)

func (*FileSystemStorage) CreatedTime 

func (s *FileSystemStorage) CreatedTime(filepath string) (time.Time, error)

CreatedTime returns the last access time.

func (*FileSystemStorage) Delete 

func (s *FileSystemStorage) Delete(filepath string) error

Delete the file from storage

func (*FileSystemStorage) Exists 

func (s *FileSystemStorage) Exists(filepath string) bool

Exists checks if the given file is in the storage

func (*FileSystemStorage) IsNotExist 

func (s *FileSystemStorage) IsNotExist(err error) bool

IsNotExist returns a boolean indicating whether the error is known to report that a file or directory does not exist.

func (*FileSystemStorage) ModifiedTime 

func (s *FileSystemStorage) ModifiedTime(filepath string) (time.Time, error)

ModifiedTime returns the last update time

func (*FileSystemStorage) Open 

func (s *FileSystemStorage) Open(filepath string) (File, error)

Open returns the file content

func (*FileSystemStorage) Save 

func (s *FileSystemStorage) Save(filepath string, file File) error

Save saves a file at the given path

func (*FileSystemStorage) SaveWithPermissions 

func (s *FileSystemStorage) SaveWithPermissions(filepath string, file File, perm os.FileMode) error

SaveWithPermissions saves a file with the given permissions to the storage

func (*FileSystemStorage) Size 

func (s *FileSystemStorage) Size(filepath string) int64

Size returns the size of the given file

func (*FileSystemStorage) URL 

func (s *FileSystemStorage) URL(filename string) string

type GCSStorage 

type GCSStorage struct {
	*BaseStorage
	// contains filtered or unexported fields
}

func (*GCSStorage) Delete 

func (gcs *GCSStorage) Delete(filepath string) error

Delete the file from the bucket

func (*GCSStorage) Exists 

func (gcs *GCSStorage) Exists(filepath string) bool

Exists checks if the given file is in the bucket

func (*GCSStorage) IsNotExist 

func (gcs *GCSStorage) IsNotExist(err error) bool

IsNotExist returns a boolean indicating whether the error is known to report that a file or directory does not exist.

func (*GCSStorage) ModifiedTime 

func (gcs *GCSStorage) ModifiedTime(filepath string) (time.Time, error)

ModifiedTime returns the last update time

func (*GCSStorage) Open 

func (gcs *GCSStorage) Open(filepath string) (File, error)

Open returns the file content in a dedicated bucket

func (*GCSStorage) Save 

func (gcs *GCSStorage) Save(path string, file File) error

Save saves a file at the given path in the bucket

func (*GCSStorage) SaveWithContentType 

func (gcs *GCSStorage) SaveWithContentType(filepath string, file File, contentType string) error

Save saves a file at the given path in the bucket

func (*GCSStorage) Size 

func (gcs *GCSStorage) Size(filepath string) int64

Size returns the size of the given file

type GCSStorageFile 

type GCSStorageFile struct {
	*storage.Reader
}

func (*GCSStorageFile) ReadAll 

func (f *GCSStorageFile) ReadAll() ([]byte, error)

type S3Storage 

type S3Storage struct {
	*BaseStorage
	AccessKeyId     string
	SecretAccessKey string
	BucketName      string
	Region          aws.Region
	ACL             s3.ACL
}

func (*S3Storage) Auth 

func (s *S3Storage) Auth() (auth aws.Auth, err error)

Auth returns a Auth instance

func (*S3Storage) Bucket 

func (s *S3Storage) Bucket() (*s3.Bucket, error)

Bucket returns a bucket instance

func (*S3Storage) Client 

func (s *S3Storage) Client() (*s3.S3, error)

Client returns a S3 instance

func (*S3Storage) Delete 

func (s *S3Storage) Delete(filepath string) error

Delete the file from the bucket

func (*S3Storage) Exists 

func (s *S3Storage) Exists(filepath string) bool

Exists checks if the given file is in the bucket

func (*S3Storage) IsNotExist 

func (s *S3Storage) IsNotExist(err error) bool

IsNotExist returns a boolean indicating whether the error is known to report that a file or directory does not exist.

func (*S3Storage) Key 

func (s *S3Storage) Key(filepath string) (*s3.Key, error)

func (*S3Storage) ModifiedTime 

func (s *S3Storage) ModifiedTime(filepath string) (time.Time, error)

ModifiedTime returns the last update time

func (*S3Storage) Open 

func (s *S3Storage) Open(filepath string) (File, error)

Open returns the file content in a dedicated bucket

func (*S3Storage) Save 

func (s *S3Storage) Save(path string, file File) error

Save saves a file at the given path in the bucket

func (*S3Storage) SaveWithContentType 

func (s *S3Storage) SaveWithContentType(filepath string, file File, contentType string) error

Save saves a file at the given path in the bucket

func (*S3Storage) Size 

func (s *S3Storage) Size(filepath string) int64

Size returns the size of the given file

type S3StorageFile 

type S3StorageFile struct {
	io.ReadCloser
	Key     *s3.Key
	Storage Storage
}

func (*S3StorageFile) ReadAll 

func (f *S3StorageFile) ReadAll() ([]byte, error)

func (*S3StorageFile) Size 

func (f *S3StorageFile) Size() int64

type Storage 

type Storage interface {
	Save(filepath string, file File) error
	Path(filepath string) string
	Exists(filepath string) bool
	Delete(filepath string) error
	Open(filepath string) (File, error)
	ModifiedTime(filepath string) (time.Time, error)
	Size(filepath string) int64
	URL(filename string) string
	HasBaseURL() bool
	IsNotExist(err error) bool
}

func NewFileSystemStorage 

func NewFileSystemStorage(location string, baseURL string) Storage

NewStorage returns a file system storage engine

func NewGCSStorage 

func NewGCSStorage(credentialsFile, bucket, location, baseURL, cacheControl string) (Storage, error)

func NewS3Storage 

func NewS3Storage(accessKeyId string, secretAccessKey string, bucketName string, location string, region aws.Region, acl s3.ACL, baseURL string) Storage

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.