google

Documentation

Overview

Package google provides a Shade storage implementation for Google Drive.

You may optionally configure a FileParentID and ChunkParentID to indicate where to store the files and chunks. These values are Drive's alphanumeric unique identifiers for directories. You can find the ID for a directory in the URL when viewing the file in the Google Drive web UI. These can be set to the same value, and AppProperties will be used to disambiguate files from chunks

To store Files and Chunks as AppData storage, so that they are not visible in the Google Drive web UI, set FileParentID and ChunkParentID to 'appDataFolder'. You can optionally reduce the scope to only 'https://www.googleapis.com/auth/drive.appfolder'.

The following configuration values are not directly supported:

MaxFiles
MaxChunkBytes
RsaPublicKey
RsaPrivateKey
Children

To encrypt the contents written to Google Drive, wrap the configuration stanza with the github.com/asjoyner/shade/drive/encrypt package.

This package supports overriding all of the OAuth configuration parameters.

Index

• func GetOAuthClient(c drive.Config) *http.Client
• func NewClient(c drive.Config) (drive.Client, error)
• type ChunkLister
  • func (c *ChunkLister) Err() error
  • func (c *ChunkLister) Next() bool
  • func (c *ChunkLister) Sha256() []byte
• type Drive
  • func (s *Drive) GetChunk(sha256sum []byte, file *shade.File) ([]byte, error)
  • func (s *Drive) GetConfig() drive.Config
  • func (s *Drive) GetFile(sha256sum []byte) ([]byte, error)
  • func (s *Drive) ListFiles() ([][]byte, error)
  • func (s *Drive) Local() bool
  • func (s *Drive) NewChunkLister() drive.ChunkLister
  • func (s *Drive) Persistent() bool
  • func (s *Drive) PutChunk(sha256sum, content []byte, f *shade.File) error
  • func (s *Drive) PutFile(sha256sum, content []byte) error
  • func (s *Drive) ReleaseChunk(sha256sum []byte) error
  • func (s *Drive) ReleaseFile(sha256sum []byte) error
  • func (s *Drive) Warm(chunks [][]byte, f *shade.File)

Functions

func GetOAuthClient

func GetOAuthClient(c drive.Config) *http.Client

func NewClient

func NewClient(c drive.Config) (drive.Client, error)

NewClient returns a new Drive client.

Types

type ChunkLister

type ChunkLister struct {
	// contains filtered or unexported fields
}

ChunkLister allows iterating the chunks in Google Drive.

func (*ChunkLister) Err

func (c *ChunkLister) Err() error

Err returns the error encountered, if any.

func (*ChunkLister) Next

func (c *ChunkLister) Next() bool

Next increments the pointer

func (*ChunkLister) Sha256

func (c *ChunkLister) Sha256() []byte

Sha256 returns the chunk pointed to by the pointer.

type Drive

type Drive struct {
	// contains filtered or unexported fields
}

Drive represents access to the Google Drive storage system.

func (*Drive) GetChunk

func (s *Drive) GetChunk(sha256sum []byte, file *shade.File) ([]byte, error)

GetChunk retrieves a chunk with a given SHA-256 sum.

func (*Drive) GetConfig

func (s *Drive) GetConfig() drive.Config

GetConfig returns the associated drive.Config object.

func (*Drive) GetFile

func (s *Drive) GetFile(sha256sum []byte) ([]byte, error)

GetFile retrieves a chunk with a given SHA-256 sum.

func (*Drive) ListFiles

func (s *Drive) ListFiles() ([][]byte, error)

ListFiles retrieves all of the File objects known to the client, and returns the corresponding sha256sum of the file object. Those may be passed to GetChunk() to retrieve the corresponding shade.File.

func (*Drive) Local

func (s *Drive) Local() bool

Local returns whether access is local.

func (*Drive) NewChunkLister

func (s *Drive) NewChunkLister() drive.ChunkLister

NewChunkLister returns an iterator which returns all chunks in Google Drive.

func (*Drive) Persistent

func (s *Drive) Persistent() bool

Persistent returns whether the storage is persistent across task restarts.

func (*Drive) PutChunk

func (s *Drive) PutChunk(sha256sum, content []byte, f *shade.File) error

PutChunk writes a chunk and returns its SHA-256 sum

func (*Drive) PutFile

func (s *Drive) PutFile(sha256sum, content []byte) error

PutFile writes the metadata describing a new file. content should be marshalled JSON, and may be encrypted.

func (*Drive) ReleaseChunk

func (s *Drive) ReleaseChunk(sha256sum []byte) error

ReleaseChunk removes a chunk file from Google Drive.

func (*Drive) ReleaseFile

func (s *Drive) ReleaseFile(sha256sum []byte) error

ReleaseFile removes a file from Google Drive.

func (*Drive) Warm

func (s *Drive) Warm(chunks [][]byte, f *shade.File)

Warm caches the Google file objects for the supplied chunks. This saves latency by batching the request, and avoiding the need to fetch them sequentially while streaming.