fakestorage

Documentation

Overview

Package fakestorage provides the server that can be used as a target on GCS-dependent tests.

The server provides a method that returns an instance of the storage client that can be used in tests.

Index

• type CreateBucketOpts
• type ListOptions
• type Object
  • func (o Object) StreamingObject() StreamingObject
• type ObjectAttrs
  • func (o ObjectAttrs) MarshalJSON() ([]byte, error)
  • func (o *ObjectAttrs) UnmarshalJSON(data []byte) error
• type Options
• type Server
  • func NewServer(objects []Object) *Server
  • func NewServerWithHostPort(objects []Object, host string, port uint16) (*Server, error)
  • func NewServerWithOptions(options Options) (*Server, error)
  • func (s *Server) Client() *storage.Client
  • func (s *Server) CreateBucket(name string)
  • func (s *Server) CreateBucketWithOpts(opts CreateBucketOpts)
  • func (s *Server) CreateObject(obj Object)
  • func (s *Server) CreateObjectStreaming(obj StreamingObject) error
  • func (s *Server) GetObject(bucketName, objectName string) (Object, error)
  • func (s *Server) GetObjectStreaming(bucketName, objectName string) (StreamingObject, error)
  • func (s *Server) GetObjectWithGeneration(bucketName, objectName string, generation int64) (Object, error)
  • func (s *Server) GetObjectWithGenerationStreaming(bucketName, objectName string, generation int64) (StreamingObject, error)
  • func (s *Server) HTTPClient() *http.Client
  • func (s *Server) ListObjects(bucketName, prefix, delimiter string, versions bool) ([]ObjectAttrs, []string, error)
  • func (s *Server) ListObjectsWithOptions(bucketName string, options ListOptions) ([]ObjectAttrs, []string, error)
  • func (s *Server) PublicURL() string
  • func (s *Server) Stop()
  • func (s *Server) URL() string
• type StreamingObject
  • func (o *StreamingObject) BufferedObject() (Object, error)
  • func (o *StreamingObject) Close() error

Examples

• Server (With_host_port)
• Server.Client

Types

type CreateBucketOpts

type CreateBucketOpts struct {
	Name              string
	VersioningEnabled bool
}

CreateBucketOpts defines the properties of a bucket you can create with CreateBucketWithOpts.

type ListOptions

type ListOptions struct {
	Prefix                   string
	Delimiter                string
	Versions                 bool
	StartOffset              string
	EndOffset                string
	IncludeTrailingDelimiter bool
}

type Object

type Object struct {
	ObjectAttrs
	Content []byte `json:"-"`
}

Object represents an object that is stored within the fake server. The content of this type is stored is buffered, i.e. it's stored in memory. Use StreamingObject to stream the content from a reader, e.g a file.

TODO: if object content and attributes were passed around separately, several functions could be simplified and separate Object and Streaming object types wouldn't be needed. Many functions that return objects only care about metadata, and some functions that take objects don't care about content. The Object type would be retained for backward compatibility of NewServer, NewServerWithHostPort and NewServerWithOptions, but internally the type wouldn't be used. This would also enable the content type to be specific to the needs of each function: io.Reader vs. io.ReadSeekCloser.

func (Object) StreamingObject

func (o Object) StreamingObject() StreamingObject

type ObjectAttrs

type ObjectAttrs struct {
	BucketName      string
	Name            string
	Size            int64
	ContentType     string
	ContentEncoding string
	// Crc32c checksum of Content. calculated by server when it's upload methods are used.
	Crc32c  string
	Md5Hash string
	Etag    string
	ACL     []storage.ACLRule
	// Dates and generation can be manually injected, so you can do assertions on them,
	// or let us fill these fields for you
	Created    time.Time
	Updated    time.Time
	Deleted    time.Time
	Generation int64
	Metadata   map[string]string
}

ObjectAttrs returns only the meta-data about an object without its contents.

func (ObjectAttrs) MarshalJSON

func (o ObjectAttrs) MarshalJSON() ([]byte, error)

MarshalJSON for ObjectAttrs to use ACLRule instead of storage.ACLRule

func (*ObjectAttrs) UnmarshalJSON

func (o *ObjectAttrs) UnmarshalJSON(data []byte) error

UnmarshalJSON for ObjectAttrs to use ACLRule instead of storage.ACLRule

type Options

type Options struct {
	InitialObjects []Object
	StorageRoot    string
	Scheme         string
	Host           string
	Port           uint16

	// when set to true, the server will not actually start a TCP listener,
	// client requests will get processed by an internal mocked transport.
	NoListener bool

	// Optional external URL, such as https://gcs.127.0.0.1.nip.io:4443
	// Returned in the Location header for resumable uploads
	// The "real" value is https://www.googleapis.com, the JSON API
	// The default is whatever the server is bound to, such as https://0.0.0.0:4443
	ExternalURL string

	// Optional URL for public access
	// An example is "storage.gcs.127.0.0.1.nip.io:4443", which will configure
	// the server to serve objects at:
	// https://storage.gcs.127.0.0.1.nip.io:4443/<bucket>/<object>
	// https://<bucket>.storage.gcs.127.0.0.1.nip.io:4443>/<object>
	// If unset, the default is "storage.googleapis.com", the XML API
	PublicHost string

	// Optional list of headers to add to the CORS header allowlist
	// An example is "X-Goog-Meta-Uploader", which will allow a
	// custom metadata header named "X-Goog-Meta-Uploader" to be
	// sent through the browser
	AllowedCORSHeaders []string

	// Destination for writing log.
	Writer io.Writer

	// EventOptions contains the events that should be published and the URL
	// of the Google cloud function such events should be published to.
	EventOptions notification.EventManagerOptions

	// Location used for buckets in the server.
	BucketsLocation string

	CertificateLocation string

	PrivateKeyLocation string
}

Options are used to configure the server on creation.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is the fake server.

It provides a fake implementation of the Google Cloud Storage API.

Example (With_host_port)

package main

import (
	"context"
	"fmt"
	"io"

	"github.com/fsouza/fake-gcs-server/fakestorage"
)

func main() {
	server, err := fakestorage.NewServerWithOptions(fakestorage.Options{
		InitialObjects: []fakestorage.Object{
			{
				ObjectAttrs: fakestorage.ObjectAttrs{
					BucketName: "some-bucket",
					Name:       "some/object/file.txt",
				},
				Content: []byte("inside the file"),
			},
		},
		Host: "127.0.0.1",
		Port: 8081,
	})
	if err != nil {
		panic(err)
	}
	defer server.Stop()
	client := server.Client()
	object := client.Bucket("some-bucket").Object("some/object/file.txt")
	reader, err := object.NewReader(context.Background())
	if err != nil {
		panic(err)
	}
	defer reader.Close()
	data, err := io.ReadAll(reader)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s", data)
}

Output:

inside the file

func NewServer

func NewServer(objects []Object) *Server

NewServer creates a new instance of the server, pre-loaded with the given objects.

func NewServerWithHostPort

func NewServerWithHostPort(objects []Object, host string, port uint16) (*Server, error)

NewServerWithHostPort creates a new server that listens on a custom host and port

Deprecated: use NewServerWithOptions.

func NewServerWithOptions

func NewServerWithOptions(options Options) (*Server, error)

NewServerWithOptions creates a new server configured according to the provided options.

func (*Server) Client

func (s *Server) Client() *storage.Client

Client returns a GCS client configured to talk to the server.

Example

package main

import (
	"context"
	"fmt"
	"io"

	"github.com/fsouza/fake-gcs-server/fakestorage"
)

func main() {
	server := fakestorage.NewServer([]fakestorage.Object{
		{
			ObjectAttrs: fakestorage.ObjectAttrs{
				BucketName: "some-bucket",
				Name:       "some/object/file.txt",
			},
			Content: []byte("inside the file"),
		},
	})
	defer server.Stop()
	client := server.Client()
	object := client.Bucket("some-bucket").Object("some/object/file.txt")
	reader, err := object.NewReader(context.Background())
	if err != nil {
		panic(err)
	}
	defer reader.Close()
	data, err := io.ReadAll(reader)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s", data)
}

Output:

inside the file

func (*Server) CreateBucket

func (s *Server) CreateBucket(name string)

CreateBucket creates a bucket inside the server, so any API calls that require the bucket name will recognize this bucket.

If the bucket already exists, this method does nothing.

Deprecated: use CreateBucketWithOpts.

func (*Server) CreateBucketWithOpts

func (s *Server) CreateBucketWithOpts(opts CreateBucketOpts)

CreateBucketWithOpts creates a bucket inside the server, so any API calls that require the bucket name will recognize this bucket. Use CreateBucketOpts to customize the options for this bucket

If the underlying backend returns an error, this method panics.

func (*Server) CreateObject

func (s *Server) CreateObject(obj Object)

CreateObject is the non-streaming version of CreateObjectStreaming.

In addition to streaming, CreateObjectStreaming returns an error instead of panicking when an error occurs.

func (*Server) CreateObjectStreaming

func (s *Server) CreateObjectStreaming(obj StreamingObject) error

CreateObjectStreaming stores the given object internally.

If the bucket within the object doesn't exist, it also creates it. If the object already exists, it overwrites the object.

func (*Server) GetObject

func (s *Server) GetObject(bucketName, objectName string) (Object, error)

GetObject is the non-streaming version of GetObjectStreaming.

func (*Server) GetObjectStreaming

func (s *Server) GetObjectStreaming(bucketName, objectName string) (StreamingObject, error)

GetObjectStreaming returns the object with the given name in the given bucket, or an error if the object doesn't exist.

func (*Server) GetObjectWithGeneration

func (s *Server) GetObjectWithGeneration(bucketName, objectName string, generation int64) (Object, error)

GetObjectWithGeneration is the non-streaming version of GetObjectWithGenerationStreaming.

func (*Server) GetObjectWithGenerationStreaming

func (s *Server) GetObjectWithGenerationStreaming(bucketName, objectName string, generation int64) (StreamingObject, error)

GetObjectWithGenerationStreaming returns the object with the given name and given generation ID in the given bucket, or an error if the object doesn't exist.

If versioning is enabled, archived versions are considered.

func (*Server) HTTPClient

func (s *Server) HTTPClient() *http.Client

HTTPClient returns an HTTP client configured to talk to the server.

func (*Server) ListObjects

func (s *Server) ListObjects(bucketName, prefix, delimiter string, versions bool) ([]ObjectAttrs, []string, error)

ListObjects returns a sorted list of objects that match the given criteria, or an error if the bucket doesn't exist.

Deprecated: use ListObjectsWithOptions.

func (*Server) ListObjectsWithOptions

func (s *Server) ListObjectsWithOptions(bucketName string, options ListOptions) ([]ObjectAttrs, []string, error)

func (*Server) PublicURL

func (s *Server) PublicURL() string

PublicURL returns the server's public download URL.

func (*Server) Stop

func (s *Server) Stop()

Stop stops the server, closing all connections.

func (*Server) URL

func (s *Server) URL() string

URL returns the server URL.

type StreamingObject

type StreamingObject struct {
	ObjectAttrs
	Content io.ReadSeekCloser `json:"-"`
}

StreamingObject is the streaming version of Object.

func (*StreamingObject) BufferedObject

func (o *StreamingObject) BufferedObject() (Object, error)

func (*StreamingObject) Close

func (o *StreamingObject) Close() error