persistence

README

Storage Location

The default storage driver url is file:///tmp/mediorum/blobs. This is the directory on your node where files will stored post transcoding.

To update the local storage location set the AUDIUS_STORAGE_DRIVER_URL environment variable

AUDIUS_STORAGE_DRIVER_URL="file:///my/directory"

The driver is determined based on the storage_url (eg s3://your-bucket uses the s3 driver)

💡 Note: URLs beginning with https:// will default to s3

Cloud Storage

To use cloud object storage instead of local storage, you will need to:

• create either an s3 bucket, gcs bucket or azure storage container
• your bucket access policy should NOT BE PUBLIC
• provision credentials that allow object storage access

S3 storage

AWS_ACCESS_KEY_ID="<my-access-key-id>"
AWS_SECRET_ACCESS_KEY="<my-secret-access-key>"
AWS_REGION="<my-aws-region>"
AUDIUS_STORAGE_DRIVER_URL="s3://my-bucket?region=us-west-1"

GCS storage

GOOGLE_APPLICATION_CREDENTIALS="/path/to/creds.json"
AUDIUS_STORAGE_DRIVER_URL="gs://my-bucket"

Azure storage

AZURE_STORAGE_ACCOUNT="<storage-account-name>"
AZURE_STORAGE_KEY="<storage-key>"
AUDIUS_STORAGE_DRIVER_URL="azblob://my-container"

Documentation

Overview

Package persistence handles taking content from the temp store and storing it in the persistence store.

Index

• Constants
• func MoveAllFiles(from, to *blob.Bucket) error
• func Open(blobDriverURL string) (*blob.Bucket, error)
• type Prefix

Constants

const (
	AWS_ACCESS_KEY_ID     = "AWS_ACCESS_KEY_ID"
	AWS_SECRET_ACCESS_KEY = "AWS_SECRET_ACCESS_KEY"
	AWS_REGION            = "AWS_REGION"

	GOOGLE_APPLICATION_CREDENTIALS = "GOOGLE_APPLICATION_CREDENTIALS"

	AZURE_STORAGE_ACCOUNT = "AZURE_STORAGE_ACCOUNT"
	AZURE_STORAGE_KEY     = "AZURE_STORAGE_KEY"
)

Functions

func MoveAllFiles

func MoveAllFiles(from, to *blob.Bucket) error

func Open

func Open(blobDriverURL string) (*blob.Bucket, error)

New creates a struct that listens to streamToStoreFrom and downloads content from the temp store to the persistent store.

Types

type Prefix

type Prefix int

const (
	FILE Prefix = iota
	HTTP
	GS
	S3
	AZBLOB
)