Documentation
¶
Overview ¶
Package object_storage provides API to fetch model files and metadata from s3/azureblob/gcs with minimum requirements to model files. Generally speaking object storage registry is not true model registry but just a way to sync models from object storage with minimum ODAHU conventions about files We expect that model files are stored whether in a directory in bucket or as .tar.gz / zip archive We expect that this directory/archive contain odahuflow.project.yaml file in root that describes next metadata about model: -- model:
name: wine version: 1.2
--
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SyncArchiveOrDir ¶
func SyncArchiveOrDir(conn v1alpha1.ConnectionSpec, relPath string, localPath string) error
SyncArchiveOrDir sync object storage directory or archive If localPath ends with "/" slash then we assume that it's a directory otherwise file If (conn.URI + relPath) ends with .tar.gz or .zip then we assume that it's compressed tarball (.zip is legacy). If local path is directory but remote path is archive then we not only fetch archive but unzip it to local path Please take a look at ObjectStorage.Download
Types ¶
type ConnGetter ¶
type ConnGetter interface {
GetConnection(id string) (*connection.Connection, error)
}
type ModelRegistry ¶
type ModelRegistry struct {
// contains filtered or unexported fields
}
func NewModelRegistry ¶
func NewModelRegistry(cg ConnGetter) *ModelRegistry
func (ModelRegistry) SyncModel ¶
func (mr ModelRegistry) SyncModel(connName string, relPath string, localPath string) (newLocalPath string, err error)
SyncModel fetch model files from registry to localPath. If localPath is nil then SyncModel set localPath to temp directory and fetch model files there Model can be whether directory or .tar.gz / .zip archive If localPath == "" then temp directory will be created and returned in newLocalPath otherwise newLocalPath == localPath
Source Files