puller

Documentation

Overview

Copyright 2021 IBM Corporation

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright 2021 IBM Corporation

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright 2021 IBM Corporation

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

• func ApplyParameterOverrides(params map[string]interface{}, overrides map[string]string) error
• type ModelKeyInfo
• type Puller
  • func NewPuller(log logr.Logger) *Puller
  • func NewPullerFromConfig(log logr.Logger, config *PullerConfiguration) *Puller
  • func (p *Puller) CleanupModel(modelID string) error
  • func (p *Puller) ListModels() ([]string, error)
  • func (s *Puller) ProcessLoadModelRequest(req *mmesh.LoadModelRequest) (*mmesh.LoadModelRequest, error)
• type PullerConfiguration
  • func GetPullerConfigFromEnv(log logr.Logger) (*PullerConfiguration, error)
  • func (config *PullerConfiguration) GetStorageConfiguration(storageKey string, log logr.Logger) (map[string]interface{}, error)
• type PullerInterface
• type StorageConfiguration

Functions

func ApplyParameterOverrides

func ApplyParameterOverrides(params map[string]interface{}, overrides map[string]string) error

Implements a simple "dotpath" style for setting values within JSON compatible configuration structs

To keep things simple: - only string values are supported - paths can only trace object keys (not arrays) - only values that are strings can be overwritten

Types

type ModelKeyInfo

type ModelKeyInfo struct {
	// Pass through model_type as-is (it's actually a json object hence interface{} type)
	ModelType     interface{}       `json:"model_type,omitempty"`
	Bucket        string            `json:"bucket,omitempty"`
	DiskSizeBytes int64             `json:"disk_size_bytes"`
	SchemaPath    *string           `json:"schema_path,omitempty"`
	StorageKey    *string           `json:"storage_key,omitempty"`
	StorageParams map[string]string `json:"storage_params,omitempty"`
}

JSON passed in ModelInfo.Key field of registration requests

type Puller

type Puller struct {
	PullerConfig *PullerConfiguration
	Log          logr.Logger
	PullManager  PullerInterface
}

Puller represents the GRPC server and its configuration

func NewPuller

func NewPuller(log logr.Logger) *Puller

NewPuller creates a new Puller instance and initializes it with configuration from the environment

func NewPullerFromConfig

func NewPullerFromConfig(log logr.Logger, config *PullerConfiguration) *Puller

NewPullerFromConfig creates a new Puller instance with the given configuration

func (*Puller) CleanupModel

func (p *Puller) CleanupModel(modelID string) error

func (*Puller) ListModels

func (p *Puller) ListModels() ([]string, error)

func (*Puller) ProcessLoadModelRequest

func (s *Puller) ProcessLoadModelRequest(req *mmesh.LoadModelRequest) (*mmesh.LoadModelRequest, error)

ProcessLoadModelRequest is for use in an mmesh serving runtime that embeds the puller

The input request is modified in place and also returned. After pulling the model files, changes to the request are: - rewrite ModelPath to a local filesystem path - rewrite ModelKey["schema_path"] to a local filesystem path - add the size of the model on disk to ModelKey["disk_size_bytes"]

type PullerConfiguration

type PullerConfiguration struct {
	RootModelDir            string // Root directory to store models
	StorageConfigurationDir string
}

PullerConfiguration stores configuration variables for the puller server

func GetPullerConfigFromEnv

func GetPullerConfigFromEnv(log logr.Logger) (*PullerConfiguration, error)

GetPullerConfigFromEnv creates a new PullerConfiguration populated from environment variables

func (*PullerConfiguration) GetStorageConfiguration

func (config *PullerConfiguration) GetStorageConfiguration(storageKey string, log logr.Logger) (map[string]interface{}, error)

GetStorageConfiguration returns configuration read from the mounted secret at the given key

type PullerInterface

type PullerInterface interface {
	Pull(context.Context, pullman.PullCommand) error
}

PullerInterface is the interface for `pullman`

useful to mock for testing

type StorageConfiguration

type StorageConfiguration struct {
	StorageType     string `json:"type"`
	AccessKeyID     string `json:"access_key_id"`
	SecretAccessKey string `json:"secret_access_key"`
	EndpointURL     string `json:"endpoint_url"`
	Region          string `json:"region"`
	Bucket          string `json:"bucket"`
	// Deprecated: for backward compatibility DefaultBucket is still populated,
	// but new code should use Bucket instead.
	DefaultBucket string `json:"default_bucket"`
	Certificate   string `json:"certificate"`
}

StorageConfiguration models the json credentials read from a storage secret