server

package
v0.2.23-0...-4aaa12c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 17, 2020 License: NCSA Imports: 66 Imported by: 11

README

MLModelScope Agent Sever Commands

Note that evaluations currently only run on datasets known by DLDataset

Running Evaluations

One can run evaluations across different frameworks and models or on a single framework and model.

Running Evaluations on all Frameworks / Models

evaluate.go is a wrapper tool exists to make it easier to run evaluations across frameworks and models. One can specify the frameworks, models, and batch sizes to use within the file and then run evaluate.go.

  • [ ]: TODO: allow one to specify the frameworks, models, and batch sizes from the command line
Running Evaluations on a single Framework / Model
Example Usage
./tensorflow_agent dataset --debug --verbose --publish=true --fail_on_error=true --gpu=true --batch_size=320 --model_name=BVLC-Reference-CaffeNet --model_version=1.0 --database_name=tx2_carml_model_trace --database_address=minsky1-1.csl.illinois.edu --publish_predictions=false --num_file_parts=8 --trace_level=FULL_TRACE
Command line options

Available Models

agent info models

Checking Divergence

  • [ ]: TODO

To compare a single prediction's divergence you use

agent database divergence --database_address=$DATABASE_ADDR --database_name=carml --source=$SOURCE_ID --target=$TARGET_ID

Analysing / Summarizing Results

  • [ ]: TODO
agent evaluation --help

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	DefaultChannelBuffer = 100000
)
View Source
var (
	DefaultRunOptions = &robustly.RunOptions{
		RateLimit:  1,
		Timeout:    time.Second,
		PrintStack: true,
		RetryDelay: 0 * time.Nanosecond,
	}
)

Functions

func GetOutboundIP

func GetOutboundIP() net.IP

Get preferred outbound ip of this machine

func NewRootCommand

func NewRootCommand(frameworkRegisterFunc FrameworkRegisterFunction, framework0 dlframework.FrameworkManifest) (*cobra.Command, error)

represents the base command when called without any subcommands

func RunRootE

func RunRootE(c *cobra.Command, framework dlframework.FrameworkManifest, args []string) (<-chan bool, error)

func SetupFlags

func SetupFlags(c *cobra.Command)

Types

type FrameworkRegisterFunction

type FrameworkRegisterFunction func()

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL