goptuna

package module
v0.8.2-0...-71703bc Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 30, 2021 License: MIT Imports: 12 Imported by: 0

README

Goptuna

Software License GoDoc Go Report Card

Decentralized hyperparameter optimization framework, inspired by Optuna [1]. This library is particularly designed for machine learning, but everything will be able to optimize if you can define the objective function (e.g. Optimizing the number of goroutines of your server and the memory buffer size of the caching systems).

Supported algorithms:

Goptuna supports various state-of-the-art Bayesian optimization, Evolution strategy and Multi-armed bandit algorithms. All algorithms are implemented in pure Go and continuously benchmarked on GitHub Actions.

  • Random search
  • TPE: Tree-structured Parzen Estimators [2]
  • CMA-ES: Covariance Matrix Adaptation Evolution Strategy [3]
  • IPOP-CMA-ES: CMA-ES with increasing population size [4]
  • BIPOP-CMA-ES: BI-population CMA-ES [5]
  • Median Stopping Rule [6]
  • ASHA: Asynchronous Successive Halving Algorithm (Optuna flavored version) [1,7,8]
  • Quasi-monte carlo sampling based on Sobol sequence [10, 11]

Built-in dashboard:

Manage optimization results Interactive live-updating graphs
state-of-the-art-algorithms visualization

Projects using Goptuna:

Installation

You can integrate Goptuna in wide variety of Go projects because of its portability of pure Go.

$ go get -u github.com/maxhora/goptuna

Usage

Goptuna supports Define-by-Run style API like Optuna. You can dynamically construct the search spaces.

Basic usage
package main

import (
    "log"
    "math"

    "github.com/maxhora/goptuna"
    "github.com/maxhora/goptuna/tpe"
)

// ① Define an objective function which returns a value you want to minimize.
func objective(trial goptuna.Trial) (float64, error) {
    // ② Define the search space via Suggest APIs.
    x1, _ := trial.SuggestFloat("x1", -10, 10)
    x2, _ := trial.SuggestFloat("x2", -10, 10)
    return math.Pow(x1-2, 2) + math.Pow(x2+5, 2), nil
}

func main() {
    // ③ Create a study which manages each experiment.
    study, err := goptuna.CreateStudy(
        "goptuna-example",
        goptuna.StudyOptionSampler(tpe.NewSampler()))
    if err != nil { ... }

    // ④ Evaluate your objective function.
    err = study.Optimize(objective, 100)
    if err != nil { ... }

    // ⑤ Print the best evaluation parameters.
    v, _ := study.GetBestValue()
    p, _ := study.GetBestParams()
    log.Printf("Best value=%f (x1=%f, x2=%f)",
        v, p["x1"].(float64), p["x2"].(float64))
}

Link: Go Playground

Furthermore, I recommend you to use RDB storage backend for following purposes.

  • Continue from where we stopped in the previous optimizations.
  • Scale studies to tens of workers that connecting to the same RDB storage.
  • Check optimization results via a built-in dashboard.
Advanced usage
Distributed optimization using MySQL

There is no complicated setup to use RDB storage backend. First, setup MySQL server like following to share the optimization result.

$ docker pull mysql:8.0
$ docker run \
  -d \
  --rm \
  -p 3306:3306 \
  -e MYSQL_USER=goptuna \
  -e MYSQL_DATABASE=goptuna \
  -e MYSQL_PASSWORD=password \
  -e MYSQL_ALLOW_EMPTY_PASSWORD=yes \
  --name goptuna-mysql \
  mysql:8.0

Then, create a study object using Goptuna CLI.

$ goptuna create-study --storage mysql://goptuna:password@localhost:3306/yourdb --study yourstudy
yourstudy

$ mysql --host 127.0.0.1 --port 3306 --user goptuna -ppassword -e "SELECT * FROM studies;"
+----------+------------+-----------+
| study_id | study_name | direction |
+----------+------------+-----------+
|        1 | yourstudy  | MINIMIZE  |
+----------+------------+-----------+
1 row in set (0.00 sec)

Finally, run the Goptuna workers which contains following code. You can execute distributed optimization by just executing this script from multiple server instances.

package main

import ...

func main() {
    db, _ := gorm.Open(mysql.Open("goptuna:password@tcp(localhost:3306)/yourdb?parseTime=true"), &gorm.Config{
        Logger: logger.Default.LogMode(logger.Silent),
    })
    storage := rdb.NewStorage(db)
    defer db.Close()

    study, _ := goptuna.LoadStudy(
        "yourstudy",
        goptuna.StudyOptionStorage(storage),
        ...,
    )
    _ = study.Optimize(objective, 50)
    ...
}

Full source code is available here.

Built-in Realtime Web Dashboard

You can check optimization results by built-in web dashboard.

SQLite3:

$ goptuna dashboard --storage sqlite:///example.db

MySQL:

$ goptuna dashboard --storage mysql://goptuna:password@127.0.0.1:3306/yourdb

goptuna dashboard

Shell script to reproduce this (SQLite3 version is here).

References:

Presentations:

Blog posts:

Status:

License

This software is licensed under the MIT license, see LICENSE for more information.

Documentation

Index

Examples

Constants

View Source 
const CategoricalDistributionName = "CategoricalDistribution"

CategoricalDistributionName is the identifier name of CategoricalDistribution

View Source 
const DiscreteUniformDistributionName = "DiscreteUniformDistribution"

DiscreteUniformDistributionName is the identifier name of DiscreteUniformDistribution

View Source 
const InMemoryStorageStudyID = 1

InMemoryStorageStudyID is a study id for in memory storage backend.

View Source 
const InMemoryStorageStudyUUID = "00000000-0000-0000-0000-000000000000"

InMemoryStorageStudyUUID is a UUID for in memory storage backend

View Source 
const IntUniformDistributionName = "IntUniformDistribution"

IntUniformDistributionName is the identifier name of IntUniformDistribution

View Source 
const LogUniformDistributionName = "LogUniformDistribution"

LogUniformDistributionName is the identifier name of LogUniformDistribution

View Source 
const StepIntUniformDistributionName = "StepIntUniformDistribution"

StepIntUniformDistributionName is the identifier name of IntUniformDistribution

View Source 
const UniformDistributionName = "UniformDistribution"

UniformDistributionName is the identifier name of UniformDistribution

Variables

View Source 
var (
	// RandomSearchSamplerOptionSeed sets seed number.
	// Deprecated: this is renamed to RandomSamplerOptionSeed.
	RandomSearchSamplerOptionSeed = RandomSamplerOptionSeed

	// NewRandomSearchSampler implements random search algorithm.
	// Deprecated: this is renamed to NewRandomSampler.
	NewRandomSearchSampler = NewRandomSampler
)
View Source 
var (
	// ErrInvalidStudyID represents invalid study id.
	ErrInvalidStudyID = errors.New("invalid study id")
	// ErrInvalidTrialID represents invalid trial id.
	ErrInvalidTrialID = errors.New("invalid trial id")
	// ErrTrialCannotBeUpdated represents trial cannot be updated.
	ErrTrialCannotBeUpdated = errors.New("trial cannot be updated")
	// ErrNoCompletedTrials represents no trials are completed yet.
	ErrNoCompletedTrials = errors.New("no trials are completed yet")
)
View Source 
var (
	// ErrTrialAlreadyDeleted means that trial is already deleted.
	ErrTrialAlreadyDeleted = errors.New("trial is already deleted")
	// ErrTrialsPartiallyDeleted means that trials are partially deleted.
	ErrTrialsPartiallyDeleted = errors.New("some trials are already deleted")
	// ErrDeleteNonFinishedTrial means that non finished trial is deleted.
	ErrDeleteNonFinishedTrial = errors.New("non finished trial is deleted")
)
View Source 
var DefaultStudyNamePrefix = "no-name-"

DefaultStudyNamePrefix is a prefix of the default study name.

View Source 
var (
	// ErrTrialPruned represents the pruned.
	ErrTrialPruned = errors.New("trial is pruned")
)
View Source 
var (
	// ErrUnknownDistribution returns the distribution is unknown.
	ErrUnknownDistribution = errors.New("unknown distribution")
)
View Source 
var (
	// ErrUnsupportedSearchSpace represents sampler does not support a given search space.
	ErrUnsupportedSearchSpace = errors.New("unsupported search space")
)
View Source 
var StudyOptionSetDirection = StudyOptionDirection

StudyOptionSetDirection change the direction of optimize Deprecated: please use StudyOptionDirection instead.

View Source 
var StudyOptionSetLogger = StudyOptionLogger

StudyOptionSetLogger sets Logger. Deprecated: please use StudyOptionLogger instead.

View Source 
var StudyOptionSetTrialNotifyChannel = StudyOptionTrialNotifyChannel

StudyOptionSetTrialNotifyChannel to subscribe the finished trials.

Functions

func DeleteStudy 

func DeleteStudy(
	name string,
	storage Storage,
) error

DeleteStudy delete a study object.

func DistributionIsSingle 

func DistributionIsSingle(distribution interface{}) (bool, error)

DistributionIsSingle whether the distribution contains just a single value.

func DistributionToJSON 

func DistributionToJSON(distribution interface{}) ([]byte, error)

DistributionToJSON serialize a distribution to JSON format.

func IntersectionSearchSpace 

func IntersectionSearchSpace(study *Study) (map[string]interface{}, error)

IntersectionSearchSpace return return the intersection search space of the Study.

Intersection search space contains the intersection of parameter distributions that have been suggested in the completed trials of the study so far. If there are multiple parameters that have the same name but different distributions, neither is included in the resulting search space (i.e., the parameters with dynamic value ranges are excluded).

func JSONToDistribution 

func JSONToDistribution(jsonBytes []byte) (interface{}, error)

JSONToDistribution deserialize a distribution in JSON format.

func ToExternalRepresentation 

func ToExternalRepresentation(distribution interface{}, ir float64) (interface{}, error)

ToExternalRepresentation converts to external representation

Types

type BlackHoleStorage 

type BlackHoleStorage struct {
	// contains filtered or unexported fields
}

BlackHoleStorage is an in-memory storage, but designed for over 100k trials. Please note that this storage just holds the 'nTrials' trials.

Methods to create a trial might return ErrDeleteNonFinishedTrial. GetAllTrials method might return ErrTrialsPartiallyDeleted. Methods to get or update a trial might return ErrTrialAlreadyDeleted.

Currently, RandomSampler and CMA-ES sampler supports this storage.

func NewBlackHoleStorage 

func NewBlackHoleStorage(n int) *BlackHoleStorage

NewBlackHoleStorage returns BlackHoleStorage.

func (*BlackHoleStorage) CloneTrial 

func (s *BlackHoleStorage) CloneTrial(studyID int, baseTrial FrozenTrial) (int, error)

CloneTrial creates new Trial from the given base Trial.

func (*BlackHoleStorage) CreateNewStudy 

func (s *BlackHoleStorage) CreateNewStudy(name string) (int, error)

CreateNewStudy creates study and returns studyID.

func (*BlackHoleStorage) CreateNewTrial 

func (s *BlackHoleStorage) CreateNewTrial(studyID int) (int, error)

CreateNewTrial creates trial and returns trialID.

func (*BlackHoleStorage) DeleteStudy 

func (s *BlackHoleStorage) DeleteStudy(studyID int) error

DeleteStudy deletes a study.

func (*BlackHoleStorage) GetAllStudySummaries 

func (s *BlackHoleStorage) GetAllStudySummaries() ([]StudySummary, error)

GetAllStudySummaries returns all study summaries.

func (*BlackHoleStorage) GetAllTrials 

func (s *BlackHoleStorage) GetAllTrials(studyID int) ([]FrozenTrial, error)

GetAllTrials returns the all trials.

func (*BlackHoleStorage) GetBestTrial 

func (s *BlackHoleStorage) GetBestTrial(studyID int) (FrozenTrial, error)

GetBestTrial returns the best trial.

func (*BlackHoleStorage) GetStudyDirection 

func (s *BlackHoleStorage) GetStudyDirection(studyID int) (StudyDirection, error)

GetStudyDirection returns study direction of the objective.

func (*BlackHoleStorage) GetStudyIDFromName 

func (s *BlackHoleStorage) GetStudyIDFromName(name string) (int, error)

GetStudyIDFromName return the study id from study name.

func (*BlackHoleStorage) GetStudyIDFromTrialID 

func (s *BlackHoleStorage) GetStudyIDFromTrialID(trialID int) (int, error)

GetStudyIDFromTrialID return the study id from trial id.

func (*BlackHoleStorage) GetStudyNameFromID 

func (s *BlackHoleStorage) GetStudyNameFromID(studyID int) (string, error)

GetStudyNameFromID return the study name from study id.

func (*BlackHoleStorage) GetStudySystemAttrs 

func (s *BlackHoleStorage) GetStudySystemAttrs(studyID int) (map[string]string, error)

GetStudySystemAttrs to restore the attributes for the system.

func (*BlackHoleStorage) GetStudyUserAttrs 

func (s *BlackHoleStorage) GetStudyUserAttrs(studyID int) (map[string]string, error)

GetStudyUserAttrs to restore the attributes for the user.

func (*BlackHoleStorage) GetTrial 

func (s *BlackHoleStorage) GetTrial(trialID int) (FrozenTrial, error)

GetTrial returns Trial.

func (*BlackHoleStorage) GetTrialNumberFromID 

func (s *BlackHoleStorage) GetTrialNumberFromID(trialID int) (int, error)

GetTrialNumberFromID returns the trial's number.

func (*BlackHoleStorage) GetTrialParam 

func (s *BlackHoleStorage) GetTrialParam(trialID int, paramName string) (float64, error)

GetTrialParam returns the internal parameter of the trial

func (*BlackHoleStorage) GetTrialParams 

func (s *BlackHoleStorage) GetTrialParams(trialID int) (map[string]interface{}, error)

GetTrialParams returns the external parameters in the trial

func (*BlackHoleStorage) GetTrialSystemAttrs 

func (s *BlackHoleStorage) GetTrialSystemAttrs(trialID int) (map[string]string, error)

GetTrialSystemAttrs to restore the attributes for the system.

func (*BlackHoleStorage) GetTrialUserAttrs 

func (s *BlackHoleStorage) GetTrialUserAttrs(trialID int) (map[string]string, error)

GetTrialUserAttrs to restore the attributes for the user.

func (*BlackHoleStorage) SetStudyDirection 

func (s *BlackHoleStorage) SetStudyDirection(studyID int, direction StudyDirection) error

SetStudyDirection sets study direction of the objective.

func (*BlackHoleStorage) SetStudySystemAttr 

func (s *BlackHoleStorage) SetStudySystemAttr(studyID int, key string, value string) error

SetStudySystemAttr to store the value for the system.

func (*BlackHoleStorage) SetStudyUserAttr 

func (s *BlackHoleStorage) SetStudyUserAttr(studyID int, key string, value string) error

SetStudyUserAttr to store the value for the user.

func (*BlackHoleStorage) SetTrialIntermediateValue 

func (s *BlackHoleStorage) SetTrialIntermediateValue(trialID int, step int, value float64) error

SetTrialIntermediateValue sets the intermediate value of trial.

func (*BlackHoleStorage) SetTrialParam 

func (s *BlackHoleStorage) SetTrialParam(
	trialID int,
	paramName string,
	paramValueInternal float64,
	distribution interface{},
) error

SetTrialParam sets the sampled parameters of trial.

func (*BlackHoleStorage) SetTrialState 

func (s *BlackHoleStorage) SetTrialState(trialID int, state TrialState) error

SetTrialState sets the state of trial.

func (*BlackHoleStorage) SetTrialSystemAttr 

func (s *BlackHoleStorage) SetTrialSystemAttr(trialID int, key string, value string) error

SetTrialSystemAttr to store the value for the system.

func (*BlackHoleStorage) SetTrialUserAttr 

func (s *BlackHoleStorage) SetTrialUserAttr(trialID int, key string, value string) error

SetTrialUserAttr to store the value for the user.

func (*BlackHoleStorage) SetTrialValue 

func (s *BlackHoleStorage) SetTrialValue(trialID int, value float64) error

SetTrialValue sets the value of trial.

type CategoricalDistribution 

type CategoricalDistribution struct {
	// Choices is a candidates of parameter values
	Choices []string `json:"choices"`
}

CategoricalDistribution is a distribution for categorical parameters

func (*CategoricalDistribution) Contains 

func (d *CategoricalDistribution) Contains(ir float64) bool

Contains to check a parameter value is contained in the range of this distribution.

func (*CategoricalDistribution) Single 

func (d *CategoricalDistribution) Single() bool

Single to test whether the range of this distribution contains just a single value.

func (*CategoricalDistribution) ToExternalRepr 

func (d *CategoricalDistribution) ToExternalRepr(ir float64) interface{}

ToExternalRepr to convert internal representation of a parameter value into external representation.

type DiscreteUniformDistribution 

type DiscreteUniformDistribution struct {
	// High is higher endpoint of the range of the distribution (included in the range).
	High float64 `json:"high"`
	// Low is lower endpoint of the range of the distribution (included in the range).
	Low float64 `json:"low"`
	// Q is a discretization step.
	Q float64 `json:"q"`
}

DiscreteUniformDistribution is a discretized uniform distribution in the linear domain.

func (*DiscreteUniformDistribution) Contains 

func (d *DiscreteUniformDistribution) Contains(ir float64) bool

Contains to check a parameter value is contained in the range of this distribution.

func (*DiscreteUniformDistribution) Single 

func (d *DiscreteUniformDistribution) Single() bool

Single to test whether the range of this distribution contains just a single value.

func (*DiscreteUniformDistribution) ToExternalRepr 

func (d *DiscreteUniformDistribution) ToExternalRepr(ir float64) interface{}

ToExternalRepr to convert internal representation of a parameter value into external representation.

type Distribution 

type Distribution interface {
	// ToExternalRepr to convert internal representation of a parameter value into external representation.
	ToExternalRepr(float64) interface{}
	// Single to test whether the range of this distribution contains just a single value.
	Single() bool
	// Contains to check a parameter value is contained in the range of this distribution.
	Contains(float64) bool
}

Distribution represents a parameter that can be optimized.

type FrozenTrial 

type FrozenTrial struct {
	ID                 int                    `json:"trial_id"`
	StudyID            int                    `json:"study_id"`
	Number             int                    `json:"number"`
	State              TrialState             `json:"state"`
	Value              float64                `json:"value"`
	IntermediateValues map[int]float64        `json:"intermediate_values"`
	DatetimeStart      time.Time              `json:"datetime_start"`
	DatetimeComplete   time.Time              `json:"datetime_complete"`
	InternalParams     map[string]float64     `json:"internal_params"`
	Params             map[string]interface{} `json:"params"`
	Distributions      map[string]interface{} `json:"distributions"`
	UserAttrs          map[string]string      `json:"user_attrs"`
	SystemAttrs        map[string]string      `json:"system_attrs"`
}

FrozenTrial holds the status and results of a Trial.

func (FrozenTrial) GetLatestStep 

func (t FrozenTrial) GetLatestStep() (step int, exist bool)

GetLatestStep returns the latest step in intermediate values.

type FuncObjective 

type FuncObjective func(trial Trial) (float64, error)

FuncObjective is a type of objective function

type InMemoryStorage 

type InMemoryStorage struct {
	// contains filtered or unexported fields
}

InMemoryStorage stores data in memory of the Go process.

func NewInMemoryStorage 

func NewInMemoryStorage() *InMemoryStorage

NewInMemoryStorage returns new memory storage.

func (*InMemoryStorage) CloneTrial 

func (s *InMemoryStorage) CloneTrial(studyID int, baseTrial FrozenTrial) (int, error)

CloneTrial creates new Trial from the given base Trial.

func (*InMemoryStorage) CreateNewStudy 

func (s *InMemoryStorage) CreateNewStudy(name string) (int, error)

CreateNewStudy creates study and returns studyID.

Example 
package main

import (
	"fmt"

	"github.com/maxhora/goptuna"
)

func main() {
	storage := goptuna.NewInMemoryStorage()
	studyID, err := storage.CreateNewStudy("")
	if err != nil {
		panic(err)
	}
	fmt.Println(studyID)

}

Output:

1

func (*InMemoryStorage) CreateNewTrial 

func (s *InMemoryStorage) CreateNewTrial(studyID int) (int, error)

CreateNewTrial creates trial and returns trialID.

func (*InMemoryStorage) DeleteStudy 

func (s *InMemoryStorage) DeleteStudy(studyID int) error

DeleteStudy deletes a study.

func (*InMemoryStorage) GetAllStudySummaries 

func (s *InMemoryStorage) GetAllStudySummaries() ([]StudySummary, error)

GetAllStudySummaries returns all study summaries.

func (*InMemoryStorage) GetAllTrials 

func (s *InMemoryStorage) GetAllTrials(studyID int) ([]FrozenTrial, error)

GetAllTrials returns the all trials.

func (*InMemoryStorage) GetBestTrial 

func (s *InMemoryStorage) GetBestTrial(studyID int) (FrozenTrial, error)

GetBestTrial returns the best trial.

func (*InMemoryStorage) GetStudyDirection 

func (s *InMemoryStorage) GetStudyDirection(studyID int) (StudyDirection, error)

GetStudyDirection returns study direction of the objective.

func (*InMemoryStorage) GetStudyIDFromName 

func (s *InMemoryStorage) GetStudyIDFromName(name string) (int, error)

GetStudyIDFromName return the study id from study name.

func (*InMemoryStorage) GetStudyIDFromTrialID 

func (s *InMemoryStorage) GetStudyIDFromTrialID(trialID int) (int, error)

GetStudyIDFromTrialID return the study id from trial id.

Example 
package main

import (
	"fmt"

	"github.com/maxhora/goptuna"
)

func main() {
	storage := goptuna.NewInMemoryStorage()
	studyID, err := storage.CreateNewStudy("")
	if err != nil {
		panic(err)
	}
	trialID, err := storage.CreateNewTrial(studyID)
	if err != nil {
		panic(err)
	}

	actual, err := storage.GetStudyIDFromTrialID(trialID)
	if err != nil {
		panic(err)
	}
	fmt.Println(actual)
}

Output:

1

func (*InMemoryStorage) GetStudyNameFromID 

func (s *InMemoryStorage) GetStudyNameFromID(studyID int) (string, error)

GetStudyNameFromID return the study name from study id.

func (*InMemoryStorage) GetStudySystemAttrs 

func (s *InMemoryStorage) GetStudySystemAttrs(studyID int) (map[string]string, error)

GetStudySystemAttrs to restore the attributes for the system.

func (*InMemoryStorage) GetStudyUserAttrs 

func (s *InMemoryStorage) GetStudyUserAttrs(studyID int) (map[string]string, error)

GetStudyUserAttrs to restore the attributes for the user.

func (*InMemoryStorage) GetTrial 

func (s *InMemoryStorage) GetTrial(trialID int) (FrozenTrial, error)

GetTrial returns Trial.

func (*InMemoryStorage) GetTrialNumberFromID 

func (s *InMemoryStorage) GetTrialNumberFromID(trialID int) (int, error)

GetTrialNumberFromID returns the trial's number.

func (*InMemoryStorage) GetTrialParam 

func (s *InMemoryStorage) GetTrialParam(trialID int, paramName string) (float64, error)

GetTrialParam returns the internal parameter of the trial

func (*InMemoryStorage) GetTrialParams 

func (s *InMemoryStorage) GetTrialParams(trialID int) (map[string]interface{}, error)

GetTrialParams returns the external parameters in the trial

func (*InMemoryStorage) GetTrialSystemAttrs 

func (s *InMemoryStorage) GetTrialSystemAttrs(trialID int) (map[string]string, error)

GetTrialSystemAttrs to restore the attributes for the system.

func (*InMemoryStorage) GetTrialUserAttrs 

func (s *InMemoryStorage) GetTrialUserAttrs(trialID int) (map[string]string, error)

GetTrialUserAttrs to restore the attributes for the user.

func (*InMemoryStorage) SetStudyDirection 

func (s *InMemoryStorage) SetStudyDirection(studyID int, direction StudyDirection) error

SetStudyDirection sets study direction of the objective.

func (*InMemoryStorage) SetStudySystemAttr 

func (s *InMemoryStorage) SetStudySystemAttr(studyID int, key string, value string) error

SetStudySystemAttr to store the value for the system.

Example 
package main

import (
	"fmt"

	"github.com/maxhora/goptuna"
)

func main() {
	storage := goptuna.NewInMemoryStorage()
	studyID, err := storage.CreateNewStudy("")
	if err != nil {
		panic(err)
	}
	err = storage.SetStudySystemAttr(studyID, "key", "value")
	if err != nil {
		panic(err)
	}
	attrs, err := storage.GetStudySystemAttrs(studyID)
	if err != nil {
		panic(err)
	}
	for k, v := range attrs {
		fmt.Println(k, v)
	}
}

Output:

key value

func (*InMemoryStorage) SetStudyUserAttr 

func (s *InMemoryStorage) SetStudyUserAttr(studyID int, key string, value string) error

SetStudyUserAttr to store the value for the user.

Example 
package main

import (
	"fmt"

	"github.com/maxhora/goptuna"
)

func main() {
	storage := goptuna.NewInMemoryStorage()
	studyID, err := storage.CreateNewStudy("")
	if err != nil {
		panic(err)
	}
	err = storage.SetStudyUserAttr(studyID, "key", "value")
	if err != nil {
		panic(err)
	}
	attrs, err := storage.GetStudyUserAttrs(studyID)
	if err != nil {
		panic(err)
	}
	for k, v := range attrs {
		fmt.Println(k, v)
	}

}

Output:

key value

func (*InMemoryStorage) SetTrialIntermediateValue 

func (s *InMemoryStorage) SetTrialIntermediateValue(trialID int, step int, value float64) error

SetTrialIntermediateValue sets the intermediate value of trial.

func (*InMemoryStorage) SetTrialParam 

func (s *InMemoryStorage) SetTrialParam(
	trialID int,
	paramName string,
	paramValueInternal float64,
	distribution interface{}) error

SetTrialParam sets the sampled parameters of trial.

func (*InMemoryStorage) SetTrialState 

func (s *InMemoryStorage) SetTrialState(trialID int, state TrialState) error

SetTrialState sets the state of trial.

func (*InMemoryStorage) SetTrialSystemAttr 

func (s *InMemoryStorage) SetTrialSystemAttr(trialID int, key string, value string) error

SetTrialSystemAttr to store the value for the system.

Example 
package main

import (
	"fmt"

	"github.com/maxhora/goptuna"
)

func main() {
	storage := goptuna.NewInMemoryStorage()
	studyID, err := storage.CreateNewStudy("")
	if err != nil {
		panic(err)
	}
	trialID, err := storage.CreateNewTrial(studyID)
	if err != nil {
		panic(err)
	}

	err = storage.SetTrialSystemAttr(trialID, "key", "value")
	if err != nil {
		panic(err)
	}
	attrs, err := storage.GetTrialSystemAttrs(trialID)
	if err != nil {
		panic(err)
	}
	for k, v := range attrs {
		fmt.Println(k, v)
	}
}

Output:

key value

func (*InMemoryStorage) SetTrialUserAttr 

func (s *InMemoryStorage) SetTrialUserAttr(trialID int, key string, value string) error

SetTrialUserAttr to store the value for the user.

Example 
package main

import (
	"fmt"

	"github.com/maxhora/goptuna"
)

func main() {
	storage := goptuna.NewInMemoryStorage()
	studyID, err := storage.CreateNewStudy("")
	if err != nil {
		panic(err)
	}
	trialID, err := storage.CreateNewTrial(studyID)
	if err != nil {
		panic(err)
	}

	err = storage.SetTrialUserAttr(trialID, "key", "value")
	if err != nil {
		panic(err)
	}
	attrs, err := storage.GetTrialUserAttrs(trialID)
	if err != nil {
		panic(err)
	}
	for k, v := range attrs {
		fmt.Println(k, v)
	}

}

Output:

key value

func (*InMemoryStorage) SetTrialValue 

func (s *InMemoryStorage) SetTrialValue(trialID int, value float64) error

SetTrialValue sets the value of trial.

type IntUniformDistribution 

type IntUniformDistribution struct {
	// High is higher endpoint of the range of the distribution (included in the range).
	High int `json:"high"`
	// Low is lower endpoint of the range of the distribution (included in the range).
	Low int `json:"low"`
}

IntUniformDistribution is a uniform distribution on integers.

func (*IntUniformDistribution) Contains 

func (d *IntUniformDistribution) Contains(ir float64) bool

Contains to check a parameter value is contained in the range of this distribution.

func (*IntUniformDistribution) Single 

func (d *IntUniformDistribution) Single() bool

Single to test whether the range of this distribution contains just a single value.

func (*IntUniformDistribution) ToExternalRepr 

func (d *IntUniformDistribution) ToExternalRepr(ir float64) interface{}

ToExternalRepr to convert internal representation of a parameter value into external representation.

type LogUniformDistribution 

type LogUniformDistribution struct {
	// High is higher endpoint of the range of the distribution (included in the range).
	High float64 `json:"high"`
	// Low is lower endpoint of the range of the distribution (included in the range).
	Low float64 `json:"low"`
}

LogUniformDistribution is a uniform distribution in the log domain.

func (*LogUniformDistribution) Contains 

func (d *LogUniformDistribution) Contains(ir float64) bool

Contains to check a parameter value is contained in the range of this distribution.

func (*LogUniformDistribution) Single 

func (d *LogUniformDistribution) Single() bool

Single to test whether the range of this distribution contains just a single value.

func (*LogUniformDistribution) ToExternalRepr 

func (d *LogUniformDistribution) ToExternalRepr(ir float64) interface{}

ToExternalRepr to convert internal representation of a parameter value into external representation.

type Logger 

type Logger interface {
	Debug(msg string, fields ...interface{})
	Info(msg string, fields ...interface{})
	Warn(msg string, fields ...interface{})
	Error(msg string, fields ...interface{})
}

Logger is the interface for logging messages. If you want to log nothing, please set Logger as nil. If you want to print more verbose logs, it might StudyOptionTrialNotifyChannel option are useful.

type LoggerLevel 

type LoggerLevel int

LoggerLevel represents Level is a logging priority. Higher levels are more important. 

const (
	// LoggerLevelDebug logs are typically voluminous, and are usually disabled in production.
	LoggerLevelDebug LoggerLevel = iota - 1
	// LoggerLevelInfo is the default logging priority.
	LoggerLevelInfo
	// LoggerLevelWarn logs are more important than Info, but don't need individual human review.
	LoggerLevelWarn
	// LoggerLevelError logs are high-priority.
	LoggerLevelError
)

type Pruner 

type Pruner interface {
	// Prune judge whether the trial should be pruned at the given step.
	Prune(study *Study, trial FrozenTrial) (bool, error)
}

Pruner is a interface for early stopping algorithms.

type RandomSampler 

type RandomSampler struct {
	// contains filtered or unexported fields
}

RandomSampler for random search

func NewRandomSampler 

func NewRandomSampler(opts ...RandomSamplerOption) *RandomSampler

NewRandomSampler implements random search algorithm.

func (*RandomSampler) Sample 

func (s *RandomSampler) Sample(
	study *Study,
	trial FrozenTrial,
	paramName string,
	paramDistribution interface{},
) (float64, error)

Sample a parameter for a given distribution.

type RandomSamplerOption 

type RandomSamplerOption func(sampler *RandomSampler)

RandomSamplerOption is a type of function to set options.

func RandomSamplerOptionSeed 

func RandomSamplerOptionSeed(seed int64) RandomSamplerOption

RandomSamplerOptionSeed sets seed number.

type RandomSearchSampler 

type RandomSearchSampler RandomSampler

RandomSearchSampler for random search Deprecated: this is renamed to RandomSampler.

type RandomSearchSamplerOption 

type RandomSearchSamplerOption RandomSamplerOption

RandomSearchSamplerOption is a type of function to set change the option. Deprecated: this is renamed to RandomSamplerOption.

type RelativeSampler 

type RelativeSampler interface {
	// SampleRelative samples multiple dimensional parameters in a given search space.
	SampleRelative(*Study, FrozenTrial, map[string]interface{}) (map[string]float64, error)
}

RelativeSampler is the interface for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

This interface is called once at the beginning of each trial, i.e., right before the evaluation of the objective function.

type Sampler 

type Sampler interface {
	// Sample a single parameter for a given distribution.
	Sample(*Study, FrozenTrial, string, interface{}) (float64, error)
}

Sampler is the interface for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

Note that if study object has RelativeSampler, this interface is used only for parameters that are not sampled by RelativeSampler.

type StdLogger 

type StdLogger struct {
	Logger *log.Logger
	Level  LoggerLevel
	Color  bool
}

StdLogger wraps 'log' standard library.

func (*StdLogger) Debug 

func (l *StdLogger) Debug(msg string, fields ...interface{})

Debug logs a message at LoggerLevelDebug which are usually disabled in production.

func (*StdLogger) Error 

func (l *StdLogger) Error(msg string, fields ...interface{})

Error logs a message at LoggerLevelError which are high-priority.

func (*StdLogger) Info 

func (l *StdLogger) Info(msg string, fields ...interface{})

Info logs a message at LoggerLevelInfo. The message includes any fields passed at the log site, as well as any fields accumulated on the logger.

func (*StdLogger) Warn 

func (l *StdLogger) Warn(msg string, fields ...interface{})

Warn logs a message at LoggerLevelWarn which more important than Info, but don't need individual human review.

type StepIntUniformDistribution 

type StepIntUniformDistribution struct {
	// High is higher endpoint of the range of the distribution (included in the range).
	High int `json:"high"`
	// Low is lower endpoint of the range of the distribution (included in the range).
	Low int `json:"low"`
	// Step is a spacing between values.
	Step int `json:"step"`
}

StepIntUniformDistribution is a uniform distribution on integers.

func (*StepIntUniformDistribution) Contains 

func (d *StepIntUniformDistribution) Contains(ir float64) bool

Contains to check a parameter value is contained in the range of this distribution.

func (*StepIntUniformDistribution) Single 

func (d *StepIntUniformDistribution) Single() bool

Single to test whether the range of this distribution contains just a single value.

func (*StepIntUniformDistribution) ToExternalRepr 

func (d *StepIntUniformDistribution) ToExternalRepr(ir float64) interface{}

ToExternalRepr to convert internal representation of a parameter value into external representation.

type Storage 

type Storage interface {
	// Basic study manipulation
	CreateNewStudy(name string) (int, error)
	DeleteStudy(studyID int) error
	SetStudyDirection(studyID int, direction StudyDirection) error
	SetStudyUserAttr(studyID int, key string, value string) error
	SetStudySystemAttr(studyID int, key string, value string) error
	// Basic study access
	GetStudyIDFromName(name string) (int, error)
	GetStudyIDFromTrialID(trialID int) (int, error)
	GetStudyNameFromID(studyID int) (string, error)
	GetStudyDirection(studyID int) (StudyDirection, error)
	GetStudyUserAttrs(studyID int) (map[string]string, error)
	GetStudySystemAttrs(studyID int) (map[string]string, error)
	GetAllStudySummaries() ([]StudySummary, error)
	// Basic trial manipulation
	CreateNewTrial(studyID int) (int, error)
	CloneTrial(studyID int, baseTrial FrozenTrial) (int, error)
	SetTrialValue(trialID int, value float64) error
	SetTrialIntermediateValue(trialID int, step int, value float64) error
	SetTrialParam(trialID int, paramName string, paramValueInternal float64,
		distribution interface{}) error
	SetTrialState(trialID int, state TrialState) error
	SetTrialUserAttr(trialID int, key string, value string) error
	SetTrialSystemAttr(trialID int, key string, value string) error
	// Basic trial access
	GetTrialNumberFromID(trialID int) (int, error)
	GetTrialParam(trialID int, paramName string) (float64, error)
	GetTrial(trialID int) (FrozenTrial, error)
	GetAllTrials(studyID int) ([]FrozenTrial, error)
	GetBestTrial(studyID int) (FrozenTrial, error)
	GetTrialParams(trialID int) (map[string]interface{}, error)
	GetTrialUserAttrs(trialID int) (map[string]string, error)
	GetTrialSystemAttrs(trialID int) (map[string]string, error)
}

Storage interface abstract a backend database and provide library internal interfaces to read/write history of studies and trials. This interface is not supposed to be directly accessed by library users.

type Study 

type Study struct {
	ID              int
	Storage         Storage
	Sampler         Sampler
	RelativeSampler RelativeSampler
	Pruner          Pruner
	// contains filtered or unexported fields
}

Study corresponds to an optimization task, i.e., a set of trials.

func CreateStudy 

func CreateStudy(
	name string,
	opts ...StudyOption,
) (*Study, error)

CreateStudy creates a new Study object.

func LoadStudy 

func LoadStudy(
	name string,
	opts ...StudyOption,
) (*Study, error)

LoadStudy loads an existing study.

func (*Study) Direction 

func (s *Study) Direction() StudyDirection

Direction returns the direction of objective function value

func (*Study) EnqueueTrial 

func (s *Study) EnqueueTrial(internalParams map[string]float64) error

EnqueueTrial to enqueue a trial with given parameter values. You can fix the next sampling parameters which will be evaluated in your objective function.

This is an EXPERIMENTAL API and may be changed in the future. Currently EnqueueTrial only accepts internal representations. This means that you need to encode categorical parameter to its index number. Furthermore, please caution that there is a concurrency problem on SQLite3.

Example 
package main

import (
	"fmt"
	"math"

	"github.com/maxhora/goptuna"
)

func main() {
	study, _ := goptuna.CreateStudy(
		"example",
		goptuna.StudyOptionLogger(nil),
	)

	objective := func(trial goptuna.Trial) (float64, error) {
		x1, _ := trial.SuggestFloat("x1", -10, 10)
		x2, _ := trial.SuggestFloat("x2", -10, 10)
		return math.Pow(x1-2, 2) + math.Pow(x2+5, 2), nil
	}

	study.EnqueueTrial(map[string]float64{"x1": 2, "x2": -5})

	err := study.Optimize(objective, 1)
	if err != nil {
		panic(err)
	}

	trials, err := study.GetTrials()
	if err != nil {
		panic(err)
	}

	fmt.Printf("x1: %.3f\n", trials[0].Params["x1"].(float64))
	fmt.Printf("x2: %.3f\n", trials[0].Params["x2"].(float64))

}

Output:

x1: 2.000
x2: -5.000

func (*Study) GetBestParams 

func (s *Study) GetBestParams() (map[string]interface{}, error)

GetBestParams return parameters of the best trial

func (*Study) GetBestValue 

func (s *Study) GetBestValue() (float64, error)

GetBestValue return the best objective value

func (*Study) GetLogger 

func (s *Study) GetLogger() Logger

GetLogger returns logger object.

func (*Study) GetSystemAttrs 

func (s *Study) GetSystemAttrs() (map[string]string, error)

GetSystemAttrs to store the value for the system.

func (*Study) GetTrials 

func (s *Study) GetTrials() ([]FrozenTrial, error)

GetTrials returns all trials in this study.

func (*Study) GetUserAttrs 

func (s *Study) GetUserAttrs() (map[string]string, error)

GetUserAttrs to store the value for the user.

func (*Study) Optimize 

func (s *Study) Optimize(objective FuncObjective, evaluateMax int) error

Optimize optimizes an objective function.

Example 
package main

import (
	"fmt"
	"math"

	"github.com/maxhora/goptuna"
)

func main() {
	sampler := goptuna.NewRandomSampler(
		goptuna.RandomSamplerOptionSeed(0),
	)
	study, _ := goptuna.CreateStudy(
		"example",
		goptuna.StudyOptionSampler(sampler),
		goptuna.StudyOptionLogger(nil),
	)

	objective := func(trial goptuna.Trial) (float64, error) {
		x1, _ := trial.SuggestFloat("x1", -10, 10)
		x2, _ := trial.SuggestFloat("x2", -10, 10)
		return math.Pow(x1-2, 2) + math.Pow(x2+5, 2), nil
	}

	if err := study.Optimize(objective, 10); err != nil {
		panic(err)
	}
	value, _ := study.GetBestValue()
	params, _ := study.GetBestParams()

	fmt.Printf("Best trial: %.5f\n", value)
	fmt.Printf("x1: %.3f\n", params["x1"].(float64))
	fmt.Printf("x2: %.3f\n", params["x2"].(float64))
}

Output:

Best trial: 0.03833
x1: 2.182
x2: -4.927

func (*Study) SetSystemAttr 

func (s *Study) SetSystemAttr(key, value string) error

SetSystemAttr to store the value for the system.

func (*Study) SetUserAttr 

func (s *Study) SetUserAttr(key, value string) error

SetUserAttr to store the value for the user.

func (*Study) WithContext 

func (s *Study) WithContext(ctx context.Context)

WithContext sets a context and it might cancel the execution of Optimize.

type StudyDirection 

type StudyDirection string

StudyDirection represents the direction of the optimization 

const (
	// StudyDirectionMaximize maximizes objective function value
	StudyDirectionMaximize StudyDirection = "maximize"
	// StudyDirectionMinimize minimizes objective function value
	StudyDirectionMinimize StudyDirection = "minimize"
)

type StudyOption 

type StudyOption func(study *Study) error

StudyOption to pass the custom option

func StudyOptionDefineSearchSpace 

func StudyOptionDefineSearchSpace(space map[string]interface{}) StudyOption

StudyOptionDefineSearchSpace to use RelativeSampler from the first trial. This option is useful for Define-and-Run interface.

func StudyOptionDirection 

func StudyOptionDirection(direction StudyDirection) StudyOption

StudyOptionDirection change the direction of optimize

func StudyOptionIgnoreError 

func StudyOptionIgnoreError(ignore bool) StudyOption

StudyOptionIgnoreError is an option to continue even if it receive error while running Optimize method.

func StudyOptionLoadIfExists 

func StudyOptionLoadIfExists(loadIfExists bool) StudyOption

StudyOptionLoadIfExists to load the study if exists.

func StudyOptionLogger 

func StudyOptionLogger(logger Logger) StudyOption

StudyOptionLogger sets Logger.

func StudyOptionPruner 

func StudyOptionPruner(pruner Pruner) StudyOption

StudyOptionPruner sets the pruner object.

func StudyOptionRelativeSampler 

func StudyOptionRelativeSampler(sampler RelativeSampler) StudyOption

StudyOptionRelativeSampler sets the relative sampler object.

func StudyOptionSampler 

func StudyOptionSampler(sampler Sampler) StudyOption

StudyOptionSampler sets the sampler object.

func StudyOptionStorage 

func StudyOptionStorage(storage Storage) StudyOption

StudyOptionStorage sets the storage object.

func StudyOptionTrialNotifyChannel 

func StudyOptionTrialNotifyChannel(notify chan FrozenTrial) StudyOption

StudyOptionTrialNotifyChannel to subscribe the finished trials.

type StudySummary 

type StudySummary struct {
	ID            int               `json:"study_id"`
	Name          string            `json:"study_name"`
	Direction     StudyDirection    `json:"direction"`
	BestTrial     FrozenTrial       `json:"best_trial"`
	UserAttrs     map[string]string `json:"user_attrs"`
	SystemAttrs   map[string]string `json:"system_attrs"`
	DatetimeStart time.Time         `json:"datetime_start"`
}

StudySummary holds basic attributes and aggregated results of Study.

type Trial 

type Trial struct {
	Study *Study
	ID    int
	// contains filtered or unexported fields
}

Trial is a process of evaluating an objective function.

This object is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial's state of the trial. Note that this object is seamlessly instantiated and passed to the objective function behind; hence, in typical use cases, library users do not care about instantiation of this object.

func (*Trial) CallRelativeSampler 

func (t *Trial) CallRelativeSampler() error

CallRelativeSampler should be called before evaluate an objective function only 1 time. Please note that this method is public for third party libraries like "Kubeflow/Katib". Goptuna users SHOULD NOT call this method.

func (*Trial) GetContext 

func (t *Trial) GetContext() context.Context

GetContext returns a context which is registered at 'study.WithContext()'.

func (*Trial) GetSystemAttrs 

func (t *Trial) GetSystemAttrs() (map[string]string, error)

GetSystemAttrs to store the value for the system.

func (*Trial) GetUserAttrs 

func (t *Trial) GetUserAttrs() (map[string]string, error)

GetUserAttrs to store the value for the user.

func (*Trial) Number 

func (t *Trial) Number() (int, error)

Number return trial's number which is consecutive and unique in a study.

func (*Trial) SetSystemAttr 

func (t *Trial) SetSystemAttr(key, value string) error

SetSystemAttr to store the value for the system.

func (*Trial) SetUserAttr 

func (t *Trial) SetUserAttr(key, value string) error

SetUserAttr to store the value for the user.

func (*Trial) ShouldPrune 

func (t *Trial) ShouldPrune(step int, value float64) error

ShouldPrune judges whether the trial should be pruned. This method calls prune method of the pruner, which judges whether the trial should be pruned at the given step. If it should be pruned, this method return ErrTrialPruned.

func (*Trial) SuggestCategorical 

func (t *Trial) SuggestCategorical(name string, choices []string) (string, error)

SuggestCategorical suggests an categorical parameter.

func (*Trial) SuggestDiscreteFloat 

func (t *Trial) SuggestDiscreteFloat(name string, low, high, q float64) (float64, error)

SuggestDiscreteFloat suggests a value for the discrete floating point parameter.

func (*Trial) SuggestDiscreteUniform 

func (t *Trial) SuggestDiscreteUniform(name string, low, high, q float64) (float64, error)

SuggestDiscreteUniform suggests a value from a discrete uniform distribution. Deprecated: This method will be removed at v1.0.0. Please use SuggestDiscreteFloat method.

func (*Trial) SuggestFloat 

func (t *Trial) SuggestFloat(name string, low, high float64) (float64, error)

SuggestFloat suggests a value for the floating point parameter.

func (*Trial) SuggestInt 

func (t *Trial) SuggestInt(name string, low, high int) (int, error)

SuggestInt suggests an integer parameter.

func (*Trial) SuggestLogFloat 

func (t *Trial) SuggestLogFloat(name string, low, high float64) (float64, error)

SuggestLogFloat suggests a value for the log-scale floating point parameter.

func (*Trial) SuggestLogUniform 

func (t *Trial) SuggestLogUniform(name string, low, high float64) (float64, error)

SuggestLogUniform suggests a value from a uniform distribution in the log domain. Deprecated: This method will be removed at v1.0.0. Please use SuggestLogFloat method.

func (*Trial) SuggestStepInt 

func (t *Trial) SuggestStepInt(name string, low, high, step int) (int, error)

SuggestStepInt suggests a step-interval integer parameter.

func (*Trial) SuggestUniform 

func (t *Trial) SuggestUniform(name string, low, high float64) (float64, error)

SuggestUniform suggests a value from a uniform distribution. Deprecated: This method will be removed at v1.0.0. Please use SuggestFloat method.

type TrialState 

type TrialState int

TrialState is a state of Trial 

const (
	// TrialStateRunning means Trial is running.
	TrialStateRunning TrialState = iota
	// TrialStateComplete means Trial has been finished without any error.
	TrialStateComplete
	// TrialStatePruned means Trial has been pruned.
	TrialStatePruned
	// TrialStateFail means Trial has failed due to an uncaught error.
	TrialStateFail
	// TrialStateWaiting means Trial has been stopped, but may be resuming.
	TrialStateWaiting
)

func (TrialState) IsFinished 

func (i TrialState) IsFinished() bool

IsFinished returns true if trial is not running.

func (TrialState) String 

func (i TrialState) String() string

type UniformDistribution 

type UniformDistribution struct {
	// High is higher endpoint of the range of the distribution (included in the range).
	High float64 `json:"high"`
	// Low is lower endpoint of the range of the distribution (included in the range).
	Low float64 `json:"low"`
}

UniformDistribution is a uniform distribution in the linear domain.

func (*UniformDistribution) Contains 

func (d *UniformDistribution) Contains(ir float64) bool

Contains to check a parameter value is contained in the range of this distribution.

func (*UniformDistribution) Single 

func (d *UniformDistribution) Single() bool

Single to test whether the range of this distribution contains just a single value.

func (*UniformDistribution) ToExternalRepr 

func (d *UniformDistribution) ToExternalRepr(ir float64) interface{}

ToExternalRepr to convert internal representation of a parameter value into external representation.

Source Files

View all Source files

Directories

Path Synopsis
_benchmarks
goptuna_solver
himmelblau_problem
rastrigin_problem
rosenbrock_problem
_examples
cmaes
cmaes/blackhole
cmaes/lowlevel
cmaes/lowlevel_bipop
cmaes/lowlevel_ipop
concurrency
enqueue_trial
gorgonia_iris
signalhandling
simple_rdb
simple_tpe
sobol
sobol/engine
trialnotify
_tools
codegen_sobol
cmd
createstudy
dashboard
deletestudy
statik
internal
random
sqlalchemy
stats
testutil

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.