README
¶
go-bayesopt 
A library for doing Bayesian Optimization using Gaussian Processes (blackbox optimizer) in Go/Golang.
This project is under active development, if you find a bug, or anything that needs correction, please let me know.
Simple Example
package main
import (
"log"
"math"
"github.com/d4l3k/go-bayesopt"
)
func main() {
X := bayesopt.UniformParam{
Max: 10,
Min: -10,
}
o := bayesopt.New(
[]Param{
X,
},
)
// minimize x^2+1
x, y, err := o.Optimize(func(params map[Param]float64) float64 {
return math.Pow(params[X], 2) + 1
})
if err != nil {
log.Fatal(err)
}
log.Println(x, y)
}
How does it work?
From https://github.com/fmfn/BayesianOptimization:
Bayesian optimization works by constructing a posterior distribution of functions (gaussian process) that best describes the function you want to optimize. As the number of observations grows, the posterior distribution improves, and the algorithm becomes more certain of which regions in parameter space are worth exploring and which are not, as seen in the picture below.
As you iterate over and over, the algorithm balances its needs of exploration and exploitation taking into account what it knows about the target function. At each step a Gaussian Process is fitted to the known samples (points previously explored), and the posterior distribution, combined with a exploration strategy (such as UCB (Upper Confidence Bound), or EI (Expected Improvement)), are used to determine the next point that should be explored (see the gif below).
This process is designed to minimize the number of steps required to find a combination of parameters that are close to the optimal combination. To do so, this method uses a proxy optimization problem (finding the maximum of the acquisition function) that, albeit still a hard problem, is cheaper (in the computational sense) and common tools can be employed. Therefore Bayesian Optimization is most adequate for situations where sampling the function to be optimized is a very expensive endeavor. See the references for a proper discussion of this method.
License
go-bayesopt is licensed under the MIT license.
Documentation
¶
Index ¶
- Constants
- Variables
- func BasicBarrier(x []float64, params []Param) float64
- type BarrierFunc
- type BoundsMethod
- type Exploration
- type ExponentialParam
- type LinearParam
- type LogBarrier
- type NormalParam
- type Optimizer
- func (o *Optimizer) ExplorationErr() error
- func (o *Optimizer) GP() *gp.GP
- func (o *Optimizer) Log(x map[Param]float64, y float64)
- func (o *Optimizer) Next() (x map[Param]float64, parallel bool, err error)
- func (o *Optimizer) Optimize(f func(map[Param]float64) float64) (x map[Param]float64, y float64, err error)
- func (o *Optimizer) Rounds() int
- func (o *Optimizer) Running() bool
- func (o *Optimizer) Stop()
- type OptimizerOption
- func WithBarrierFunc(bf BarrierFunc) OptimizerOption
- func WithExploration(exploration Exploration) OptimizerOption
- func WithMinimize(minimize bool) OptimizerOption
- func WithOutputName(name string) OptimizerOption
- func WithRandomRounds(rounds int) OptimizerOption
- func WithRounds(rounds int) OptimizerOption
- type Param
- type RejectionParam
- type UCB
- type UniformParam
Constants ¶
const ( // DefaultRounds is the default number of rounds to run. DefaultRounds = 20 // DefaultRandomRounds is the default number of random rounds to run. DefaultRandomRounds = 5 // DefaultMinimize is the default value of minimize. DefaultMinimize = true NumRandPoints = 100000 NumGradPoints = 256 )
Variables ¶
var ( // DefaultExploration uses UCB with 95 confidence interval. DefaultExploration = UCB{Kappa: 1.96} // DefaultBarrierFunc sets the default barrier function to use. DefaultBarrierFunc = LogBarrier{} )
var SampleTries = 1000
SampleTries is the number of tries a sample function should try before truncating the samples to the boundaries.
Functions ¶
func BasicBarrier ¶
BasicBarrier returns -Inf if an x value is outside the param range.
Types ¶
type BarrierFunc ¶
type BarrierFunc interface {
Val(x []float64, params []Param) float64
Grad(x []float64, params []Param) []float64
}
BarrierFunc returns a value that is added to the value to bound the optimization.
type BoundsMethod ¶
func (BoundsMethod) Init ¶
func (m BoundsMethod) Init(dims, tasks int) int
type Exploration ¶
Exploration is the strategy to use for exploring the Gaussian process.
type ExponentialParam ¶
ExponentialParam is an exponentially distributed parameter between 0 and in the range (0, +math.MaxFloat64] whose rate parameter (lambda) is Rate and whose mean is 1/lambda (1). The Max and Min parameters use discard sampling to find a point between them. Set them to be math.Inf(1) and math.Inf(-1) to disable the bounds.
func (ExponentialParam) GetMax ¶
func (p ExponentialParam) GetMax() float64
GetMax implements Param.
func (ExponentialParam) GetMin ¶
func (p ExponentialParam) GetMin() float64
GetMin implements Param.
func (ExponentialParam) GetName ¶
func (p ExponentialParam) GetName() string
GetName implements Param.
func (ExponentialParam) Sample ¶
func (p ExponentialParam) Sample() float64
Sample implements Param.
type LogBarrier ¶
type LogBarrier struct{}
LogBarrier implements a logarithmic barrier function.
type NormalParam ¶
NormalParam is a normally distributed parameter with Mean and StdDev. The Max and Min parameters use discard sampling to find a point between them. Set them to be math.Inf(1) and math.Inf(-1) to disable the bounds.
type Optimizer ¶
type Optimizer struct {
// contains filtered or unexported fields
}
Optimizer is a blackbox gaussian process optimizer.
func New ¶
func New(params []Param, opts ...OptimizerOption) *Optimizer
New creates a new optimizer with the specified optimizable parameters and options.
func (*Optimizer) ExplorationErr ¶
func (*Optimizer) GP ¶
GP returns the underlying gaussian process. Primary for use with plotting behavior.
func (*Optimizer) Next ¶
Next returns the next best x values to explore. If more than rounds have elapsed, nil is returned. If parallel is true, that round can happen in parallel to other rounds.
func (*Optimizer) Optimize ¶
func (o *Optimizer) Optimize(f func(map[Param]float64) float64) (x map[Param]float64, y float64, err error)
Optimize will call f the fewest times as possible while trying to maximize the output value. It blocks until all rounds have elapsed, or Stop is called.
type OptimizerOption ¶
type OptimizerOption func(*Optimizer)
OptimizerOption sets an option on the optimizer.
func WithBarrierFunc ¶
func WithBarrierFunc(bf BarrierFunc) OptimizerOption
WithBarrierFunc sets the barrier function to use.
func WithExploration ¶
func WithExploration(exploration Exploration) OptimizerOption
WithExploration sets the exploration function to use.
func WithMinimize ¶
func WithMinimize(minimize bool) OptimizerOption
WithMinimize sets whether or not to minimize. Passing false, maximizes instead.
func WithOutputName ¶
func WithOutputName(name string) OptimizerOption
WithOutputName sets the outputs name. Only really matters if you're planning on using gp/plot.
func WithRandomRounds ¶
func WithRandomRounds(rounds int) OptimizerOption
WithRandomRounds sets the number of random rounds to run.
func WithRounds ¶
func WithRounds(rounds int) OptimizerOption
WithRounds sets the total number of rounds to run.
type Param ¶
type Param interface {
// GetName returns the name of the parameter.
GetName() string
// GetMax returns the maximum value.
GetMax() float64
// GetMin returns the minimum value.
GetMin() float64
// Sample returns a random point within the bounds. It doesn't have to be
// uniformly distributed.
Sample() float64
}
Param represents a parameter that can be optimized.
type RejectionParam ¶
RejectionParam samples from Param and then uses F to decide whether or not to reject the sample. This is typically used with a UniformParam. F should output a value between 0 and 1 indicating the proportion of samples this point should be accepted. If F always outputs 0, Sample will get stuck in an infinite loop.
type UniformParam ¶
UniformParam is a uniformly distributed parameter between Max and Min.
Source Files
Directories