README
¶
Introduction
scientist helps you refactor your Go code with confidence.
Start by creating a new experiment:
experiment := scientist.NewQuickExperiment()
Wrap the current behavior into the control function:
// I wonder why this code is so slow :/
control := func(ctx context.Context) (interface{}, error) {
time.Sleep(10000000 * time.Second)
return "done", nil
}
experiment.Use(control)
Then, create one or more candidate behaviors to compare results:
// This is slightly faster, but I'm getting different results :(
slightlyFasterButWrongResult := func(ctx context.Context) (interface{}, error) {
time.Sleep(1 * time.Second)
return "exit", nil
}
experiment.Try("slightly faster call", slightlyFasterWrongResult)
// I think this is what I want \m/
superFast := func(ctx context.Context) (interface{}, error) {
return "done", nil
}
experiment.Try("super fast call", superFast)
Finally, run the experiment:
value, err := scientist.Run(experiment)
This call always returns the result of calling the control function. It randomizes the call between all three behaviors and measures their duration. It compares the results and publishes all this information somewhere else to analyze.
Creating your own experiments
You can create your own experiments by implementing the interface Experiment.
The easiest way to do this is by composing your own experiments with QuickExperiment
and implementing the methods you want to change, most likely Name, IsEnabled, Ignore,
Compare and Publish. You can see several examples of this in the samples package.
Failing with mismatches
scientist.Run guarantees that the control behavior, your old code, always returns its values.
It might be useful, mostly on testing, to fail the execution when the behaviors don't match,
that way you can test that your experiments are more robust.
To enable this, you can set the global variable scientist.ErrorOnMismatch to true.
In case of mismatched observations, scientist.Run returns scientist.MismatchResult as error,
giving you access to all the information about the observations.
Adding context information
Giving extra information to your experiments is easy using a context.Context object.
Use scientist.RunWithContext to run your experiment and each behavior will get a copy
of your context object to gather more information.
ctx := context.Background()
ctx = context.WithValue(ctx, "user", models.User{})
control := func(ctx context.Context) (interface{}, error) {
return ctx.Value("user").(models.User).Login, nil
}
experiment := scientist.NewQuickExperiment()
experiment.Use(control)
login, err := scientist.RunWithContext(ctx, experiment)
This package was inspired by GitHub's ruby scientist: https://github.com/github/scientist.
Documentation
¶
Overview ¶
Package scientist helps you refactor your Go code with confidence.
Start by creating a new experiment:
experiment := scientist.NewQuickExperiment()
Wrap the current behavior into the control function:
// I wonder why this code is so slow :/
control := func(ctx context.Context) (interface{}, error) {
time.Sleep(10000000 * time.Second)
return "done", nil
}
experiment.Use(control)
Then, create one or more candidate behaviors to compare results:
// This is slightly faster, but I'm getting different results :(
slightlyFasterButWrongResult := func(ctx context.Context) (interface{}, error) {
time.Sleep(1 * time.Second)
return "exit", nil
}
experiment.Try("slightly faster call", slightlyFasterWrongResult)
// I think this is what I want \m/
superFast := func(ctx context.Context) (interface{}, error) {
return "done", nil
}
experiment.Try("super fast call", superFast)
Finally, run the experiment:
value, err := scientist.Run(experiment)
This call always returns the result of calling the control function. It randomizes the call between all three behaviors and measures their duration. It compares the results and publishes all this information somewhere else to analyze.
Creating your own experiments ¶
You can create your own experiments by implementing the interface `Experiment`. The easiest way to do this is by composing your own experiments with `QuickExperiment` and implementing the methods you want to change, most likely `Name`, `IsEnabled`, `Ignore`, `Compare` and `Publish`. You can see several examples of this in the `samples` package.
Failing with mismatches ¶
`scientist.Run` guarantees that the control behavior, your old code, always returns its values. It might be useful, mostly on testing, to fail the execution when the behaviors don't match, that way you can test that your experiments are more robust. To enable this, you can set the global variable `scientist.ErrorOnMismatch` to `true`.
In case of mismatched observations, `scientist.Run` returns `scientist.MismatchResult` as error, giving you access to all the information about the observations.
Adding context information ¶
Giving extra information to your experiments is easy using a `context.Context` object. Use `scientist.RunWithContext` to run your experiment and each behavior will get a copy of your context object to gather more information.
ctx := context.Background()
ctx = context.WithValue(ctx, "user", models.User{})
control := func(ctx context.Context) (interface{}, error) {
return ctx.Value("user").(models.User).Login, nil
}
experiment := scientist.NewQuickExperiment()
experiment.Use(control)
login, err := scientist.RunWithContext(ctx, experiment)
This package was inspired by GitHub's ruby scientist: https://github.com/github/scientist.
Index ¶
- Variables
- func IsBehaviorExist(err error) bool
- func IsControlNotExist(err error) bool
- func IsRecoverFromBadBehavior(err error) bool
- func Run(e Experiment) (interface{}, error)
- func RunWithContext(ctx context.Context, e Experiment) (interface{}, error)
- type Behavior
- type Experiment
- type Facts
- type MismatchError
- type Observation
- type QuickExperiment
- func (e QuickExperiment) Compare(ctx context.Context, control, candidate *Observation) bool
- func (e QuickExperiment) Ignore(ctx context.Context, control, candidate *Observation) bool
- func (e QuickExperiment) IsEnabled(ctx context.Context) bool
- func (e QuickExperiment) Publish(ctx context.Context, result Result) error
- type Result
Constants ¶
This section is empty.
Variables ¶
var ErrorOnMismatch = false
ErrorOnMismatch tells scientist to return errors when experiments have mismatches. Use this to make your tests fail while preserving the control candidate behavior intact in production.
Functions ¶
func IsBehaviorExist ¶
IsBehaviorExist returns true if the error was caused because a behavior with a given name already exists in the experiment.
func IsControlNotExist ¶
IsControlNotExist returns true if the experiment doesn't have any control behavior.
func IsRecoverFromBadBehavior ¶
IsRecoverFromBadBehavior returns true if one of the behaviors panicked.
func Run ¶
func Run(e Experiment) (interface{}, error)
Run executes the experiment and publishes the results. It always returns the result of the control behavior, unless ErrorOnMismatch is true and there are mismatches. The order of execution between control and candidates is always random.
func RunWithContext ¶
func RunWithContext(ctx context.Context, e Experiment) (interface{}, error)
RunWithContext executes the experiment and publishes the results. It allows to set additional information via the context object. It always returns the result of the control behavior, unless ErrorOnMismatch is true and there are mismatches. The order of execution between control and candidates is always random.
Types ¶
type Behavior ¶
Behavior is the type of function that defines how your experiment behaves. See Experiment.Use and Experiment.Try to set those behaviors.
type Experiment ¶
type Experiment interface {
Name() string
Control() Behavior
Shuffle() []string
Behavior(name string) Behavior
IsEnabled(ctx context.Context) bool
Ignore(ctx context.Context, control, candidate *Observation) bool
Compare(ctx context.Context, control, candidate *Observation) bool
Publish(ctx context.Context, result Result) error
}
Experiment is an interface that defines how an experiment behaves.
type Facts ¶
type Facts struct {
// contains filtered or unexported fields
}
Facts holds behavior information for an experiment.
type MismatchError ¶
type MismatchError struct {
// contains filtered or unexported fields
}
MismatchError holds the result information to inspect when observations don't match.
func (MismatchError) Error ¶
func (m MismatchError) Error() string
Error returns the string representation of the MismatchError.
func (MismatchError) MismatchResult ¶
func (m MismatchError) MismatchResult() Result
MismatchResult returns the result of the experiment.
type Observation ¶
type Observation struct {
// Name is the name of the behavior executed.
Name string
// Start is the time when the behavior was executed.
Start time.Time
// Duration is the time that take the behavior to run.
Duration time.Duration
// Value is the value returned by the behavior if any.
Value interface{}
// Error is the error returned by the behavior, if any.
Error error
}
Observation holds information about an executed behavior.
type QuickExperiment ¶
type QuickExperiment struct {
*Facts
}
QuickExperiment is an experiment with a very basic behavior. It's always enabled and it does not publishes results anywhere.
func NewQuickExperiment ¶
func NewQuickExperiment() QuickExperiment
NewQuickExperiment creates a new Experiment with a given name. It creates an empty context for the experiment.
func (QuickExperiment) Compare ¶
func (e QuickExperiment) Compare(ctx context.Context, control, candidate *Observation) bool
Compare returns true if the result of the control behavior is the same as the result of a candidate behavior.
func (QuickExperiment) Ignore ¶
func (e QuickExperiment) Ignore(ctx context.Context, control, candidate *Observation) bool
Ignore returns true if a candidate behavior can be ignored. By default there are no behaviors ignored.
type Result ¶
type Result struct {
// Control is the result of executing the control behavior.
Control *Observation
// Candidates are the results of executing all the candidate behaviors.
Candidates []*Observation
// Mismatches are the results of behaviors that don't match the control.
Mistmaches []*Observation
// Ignored are the results of behaviors that can be ignored.
Ignored []*Observation
// contains filtered or unexported fields
}
Result holds information about an executed experiment.
Source Files
Directories