README
¶
(Multi Variate Testing) interpreter for PlanOut code written in Golang
What what ?
PlanOut is a framework for providing randomised parameter assignment for controlling parameters and defaults used in code. It exists as a combination of both a generalised methodology and as a DSL for constructing online field experiments.
An excellent introduction can be found both in the original research, Designing and Deploying Online Field Experiments (Bakshy, Eckles and Bernstein), and in the following lecture https://www.youtube.com/watch?v=Ayd4sqPH2DE.
So what is this ?
This is an interpreter that provides the basic functionality for running PlanOut interpreter code, allowing for integrating experiments into GoLang applications.
This is not a full implementation of a complete PlanOut stack, as such it lacks the compiler needed to turn PlanOut DSL into the interpreter code, as well as the general test management tooling needed.
Much of the additional tooling for PlanOut can be found the in original project.
This code will however run PlanOut programs in an idiomatic Golang fashion.
How to run a basic experiment ?
Here's an example program that consumes compiled PlanOut code and executes the associated experiment using the Golang interpreter.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"math/rand"
"github.com/URXtech/planout-golang"
)
// Helper function to generate random string.
func generateString() string {
s := make([]byte, 10)
for j := 0; j < 10; j++ {
s[j] = 'a' + byte(rand.Int()%26)
}
return string(s)
}
func main() {
// Read PlanOut code from file on disk.
data, _ := ioutil.ReadFile("test/simple_ops.json")
// The PlanOut code is expected to use json.
// This format is the same as the output of
// the PlanOut compiler webapp
// http://facebook.github.io/planout/demo/planout-compiler.html
var js map[string]interface{}
json.Unmarshal(data, &js)
// Set the necessary input parameters required to run
// the experiments. For instance, simple_ops.json expects
// the value for 'userid' to be set.
params := make(map[string]interface{})
params["experiment_salt"] = "expt"
params["userid"] = generateString()
// Construct an instance of the Interpreter object.
// Initialize Salt and set Inputs to params.
expt := &planout.Interpreter{
Salt: "global_salt",
Evaluated: false,
Inputs: params,
Outputs: map[string]interface{}{},
Overrides: map[string]interface{}{},
Code: js,
}
// Call the Run() method on the Interpreter instance.
// The output of the run will contain the dictionary
// of variables and associated values that were evaluated
// as part of the experiment.
output, ok := expt.Run()
if !ok {
fmt.Println("Failed to run the experiment")
} else {
fmt.Printf("Params: %v\n", params)
}
fmt.Println(output)
}
Suppose we want to run the following experiment:
id = uniformChoice(choices=[1, 2, 3, 4], unit=userid);
The PlanOut code generated by the compiler looks like:
{
"op": "seq",
"seq": [
{
"op": "set",
"var": "id",
"value": {
"choices": {
"op": "array",
"values": [
1,
2,
3,
4
]
},
"unit": {
"op": "get",
"var": "userid"
},
"op": "uniformChoice"
}
}
]
}
Each execution of the above experiment will result in setting the variable 'id'. The output to stdout will look like:
Params: map[experiment_salt:expt userid:noocavzddw salt:id id:2]
Params: map[experiment_salt:expt userid:cuncjyqmmz salt:id id:1]
How to run a experiments in an allocated namespace ?
This example consumes multiple compiled PlanOut experiments and executes within a namespace.
package main
func main() {
js1 := readTest("test/simple_ops.json")
js2 := readTest("test/random_ops.json")
js3 := readTest("test/simple.json")
inputs := make(map[string]interface{})
inputs["userid"] = "test-id"
n := planout.NewSimpleNamespace("simple_namespace", 100, "userid", inputs)
n.AddExperiment("simple ops", js1, 10)
n.AddExperiment("random ops", js2, 10)
n.AddExperiment("simple", js3, 80)
out, ok = := n.Run()
}
Documentation
¶
Index ¶
- func FisherYatesShuffle(inputs []interface{}, nhash ...uint64)
- type Interpreter
- func (interpreter *Interpreter) Evaluate(code interface{}) interface{}
- func (interpreter *Interpreter) Get(name string) (interface{}, bool)
- func (interpreter *Interpreter) GetOperator(expr interface{}) (Operator, bool)
- func (interpreter *Interpreter) Run(force ...bool) (map[string]interface{}, bool)
- type Namespace
- type Operator
- type PlanOutCode
- type SimpleNamespace
- type TypedMap
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func FisherYatesShuffle ¶
func FisherYatesShuffle(inputs []interface{}, nhash ...uint64)
Types ¶
type Interpreter ¶
type Interpreter struct {
Name string
Salt string
Inputs, Outputs, Overrides map[string]interface{}
Code interface{}
Evaluated, InExperiment bool
ParameterSalt string
OperatorOverrides map[string]Operator
}
func (*Interpreter) Evaluate ¶
func (interpreter *Interpreter) Evaluate(code interface{}) interface{}
func (*Interpreter) Get ¶
func (interpreter *Interpreter) Get(name string) (interface{}, bool)
func (*Interpreter) GetOperator ¶
func (interpreter *Interpreter) GetOperator(expr interface{}) (Operator, bool)
type Operator ¶
type Operator interface {
Execute(map[string]interface{}, *Interpreter) interface{}
}
type PlanOutCode ¶
type SimpleNamespace ¶
type SimpleNamespace struct {
Name string
PrimaryUnit string
NumSegments int
Inputs map[string]interface{}
// contains filtered or unexported fields
}
func NewSimpleNamespace ¶
func NewSimpleNamespace(name string, numSegments int, primaryUnit string, inputs map[string]interface{}) SimpleNamespace
func (*SimpleNamespace) AddDefaultExperiment ¶
func (n *SimpleNamespace) AddDefaultExperiment(defaultExperiment *Interpreter)
func (*SimpleNamespace) AddExperiment ¶
func (n *SimpleNamespace) AddExperiment(name string, interpreter *Interpreter, segments int) error
func (*SimpleNamespace) RemoveExperiment ¶
func (n *SimpleNamespace) RemoveExperiment(name string) error
func (*SimpleNamespace) Run ¶
func (n *SimpleNamespace) Run() *Interpreter
Source Files