README
¶
runn ( means "Run N" ) is a package/tool for running operations following a scenario.
Key features of runn are:
- As a tool for scenario based testing.
- As a test helper package for the Go language.
- As a tool for workflow automation.
- Support HTTP request, gRPC request, DB query, Chrome DevTools Protocol, and command execution
- OpenAPI Document-like syntax for HTTP request testing.
Quickstart
You can use the runn new command to quickly start creating scenarios (runbooks).
🚀 Create and run scenario using curl or grpcurl commands:
Command details
$ curl https://httpbin.org/json -H "accept: application/json"
{
"slideshow": {
"author": "Yours Truly",
"date": "date of publication",
"slides": [
{
"title": "Wake up to WonderWidgets!",
"type": "all"
},
{
"items": [
"Why <em>WonderWidgets</em> are great",
"Who <em>buys</em> WonderWidgets"
],
"title": "Overview",
"type": "all"
}
],
"title": "Sample Slide Show"
}
}
$ runn new --and-run --desc 'httpbin.org GET' --out http.yml -- curl https://httpbin.org/json -H "accept: application/json"
$ grpcurl -d '{"greeting": "alice"}' grpcb.in:9001 hello.HelloService/SayHello
{
"reply": "hello alice"
}
$ runn new --and-run --desc 'grpcb.in Call' --out grpc.yml -- grpcurl -d '{"greeting": "alice"}' grpcb.in:9001 hello.HelloService/SayHello
$ runn list *.yml
Desc Path If
---------------------------------
grpcb.in Call grpc.yml
httpbin.org GET http.yml
$ runn run *.yml
grpcb.in Call ... ok
httpbin.org GET ... ok
2 scenarios, 0 skipped, 0 failures
🚀 Create scenario using access log:
Command details
$ cat access_log
183.87.255.54 - - [18/May/2019:05:37:09 +0200] "GET /?post=%3script%3ealert(1); HTTP/1.0" 200 42433
62.109.16.162 - - [18/May/2019:05:37:12 +0200] "GET /core/files/js/editor.js/?form=\xeb\x2a\x5e\x89\x76\x08\xc6\x46\x07\x00\xc7\x46\x0c\x00\x00\x00\x80\xe8\xdc\xff\xff\xff/bin/sh HTTP/1.0" 200 81956
87.251.81.179 - - [18/May/2019:05:37:13 +0200] "GET /login.php/?user=admin&amount=100000 HTTP/1.0" 400 4797
103.36.79.144 - - [18/May/2019:05:37:14 +0200] "GET /authorize.php/.well-known/assetlinks.json HTTP/1.0" 200 9436
$ cat access_log| runn new --out axslog.yml
$ cat axslog.yml| yq
desc: Generated by `runn new`
runners:
req: https://dummy.example.com
steps:
- req:
/?post=%3script%3ealert(1);:
get:
body: null
- req:
/core/files/js/editor.js/?form=xebx2ax5ex89x76x08xc6x46x07x00xc7x46x0cx00x00x00x80xe8xdcxffxffxff/bin/sh:
get:
body: null
- req:
/login.php/?user=admin&amount=100000:
get:
body: null
- req:
/authorize.php/.well-known/assetlinks.json:
get:
body: null
$
Usage
runn can run a multi-step scenario following a runbook written in YAML format.
As a tool for scenario based testing / As a tool for automation.
runn can run one or more runbooks as a CLI tool.
$ runn list path/to/**/*.yml
Desc Path If
---------------------------------------------------------------------------------
Login and get projects. pato/to/book/projects.yml
Login and logout. pato/to/book/logout.yml
Only if included. pato/to/book/only_if_included.yml included
$ runn run path/to/**/*.yml
Login and get projects. ... ok
Login and logout. ... ok
Only if included. ... skip
3 scenarios, 1 skipped, 0 failures
As a test helper package for the Go language.
runn can also behave as a test helper for the Go language.
Run N runbooks using httptest.Server and sql.DB
func TestRouter(t *testing.T) {
ctx := context.Background()
dsn := "username:password@tcp(localhost:3306)/testdb"
db, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
dbr, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
ts := httptest.NewServer(NewRouter(db))
t.Cleanup(func() {
ts.Close()
db.Close()
dbr.Close()
})
opts := []runn.Option{
runn.T(t),
runn.Runner("req", ts.URL),
runn.DBRunner("db", dbr),
}
o, err := runn.Load("testdata/books/**/*.yml", opts...)
if err != nil {
t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
t.Fatal(err)
}
}
Run single runbook using httptest.Server and sql.DB
func TestRouter(t *testing.T) {
ctx := context.Background()
dsn := "username:password@tcp(localhost:3306)/testdb"
db, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
dbr, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
ts := httptest.NewServer(NewRouter(db))
t.Cleanup(func() {
ts.Close()
db.Close()
dbr.Close()
})
opts := []runn.Option{
runn.T(t),
runn.Book("testdata/books/login.yml"),
runn.Runner("req", ts.URL),
runn.DBRunner("db", dbr),
}
o, err := runn.New(opts...)
if err != nil {
t.Fatal(err)
}
if err := o.Run(ctx); err != nil {
t.Fatal(err)
}
}
Run N runbooks using grpc.Server
func TestServer(t *testing.T) {
addr := "127.0.0.1:8080"
l, err := net.Listen("tcp", addr)
if err != nil {
t.Fatal(err)
}
ts := grpc.NewServer()
myapppb.RegisterMyappServiceServer(s, NewMyappServer())
reflection.Register(s)
go func() {
s.Serve(l)
}()
t.Cleanup(func() {
ts.GracefulStop()
})
opts := []runn.Option{
runn.T(t),
runn.Runner("greq", fmt.Sprintf("grpc://%s", addr),
}
o, err := runn.Load("testdata/books/**/*.yml", opts...)
if err != nil {
t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
t.Fatal(err)
}
}
Run N runbooks with http.Handler and sql.DB
func TestRouter(t *testing.T) {
ctx := context.Background()
dsn := "username:password@tcp(localhost:3306)/testdb"
db, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
dbr, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
t.Cleanup(func() {
db.Close()
dbr.Close()
})
opts := []runn.Option{
runn.T(t),
runn.HTTPRunnerWithHandler("req", NewRouter(db)),
runn.DBRunner("db", dbr),
}
o, err := runn.Load("testdata/books/**/*.yml", opts...)
if err != nil {
t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
t.Fatal(err)
}
}
Runbook ( runn scenario file )
The runbook file has the following format.
step: section accepts list or ordered map.
List:
desc: Login and get projects.
runners:
req: https://example.com/api/v1
db: mysql://root:mypass@localhost:3306/testdb
vars:
username: alice
password: ${TEST_PASS}
steps:
-
db:
query: SELECT * FROM users WHERE name = '{{ vars.username }}'
-
req:
/login:
post:
body:
application/json:
email: "{{ steps[0].rows[0].email }}"
password: "{{ vars.password }}"
test: steps[1].res.status == 200
-
req:
/projects:
get:
headers:
Authorization: "token {{ steps[1].res.body.session_token }}"
body: null
test: steps[2].res.status == 200
-
test: len(steps[2].res.body.projects) > 0
Map:
desc: Login and get projects.
runners:
req: https://example.com/api/v1
db: mysql://root:mypass@localhost:3306/testdb
vars:
username: alice
password: ${TEST_PASS}
steps:
find_user:
db:
query: SELECT * FROM users WHERE name = '{{ vars.username }}'
login:
req:
/login:
post:
body:
application/json:
email: "{{ steps.find_user.rows[0].email }}"
password: "{{ vars.password }}"
test: steps.login.res.status == 200
list_projects:
req:
/projects:
get:
headers:
Authorization: "token {{ steps.login.res.body.session_token }}"
body: null
test: steps.list_projects.res.status == 200
count_projects:
test: len(steps.list_projects.res.body.projects) > 0
Grouping of related parts by color
List:
Map:
desc:
Description of runbook.
runners:
Mapping of runners that run steps: of runbook.
In the steps: section, call the runner with the key specified in the runners: section.
Built-in runners such as test runner do not need to be specified in this section.
runners:
ghapi: ${GITHUB_API_ENDPOINT}
idp: https://auth.example.com
db: my:dbuser:${DB_PASS}@hostname:3306/dbname
In the example, each runner can be called by ghapi:, idp: or db: in steps:.
vars:
Mapping of variables available in the steps: of runbook.
vars:
username: alice@example.com
token: ${SECRET_TOKEN}
In the example, each variable can be used in {{ vars.username }} or {{ vars.token }} in steps:.
debug:
Enable debug output for runn.
debug: true
if:
Conditions for skip all steps.
if: included # Run steps only if included
skipTest:
Skip all test: sections
skipTest: true
steps:
Steps to run in runbook.
The steps are invoked in order from top to bottom.
Any return values are recorded for each step.
When steps: is array, recorded values can be retrieved with {{ steps[*].* }}.
steps:
-
db:
query: SELECT * FROM users WHERE name = '{{ vars.username }}'
-
req:
/users/{{ steps[0].rows[0].id }}:
get:
body: null
When steps: is map, recorded values can be retrieved with {{ steps.<key>.* }}.
steps:
find_user:
db:
query: SELECT * FROM users WHERE name = '{{ vars.username }}'
user_info:
req:
/users/{{ steps.find_user.rows[0].id }}:
get:
body: null
steps[*].desc: steps.<key>.desc:
Description of step.
steps[*].if: steps.<key>.if:
Conditions for skip step.
steps:
login:
if: 'len(vars.token) == 0' # Run step only if var.token is not set
req:
/login:
post:
body:
[...]
steps[*].loop: steps.<key>.loop:
Loop settings for steps.
Simple loop usage
steps:
multicartin:
loop: 10
req:
/cart/in:
post:
body:
application/json:
product_id: "{{ i }}" # The loop count (0..9) is assigned to `i`.
[...]
or
steps:
multicartin:
loop:
count: 10
req:
/cart/in:
post:
body:
application/json:
product_id: "{{ i }}" # The loop count (0..9) is assigned to `i`.
[...]
Retry
It can be used as a retry mechanism by setting a condition in the until: section.
If the condition of until: is met, the loop is broken without waiting for the number of count: to be run.
Also, if the run of the number of count: completes but does not satisfy the condition of until:, then the step is considered to be failed.
steps:
waitingroom:
loop:
count: 10
until: 'steps.waitingroom.res.status == "201"' # Store values of latest loop
minInterval: 0.5 # sec
maxInterval: 10 # sec
# jitter: 0.0
# interval: 5
# multiplier: 1.5
req:
/cart/in:
post:
body:
[...]
( steps[*].retry: steps.<key>.retry: are deprecated )
Runner
HTTP Runner: Do HTTP request
Use https:// or http:// scheme to specify HTTP Runner.
When the step is invoked, it sends the specified HTTP Request and records the response.
runners:
req: https://example.com
steps:
-
desc: Post /users # description of step
req: # key to identify the runner. In this case, it is HTTP Runner.
/users: # path of http request
post: # method of http request
headers: # headers of http request
Authorization: 'Bearer xxxxx'
body: # body of http request
application/json: # Content-Type specification. In this case, it is "Content-Type: application/json"
username: alice
password: passw0rd
test: | # test for current step
current.res.status == 201
Structure of recorded responses
The following response
HTTP/1.1 200 OK
Content-Length: 29
Content-Type: application/json
Date: Wed, 07 Sep 2022 06:28:20 GMT
{"data":{"username":"alice"}}
is recorded with the following structure.
[`step key` or `current` or `previous`]:
res:
status: 200 # current.res.status
headers:
Content-Length:
- '29' # current.res.headers["Content-Length"][0]
Content-Type:
- 'application/json' # current.res.headers["Content-Type"][0]w
Date:
- 'Wed, 07 Sep 2022 06:28:20 GMT' # current.res.headers["Date"][0]
body:
data:
username: 'alice' # current.res.body.data.username
rawBody: '{"data":{"username":"alice"}}' # current.res.rawBody
Do not follow redirect
The HTTP Runner interprets HTTP responses and automatically redirects.
To disable this, set notFollowRedirect to true.
runners:
req:
endpoint: https://example.com
notFollowRedirect: true
Validation of HTTP request and HTTP response
HTTP requests sent by runn and their HTTP responses can be validated.
OpenAPI v3:
runners:
myapi:
endpoint: https://api.github.com
openapi3: path/to/openapi.yaml
# skipValidateRequest: false
# skipValidateResponse: false
gRPC Runner: Do gRPC request
Use grpc:// scheme to specify gRPC Runner.
When the step is invoked, it sends the specified gRPC Request and records the response.
runners:
greq: grpc://grpc.example.com:80
steps:
-
desc: Request using Unary RPC # description of step
greq: # key to identify the runner. In this case, it is gRPC Runner.
grpctest.GrpcTestService/Hello: # package.Service/Method of rpc
headers: # headers of rpc
authentication: tokenhello
message: # message of rpc
name: alice
num: 3
request_time: 2022-06-25T05:24:43.861872Z
-
desc: Request using Client streaming RPC
greq:
grpctest.GrpcTestService/MultiHello:
headers:
authentication: tokenmultihello
messages: # messages of rpc
-
name: alice
num: 5
request_time: 2022-06-25T05:24:43.861872Z
-
name: bob
num: 6
request_time: 2022-06-25T05:24:43.861872Z
runners:
greq:
addr: grpc.example.com:8080
tls: true
cacert: path/to/cacert.pem
cert: path/to/cert.pem
key: path/to/key.pem
# skipVerify: false
Structure of recorded responses
The following response
message HelloResponse {
string message = 1;
int32 num = 2;
google.protobuf.Timestamp create_time = 3;
}
{"create_time":"2022-06-25T05:24:43.861872Z","message":"hello","num":32}
and headers
content-type: ["application/grpc"]
hello: ["this is header"]
and trailers
hello: ["this is trailer"]
are recorded with the following structure.
[`step key` or `current` or `previous`]:
res:
status: 0 # current.res.status
headers:
content-type:
- 'application/grpc' # current.res.headers[0].content-type
hello:
- 'this is header' # current.res.headers[0].hello
trailers:
hello:
- 'this is trailer' # current.res.trailers[0].hello
message:
create_time: '2022-06-25T05:24:43.861872Z' # current.res.message.create_time
message: 'hello' # current.res.message.message
num: 32 # current.res.message.num
messages:
-
create_time: '2022-06-25T05:24:43.861872Z' # current.res.messages[0].create_time
message: 'hello' # current.res.messages[0].message
num: 32 # current.res.messages[0].num
DB Runner: Query a database
Use dsn (Data Source Name) to specify DB Runner.
When step is executed, it executes the specified query the database.
runners:
db: postgres://dbuser:dbpass@hostname:5432/dbname
steps:
-
desc: Select users # description of step
db: # key to identify the runner. In this case, it is DB Runner.
query: SELECT * FROM users; # query to execute
See testdata/book/db.yml.
Structure of recorded responses
If the query is a SELECT clause, it records the selected rows,
[`step key` or `current` or `previous`]:
rows:
-
id: 1 # current.rows[0].id
username: 'alice' # current.rows[0].username
password: 'passw0rd' # current.rows[0].password
email: 'alice@example.com' # current.rows[0].email
created: '2017-12-05T00:00:00Z' # current.rows[0].created
-
id: 2 # current.rows[1].id
username: 'bob' # current.rows[1].username
password: 'passw0rd' # current.rows[1].password
email: 'bob@example.com' # current.rows[1].email
created: '2022-02-22T00:00:00Z' # current.rows[1].created
otherwise it records last_insert_id and rows_affected .
[`step key` or `current` or `previous`]:
last_insert_id: 3 # current.last_insert_id
rows_affected: 1 # current.rows_affected
Support Databases
PostgreSQL:
runners:
mydb: postgres://dbuser:dbpass@hostname:5432/dbname
runners:
db: pg://dbuser:dbpass@hostname:5432/dbname
MySQL:
runners:
testdb: mysql://dbuser:dbpass@hostname:3306/dbname
runners:
db: my://dbuser:dbpass@hostname:3306/dbname
SQLite3:
runners:
db: sqlite:///path/to/dbname.db
runners:
local: sq://dbname.db
CDP Runner: Control browser using Chrome DevTools Protocol (CDP)
Use cdp:// or chrome:// scheme to specify CDP Runner.
When the step is invoked, it controls browser via Chrome DevTools Protocol.
runners:
cc: chrome://new
steps:
-
desc: Navigate, click and get h1 using CDP # description of step
cc: # key to identify the runner. In this case, it is CDP Runner.
actions: # actions to control browser
- navigate: https://pkg.go.dev/time
- click: 'body > header > div.go-Header-inner > nav > div > ul > li:nth-child(2) > a'
- waitVisible: 'body > footer'
- text: 'h1'
-
test: |
previous.text == 'Install the latest version of Go'
Functions for action to control browser
attributes (aliases: getAttributes, attrs, getAttrs)
Get the element attributes for the first element node matching the selector (sel).
actions:
- attributes:
sel: 'h1'
# record to current.attrs:
or
actions:
- attributes: 'h1'
click
Send a mouse click event to the first element node matching the selector (sel).
actions:
- click:
sel: 'nav > div > a'
or
actions:
- click: 'nav > div > a'
doubleClick
Send a mouse double click event to the first element node matching the selector (sel).
actions:
- doubleClick:
sel: 'nav > div > li'
or
actions:
- doubleClick: 'nav > div > li'
evaluate (aliases: eval)
Evaluate the Javascript expression (expr).
actions:
- evaluate:
expr: 'document.querySelector("h1").textContent = "hello"'
or
actions:
- evaluate: 'document.querySelector("h1").textContent = "hello"'
fullHTML (aliases: getFullHTML, getHTML, html)
Get the full html of page.
actions:
- fullHTML:
# record to current.html:
innerHTML (aliases: getInnerHTML)
Get the inner html of the first element node matching the selector (sel).
actions:
- innerHTML:
sel: 'h1'
# record to current.html:
or
actions:
- innerHTML: 'h1'
location (aliases: getLocation)
Get the document location.
actions:
- location:
# record to current.url:
navigate
Navigate the current frame to url page.
actions:
- navigate:
url: 'https://pkg.go.dev/time'
or
actions:
- navigate: 'https://pkg.go.dev/time'
outerHTML (aliases: getOuterHTML)
Get the outer html of the first element node matching the selector (sel).
actions:
- outerHTML:
sel: 'h1'
# record to current.html:
or
actions:
- outerHTML: 'h1'
screenshot (aliases: getScreenshot)
Take a full screenshot of the entire browser viewport.
actions:
- screenshot:
# record to current.png:
scroll (aliases: scrollIntoView)
Scroll the window to the first element node matching the selector (sel).
actions:
- scroll:
sel: 'body > footer'
or
actions:
- scroll: 'body > footer'
sendKeys
Send keys (value) to the first element node matching the selector (sel).
actions:
- sendKeys:
sel: 'input[name=username]'
value: 'k1lowxb@gmail.com'
setUploadFile (aliases: setUpload)
Set upload file (path) to the first element node matching the selector (sel).
actions:
- setUploadFile:
sel: 'input[name=avator]'
path: '/path/to/image.png'
submit
Submit the parent form of the first element node matching the selector (sel).
actions:
- submit:
sel: 'form.login'
or
actions:
- submit: 'form.login'
text (aliases: getText)
Get the visible text of the first element node matching the selector (sel).
actions:
- text:
sel: 'h1'
# record to current.text:
or
actions:
- text: 'h1'
textContent (aliases: getTextContent)
Get the text content of the first element node matching the selector (sel).
actions:
- textContent:
sel: 'h1'
# record to current.text:
or
actions:
- textContent: 'h1'
title (aliases: getTitle)
Get the document title.
actions:
- title:
# record to current.title:
value (aliases: getValue)
Get the Javascript value field of the first element node matching the selector (sel).
actions:
- value:
sel: 'input[name=address]'
# record to current.value:
or
actions:
- value: 'input[name=address]'
wait (aliases: sleep)
Wait for the specified time.
actions:
- wait:
time: '10sec'
or
actions:
- wait: '10sec'
waitReady
Wait until the element matching the selector (sel) is ready.
actions:
- waitReady:
sel: 'body > footer'
or
actions:
- waitReady: 'body > footer'
waitVisible
Wait until the element matching the selector (sel) is visible.
actions:
- waitVisible:
sel: 'body > footer'
or
actions:
- waitVisible: 'body > footer'
Exec Runner: execute command
The exec runner is a built-in runner, so there is no need to specify it in the runners: section.
It execute command using command: and stdin:
-
exec:
command: grep hello
stdin: '{{ steps[3].res.rawBody }}'
Structure of recorded responses
The response to the run command is always stdout, stderr and exit_code.
[`step key` or `current` or `previous`]:
stdout: 'hello world' # current.stdout
stderr: '' # current.stderr
exit_code: 0 # current.exit_code
Test Runner: test using recorded values
The test runner is a built-in runner, so there is no need to specify it in the runners: section.
It evaluates the conditional expression using the recorded values.
-
test: steps[3].res.status == 200
The test runner can run in the same steps as the other runners.
Dump Runner: dump recorded values
The dump runner is a built-in runner, so there is no need to specify it in the runners: section.
It dumps the specified recorded values.
-
dump: steps[4].rows
or
-
dump:
expr: steps[4].rows
out: path/to/dump.out
The dump runner can run in the same steps as the other runners.
Include Runner: include other runbook
The include runner is a built-in runner, so there is no need to specify it in the runners: section.
Include runner reads and runs the runbook in the specified path.
Recorded values are nested.
-
include: path/to/get_token.yml
It is also possible to override vars: of included runbook.
-
include:
path: path/to/login.yml
vars:
username: alice
password: alicepass
-
include:
path: path/to/login.yml
vars:
username: bob
password: bobpass
It is also possible to skip all test: sections in the included runbook.
-
include:
path: path/to/signup.yml
skipTest: true
Bind Runner: bind variables
The bind runner is a built-in runner, so there is no need to specify it in the runners: section.
It bind runner binds any values with another key.
-
req:
/users/k1low:
get:
body: null
-
bind:
user_id: steps[0].res.body.data.id
-
dump: user_id
The bind runner can run in the same steps as the other runners.
Expression evaluation engine
runn has embedded antonmedv/expr as the evaluation engine for the expression.
See Language Definition.
Built-in functions
urlencode... url.QueryEscapestring... cast.ToStringint... cast.ToIntbool... cast.ToBoolcompare... Compare two values (func(x, y interface{}, ignoreKeys ...string) bool).diff... Difference between two values (func(x, y interface{}, ignoreKeys ...string) string).
Option
See https://pkg.go.dev/github.com/k1LoW/runn#Option
Example: Run as a test helper ( func T )
https://pkg.go.dev/github.com/k1LoW/runn#T
o, err := runn.Load("testdata/**/*.yml", runn.T(t))
if err != nil {
t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
t.Fatal(err)
}
Example: Add custom function ( func Func )
https://pkg.go.dev/github.com/k1LoW/runn#Func
desc: Test using GitHub
runners:
req:
endpoint: https://github.com
steps:
-
req:
/search?l={{ urlencode('C++') }}&q=runn&type=Repositories:
get:
body:
application/json:
null
test: 'steps[0].res.status == 200'
o, err := runn.Load("testdata/**/*.yml", runn.Func("urlencode", url.QueryEscape))
if err != nil {
t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
t.Fatal(err)
}
Filter runbooks to be executed by the environment variable RUNN_RUN
Run only runbooks matching the filename "login".
$ env RUNN_RUN=login go test ./... -run TestRouter
Measure elapsed time as profile
opts := []runn.Option{
runn.T(t),
runn.Book("testdata/books/login.yml"),
runn.Profile(true)
}
o, err := runn.New(opts...)
if err != nil {
t.Fatal(err)
}
if err := o.Run(ctx); err != nil {
t.Fatal(err)
}
f, err := os.Open("profile.json")
if err != nil {
t.Fatal(err)
}
if err := o.DumpProfile(f); err != nil {
t.Fatal(err)
}
or
$ runn run testdata/books/login.yml --profile
The runbook run profile can be read with runn rprof command.
$ runn rprof runn.prof
runbook[login site](t/b/login.yml) 2995.72ms
steps[0].req 747.67ms
steps[1].req 185.69ms
steps[2].req 192.65ms
steps[3].req 188.23ms
steps[4].req 569.53ms
steps[5].req 299.88ms
steps[6].test 0.14ms
steps[7].include 620.88ms
runbook[include](t/b/login_include.yml) 605.56ms
steps[0].req 605.54ms
steps[8].req 190.92ms
[total] 2995.84ms
Capture runbook runs
opts := []runn.Option{
runn.T(t),
runn.Capture(capture.Runbook("path/to/dir")),
}
o, err := runn.Load("testdata/books/**/*.yml", opts...)
if err != nil {
t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
t.Fatal(err)
}
or
$ runn run path/to/**/*.yml --capture path/to/dir
Install
As a CLI tool
deb:
$ export RUNN_VERSION=X.X.X
$ curl -o runn.deb -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.deb
$ dpkg -i runn.deb
RPM:
$ export RUNN_VERSION=X.X.X
$ yum install https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.rpm
apk:
$ export RUNN_VERSION=X.X.X
$ curl -o runn.apk -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.apk
$ apk add runn.apk
homebrew tap:
$ brew install k1LoW/tap/runn
manually:
Download binary from releases page
docker:
$ docker container run -it --rm --name runn -v $PWD:/books ghcr.io/k1low/runn:latest list /books/*.yml
go install:
$ go install github.com/k1LoW/runn/cmd/runn@latest
As a test helper
$ go get github.com/k1LoW/runn
Alternatives
- zoncoen/scenarigo: An end-to-end scenario testing tool for HTTP/gRPC server.
References
- zoncoen/scenarigo: An end-to-end scenario testing tool for HTTP/gRPC server.
- fullstorydev/grpcurl: Like cURL, but for gRPC: Command-line tool for interacting with gRPC servers
Documentation
¶
Index ¶
- Constants
- Variables
- func CACert(path string) grpcRunnerOption
- func CACertFromData(b []byte) grpcRunnerOption
- func Cert(path string) grpcRunnerOption
- func CertFromData(b []byte) grpcRunnerOption
- func CreateHTTPStepMapSlice(key string, req *http.Request) (yaml.MapSlice, error)
- func GetDesc(opt Option) (string, error)
- func Key(path string) grpcRunnerOption
- func KeyFromData(b []byte) grpcRunnerOption
- func Load(pathp string, opts ...Option) (*operators, error)
- func LoadBook(path string) (*book, error)
- func LoadRunbook(in io.Reader) (*runbook, error)
- func New(opts ...Option) (*operator, error)
- func NewCmdOut(out io.Writer) *cmdOut
- func NewDebugger(out io.Writer) *debugger
- func NewRunbook(desc string) *runbook
- func NotFollowRedirect(nf bool) httpRunnerOption
- func OpenApi3(l string) httpRunnerOption
- func OpenApi3FromData(d []byte) httpRunnerOption
- func Paths(pathp string) ([]string, error)
- func ShortenPath(p string) string
- func SkipValidateRequest(skip bool) httpRunnerOption
- func SkipValidateResponse(skip bool) httpRunnerOption
- func TLS(useTLS bool) grpcRunnerOption
- type CDPAction
- type CDPActions
- type CDPArgType
- type CDPFn
- type CDPFnArg
- type CDPFnArgs
- type Capturer
- type DBResponse
- type GRPCOp
- type GRPCType
- type ID
- type IDType
- type IDs
- type Loop
- type Option
- func AfterFunc(fn func() error) Option
- func BeforeFunc(fn func() error) Option
- func Book(path string) Option
- func Books(pathp string) ([]Option, error)
- func Capture(c Capturer) Option
- func DBRunner(name string, client Querier) Option
- func Debug(debug bool) Option
- func Desc(desc string) Option
- func FailFast(enable bool) Option
- func Func(k string, v interface{}) Option
- func GRPCNoTLS(noTLS bool) Option
- func GrpcRunner(name string, cc *grpc.ClientConn, opts ...grpcRunnerOption) Option
- func HTTPRunner(name, endpoint string, client *http.Client, opts ...httpRunnerOption) Option
- func HTTPRunnerWithHandler(name string, h http.Handler, opts ...httpRunnerOption) Option
- func Interval(d time.Duration) Option
- func Overlay(path string) Option
- func Profile(profile bool) Option
- func RunMatch(m string) Option
- func RunParallel(enable bool, max int64) Option
- func RunRandom(n int) Option
- func RunSample(n int) Option
- func RunShard(n, i int) Option
- func RunShuffle(enable bool, seed int64) Option
- func Runner(name, dsn string, opts ...httpRunnerOption) Option
- func SkipIncluded(enable bool) Option
- func SkipTest(enable bool) Option
- func T(t *testing.T) Option
- func Underlay(path string) Option
- func Var(k interface{}, v interface{}) Option
- type Querier
- type RunnerType
- type TxQuerier
- type UnsupportedError
Constants ¶
View Source
const ( MediaTypeApplicationJSON = "application/json" MediaTypeTextPlain = "text/plain" MediaTypeApplicationFormUrlencoded = "application/x-www-form-urlencoded" )
Variables ¶
View Source
var ( AsTestHelper = T Runbook = Book RunPart = RunShard )
View Source
var CDPFnMap = map[string]CDPFn{ "navigate": { Desc: "Navigate the current frame to `url` page.", Fn: chromedp.Navigate, Args: CDPFnArgs{ {CDPArgTypeArg, "url", "https://pkg.go.dev/time"}, }, }, "click": { Desc: "Send a mouse click event to the first element node matching the selector (`sel`).", Fn: chromedp.Click, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "nav > div > a"}, }, }, "doubleClick": { Desc: "Send a mouse double click event to the first element node matching the selector (`sel`).", Fn: chromedp.DoubleClick, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "nav > div > li"}, }, }, "sendKeys": { Desc: "Send keys (`value`) to the first element node matching the selector (`sel`).", Fn: chromedp.SendKeys, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "input[name=username]"}, {CDPArgTypeArg, "value", "k1lowxb@gmail.com"}, }, }, "submit": { Desc: "Submit the parent form of the first element node matching the selector (`sel`).", Fn: chromedp.Submit, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "form.login"}, }, }, "scroll": { Desc: "Scroll the window to the first element node matching the selector (`sel`).", Fn: chromedp.ScrollIntoView, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "body > footer"}, }, Aliases: []string{"scrollIntoView"}, }, "wait": { Desc: "Wait for the specified `time`.", Fn: func(d string) chromedp.Action { return &waitAction{d: d} }, Args: CDPFnArgs{ {CDPArgTypeArg, "time", "10sec"}, }, Aliases: []string{"sleep"}, }, "waitReady": { Desc: "Wait until the element matching the selector (`sel`) is ready.", Fn: chromedp.WaitReady, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "body > footer"}, }, }, "waitVisible": { Desc: "Wait until the element matching the selector (`sel`) is visible.", Fn: chromedp.WaitVisible, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "body > footer"}, }, }, "text": { Desc: "Get the visible text of the first element node matching the selector (`sel`).", Fn: chromedp.Text, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "h1"}, {CDPArgTypeRes, "text", "Install the latest version of Go"}, }, Aliases: []string{"getText"}, }, "textContent": { Desc: "Get the text content of the first element node matching the selector (`sel`).", Fn: chromedp.TextContent, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "h1"}, {CDPArgTypeRes, "text", "Install the latest version of Go"}, }, Aliases: []string{"getTextContent"}, }, "innerHTML": { Desc: "Get the inner html of the first element node matching the selector (`sel`).", Fn: chromedp.InnerHTML, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "h1"}, {CDPArgTypeRes, "html", "Install the latest version of Go"}, }, Aliases: []string{"getInnerHTML"}, }, "outerHTML": { Desc: "Get the outer html of the first element node matching the selector (`sel`).", Fn: chromedp.OuterHTML, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "h1"}, {CDPArgTypeRes, "html", "<h1>Install the latest version of Go</h1>"}, }, Aliases: []string{"getOuterHTML"}, }, "fullHTML": { Desc: "Get the full html of page.", Fn: func(html *string) chromedp.Action { expr := "new XMLSerializer().serializeToString(document);" return chromedp.Evaluate(expr, html) }, Args: CDPFnArgs{ {CDPArgTypeRes, "html", "<!DOCTYPE html><html><body><h1>hello</h1></body></html>"}, }, Aliases: []string{"getFullHTML", "getHTML", "html"}, }, "value": { Desc: "Get the Javascript value field of the first element node matching the selector (`sel`).", Fn: chromedp.Value, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "input[name=address]"}, {CDPArgTypeRes, "value", "Fukuoka"}, }, Aliases: []string{"getValue"}, }, "setUploadFile": { Desc: "Set upload file (`path`) to the first element node matching the selector (`sel`).", Fn: func(sel, path string) chromedp.Action { abs, err := filepath.Abs(path) if err != nil { return &errAction{err: err} } if _, err := os.Stat(abs); err != nil { return &errAction{err: err} } return chromedp.SetUploadFiles(sel, []string{abs}) }, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "input[name=avator]"}, {CDPArgTypeArg, "path", "/path/to/image.png"}, }, Aliases: []string{"setUpload"}, }, "title": { Desc: "Get the document `title`.", Fn: chromedp.Title, Args: CDPFnArgs{ {CDPArgTypeRes, "title", "GitHub"}, }, Aliases: []string{"getTitle"}, }, "location": { Desc: "Get the document location.", Fn: chromedp.Location, Args: CDPFnArgs{ {CDPArgTypeRes, "url", "https://github.com"}, }, Aliases: []string{"getLocation"}, }, "attributes": { Desc: "Get the element attributes for the first element node matching the selector (`sel`).", Fn: chromedp.Attributes, Args: CDPFnArgs{ {CDPArgTypeArg, "sel", "h1"}, {CDPArgTypeRes, "attrs", `{"class": "sr-only"}`}, }, Aliases: []string{"getAttributes", "attrs", "getAttrs"}, }, "screenshot": { Desc: "Take a full screenshot of the entire browser viewport.", Fn: func(b *[]byte) chromedp.Action { return chromedp.FullScreenshot(b, 100) }, Args: CDPFnArgs{ {CDPArgTypeRes, "png", "[]byte"}, }, Aliases: []string{"getScreenshot"}, }, "evaluate": { Desc: "Evaluate the Javascript expression (`expr`).", Fn: func(expr string) chromedp.Action { return chromedp.Evaluate(expr, nil) }, Args: CDPFnArgs{ {CDPArgTypeArg, "expr", `document.querySelector("h1").textContent = "hello"`}, }, Aliases: []string{"eval"}, }, }
Functions ¶
func CACertFromData ¶ added in v0.26.0
func CACertFromData(b []byte) grpcRunnerOption
func CertFromData ¶ added in v0.26.0
func CertFromData(b []byte) grpcRunnerOption
func CreateHTTPStepMapSlice ¶ added in v0.40.0
func KeyFromData ¶ added in v0.26.0
func KeyFromData(b []byte) grpcRunnerOption
func LoadRunbook ¶ added in v0.42.0
func NewDebugger ¶ added in v0.33.0
func NewRunbook ¶ added in v0.40.0
func NewRunbook(desc string) *runbook
func NotFollowRedirect ¶ added in v0.38.0
func NotFollowRedirect(nf bool) httpRunnerOption
func OpenApi3FromData ¶ added in v0.13.0
func OpenApi3FromData(d []byte) httpRunnerOption
func ShortenPath ¶ added in v0.45.0
func SkipValidateRequest ¶ added in v0.13.0
func SkipValidateRequest(skip bool) httpRunnerOption
func SkipValidateResponse ¶ added in v0.13.0
func SkipValidateResponse(skip bool) httpRunnerOption
Types ¶
type CDPActions ¶ added in v0.47.0
type CDPActions []CDPAction
type CDPArgType ¶ added in v0.47.0
type CDPArgType string
const ( CDPArgTypeArg CDPArgType = "arg" CDPArgTypeRes CDPArgType = "res" )
type CDPFnArg ¶ added in v0.47.0
type CDPFnArg struct {
Typ CDPArgType
Key string
Example string
}
type Capturer ¶ added in v0.33.0
type Capturer interface {
CaptureStart(ids IDs, bookPath, desc string)
CaptureFailed(ids IDs, bookPath, desc string, err error)
CaptureSkipped(ids IDs, bookPath, desc string)
CaptureSuccess(ids IDs, bookPath, desc string)
CaptureEnd(ids IDs, bookPath, desc string)
CaptureHTTPRequest(name string, req *http.Request)
CaptureHTTPResponse(name string, res *http.Response)
CaptureGRPCStart(name string, typ GRPCType, service, method string)
CaptureGRPCRequestHeaders(h map[string][]string)
CaptureGRPCRequestMessage(m map[string]interface{})
CaptureGRPCResponseStatus(status int)
CaptureGRPCResponseHeaders(h map[string][]string)
CaptureGRPCResponseMessage(m map[string]interface{})
CaptureGRPCResponseTrailers(t map[string][]string)
CaptureGRPCClientClose()
CaptureGRPCEnd(name string, typ GRPCType, service, method string)
CaptureCDPStart(name string)
CaptureCDPAction(a CDPAction)
CaptureCDPResponse(a CDPAction, res map[string]interface{})
CaptureCDPEnd(name string)
CaptureDBStatement(name string, stmt string)
CaptureDBResponse(name string, res *DBResponse)
CaptureExecCommand(command string)
CaptureExecStdin(stdin string)
CaptureExecStdout(stdin string)
CaptureExecStderr(stderr string)
SetCurrentIDs(ids IDs)
Errs() error
}
type DBResponse ¶ added in v0.33.0
type ID ¶ added in v0.45.0
type ID struct {
Type IDType `json:"type"`
Desc string `json:"desc,omitempty"`
RunbookID string `json:"id,omitempty"`
RunbookPath string `json:"path,omitempty"`
StepKey string `json:"key,omitempty"`
StepRunnerType RunnerType `json:"runner_type,omitempty"`
StepRunnerKey string `json:"runner_key,omitempty"`
FuncIndex int `json:"func_index,omitempty"`
}
ID - ID and context of each element in the runbook
type Loop ¶ added in v0.29.0
type Loop struct {
Count string `yaml:"count,omitempty"`
Interval string `yaml:"interval,omitempty"`
MinInterval string `yaml:"minInterval,omitempty"`
MaxInterval string `yaml:"maxInterval,omitempty"`
Jitter *float64 `yaml:"jitter,omitempty"`
Multiplier *float64 `yaml:"multiplier,omitempty"`
Until string `yaml:"until"`
// contains filtered or unexported fields
}
type Option ¶
type Option func(*book) error
func AfterFunc ¶ added in v0.20.0
AfterFunc - Register the function to be run after the runbook is run.
func BeforeFunc ¶ added in v0.20.0
BeforeFunc - Register the function to be run before the runbook is run.
func GrpcRunner ¶ added in v0.24.0
func GrpcRunner(name string, cc *grpc.ClientConn, opts ...grpcRunnerOption) Option
GrpcRunner - Set grpc runner to runbook
func HTTPRunner ¶
HTTPRunner - Set http runner to runbook
func HTTPRunnerWithHandler ¶ added in v0.6.0
HTTPRunnerWithHandler - Set http runner to runbook with http.Handler
func RunParallel ¶ added in v0.39.0
RunParallel - Parallelize runs of runbooks.
func RunRandom ¶ added in v0.46.0
RunRandom - Run the specified number of runbooks at random. Sometimes the same runbook is run multiple times.
func RunShard ¶ added in v0.23.0
RunShard - Distribute runbooks into a specified number of shards and run the specified shard of them.
func RunShuffle ¶ added in v0.39.0
RunShuffle - Randomize the order of running runbooks.
func SkipIncluded ¶ added in v0.19.0
SkipIncluded - Skip running the included step by itself.
type RunnerType ¶ added in v0.45.0
type RunnerType string
const ( RunnerTypeHTTP RunnerType = "http" RunnerTypeDB RunnerType = "db" RunnerTypeGRPC RunnerType = "grpc" RunnerTypeCDP RunnerType = "cdp" RunnerTypeExec RunnerType = "exec" RunnerTypeTest RunnerType = "test" RunnerTypeDump RunnerType = "dump" RunnerTypeInclude RunnerType = "include" RunnerTypeBind RunnerType = "bind" )
type UnsupportedError ¶ added in v0.17.1
type UnsupportedError struct {
Cause error
}
func (*UnsupportedError) Error ¶ added in v0.17.1
func (e *UnsupportedError) Error() string
func (UnsupportedError) Unwrap ¶ added in v0.17.1
func (e UnsupportedError) Unwrap() error
Source Files
¶
Directories
Click to show internal directories.
Click to hide internal directories.