k6 extension to test and benchmark FrostFS related protocols.
xk6-frostfs
Build
To build a k6
binary with this extension, first ensure you have the prerequisites:
- Install
xk6
framework for extending k6
:
go install go.k6.io/xk6/cmd/xk6@latest
- Clone this repository
git clone git.frostfs.info/TrueCloudLab/xk6-frostfs
cd xk6-frostfs
- Build the binary:
xk6 build --with git.frostfs.info/TrueCloudLab/xk6-frostfs=.
- Run k6:
./k6 run test-script.js
API
Native
Create native client with connect
method. Arguments:
- frostfs storage node endpoint
- hex encoded private key (empty value produces random key)
- dial timeout in seconds (0 for the default value)
- stream timeout in seconds (0 for the default value)
- generate object header on the client side (for big object - split locally too)
- max size for generated object header on the client side (for big object - the size that the object is splitted into)
import native from 'k6/x/frostfs/native';
const frostfs_cli = native.connect("s01.frostfs.devenv:8080", "", 0, 0, false, 0)
Methods
putContainer(params)
. The params
is a dictionary (e.g.
{placement_policy:'REP 3',name:'container-name',name_global_scope:'false'}
).
Returns dictionary with success
boolean flag, container_id
string, and error
string.
setBufferSize(size)
. Sets internal buffer size for data upload and
download. Default is 64 KiB.
put(container_id, headers, payload)
. Returns dictionary with success
boolean flag, object_id
string, and error
string.
get(container_id, object_id)
. Returns dictionary with success
boolean
flag, and error
string.
onsite(container_id, payload)
. Returns FrostFS object instance with prepared
headers. Invoke put(headers)
method on this object to upload it into FrostFS.
It returns dictionary with success
boolean flag, object_id
string and
error
string.
Local
Create a local client with connect
method. Arguments:
- local path to frostfs storage node configuration file
- local path to frostfs storage node configuration directory
- hex encoded private key (empty value produces random key)
- whether to use the debug logger (warning: very verbose)
import local from 'k6/x/frostfs/local';
const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", "", false)
Methods
put(container_id, headers, payload)
. Returns dictionary with success
boolean flag, object_id
string, and error
string.
get(container_id, object_id)
. Returns dictionary with success
boolean
flag, and error
string.
delete(container_id, object_id)
. Returns dictionary with success
boolean
flag, and error
string.
S3
Create s3 client with connect
method. Arguments:
Credentials are taken from default AWS configuration files and ENVs.
import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080")
You can also provide additional options:
import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080", {'no_verify_ssl': 'true', 'timeout': '60s'})
no_verify_ss
- Bool. If true
- skip verifying the s3 certificate chain and host name (useful if s3 uses self-signed certificates)
timeout
- Duration. Set timeout for requests (in http client). If omitted or zero - timeout is infinite.
Methods
createBucket(bucket, params)
. Returns dictionary with success
boolean flag
and error
string. The params
is a dictionary (e.g. {acl:'private',lock_enabled:'true',location_constraint:'ru'}
)
put(bucket, key, payload)
. Returns dictionary with success
boolean flag
and error
string.
get(bucket, key)
. Returns dictionary with success
boolean flag and error
string.
S3 Local
Create local s3 client with connect
method. Arguments:
- local path to frostfs storage node configuration file
- local path to frostfs storage node configuration directory
- parameter map with the following options:
hex_key
: private key to use as a hexadecimal string. A random one is created if none is provided.
node_position
: position of this node in the node array if loading multiple nodes independently (default: 0).
node_count
: number of nodes in the node array if loading multiple nodes independently (default: 1).
debug_logger
: whether to use the development logger instead of the default. Helpful for debugging (default: false).
- bucket-container mapping, which is needed to resolve the container id for a given bucket name. Any bucket
used by the client must have an entry here.
import local from 'k6/x/frostfs/local';
const params = {'node_position': 1, 'node_count': 3}
const bucketMapping = {'mytestbucket': 'GBQDDUM1hdodXmiRHV57EUkFWJzuntsG8BG15wFSwam6'}
const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", params, bucketMapping)
Methods
put(bucket, key, payload)
. Returns dictionary with success
boolean flag
and error
string.
get(bucket, key)
. Returns dictionary with success
boolean flag and error
string.
Examples
See native protocol and s3 test suite examples in examples dir.
Command line utils
To build all command line utils just run:
$ make
All binaries will be in bin
directory.
Export registry db
You can export registry bolt db to json file, that can be used as pregen for scenarios (see docs).
To do this use frostfs-xk6-registry-exporter
, available flags can be seen in help:
$ ./bin/frostfs-xk6-registry-exporter -h
Registry exporter for xk6
Usage:
registry-exporter [flags]
Examples:
registry-exporter registry.bolt
registry-exporter --status created --out out.json registry.bolt
Flags:
--age int Object age
--format string Output format (default "json")
-h, --help help for registry-exporter
--out string Path to output file (default "dumped-registry.json")
--status string Object status (default "created")
-v, --version version for registry-exporter
Import pregen into registry db
You can import pregenerated json files into registry bolt db. Use frostfs-xk6-registry import
. Usage examples are in help:
$ ./bin/frostfs-xk6-registry import -h
Import objects into registry from pregenerated files
Usage:
xk6-registry import [flags]
Examples:
xk6-registry import registry.bolt preset.json
xk6-registry import registry.bolt preset.json another_preset.json
Flags:
-h, --help help for import
License