README
¶
natstest
NATS Bus Functional Test Component
- category: Service
- author: Nicola Asuni info@tecnick.com
- copyright: 2016-2017 MIRACL UK LTD
- license: ASL 2.0 - http://www.apache.org/licenses/LICENSE-2.0
- link: https://github.com/tecnickcom/natstest
Description
NATS Bus Test Component.
This program provides a RESTful HTTP JSON API service to test services connected to a NATS bus. It has been designed to test any system composed by services that exchange JSON messages via a NATS bus.
It is capable of sending arbitrary sequences of raw JSON messages to the NATS bus and compare the responses with the expected ones. The test sequences are defined using the JSON format and can be either stored as configuration files or sent via the HTTP interface.
Requirements
An additional Python program is used to check the validity of the JSON configuration files against a JSON schema:
sudo pip install json-spec
Quick Start
This project includes a Makefile that allows you to test and build the project in a Linux-compatible system with simple commands.
All the artifacts and reports produced using this Makefile are stored in the target folder.
All the packages listed in the resources/DockerDev/Dockerfile file are required in order to build and test all the library options in the current environment. Alternatively, everything can be built inside a Docker container using the command "make dbuild".
To see all available options:
make help
To build the project inside a Docker container (requires Docker):
make dbuild
To build a particular set of options inside a Docker container:
MAKETARGET='buildall' make dbuild
The list of pre-defined options can be listed by typing make
The base Docker building environment is defined in the following Dockerfile:
resources/DockerDev/Dockerfile
To execute all the default test builds and generate reports in the current environment:
make qa
To format the code (please use this command before submitting any pull request):
make format
Useful Docker commands
To manually create the container you can execute:
docker build --tag="tecnickcom/natstestdev" .
To log into the newly created container:
docker run -t -i tecnickcom/natstestdev /bin/bash
To get the container ID:
CONTAINER_ID=`docker ps -a | grep tecnickcom/natstestdev | cut -c1-12`
To delete the newly created docker container:
docker rm -f $CONTAINER_ID
To delete the docker image:
docker rmi -f tecnickcom/natstestdev
To delete all containers
docker rm $(docker ps -a -q)
To delete all images
docker rmi $(docker images -q)
Usage
Usage:
natstest [flags]
natstest [command]
Available Commands:
version print this program version
Flags:
-c, --configDir string Configuration directory to be added on top of the search list
-l, --logLevel string Log level: EMERGENCY, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG
-n, --natsAddress string NATS bus Address (nats://ip:port) (default "nats://127.0.0.1:4222")
-s, --serverAddress string HTTP API URL (ip:port) or just (:port) (default ":8081")
Use "natstest [command] --help" for more information about a command.
How it works
The service can be started by issuing the following command (with the right parameters):
natstest --serverAddress=":8080" --natsAddress="nats://127.0.0.1:4222 --logLevel=DEBUG"
If no command-line parameters are specified, then the ones in the configuration file (config.json) will be used.
The configuration files can be stored in the current directory or in any of the following (in order of precedence):
- ./
- config/
- $HOME/natstest/
- /etc/natstest/
This service also support secure remote configuration via Consul or Etcd.
The remote configuration server can be defined either in the local configuration file using the following parameters, or with environment variables:
- remoteConfigProvider : remote configuration source ("consul", "etcd");
- remoteConfigEndpoint : remote configuration URL (ip:port);
- remoteConfigPath : remote configuration path where to search fo the configuration file (e.g. "/config/natstest");
- remoteConfigSecretKeyring : path to the openpgp secret keyring used to decript the remote configuration data (e.g. "/etc/natstest/configkey.gpg"); if empty a non secure connection will be used instead;
The equivalent environment variables are:
- NATSTEST_REMOTECONFIGPROVIDER
- NATSTEST_REMOTECONFIGENDPOINT
- NATSTEST_REMOTECONFIGPATH
- NATSTEST_REMOTECONFIGSECRETKEYRING
Once the service is started without errors, a RESTful HTTP JSON API server will listen on the configured address:port for the following entry points:
| ENTRY POINT | METHOD | DESCRIPTION |
|---|---|---|
| / | GET | return a list of available entry points and tests |
| /status | GET | return the status of the service |
| /test/TESTNAME | GET | execute the test identified by the TESTNAME |
| /test/all | GET | execute all the available tests |
| /reload | GET | reload and reset all test configuration files |
| /new/TESTNAME | PUT | upload and execute a new test |
| /delete/TESTNAME | DELETE | delete the specified test |
The natstest HTTP RESTful API always returns a JSON message with the following fields:
The available tests are specified using JSON configuration files with the following naming syntax:
test_TESTNAME.json
The TESTNAME may contains characters from 'a' to 'z', numbers from '0' to '9' and the character '@' used only for internal testing.
Each configuration file contains a sequence (an array or slice) of RAW messages for the NATS bus with:
- Topic : the message will be processed by the service listening to the specified topic;
- Request : the raw JSON message content to send;
- Response : the expected response message template.
Each field in the Request and Response section of a test message supports templates in addition to fixed values:
-
Regular Expression (only for Response)
A regular expression is identified by the “~re:” prefix.
For example, the following regular expression matches any integer number:
"fieldA" : "~re:[0-9]+" -
External Comparison (only for Response)
An external command-line tool used to compare the expected with the actual value is identified by the “~xc:” prefix, followed by the command (e.g. /usr/bin/mycomparetool), a colon (:) and the string-encoded expected value. The external tool must accept two arguments, the first one is the expected value and the second one is the actual value. The actual value will be automatically JSON-encoded if it is not a simple string.
For example:
"fieldA" : "~xc:/usr/bin/mycomparetool.sh:expected_value" -
TimeStamp
We can add the current UTC timestamp by using the “~ts:” prefix followed by the time format as defined in https://golang.org/pkg/time, or without format to get the Unix timestamp in seconds.
For example:
*"unixTimestamp" : "~ts:",
*"time" : "~ts:15:04:05" -
Previous Value
We can refer to any previously returned value by using the “~pv:” prefix followed by the path to the the reference field.
For example, the following refers to the value of someField in the Response section of the fourth message (the message index starts from zero):
"fieldB" : "~pv:3.Response.someField" -
Tranformed Previous Value
The Previous Values as described above can be transformed by an external command-line application using the syntax as in the following example:
"fieldC" : "~pv:6.Response.anotherField>/bin/echo -n %v"
In this example the command line application is “/bin/echo” and the previous value is passed as argument (”%v” placeholder).
The allowed external command-line applications are defined in the configuration file. If the argument is not a single value, then it will be passed as JSON string.
Command-line API Examples
$ curl -X GET http://127.0.0.1:8080/
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":{"busy":false,"duration":40.08854651,"routes":[{"method":"GET","path":"/status","description":"check this service status"},{"method":"GET","path":"/test/:name","description":"execute the specified test"},{"method":"GET","path":"/reload","description":"reset and reload the test configuration files"},{"method":"PUT","path":"/new/:name","description":"load and execute the specified test configuration"},{"method":"DELETE","path":"/delete/:name","description":"remove the specified test configuration"}],"tests":["@cli","@example","@internal"]}}
$ curl -X GET http://127.0.0.1:8080/status
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":{"busy":false,"duration":40.230439379,"nats":true,"message":"The service is healthy"}}
$ curl -X GET http://127.0.0.1:8080/test/@internal
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":{"tests":1,"duration":0.00665151,"message":"All tests completed successfully"}}
$ curl -H "Content-Type: application/json" -X PUT -d '[{"Topic":"@ab.cd","Request":{"a":"b"},"Response":{"a":"b"}}]' http://127.0.0.1:8080/new/@abcd
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":{"tests":1,"duration":0.00424797,"message":"All tests completed successfully"}}
$ curl -X GET http://127.0.0.1:8080/test/all
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":{"tests":0,"duration":1.8450000000000001e-06,"message":"All tests completed successfully"}}
$ curl -X DELETE http://127.0.0.1:8080/delete/@abcd
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":"the test @abcd has been successfully removed"}
$ curl -X GET http://127.0.0.1:8080/reload
{"program":"natstest","version":"4.7.1","release":"1","datetime":"2016-10-06T15:00:39Z","timestamp":1475766039290853316,"status":"success","code":200,"message":"OK","data":"the test configuration files were successfully reloaded"}
Testing approach
Even if natstest allows to add and delete tests dynamically at runtime, the preferred approach is to store the tests in the configuration directory, so they can be used consistently as regression tests. The ability to add and delete tests via the HTTP interface should be used mainly during the development of the tests and for debugging.
The tests can be triggered manually via the HTTP interface, automatically via CI/CD when a new version of a service is deployed, or periodically using a cron job.
The types of test to be performed can be divided in two categories:
-
Topic testing
The aim of this is to test in isolation each topic of each service attached to the NATS bus.
This requires a test configuration file for each individual service, containing one or more messages for each topic to test. The test should be named: test_servicename.json
This type of test can be seen as analogue to unit testing, so the developer should be responsible for keeping it consistent with the code base.
If a particular topic message requires the information provided by another service and it is not possible to mock it, then it should be tested using the functional testing described hereinafter. -
Functional testing
The aim of this is to test functionalities which involve the use of multiple services.
This requires a test configuration file for each functionality to test. The test should be named: test_functionname.json
As in the traditional functional testing, each test is equivalent to a scenario.
It is likely that the Request messages contains values or transformation of values returned by previous message interactions. In this case the developers should provide the required command-line transformation applications. The full path of each authorized transformation application needs to be added in the "validTransfCmd" list inside the config.json configuration file.
Logs
This service logs the log messages in JSON format. For example:
{"URI":"/new/@alpha","code":200,"datetime":"2016-10-06T15:00:39Z","hostname":"myserver","level":"info","msg":"request","program":"natstest","release":"1","timestamp":1475766039290853316,"type":"PUT","version":"4.7.1"}
Developer(s) Contact
- Nicola Asuni info@tecnick.com
Directories