tests/

README

Preparations

1. The following executables must be copied or generated or linked into these locations.

   • bin/tidb-server can be downloaded from tidb-master-linux-amd64 or installed by tiup, you can use the command find ~/.tiup -name tidb-server to locate tidb-server binary file and copy it
   • bin/sync_diff_inspector # can be downloaded from tidb-enterprise-tools-latest-linux-amd64 or build from source code
   • bin/minio can be build from (https://github.com/minio/minio)
   • bin/dm-master.test # generated by make dm_integration_test_build
   • bin/dm-worker.test # generated by make dm_integration_test_build
   • bin/dm-master.test.current # generated from bin/dm-master.test by make compatibility_test
   • bin/dm-worker.test.current # generated from bin/dm-worker.test by make compatibility_test
   • bin/dm-master.test.previous # generated from bin/dm-master.test by make compatibility_test
   • bin/dm-worker.test.previous # generated from bin/dm-worker.test by make compatibility_test
   • gh-ost # must be added to path, or you can export GHOST_BINARY=/path/to/gh-ost-binary
   • pt-online-schema-change # must be added to path, or you can export PTOSC_BINARY=/path/to/pt-osc-binary
   • GNU sed # must be added to path, BSD sed user should be attention
   • GNU awk # must be added to path

2. The following programs must be installed:

   • mysql (the CLI client, currently not supported for mysql client 8.0)
   • python2.7 or python3.x

3. The user executing the tests must have permission to create the folder /tmp/dm_test. All test artifacts will be written into this folder.

Running

Unit Test

1. Run make unit_test to run unit test

Integration Test

1. Run make dm_integration_test_build to generate DM related binary for integration test

2. Setup two MySQL servers (the first one: 5.6 ~ 5.7; the second one: 8.0.21, suggest you are same as CI) with binlog enabled first and set GTID_MODE=ON, You need set the mysql port and root password according to the following table.

   MySQL	Host	Port	PASSWORD
   MySQL1	127.0.0.1	3306	123456
   MySQL2	127.0.0.1	3307	123456

   For the second MySQL server (with 8.0 version), you should setup it with --default-authentication-plugin=mysql_native_password.

3. Run make integration_test to execute the integration tests. This command will

   1. Check that all required executables exist.
   2. Execute tests/run.sh

   If want to run one integration test case only, just pass the CASE parameter, such as make integration_test CASE=sharding.

   There exists some environment variables that you can set by yourself, including RESET_MASTER,ONLINE_DDL_ENABLE. If RESET_MASTER is not set or set to true, RESET MASTER will be executed at upstream MySQL before each case.

   The online DDL test using pt-osc doesn't work if the upstream MySQL has different connect port and bind port (often caused by port forwarding via NAT). In this case, you must specify the real IP and port of MySQL. Otherwise you can skip online DDL test by export ONLINE_DDL_ENABLE=false.

4. After executing the tests, run make coverage to get a coverage report at /tmp/dm_test/all_cov.html.

Compatibility Test

1. Run Compatibility test

   Run make compatibility_test will run compatibility test.

2. Setup two MySQL servers with binlog enabled first and set GTID_MODE=ON, export proper environment variable. Variables and their default values are showed in following table.

   MySQL	Host	Port	PASSWORD
   MySQL1	MYSQL_HOST1 / 127.0.0.1	MYSQL_PORT1 / 3306	MYSQL_PASSWORD1 / 123456
   MySQL2	MYSQL_HOST2 / 127.0.0.1	MYSQL_PORT2 / 3307	MYSQL_PASSWORD2 / 123456

3. Run other test with different version's dm-master and dm-worker

   Run make compatibility_test CASE={test_case}, for example, make compatibility_test CASE=http_apis will run test http_apis with previous dm-master and current dm-worker, and also current dm-master and previous dm-worker. The test_case can be all, and will run all the test cases.

Writing new tests

1. New integration tests can be written as shell scripts in tests/TEST_NAME/run.sh. The script should exit with a nonzero error code on failure.
2. Add TEST_NAME to existing group in run_group.sh, or add a new group for it.
3. If you add a new group, the name of the new group must be added to CI.
   • dm-integration-test

Note the integration test runs in parallel, and new added test case is not listed in CI, so remember to add the TEST_NAME of new added test case to file others_integration_2.txt in a newline.

Several convenient commands are provided:

• run_dm_master <WORKDIR> <PORT> <CONFIG> — Starts dm-master using config provided, on given port, running in workdir.
• run_dm_worker <WORKDIR> <PORT> <CONFIG> — Starts dm-worker using config provided, on given port, running in workdir.
• run_dm_ctl <WORKDIR> <MASTER_ADDR> <DMCTL_COMMAND> - Runs dmctl with given command in non-interactive mode.
• run_sql <SQL> <PORT> <PASSWORD> — Executes an SQL query in database based on port provided
• run_sql_file <path_to_SQL_file> <HOST> <PORT> <PASSWORD> — Executes all SQLs in given file to the database on port provided
• run_sql_file_online_ddl <path_to_SQL_file> <HOST> <PORT> <PASSWORD> <DB> <ONLINE DDL TOOL> — Executes all SQLs in given file, will auto switch DDL to online DDL command.
• check_contains <TEXT> — Checks if the previous run_sql/run_sql_file result contains the given text (in -E format)
• check_sync_diff <WORKDIR> <CONFIG> - Runs sync_diff_inspector to check diff between databases, using config file provided
• check_rpc_alive <CHECK_TOOL> <ADDR> - Wrapper to check a rpc address is available, at most 10 times.
• check_port_alive <PORT> - Wrapper to check a port is alive, at most 20 times.
• check_port <HOST> <PORT> - Checks a host:port is alive.
• wait_process_exit <process_name> - Wait for one or more processes to exit by given process name.
• check_metric <PORT> <METRIC_NAME> <RETRY_COUNT> <LOWER BOUND> <UPPER BOUND> - check metric value from prometheus.
• truncate_trace_events <PORT> - truncate trace server events records.