performance

README

Spanner migration tool: Performance benchmarking

Spanner migration tool (formerly known as HarbourBridge) is a stand-alone open source tool for Cloud Spanner evaluation and migration. This module can be used to run benchmark tests on single table and multiple table database migration using Spanner migration tool.

For executing the benchmark test run the following command:

bash ./performance/test.sh

Steps

The benchmarking script comprises of the following steps:

• Authenticate to gcloud:

  gcloud auth application-default login
  

• Populate database: We can populate either single table database or multiple table database based on the flag and the flag and specify the number of records to be inserted to the database.

  For example, if we want to insert 1000 records to a single table database then run the following command:

  go run ./performance/populate_database/populate_database.go -record-count 1000
  

  If we want to insert 1000 records to a multiple table database then run the below command:

  go run ./performance/populate_database/populate_database.go -record-count 1000 -multiple-table-db
  

• Create spanner instance:

  gcloud spanner instances create new-test-instance --config=regional-us-central1 --description="New test Instance" --nodes=1
  gcloud config set spanner/instance new-test-instance
  

• Run migration: Run migration for the test database created. We can specify the maximum number of writer threads in write-limit flag. Enter the password for connecting to MYSQL db and run the below command.

  go run main.go schema-and-data -source=mysql -source-profile='host=localhost,user=root,dbName=testdb,password='
   -target-profile='instance=new-test-instance,dbName=testdb' -write-limit 40
  

• Cleanup: Once the migration is complete, the last step is cleanup. It involves deleting the spanner instance and droping the MYSQL database.

  For deleting the spanner instance, run:

  gcloud spanner instances delete test-instance
  

  For cleaning up MYSQL resources, run:

  go run ./performance/cleanup_resource/cleanup_resource.go
  

The benchmarking process runs the above steps for different database sizes, spanner instances and different number of writer threads. The time taken for schema and data migration respectively are printed in the report generated after each migration.

Documentation

Overview

Utils.go contains common helper functions used across multiple other packages under performance folder.

Index

• func CurrentTimestamp() string
• func GetMYSQLConnectionStr(server, port, user, password, dbName string) string
• func RandomBool() bool
• func RandomDate() string
• func RandomFloat(min, max int64) float64
• func RandomInt(min, max int64) int64
• func RandomString(n int) string

Functions

func CurrentTimestamp

func CurrentTimestamp() string

func GetMYSQLConnectionStr

func GetMYSQLConnectionStr(server, port, user, password, dbName string) string

Generate MYSQL connection string with server, port, user, password, database specified

func RandomBool

func RandomBool() bool

func RandomDate

func RandomDate() string

Generate random date in the format: 2006-01-02

func RandomFloat

func RandomFloat(min, max int64) float64

Generate random float in the range of (min, max).

func RandomInt

func RandomInt(min, max int64) int64

Generate random integer in the range of (min, max).

func RandomString

func RandomString(n int) string

Generate random string of specific length.