temporal-sqls

module
v1.16.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2022 License: MIT

README

temporal-sqls

temporal.io server with MS SQL Server persistence

Status

Status: Stable on Windows (compiles, upgraded db schema, all unit test suites green, server runs, tctl runs, wfping sample workflow works). I did no testing on other OSs like linux but there is no reason to assume it won't work, besides SQL Server single sign on maybe.

See tag v1.7 for the previous release of the SQL Server driver based on temporal v1.7, which is quite old now as of May 2022.

Notes for v1.16.1

  • the server appears to be working properly for the v1.16.1 version of temporal as of May 2022
  • temporal moved tctl to its own repository https://github.com/temporalio/tctl, but we keep it at cmd/tctl for now
  • sql scripts syntax for setup and upgrade are OK
  • added cmd wfping which runs a simple workflow with an activity and a signal to have a simple top level integration test
  • the server throws no errors during idle
  • tctl can register a namespace without an error
  • tctl recognizes a duplicate namespace without an error
  • wfping works as expected
  • temporal.io provided test suite for v1.16.1 works with this plugin and has some coverage of ~65% including most sql statements for single server use cases except some operations denoted as DeleteAll*, server replecation seems to have only little coverage: gitlab.com/lercher/temporal-sqls/sqlserver 14.636s coverage: 65.1% of statements
  • This repo adds no own unit tests
TODO
  • tackle server log messages stating deadlock victims. This happens sporadically in visibility at SQL DELETE in a v1.7 method that is no more present in v1.16
  • load testing

Registering the namespace default

1st run

C:\git\src\gitlab.com\lercher\temporal-sqls\cmd\tctl>tctl --ns default namespace register
Namespace default successfully registered.

2nd run

C:\git\src\gitlab.com\lercher\temporal-sqls\cmd\tctl>tctl --ns default namespace register
Error: Register namespace operation failed.
Error Details: rpc error: code = AlreadyExists desc = Namespace already exists.
('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)

which is a good sign.

Notes on Unit Tests

In \git\src\github.com\temporalio\temporal\common\persistence\sql\sqlPersistenceTest.go line 100 we need to change if !strings.HasPrefix(schemaDir, "/") && !strings.HasPrefix(schemaDir, "../") { to

if false && !strings.HasPrefix(schemaDir, "/") && !strings.HasPrefix(schemaDir, "../") {

otherwise (schemaDir is C:/git/... on Windows) we get a concatenation of two absolute file paths to the mock schema reset scripts.

With replace go.temporal.io/server v1.16.1 => ../../../github.com/temporalio/temporal in go.mod and the local sources at git status -> HEAD detached at 6deeed5a6 which is tag v1.16.1, go test works with CGO turned off.

Otherwise we Get

Temporal tests moved after v1.7 from the plugin code out to a central package persistencetests around \git\src\github.com\temporalio\temporal\common\persistence\persistence-tests\postgres_test.go

Current code in this repo leads to:

C:\git\src\gitlab.com\lercher\temporal-sqls>set CGO_ENABLED=1

C:\git\src\gitlab.com\lercher\temporal-sqls>go test ./...
# runtime/cgo
cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
?       gitlab.com/lercher/temporal-sqls/cmd/server     [no test files]
?       gitlab.com/lercher/temporal-sqls/cmd/tctl       [no test files]
?       gitlab.com/lercher/temporal-sqls/instrumentedsql        [no test files]
FAIL    gitlab.com/lercher/temporal-sqls/sqlserver [build failed]
FAIL

C:\git\src\gitlab.com\lercher\temporal-sqls>set CGO_ENABLED=0

C:\git\src\gitlab.com\lercher\temporal-sqls>go test ./...
# gitlab.com/lercher/temporal-sqls/sqlserver
package gitlab.com/lercher/temporal-sqls/sqlserver_test
        imports go.temporal.io/server/common/persistence/persistence-tests
        imports go.temporal.io/server/common/persistence/sql/sqlplugin/sqlite: build constraints exclude all Go files in C:\Users\Megaport\go\pkg\mod\go.temporal.io\server@v1.16.1\common\persistence\sql\sqlplugin\sqlite
FAIL    gitlab.com/lercher/temporal-sqls/sqlserver [setup failed]
?       gitlab.com/lercher/temporal-sqls/cmd/server     [no test files]
?       gitlab.com/lercher/temporal-sqls/cmd/tctl       [no test files]
?       gitlab.com/lercher/temporal-sqls/instrumentedsql        [no test files]
FAIL

So we see a probably left-over CGO dependency.

On the Particular Tests Themselves

Temporal unit tests use a non-idiomatic testing library (github.com/stretchr/testify) and also nearly all tests cases are implemented in a shared temporal package, which is normally a good thing to test plug-ins against the contract. However, inspecting test runs with VS Code is not straigt forward.

Important note: Unit testing erases all temporal data in the test database without question.

License

License is The MIT License

Build

On Windows as follows, on sh use export CGO_ENABLED=0 instead of set ...:

set CGO_ENABLED=0
cd cmd
cd server
go build
cd ..
cd tctl
go build
cd ..
cd wfping
go build

Configuration

See cmd/server/config/development.yaml, esp. the sql section.

For more information see https://github.com/temporalio/temporal/tree/master/config.

Setup

Despite temporal.io's database configuration procedures, this sql server's database needs to be set up via an SQL script:

Running Services Locally

Note: this section is adapted from temporal.io's CONTRIBUTING.md

Run the server:

cd cmd
cd server
go build && server start

Now you can create the default namespace with tctl in a new terminal window:

cd cmd
cd tctl
go build && tctl --ns default namespace register

Then run samples from Go and Java samples repos.

You can also access the web UI at localhost:8088 if the docker image temporalio/web:1.4.0 is started separately.

When you are done, press Ctrl+C to stop the server. Don't forget to stop any dependencies as well such as the web UI.

WFPing

The simple integration test wfping logs sth like this:

C:\git\src\gitlab.com\lercher\temporal-sqls\cmd\wfping>go run .
2022/05/01 19:14:08 This is WFPing. (C) 2022 by Martin Lercher.
2022/05/01 19:14:08 
2022/05/01 19:14:08 It hosts and runs a simple temporal workflow with a trivial action
2022/05/01 19:14:08 and some signal processing. It is usefull to test if a temporal.io
2022/05/01 19:14:08 infrastructure service is up and running.
2022/05/01 19:14:08
2022/05/01 19:14:08 usage: wfping [temporal-frontend-service-host[:port]]
2022/05/01 19:14:08
2022/05/01 19:14:08 Connecting with temporal.io at localhost:7233 ...
2022/05/01 19:14:08 INFO  No logger configured for temporal client. Created default one.
2022/05/01 19:14:09 registering namespace default: Namespace already exists.
2022/05/01 19:14:09 new worker, registering wf and activity ...
2022/05/01 19:14:09 INFO  Started Worker Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@
2022/05/01 19:14:09 Starting the ping WF ...
2022/05/01 19:14:09 Waiting a second to send the signal ...
2022/05/01 19:14:09 INFO  ping-workflow started Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkflowType PingWorkflow WorkflowID ping-workflow-id RunID 7e8f33c3-d803-480c-ac95-dca6bfed8778 Attempt 1 param 
2022-05-01T19:14:09+02:00
2022/05/01 19:14:09 DEBUG ExecuteActivity Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkflowType PingWorkflow WorkflowID ping-workflow-id RunID 7e8f33c3-d803-480c-ac95-dca6bfed8778 Attempt 1 ActivityID 5 ActivityType PingActivity
2022/05/01 19:14:09 INFO  ping-activity called Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ ActivityID 5 ActivityType PingActivity Attempt 1 WorkflowType PingWorkflow WorkflowID ping-workflow-id RunID 7e8f33c3-d803-480c-ac95-dca6bfed8778 value I'm active with the umlauts äöüßÄÖÜ. activity-id 5
2022/05/01 19:14:09 INFO  waiting for signal Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkflowType PingWorkflow WorkflowID ping-workflow-id RunID 7e8f33c3-d803-480c-ac95-dca6bfed8778 Attempt 1
2022/05/01 19:14:10 Sending the signal ..
2022/05/01 19:14:10 Get WF result (max 5s) ...
2022/05/01 19:14:10 INFO  received signal Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkflowType PingWorkflow WorkflowID ping-workflow-id RunID 7e8f33c3-d803-480c-ac95-dca6bfed8778 Attempt 1 value ping-signal-content
2022/05/01 19:14:10 INFO  completed Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkflowType PingWorkflow WorkflowID ping-workflow-id RunID 7e8f33c3-d803-480c-ac95-dca6bfed8778 Attempt 1 return-value ping-wf "2022-05-01T19:14:09+02:00" completed-with: "activity-value:I'm active with the umlauts äöüßÄÖÜ." and signal value ping-signal-content
2022/05/01 19:14:10 WF run OK, result: ping-wf "2022-05-01T19:14:09+02:00" completed-with: "activity-value:I'm active with the umlauts äöüßÄÖÜ." and signal value ping-signal-content
2022/05/01 19:14:10 WARN  Failed to poll for task. Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkerType WorkflowWorker Error worker stopping
2022/05/01 19:14:10 INFO  Stopped Worker Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@
2022/05/01 19:14:10 WARN  Failed to poll for task. Namespace default TaskQueue ping-tq WorkerID 12080@LPOD@ WorkerType ActivityWorker Error worker stopping
2022/05/01 19:14:10 ================= DONE sucessfully =================

Caveat when Changing SQL Statements

SQL queries executed by sqlx's pdb.conn.NamedExecContext(...) need parameters denoted by a colon (:) prefix even though SQL Server uses an at sign (@) for parameters in query text. sqlx compiles the text for the drivers it knows of.

Ressources

Directories

Path Synopsis
cmd
server
tctl
wfping
Package wfping implements a simple temporal workflow with a trivial action and some signal processing.
 Package wfping implements a simple temporal workflow with a trivial action and some signal processing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.