iwf

module
v1.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 9, 2023 License: MIT

README

iWF project - main & server repo

Build status Build status

iWF will make you a 10x developer!

iWF is a platform providing all-in-one tooling for building long-running business application. It provides an abstraction for persistence(database, elasticSearch) and more, with clean, simple and easy to use interface.

It's a simple and powerful WorkflowAsCode general purpose workflow engine. The server is back by Cadence/Temporal as an interpreter, preserved the same power of Cadence/Temporal(including scalability/reliability).

Related projects:

Table of contents

Community & Help

What is iWF

Architecture

An iWF application includes a set of iWF workflow workers which host two REST APIs of WorkflowState start and decide. The application calls an iWF server to operate on workflow executions -- start, stop, signal, get results, etc, using iWF SDKs.

The iWF server hosts those APIs(also REST) as a iWF API service. Internally the API service will call Cadence/Temporal service as the backend.

The iWF server also includes Cadence/Temporal workers which host an interpreter workflow. Internally, any iWF workflows are interpreted into this Cadence/Temporal workflow. The interpreter workflow will invoke the two application worker APIs(start and decide). The API invocations are implemented by Cadence/Temporal activities. Therefore, all the REST API request/response with the worker are stored in history events which are useful for debugging/troubleshooting, and no replay is needed for application workflow code.

architecture diagram

Basic Concepts

Workflow and WorkflowState definition

A long-running process is called Workflow.

iWF lets you build long-running applications by implementing the workflow interface, e.g. Java Workflow interface or Golang Workflow interface. An instance of the interface is a WorkflowDefinition. User applications use IwfWorkflowType to differentiate WorkflowDefinitions.

A WorkflowDefinition contains several WorkflowState e.g. Java WorkflowState interface or Golang WorkflowState interface. A WorkflowState is implemented with two APIs: start and decide.

  • start API is invoked immediately when a WorkflowState is started. It will return some Commands to server. When the requested Commands are completed, decide API will be triggered. The number of commands can be zero, one or multiple.
  • decide API will decide next states to execute. Next states can be zero, one or multiple, and can be re-executed as different stateExecutions.

workflow diagram example

Workflow execution and WorkflowState execution

Application can start a workflow instance with a workflowId for any workflow definition. A workflow instance is called WorkflowExecution. iWF server returns runId of UUID as the identifier of the WorkflowExecution. The runId is globally unique.

⚠ Note:

Depends on the context, the only word workflow may mean WorkflowExecution(most commonly), WorkflowDefinition or both.

For a running WorkflowExecution, there must be at least one WorkflowState being executed, otherwise the workflow execution will complete. An execution instance of WorkflowState is called StateExecution, which by identified StateExecutionId. A StateExecutionId is formatted as <StateId>-<Number>. StateId is defined by workflow state definition, while Number is how many times this StateId has been executed. StateExecutionId is only unique within the workflow execution.

WorkflowId uniqueness and reuse: For the same workflowId, there must be at most one WorkflowExecution running at anytime. However, after a previous WorkflowExecution finished running (in any closed status), application may start a new WorkflowExecution with the same workflowId using appropriate IdReusePolicy.

Commands

These are the three command types:

  • SignalCommand: will be waiting for a signal from external to the workflow signal channel. External application can use SignalWorkflow API to signal a workflow.
  • TimerCommand: will be waiting for a durable timer to fire.
  • InterStateChannelCommand: will be waiting for a value being published from another state in the same workflow execution

Note that start API can return multiple commands, and choose different DeciderTriggerType for triggering decide API:

  • AllCommandCompleted: this will wait for all command completed
  • AnyCommandCompleted: this will wait for any command completed
  • AnyCommandCombinationCompleted: this will wait for a list of command combinations on any combination completed
Persistence

iWF provides super simple persistence abstraction. Developers don't need to touch any database system to register/maintain the schemas. The only schema is defined in the workflow code.

  • DataObject is
    • sharing some data values across the workflow
    • can be retrieved by external application using GetDataObjects API
    • can be viewed in Cadence/Temporal WebUI in QueryHandler tab
  • SearchAttribute is similarly:
    • sharing some data values across the workflow
    • can be retrieved by external application using GetSearchAttributes API
    • search for workflows by external application using SearchWorkflow API
    • search for workflows in Cadence/Temporal WebUI in Advanced tab
    • search attribute type must be registered in Cadence/Temporal server before using for searching because it is backed up ElasticSearch
    • the data types supported are limited as server has to understand the value for indexing
    • See Temporal doc and Cadence doc to understand more about SearchAttribute
  • StateLocal is for
    • passing some data values from state API to decide API in the same WorkflowState execution
  • RecordEvent is for
    • recording some events within the state execution. They are useful for debugging using Workflow history. Usually you may want to record the input/output of the dependency RPC calls.

Logically, each workflow type will have a persistence schema like below:

+-------------+-------+-----------------+-----------------+----------------------+----------------------+-----+
| workflowId  | runId | dataObject key1 | dataObject key2 | searchAttribute key1 | searchAttribute key2 | ... |
+-------------+-------+-----------------+-----------------+----------------------+----------------------+-----+
| your-wf-id1 | uuid1 | valu1           | value2          | keyword-value1       | 123(integer)         | ... |
+-------------+-------+-----------------+-----------------+----------------------+----------------------+-----+
| your-wf-id1 | uuid2 | value3          | value4          | keyword-value2       | 456(integer)         | ... |
+-------------+-------+-----------------+-----------------+----------------------+----------------------+-----+
| your-wf-id2 | uuid3 | value5          | value5          | keyword-value3       | 789(integer)         | ... |
+-------------+-------+-----------------+-----------------+----------------------+----------------------+-----+
| ...         | ...   | ...             | ...             | ...                  | ...                  | ... |
+-------------+-------+-----------------+-----------------+----------------------+----------------------+-----+
Communication

There are two major communication mechanism in iWF:

  • SignalChannel is for receiving input from external asynchronously. It's used with SignalCommand.
  • InterStateChannel: for interaction between state executions. It's used with InterStateChannelCommand.
iWF workflow design diagram

When designing an iWF workflow, it's useful to use iWF state diagrams like this template for visualization.

state diagram template

For example, the subscription workflow diagram:

subscription state diagram

Client APIs

Client APIs are hosted by iWF server for user workflow application to interact with their workflow executions.

  • Start workflow: start a new workflow execution
  • Stop workflow: stop a workflow execution
  • Signal workflow: send a signal to a workflow execution
  • Search workflow: search for workflows using a query language like SQL with search attributes
  • Get workflow: get basic information about a workflow like status and results(if completed or waiting for completed)
  • Get workflow data objects: get the dataObjects of a workflow execution
  • Get workflow search attributes: get the search attributes of a workflow execution
  • Reset workflow: reset a workflow to previous states
  • Skip timer: skip a timer of a workflow (usually for testing or operation)

Why iWF

If you are familiar with Cadence/Temporal/AWS SWF/Azure Durable Functions

That article should still apply to AWS SWF and Azure Durable Functions:

  • AWS SWF is the precedent of Cadence/Temporal with the same APIs but limited features/power compared to Cadence/Temporal.
    • See this post for comparison between SWF vs Cadence
  • Azure Durable Functions shared the same programming model(replay workflow code) but also with limited features compared to Cadence/Temporal.
    • Also read the article about the pitfall about the programming model.

If you are not

  • Check out this article to understand difference between iWF and other workflow engines.

iWF is an application platform that provides you a comprehensive tooling:

  • WorkflowAsCode for highly flexible/customizable business logic, highly testable and easy to maintain
  • Parallel execution of multiple threads of business
  • Persistence storage for intermediate states stored as "dataObjects"
  • Persistence searchable attributes that can be used for flexible searching, even full text searching, backed by ElasticSearch
  • Receiving data from external system by Signal
  • Durable timer, and cron job scheduling
  • Reset workflow to let you recover the workflows from bad states easily
  • Troubleshooting/debugging is easy
  • Scalability/reliability
  • ...

How to run this server

Using docker image & docker-compose

Checkout this repo, go to the docker-compose folder and run it:

cd docker-compose && docker-compose up

This by default will run Temporal server with it. And it will also register a default namespace and required search attributes by iWF. Link to WebUI: http://localhost:8233/namespaces/default/workflows

By default, iWF server is serving port 8801, server URL is http://localhost:8801/ )

NOTE:

Use docker pull iworkflowio/iwf-server:latest to update the latest image.Or update the docker-compose file to specify the version tag.

How to build & run locally

  • Run make bins to build the binary iwf-server
  • Make sure you have registered the system search attributes required by iWF server:
    • Keyword: IwfWorkflowType
    • Int: IwfGlobalWorkflowVersion
    • Keyword: IwfExecutingStateIds
    • See Contribution for more detailed commands.
  • Then run ./iwf-server start to run the service . This defaults to serve workflows APIs with Temporal interpreter implementation. It requires to have local Temporal setup. See Run with local Temporal.
  • Alternatively, run ./iwf-server --config config/development_cadence.yaml start to run with local Cadence. See below instructions for setting up local Cadence.

How to use in production

You can customize the docker image, or just use the api and interpreter that are exposed as the api service and workflow service.

For more info, contact qlong.seattle@gmail.com

Monitoring and Operations

iWF server

There are two components for iWF server: API service and interpreter worker service.

For API service, set up monitors/dashboards:

  • API availability
  • API latency

The interpreter worker service is just a standard Cadence/Temporal workflow application. Follow the developer guides:

iWF application

As you may realize, iWF application is a typical REST microservice. You just need the standard ways to operate it.

Usually, you need to set up monitors/dashboards:

  • API availability
  • API latency

Troubleshooting

When something goes wrong in your applications, here are the tips:

  • Use query handlers like (DumpAllInternal or GetCurrentTimerInfos) in Cadence/Temporal WebUI to quickly understand the current status of the workflows.
    • DumpAllInternal will return all the internal status or the pending states
    • GetCurrentTimerInfos will return all the timers of the pending states
  • Let your worker service return error stacktrace as the response body to iWF server. E.g. like this example of Spring Boot using ExceptionHandler.
  • If you return the full stacktrace in response body, the pending activity view will show it to you! Then use Cadence/Temporal WebUI to debug your application.
  • All the input/output to your workflow are stored in the activity input/output of history event. The input is in ActivityTaskScheduledEvent, output is in ActivityTaskCompletedEvent or in pending activity view if having errors.

Operation

In additional of using Cadence/Temporal CLI, you can just use some HTTP script like this to operate on workflows to:

  • Start a workflow
  • Stop a workflow
  • Reset a workflow
  • Skip a timer
  • etc

How to migrate from Cadence/Temporal

Check this wiki for how to migrate from Cadence/Temporal.

Development Plan

1.0
  • Start workflow API
  • Executing start/decide APIs and completing workflow
  • Parallel execution of multiple states
  • Timer command
  • Signal command
  • SearchAttributeRW
  • DataObjectRW
  • StateLocal
  • Signal workflow API
  • Get DataObjects/SearchAttributes API
  • Get workflow info API
  • Search workflow API
  • Stop workflow API
  • Reset workflow API
  • Command type(s) for inter-state communications (e.g. internal channel)
  • AnyCommandCompleted Decider trigger type
  • More workflow start options: IdReusePolicy, cron schedule, retry
  • StateOption: Start/Decide API timeout and retry policy
  • Reset workflow by stateId or stateExecutionId
  • StateOption.PersistenceLoadingPolicy: LOAD_PARTIAL_WITHOUT_LOCKING
1.1
  • More Search attribute types: Datetime, double, bool, keyword array, text
  • More workflow start options: initial search attributes
1.2
  • Skip timer API for testing/operation
  • Decider trigger type: any command combination
Future
  • Auto continueAsNew(WIP)
  • WaitForMoreResults in StateDecision
  • LongRunningActivityCommand
  • More Decider trigger type
  • Failing workflow details
  • StateOption.PersistenceLoadingPolicy: LOAD_ALL_WITH_EXCLUSIVE_LOCK and LOAD_PARTIAL_WITH_EXCLUSIVE_LOCK

Some history

AWS published SWF in 2012 and then moved to Step Functions in 2016 because they found it’s too hard to support SWF. Cadence & Temporal continued the idea of SWF and became much more powerful. However, AWS is right that the programming of SWF/Cadence/Temporal is hard to adopt because of leaking too many internals. Inspired by Step Function, iWF is created to provide equivalent power of Cadence/Temporal, but hiding all the internal details and provide clean and simple API to use.

Read this doc for more.

history diagram

Posts & Articles

Directories

Path Synopsis
cmd
server
server/iwf
gen
iwfidl
workflow/any_command_close
workflow/any_command_combination
workflow/any_timer_signal
workflow/basic
workflow/common
workflow/interstate
workflow/parallel
workflow/persistence
workflow/signal
workflow/timer
workflow/wf_force_fail
workflow/wf_state_api_fail
workflow/wf_state_api_timeout
api
api/cadence
api/temporal
common/config
common/errors
common/log
common/log/loggerimpl
common/log/tag
common/mapper
common/ptr
common/retry
common/timeparser
interpreter
interpreter/cadence
interpreter/temporal

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.