machine

package
v1.19.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 10, 2023 License: MIT Imports: 7 Imported by: 2

README

mkunion and state machines

Package models state machines as a union of states, and transition functions as a union of commands. Package provides an inferring method to visualize state machines as a mermaid diagram.

TODO

  • explain field management
  • dependency injection
  • demonstrate process orchestration
  • demonstrate states managing states
  • visualize error state
  • describe why not to start with state diagram first? TDD, fuzzy, self documenting and context coherence vs switching context
  • describe storage with version use case as optimistic locking

Example

Look into simple_machine_test.go directory for a complete example.

Example implements such state machine:

stateDiagram
	[*] --> "*state.Candidate": "*state.CreateCandidateCMD"
	"*state.Candidate" --> "*state.Unique": "*state.MarkAsUniqueCMD"
	"*state.Candidate" --> "*state.Canonical": "*state.MarkAsCanonicalCMD"
	"*state.Candidate" --> "*state.Duplicate": "*state.MarkAsDuplicateCMD"

Below is a code sample that demonstrate how to implement such state machine in golang. Code shows how to use mkunion to generate union of commands and states

//go:generate mkunion -name State
type (
    Candiate struct {}
    Duplicate struct {}
    Canonical struct {}
    Unique struct {}
)

//go:generate mkunion -name Command
type (
    CreateCandidateCMD struct {}
    MarkAsDuplicateCMD struct {}
    MarkAsCanonicalCMD struct {}
    MarkAsUniqueCMD struct {}
)

var (
    ErrCannotChangeDuplicateToCanonical = errors.New("cannot change duplicate to canonical")
)

func Transition(cmd Command, state State) (State, error) {
    return MustMatchCommandR2(
        cmd,
        func (cmd *CreateCandidateCMD) (State, error) {/* ... */},
        func (cmd *MarkAsDuplicateCMD) (State, error) {
            if state.(*Canonical) {
                return nil, ErrCannotChangeDuplicateToCanonical
            }

            return &Duplicate{}, nil
        }, 
        func (cmd *MarkAsCanonicalCMD) (State, error) {/* .. */}, 
        func (cmd *MarkAsUniqueCMD) (State, error) {/* ... */},
    )
}

Testing state machines & self documenting

Library provides a way to test state machines and generate a mermaid diagram from tests. Above diagram that you see is generated from the following test.

func TestSuite(t *testing.T) {
    suite := machine.NewTestSuite(NewMachine)
    suite.Case(
        "happy path of transitions",
        func(c *machine.Case[Command, State]) {
            c.GivenCommand(&CreateCandidateCMD{ID: "123"}).
                ThenState(&Candidate{ID: "123"}).
                ThenNext("can mark as canonical", func(c *machine.Case[Command, State]) {
                    c.GivenCommand(&MarkAsCanonicalCMD{}).
                        ThenState(&Canonical{ID: "123"})
                }).
                ThenNext("can mark as duplicate", func(c *machine.Case[Command, State]) {
                    c.GivenCommand(&MarkAsDuplicateCMD{CanonicalID: "456"}).
                        ThenState(&Duplicate{ID: "123", CanonicalID: "456"})
                }).
                ThenNext("can mark as unique", func(c *machine.Case[Command, State]) {
                    c.GivenCommand(&MarkAsUniqueCMD{}).
                        ThenState(&Unique{ID: "123"})
                })
        },
    )
    suite.Run(t)
    suite.Fuzzy(t)

    // this line will generate a mermaid diagram, for TDD cycles use line below
    // if true || suite.AssertSelfDocumentStateDiagram(t, "simple_machine_test.go")
    if suite.AssertSelfDocumentStateDiagram(t, "simple_machine_test.go") {
        suite.SelfDocumentStateDiagram(t, "simple_machine_test.go")
    }
}

Infer stat diagram from tests - self documenting

Note in above example function suite.Fuzzy(t). This function explore how machine will act on transition between states, that are not explicitly defined. Using Fuzzy help to discover edge cases, that can be inspected visually.

suite.SelfDocumentStateDiagram will create two diagrams.

  • First one is a diagram of ONLY successful transitions, that are easier to read (first diagram in this document).
  • Second one is a diagram that includes transition that resulted in an error, such diagram is much more visually complex, yet also valuable.
stateDiagram
	[*] --> "*state.Candidate": "*state.CreateCandidateCMD"
 %% error=state is not candidate, state: *state.Canonical; invalid cmds 
	"*state.Canonical" --> "*state.Canonical": "❌*state.MarkAsDuplicateCMD"
 %% error=state is not candidate, state: <nil>; invalid cmds 
	[*] --> [*]: "❌*state.MarkAsDuplicateCMD"
 %% error=candidate already created, state: *state.Canonical; invalid cmds 
	"*state.Canonical" --> "*state.Canonical": "❌*state.CreateCandidateCMD"
 %% error=candidate already created, state: *state.Candidate; invalid cmds 
	"*state.Candidate" --> "*state.Candidate": "❌*state.CreateCandidateCMD"
	"*state.Candidate" --> "*state.Unique": "*state.MarkAsUniqueCMD"
 %% error=candidate already created, state: *state.Unique; invalid cmds 
	"*state.Unique" --> "*state.Unique": "❌*state.CreateCandidateCMD"
 %% error=state is not candidate, state: *state.Unique; invalid cmds 
	"*state.Unique" --> "*state.Unique": "❌*state.MarkAsDuplicateCMD"
 %% error=candidate already created, state: *state.Duplicate; invalid cmds 
	"*state.Duplicate" --> "*state.Duplicate": "❌*state.CreateCandidateCMD"
	"*state.Candidate" --> "*state.Canonical": "*state.MarkAsCanonicalCMD"
 %% error=state is not candidate, state: *state.Unique; invalid cmds 
	"*state.Unique" --> "*state.Unique": "❌*state.MarkAsCanonicalCMD"
 %% error=state is not candidate, state: *state.Unique; invalid cmds 
	"*state.Unique" --> "*state.Unique": "❌*state.MarkAsUniqueCMD"
 %% error=state is not candidate, state: *state.Canonical; invalid cmds 
	"*state.Canonical" --> "*state.Canonical": "❌*state.MarkAsUniqueCMD"
 %% error=state is not candidate, state: *state.Canonical; invalid cmds 
	"*state.Canonical" --> "*state.Canonical": "❌*state.MarkAsCanonicalCMD"
 %% error=state is not candidate, state: *state.Duplicate; invalid cmds 
	"*state.Duplicate" --> "*state.Duplicate": "❌*state.MarkAsCanonicalCMD"
 %% error=state is not candidate, state: <nil>; invalid cmds 
	[*] --> [*]: "❌*state.MarkAsUniqueCMD"
 %% error=state is not candidate, state: <nil>; invalid cmds 
	[*] --> [*]: "❌*state.MarkAsCanonicalCMD"
 %% error=state is not candidate, state: *state.Duplicate; invalid cmds 
	"*state.Duplicate" --> "*state.Duplicate": "❌*state.MarkAsDuplicateCMD"
	"*state.Candidate" --> "*state.Duplicate": "*state.MarkAsDuplicateCMD"
 %% error=state is not candidate, state: *state.Duplicate; invalid cmds 
	"*state.Duplicate" --> "*state.Duplicate": "❌*state.MarkAsUniqueCMD"

Persisting state in database

sequenceDiagram
    participant R as Request
    participant Store as Store

    activate R
    R->>R: Validate(request) -> error
    
    R->>Store: Load state from database by request.ObjectId
    activate Store
    Store->>R: Ok(State)
    deactivate Store
    
    R->>R: Create machine with state
    R->>R: Apply command on a state
    
    R->>Store: Save state in database under request.ObjectId
    activate Store
    Store->>R: Ok()
    deactivate Store
    
    deactivate R

Example implementation of such sequence diagram:

func Handle(rq Request, response Resopnse) {
	ctx := rq.Context()
	
	// extract objectId and command from request + do some validation
    id := rq.ObjectId
	command := rq.Command
	
    // Load state from store
    state, err := store.Find(ctx, id)
	if err != nil { /*handle error*/ }

    machine := NewSimpleMachineWithState(Transition, state)
    newState, err := machine.Apply(cmd, state)
    if err != nil { /*handle error*/ }
	
    err := store.Save(ctx, newState)
    if err != nil { /*handle error*/ }
	
	// serialize response
	response.Write(newState)
}

Error as state. Self-healing systems.

In request-response situation, handing errors is easy, but what if in some long-lived process something goes wrong? How to handle errors in such situation? Without making what we learn about state machines useless or hard to use?

One solution is to treat errors as state. In such case, our state machines will never return error, but instead will return new state, that will represent error.

When we introduce explicit command responsible for correcting RecoverableError, we can create self-healing systems. Thanks to that, even in situation when errors are unknown, we can retroactivly introduce self-healing logic that correct states.

Because there is always there is only one error state, it makes such state machines easy to reason about.

//go:generate mkunion -name State
type (
    // ...
    RecoverableError struct {
        ErrCode int
        PrevState State
        RetryCount int
    }
)

//go:generate mkunion -name Command
type (
    // ...
    CorrectStateCMD struct {}
)

Now, we have to implement recoverable logic in our state machine. We show example above how to do it in Transition function.

Here is example implementation of such transition function:

func Transition(cmd Command, state State) (State, error) {
return MustMatchCommandR2(
    cmd,
    /* ... */
    func(cmd *CorrectStateCMD) (State, error) {
        switch state := state.(type) {
        case *RecoverableError:
            state.RetryCount = state.RetryCount + 1
			
            // here we can do some self-healing logic
            if state.ErrCode == DuplicateServiceUnavailable {
                newState, err := Transition(&MarkAsDuplicateCMD{}, state.PrevState)
                 if err != nil {
                    // we failed to correct error, so we return error state 
                     return &RecoverableError{
                        ErrCode: err,
                        PrevState: state.PrevState,
                        RetryCount: state.RetryCount,
                    }, nil
                }
				
                 // we manage to fix state, so we return new state
                 return newState, nil
             } else {
                 // log information that we have new code, that we don't know how to handle
             }
			
            // try to correct error in next iteration
            return state, nil
        }
    }
}

Now, to correct states we have to select from database all states that are in error state. It can be use in many ways, example below use a abstraction called TaskQueue that is responsible for running tasks in background.

This abstraction guaranties that all records (historical and new ones) will be processed. You can think about it, as a queue that is populated by records from database, that meet SQL query criteria.

You can use CRON job and pull database.

//go:generate mms deployyml -type=TaskQueue -name=CorrectMSPErrors -autoscale=1,10 -memory=128Mi -cpu=100m -timeout=10s -schedule="0 0 * * *"
func main()
    sql := "SELECT * FROM ObjectState WHERE RecoverableError.RetryCount < 3"
    store := datalayer.DefaultStore()
    queue := TaskQueueFrom("correct-msp-errors", sql, store)
    queue.OnTask(func (ctx context.Context, task Task) error {
        state := task.State()
        cmd := &CorrectStateCMD{}
        machine := NewSimpleMachineWithState(Transition, state)
        newState, err := machine.Apply(cmd, state)
        if err != nil {
            return err
        }
        return task.Save(ctx, newState)
    })
    err := queue.Run(ctx)
    if err != nil {
        log.Panic(err)
    }
}

State machines and command queues and workflows

What if command would initiate state "to process" and save it in db What if task queue would take such state and process it Woudn't this be something like command queue?

When to make a list of background processes that transition such states?

processors per state

It's like micromanage TaskQueue, where each state has it's own state, and it knows what command to apply to given state This could be good starting point, when there is not a lot of good tooling

processor for state machine

With good tooling, transition of states can be declared in one place, and deployment to task queue could be done automatically.

Note, that only some of the transitions needs to happen in background, other can be done in request-response manner.

processor for state machine with workflow

State machine could be generalized to workflow. We can think about it as set of generic Command and State (like a turing machine).

States like Pending, Completed, Failed Commands like Process, Retry, Cancel

And workflow DSL with commands like: Invoke, Choose, Assign Where function is some ID string, and functions needs to be either pulled from registry, or called remotely (InvokeRemote). some operations would require callback (InvokeAndAwait)

Then background processor would be responsible for executing such workflow (using task queue) Program would be responsible for defining workflow, and registering functions.

Such programs could be also optimised for deployment, if some function would be better to run on same machine that do RPC call like function doing RPC call to database, and caching result in memory or in cache cluster dedicated to specific BFF

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Case added in v1.17.0 

type Case[TCommand, TState any] struct {
	// contains filtered or unexported fields
}

func (*Case[TCommand, TState]) ForkCase added in v1.17.1 

func (c *Case[TCommand, TState]) ForkCase(name string, definition func(c *Case[TCommand, TState])) *Case[TCommand, TState]

ForkCase takes previous state of machine and allows to apply another case from this point onward there can be many forks from one state

func (*Case[TCommand, TState]) GivenCommand added in v1.17.0 

func (c *Case[TCommand, TState]) GivenCommand(cmd TCommand, opts ...InitCaseOptions) *Case[TCommand, TState]

GivenCommand starts building assertion that when command is applied to machine, it will result in given state or error. Use this method always with ThenState or ThenStateAndError

func (*Case[TCommand, TState]) ThenState added in v1.17.0 

func (c *Case[TCommand, TState]) ThenState(state TState) *Case[TCommand, TState]

ThenState asserts that command applied to machine will result in given state

func (*Case[TCommand, TState]) ThenStateAndError added in v1.17.0 

func (c *Case[TCommand, TState]) ThenStateAndError(state TState, err error)

type InferTransition added in v1.16.1 

type InferTransition[Transition, State any] struct {
	// contains filtered or unexported fields
}

func NewInferTransition added in v1.16.1 

func NewInferTransition[Transition, State any]() *InferTransition[Transition, State]

NewInferTransition creates a helper to infer state machine transitions.

func (*InferTransition[Transition, State]) Record added in v1.16.1 

func (t *InferTransition[Transition, State]) Record(tr Transition, prev, curr State, errAfterTransition error)

Record records a transition.

func (*InferTransition[Transition, State]) ToMermaid added in v1.16.1 

func (t *InferTransition[Transition, State]) ToMermaid() string

ToMermaid returns a string in Mermaid format. https://mermaid-js.github.io/mermaid/#/stateDiagram

func (*InferTransition[Transition, State]) WithErrorTransitions added in v1.16.1 

func (t *InferTransition[Transition, State]) WithErrorTransitions(flag bool) *InferTransition[Transition, State]

func (*InferTransition[Transition, State]) WithTitle added in v1.16.1 

func (t *InferTransition[Transition, State]) WithTitle(name string) *InferTransition[Transition, State]

type InitCaseOptions added in v1.19.0 

type InitCaseOptions func(o *caseOption)

func WithAfter added in v1.19.0 

func WithAfter(f func()) InitCaseOptions

func WithBefore added in v1.19.0 

func WithBefore(f func()) InitCaseOptions

type Machine 

type Machine[C, S any] struct {
	// contains filtered or unexported fields
}

func NewSimpleMachine 

func NewSimpleMachine[C, S any](f func(C, S) (S, error)) *Machine[C, S]

func NewSimpleMachineWithState 

func NewSimpleMachineWithState[C, S any](f func(C, S) (S, error), state S) *Machine[C, S]

func (*Machine[C, S]) Handle 

func (o *Machine[C, S]) Handle(cmd C) error

func (*Machine[C, S]) State 

func (o *Machine[C, S]) State() S

type Suite added in v1.17.0 

type Suite[TCommand, TState any] struct {
	// contains filtered or unexported fields
}

func NewTestSuite added in v1.17.0 

func NewTestSuite[TCommand, TState any](mkMachine func() *Machine[TCommand, TState]) *Suite[TCommand, TState]

func (*Suite[TCommand, TState]) AssertSelfDocumentStateDiagram added in v1.17.0 

func (suite *Suite[TCommand, TState]) AssertSelfDocumentStateDiagram(t *testing.T, baseFileName string) (shouldSelfDocument bool)

AssertSelfDocumentStateDiagram help to self document state machine transitions, just by running tests. It will compare current state diagram with stored in file. It will fail assertion if they are not equal. This may happen, when tests are changed, or state machine is changed. In both then visual inspection of state diagram helps to double-check if changes are correct. And use diagrams in documentation.

If file does not exist, function will return true, to indicate that file should be created. For this purpose call SelfDocumentStateDiagram.

func (*Suite[TCommand, TState]) Case added in v1.17.0 

func (suite *Suite[TCommand, TState]) Case(name string, definition func(c *Case[TCommand, TState]))

func (*Suite[TCommand, TState]) Fuzzy added in v1.17.0 

func (suite *Suite[TCommand, TState]) Fuzzy(t *testing.T)

Fuzzy takes commands and states from recorded transitions and tries to find all possible combinations of commands and states. This can help complete state diagrams with missing transitions, or find errors in state machine that haven't been tested yet. It's useful when connected with AssertSelfDocumentStateDiagram, to automatically update state diagram.

func (*Suite[TCommand, TState]) Run added in v1.17.0 

func (suite *Suite[TCommand, TState]) Run(t *testing.T)

Run runs all test then that describe state machine transitions

func (*Suite[TCommand, TState]) SelfDocumentStateDiagram added in v1.17.0 

func (suite *Suite[TCommand, TState]) SelfDocumentStateDiagram(t *testing.T, baseFileName string)

SelfDocumentStateDiagram help to self document state machine transitions, just by running tests. It will always overwrite stored state diagram files, useful in TDD loop, when tests are being written.

func (*Suite[TCommand, TState]) SelfDocumentTitle added in v1.17.0 

func (suite *Suite[TCommand, TState]) SelfDocumentTitle(title string)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.