webhooks

README

Webhooks

• The package is meant to be used across caraml components (e.g. merlin, turing, mlp) to call webhooks when specific events occur.
• The package contains the webhook client implementation and abstracts the logic from the user. It provides some helper functions for the user to call in their code when specific events occur.
• The payload to the webhook server and the response can be arbitrary, and it is up to the user to choose what payload to send to the webhook server(s), but only 1 response will be used in the callback

How to use?

1. In the caller package (eg, mlp, merlin), define the list of events that requires webhooks. For example:

const (
	ProjectCreatedEvent wh.EventType = "OnProjectCreated"
	ProjectUpdatedEvent wh.EventType = "OnProjectUpdated"
)

var EventList = []wh.EventType{
	ProjectCreatedEvent,
	ProjectUpdatedEvent,
}

1. Define the event to webhook configuration. Optionally, the configuration can be provided in a yaml file and parsed via the Config struct. In the config file, define the event to webhook mapping for those events as required. For example, if projects need extra labels from an external source, we define the webhook config for the OnProjectCreated event

webhooks:
  enabled: true
  config:
    OnProjectCreated:
      - url: http://localhost:8081/project_created
        method: POST
        finalResponse: true
        name: webhook1

1. Call InitializeWebhooks() to get a WebhookManager instance. This method will initialize the webhook clients for each event type based on the mapping provided

projectsWebhookManager, err := webhooks.InitializeWebhooks(cfg.Webhooks, service.EventList)

1. Call

InvokeWebhooks(context.Context, EventType, payload interface{}, onSuccess func([]byte) error, onError func(error) error) error

method in the caller code based on the event.

Optional webhooks events

In the event that there are multiple events to be configured, for example OnProjectCreated and OnProjectUpdated, use the IsEventConfigured() method provided by the WebhookManager to check if the event OnProjectUpdated is set before calling InvokeWebhooks() for the OnProjectUpdated. If this check is not performed, the OnProjectUpdated event must always be configured in the webhooks configuration if webhooks are enabled.

For example:

	if webhookManager == nil || !webhookManager.IsEventConfigured(ProjectUpdatedEvent) {
		// do step if webhooks disabled, or event not set
        ...
	} else {
        err := webhookManager.InvokeWebhooks(ctx, ProjectUpdatedEvent, project, func(p []byte) error {
            // onSuccess steps
            ...
        }, webhooks.NoOpErrorHandler)
    }

example config:

webhooks:
  enabled: true
  config:
    OnProjectCreated:
      - url: http://localhost:8081/project_created
        method: POST
        finalResponse: true
        name: webhook1
    OnProjectUpdated: # <-- this must always be set if no check is performed before InvokeWebhooks() is called
      - url: http://localhost:8081/project_updated
        method: POST
        finalResponse: true
        name: webhook2

Checking if the event exists allows users to just specify a subset of the events available, in this case only OnProjectCreated is set.

webhooks:
  enabled: true
  config:
    OnProjectCreated:
      - url: http://localhost:8081/project_created
        method: POST
        finalResponse: true
        name: webhook1

Single Webhook Configuration

webhooks:
  enabled: true
  config:
    OnProjectCreated:
      - name: webhook1
        url: http://webhook1
        method: POST
        finalResponse: true

• This configuration is the most straight forward. It configures 1 webhook client to be called when the OnProjectCreated event happens.
• The payload to the webhook is the json payload of the payload argument passed to InvokeWebhooks.
• The response from this webhook is used as the final response to the callback passed to the onSuccess argument.

Multiple Webhooks use case

• The library supports multiple webhooks per event to a certain extent.

Use case 1

• sync and async webhook
• This can be specified by:

webhooks:
  enabled: true
  config:
    OnProjectCreated:
      - name: webhook1
        url: http://webhook1
        method: POST
        finalResponse: true
      - name: webhook2
        url: http://webhook2
        method: POST
        async: true

• The async webhook2 will be called only after webhook1 completes.
• If there are multiple sync and async webhooks, the async webhooks will be called only after all sync webhooks have completed.

Use case 2

• 3 sync clients, where the response of the first webhook is used as the payload for the second webhook.
• This can be specified by:

webhooks:
  enabled: true
  config:
    OnProjectCreated:
      - url: http://webhook1
        method: POST
        finalResponse: true
        name: webhook1
      - url: http://webhook2
        method: POST
        useDataFrom: webhook1 # <-- specify to use data from webhook1
        name: webhook2
      - url: http://webhook3
        method: POST
        name: webhook3

• The order of webhook matters, webhook1 will be called before webhook2. If webhook2 is defined before webhook1 but uses the response from webhook1, there will be a validation error on initialization.
• Since useDataFrom for webhook1 is not set, webhook1 uses the original payload passed to InvokeWebhooks function.
• webhook2 will use the response from webhook1 as its payload. The response from webhook2 is not used.
• webhook3 will use the same payload as webhook1, but will only be called after webhook2
• Here, the finalResponse is set to true for webhook1. This means that the response from webhook1 will be passed as an argument to the onSuccess function

Error Handling

• For synchronous webhooks, all webhooks must be successful before the onSuccess handler is called. This means that the caller of this package only needs to consider how to handle the successful response.
• In the event any sync webhooks fail, the onError handler is called
• For webhooks that do not need to succeed (for whatever reason), pass them as async webhooks.

Documentation

Index

• func NoOpCallback([]byte) error
• func NoOpErrorHandler(err error) error
• type Config
• type EventType
• type MockWebhookClient
  • func NewMockWebhookClient(t interface{ ... }) *MockWebhookClient
  • func (_m *MockWebhookClient) GetName() string
  • func (_m *MockWebhookClient) GetUseDataFrom() string
  • func (_m *MockWebhookClient) Invoke(_a0 context.Context, _a1 []byte) ([]byte, error)
  • func (_m *MockWebhookClient) IsAsync() bool
  • func (_m *MockWebhookClient) IsFinalResponse() bool
• type MockWebhookManager
  • func NewMockWebhookManager(t interface{ ... }) *MockWebhookManager
  • func (_m *MockWebhookManager) InvokeWebhooks(_a0 context.Context, _a1 EventType, _a2 interface{}, _a3 func([]byte) error, ...) error
• type ServiceType
• type SimpleWebhookManager
  • func (w *SimpleWebhookManager) InvokeWebhooks(ctx context.Context, event EventType, p interface{}, ...) error
  • func (w *SimpleWebhookManager) IsEventConfigured(event EventType) bool
• type WebhookClient
• type WebhookConfig
• type WebhookManager
  • func InitializeWebhooks(cfg *Config, eventList []EventType) (WebhookManager, error)
• type WebhookType

Functions

func NoOpCallback

func NoOpCallback([]byte) error

func NoOpErrorHandler

func NoOpErrorHandler(err error) error

Types

type Config

type Config struct {
	Enabled bool
	Config  map[EventType][]WebhookConfig `validate:"required_if=Enabled True"`
}

Config is a helper struct to define the webhook config in a configuration file

type EventType

type EventType string

type MockWebhookClient

type MockWebhookClient struct {
	mock.Mock
}

MockWebhookClient is an autogenerated mock type for the WebhookClient type

func NewMockWebhookClient

func NewMockWebhookClient(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockWebhookClient

NewMockWebhookClient creates a new instance of MockWebhookClient. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockWebhookClient) GetName

func (_m *MockWebhookClient) GetName() string

GetName provides a mock function with given fields:

func (*MockWebhookClient) GetUseDataFrom

func (_m *MockWebhookClient) GetUseDataFrom() string

GetUseDataFrom provides a mock function with given fields:

func (*MockWebhookClient) Invoke

func (_m *MockWebhookClient) Invoke(_a0 context.Context, _a1 []byte) ([]byte, error)

Invoke provides a mock function with given fields: _a0, _a1

func (*MockWebhookClient) IsAsync

func (_m *MockWebhookClient) IsAsync() bool

IsAsync provides a mock function with given fields:

func (*MockWebhookClient) IsFinalResponse

func (_m *MockWebhookClient) IsFinalResponse() bool

IsFinalResponse provides a mock function with given fields:

type MockWebhookManager

type MockWebhookManager struct {
	mock.Mock
}

MockWebhookManager is an autogenerated mock type for the WebhookManager type

func NewMockWebhookManager

func NewMockWebhookManager(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockWebhookManager

NewMockWebhookManager creates a new instance of MockWebhookManager. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockWebhookManager) InvokeWebhooks

func (_m *MockWebhookManager) InvokeWebhooks(_a0 context.Context, _a1 EventType, _a2 interface{}, _a3 func([]byte) error, _a4 func(error) error) error

InvokeWebhooks provides a mock function with given fields: _a0, _a1, _a2, _a3, _a4

type ServiceType

type ServiceType string

type SimpleWebhookManager

type SimpleWebhookManager struct {
	SyncClients  map[EventType][]WebhookClient
	AsyncClients map[EventType][]WebhookClient
}

func (*SimpleWebhookManager) InvokeWebhooks

func (w *SimpleWebhookManager) InvokeWebhooks(
	ctx context.Context,
	event EventType,
	p interface{},
	onSuccess func([]byte) error,
	onError func(error) error,
) error

InvokeWebhooks iterates through the webhooks for a given event and invokes them. Sync webhooks are called first, and only after all of them succeed, the async webhooks are called. Sync webhooks are called in the order that they are defined. The call order of async webhooks are is not guaranteed. If any of the sync clients are set to abort, the whole chain aborts as long as 1 sync request returns error. onSuccess and onError are callbacks that are called after all webhooks are invoked. For sync clients, the payload can be either the original input payload, or the response from another sync webhook. This can be specified in the UseDataFrom field For async clients, the payload is only the original input payload. Only one webhook's response can be used as the finalResponse

func (*SimpleWebhookManager) IsEventConfigured

func (w *SimpleWebhookManager) IsEventConfigured(event EventType) bool

IsEventConfigured checks if the event is configured in the webhook manager Use this method before calling InvokeWebhooks if it is optional to set webhooks for an event

type WebhookClient

type WebhookClient interface {
	Invoke(context.Context, []byte) ([]byte, error)
	IsAsync() bool
	IsFinalResponse() bool
	GetUseDataFrom() string
	GetName() string
}

type WebhookConfig

type WebhookConfig struct {
	Name        string `yaml:"name"        validate:"required"`
	URL         string `yaml:"url"         validate:"required,url"`
	Method      string `yaml:"method"`
	AuthEnabled bool   `yaml:"authEnabled"`
	AuthToken   string `yaml:"authToken"   validate:"required_if=AuthEnabled True"`
	Async       bool   `yaml:"async"`
	NumRetries  int    `yaml:"numRetries"`
	Timeout     *int   `yaml:"timeout"`
	// UseDataFrom is the name of the webhook whose response will be used as input to this webhook
	UseDataFrom string `yaml:"useDataFrom"`

	// FinalResponse can be set to use the response from this webhook to the onSuccess callback function
	FinalResponse bool `yaml:"finalResponse"`
}

WebhookConfig struct is the configuration for each webhook to be called

type WebhookManager

type WebhookManager interface {
	InvokeWebhooks(
		context.Context,
		EventType,
		interface{},
		func(payload []byte) error,
		func(error) error,
	) error

	IsEventConfigured(EventType) bool
}

func InitializeWebhooks

func InitializeWebhooks(cfg *Config, eventList []EventType) (WebhookManager, error)

InitializeWebhooks is a helper method to initialize a webhook manager based on the eventList provided. It returns an error if the configuration is invalid

type WebhookType

type WebhookType string

const (
	Async WebhookType = "async"
	Sync  WebhookType = "sync"
)