rudder-plugins-manager

README

The Customer Data Platform for Developers

Website · Documentation · Community Slack

---

Rudder Plugins Manager

We process different customer events and then send them to 200+ destinations. We need to write custom code specific to the customer events. This library helps to implement such requirements as plugins or workflows. This is designed as per our needs but we believe this is a general tool for defining any custom data pipelines.

Use cases

• Alert Management
  • Different type of alerts may need to be enriched differently
  • Based on context of the alert, alert may need to be routed to different destinations.
• Customer Notifications
  • Similar to alert management, different notifications require different processing.
• Data pipelines
  • If you receive from or deliver to multiple third-party services then this can be used to standardize the incoming event and then do common processing and then again customize outgoing events as per different services API contracts.
• General automation
  • Database cleanup workflows
  • Report generation workflows.

Features

• Plugin Interfaces
• Useful Plugins to get started
  • Base Plugin
  • Bloblang Plugin (using bloblang)
  • Workflow Plugin
    • Supports retrying from last failed step.
  • Retryable Plugin
  • Orchestrator Plugin
• Managers
  • Plugin Manager
  • Workflow Manager

Getting started

• Install go get github.com/rudderlabs/rudder-plugins-manager
• Create a plugin

plugin := plugins.NewBasePlugin("no-op",
		plugins.ExecuteFunc(
			func(ctx context.Context, data *plugins.Message) (*plugins.Message, error) {
				return data, nil
			},
		),
	)
// Add to Manager
pluginManager := plugins.NewBasePluginManager()
pluginManager.Add(plugin)
// Execute Plugin
pluginManager.Execute(context.Background(), "no-op", plugins.NewMessage("some data"))

• Create a workflow

workflow := lo.Must(plugins.NewBaseWorkflowPlugin(pluginManager, plugins.WorkflowConfig{
	Name: "test",
	Steps: []plugins.StepConfig{
		{
			Name:     "blobl",
			Bloblang: `root.test = "test"`,
		},
    {
			Name:     "plugin",
			Plugin: "no-op",
		},
	},
}))

// Add to Manager
workflowManager := plugins.NewBaseWorkflowManager()
workflowManager.Add(workflow)
// Execute Plugin
workflowManager.Execute(context.Background(), "test", plugins.NewMessage("some data"))

Examples

• Base Plugin
• Bloblang Plugin
• Workflow Plugin
  • Sample Workflow
• Retryable Plugin
• Orchestrator Plugin
• Managers
  • Plugin Manager
  • Workflow Manager

Contribute

We would love to see you contribute to RudderStack. Get more information on how to contribute here.

License

The RudderStack Plugins Manager is released under the MIT License.