convoy

package module
v0.2.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 13, 2022 License: MPL-2.0 Imports: 14 Imported by: 22

README

Convoy

golangci-lint Build and run all tests

convoy image

Convoy is a fast & secure webhooks service. It receives event data from a HTTP API and sends these event data to the configured endpoints. To get started download the openapi spec into Postman or Insomnia.

It includes the following features

  • Sign payload: Configure hash function to use in signing payload.
  • Retry events: Retry events to endpoints.
  • Delivery Attempt Logs: View request headers and body as well as response headers and body.
  • Rich UI: To easily debug and retry failed events.

Install

There are various ways of installing Convoy.

Precompiled binaries

Precompiled binaries for released versions are available in the releases section on Github.

Docker images

Docker images are available on Github Container Registry.

You can launch a Convoy Container to try it out with

$ docker run \
	-p 5005:5005 \
	--name convoy-server \ 
	-v `pwd`/convoy.json:convoy.json \
	ghcr.io/frain-dev/convoy

You can download a sample configuration of convoy.json.

Building from source

To build Convoy from source code, you need:

git clone https://github.com/frain-dev/convoy.git && cd convoy
go build -o convoy ./convoy

Concepts

  1. Apps: An app is an abstraction representing a user who wants to receive webhooks. Currently, an app contains one endpoint to receive webhooks.
  2. Events: An event represents a webhook event to be sent to an app.
  3. Delivery Attempts: A delivery attempt represents an attempt to send an event to it's respective app's endpoint. It contains the event body, status code and response body received on attempt. The amount of attempts on a failed delivery depends on your configured retry strategy.

Configuration

Convoy is configured using a json file with a sample configuration below:

{
	"database": {
		"dsn": "mongo-url-with-username-and-password"
	},
	"queue": {
		"type": "redis",
		"redis": {
			"dsn": "redis-url-with-username-and-password"
		}
	},
	"server": {
		"http": {
			"port": 5005
		}
	},
	"auth": {
		"type": "none"
	},
	"strategy": {
		"type": "default",
		"default": {
			"intervalSeconds": 125,
			"retryLimit": 15
		}
	},
	"signature": {
		"header": "X-Company-Event-Webhook-Signature"
	}
}
Notes to Configuration
  • You can set basic auth mechanism with the following:
{
	"auth": {
		"type": "basic",
		"basic": {
			"username": "username",
			"password": "password"
		}
	}
}

License

Mozilla Public License v2.0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrApplicationNotFound = errors.New("application not found")

	ErrEndpointNotFound = errors.New("endpoint not found")
)
View Source 
var (
	ErrMessageNotFound = errors.New("event not found")

	ErrMessageDeliveryAttemptNotFound = errors.New("delivery attempt not found")
)
View Source 
var DefaultLevels = []log.Level{
	log.ErrorLevel,
	log.PanicLevel,
	log.FatalLevel,
	log.WarnLevel,
}
View Source 
var ErrGroupNotFound = errors.New("group not found")
View Source 
var PeriodValues = map[string]Period{
	"daily":   Daily,
	"weekly":  Weekly,
	"monthly": Monthly,
	"yearly":  Yearly,
}

Functions

func IsValidPeriod added in v0.1.0 

func IsValidPeriod(period string) bool

Types

type AppMetadata added in v0.1.0 

type AppMetadata struct {
	GroupID      string `json:"group_id" bson:"group_id"`
	Secret       string `json:"secret" bson:"secret"`
	SupportEmail string `json:"support_email" bson:"support_email"`

	Endpoints []EndpointMetadata `json:"endpoints" bson:"endpoints"`
}

type Application added in v0.1.0 

type Application struct {
	ID           primitive.ObjectID `json:"-" bson:"_id"`
	UID          string             `json:"uid" bson:"uid"`
	GroupID      string             `json:"group_id" bson:"group_id"`
	Title        string             `json:"name" bson:"title"`
	SupportEmail string             `json:"support_email" bson:"support_email"`

	Secret string `json:"secret" bson:"secret"`

	Endpoints []Endpoint         `json:"endpoints" bson:"endpoints"`
	CreatedAt primitive.DateTime `json:"created_at,omitempty" bson:"created_at,omitempty" swaggertype:"string"`
	UpdatedAt primitive.DateTime `json:"updated_at,omitempty" bson:"updated_at,omitempty" swaggertype:"string"`
	DeletedAt primitive.DateTime `json:"deleted_at,omitempty" bson:"deleted_at,omitempty" swaggertype:"string"`

	Events int64 `json:"events" bson:"-"`

	DocumentStatus DocumentStatus `json:"-" bson:"document_status"`
}

type ApplicationRepository added in v0.1.0 

type ApplicationRepository interface {
	CreateApplication(context.Context, *Application) error
	LoadApplicationsPaged(context.Context, string, models.Pageable) ([]Application, pager.PaginationData, error)
	FindApplicationByID(context.Context, string) (*Application, error)
	UpdateApplication(context.Context, *Application) error
	DeleteApplication(context.Context, *Application) error
	LoadApplicationsPagedByGroupId(context.Context, string, models.Pageable) ([]Application, pager.PaginationData, error)
	SearchApplicationsByGroupId(context.Context, string, models.SearchParams) ([]Application, error)
	FindApplicationEndpointByID(context.Context, string, string) (*Endpoint, error)
	UpdateApplicationEndpointsStatus(context.Context, string, []string, EndpointStatus) error
}

type Datastore added in v0.1.0 

type Datastore interface {
	GroupRepository
	ApplicationRepository
	// EndpointRepository
	io.Closer
	Migrate() error
}

Datastore provides an abstraction for all database related operations

type DocumentStatus added in v0.1.0 

type DocumentStatus string
const (
	ActiveDocumentStatus   DocumentStatus = "Active"
	InactiveDocumentStatus DocumentStatus = "Inactive"
	DeletedDocumentStatus  DocumentStatus = "Deleted"
)

type Endpoint added in v0.1.0 

type Endpoint struct {
	UID         string         `json:"uid" bson:"uid"`
	TargetURL   string         `json:"target_url" bson:"target_url"`
	Description string         `json:"description" bson:"description"`
	Status      EndpointStatus `json:"status" bson:"status"`

	CreatedAt primitive.DateTime `json:"created_at,omitempty" bson:"created_at,omitempty" swaggertype:"string"`
	UpdatedAt primitive.DateTime `json:"updated_at,omitempty" bson:"updated_at,omitempty" swaggertype:"string"`
	DeletedAt primitive.DateTime `json:"deleted_at,omitempty" bson:"deleted_at,omitempty" swaggertype:"string"`

	DocumentStatus DocumentStatus `json:"-" bson:"document_status"`
}

type EndpointMetadata added in v0.1.0 

type EndpointMetadata struct {
	UID       string         `json:"uid" bson:"uid"`
	TargetURL string         `json:"target_url" bson:"target_url"`
	Status    EndpointStatus `json:"status" bson:"status"`

	Sent bool `json:"sent" bson:"sent"`
}

type EndpointStatus added in v0.2.0 

type EndpointStatus string
const (
	ActiveEndpointStatus   EndpointStatus = "active"
	InactiveEndpointStatus EndpointStatus = "inactive"
	PendingEndpointStatus  EndpointStatus = "pending"
)

type EventType added in v0.1.0 

type EventType string

EventType is used to identify an specific event. This could be "user.new" This will be used for data indexing Makes it easy to filter by a list of events

type Group added in v0.2.0 

type Group struct {
	ID      primitive.ObjectID `json:"-" bson:"_id"`
	UID     string             `json:"uid" bson:"uid"`
	Name    string             `json:"name" bson:"name"`
	LogoURL string             `json:"logo_url" bson:"logo_url"`

	CreatedAt primitive.DateTime `json:"created_at,omitempty" bson:"created_at,omitempty" swaggertype:"string"`
	UpdatedAt primitive.DateTime `json:"updated_at,omitempty" bson:"updated_at,omitempty" swaggertype:"string"`
	DeletedAt primitive.DateTime `json:"deleted_at,omitempty" bson:"deleted_at,omitempty" swaggertype:"string"`

	DocumentStatus DocumentStatus `json:"-" bson:"document_status"`
}

func (*Group) IsDeleted added in v0.2.0 

func (o *Group) IsDeleted() bool

func (*Group) IsOwner added in v0.2.0 

func (o *Group) IsOwner(a *Application) bool

type GroupFilter added in v0.2.0 

type GroupFilter struct {
	Name string `json:"name" bson:"name"`
}

type GroupRepository added in v0.2.0 

type GroupRepository interface {
	LoadGroups(context.Context, *GroupFilter) ([]*Group, error)
	CreateGroup(context.Context, *Group) error
	UpdateGroup(context.Context, *Group) error
	FetchGroupByID(context.Context, string) (*Group, error)
}

type HttpHeader added in v0.2.0 

type HttpHeader map[string]string

type HttpMethod 

type HttpMethod string
const (
	HttpPost HttpMethod = "POST"
)

type Message added in v0.1.0 

type Message struct {
	ID        primitive.ObjectID `json:"-" bson:"_id"`
	UID       string             `json:"uid" bson:"uid"`
	AppID     string             `json:"app_id" bson:"app_id"`
	EventType EventType          `json:"event_type" bson:"event_type"`

	// ProviderID is a custom ID that can be used to reconcile this message
	// with your internal systems.
	// This is optional
	// If not provided, we will generate one for you
	ProviderID string `json:"provider_id" bson:"provider_id"`

	// Data is an arbitrary JSON value that gets sent as the body of the
	// webhook to the endpoints
	Data json.RawMessage `json:"data" bson:"data"`

	Metadata *MessageMetadata `json:"metadata" bson:"metadata"`

	Description string `json:"description,omitempty" bson:"description"`

	Status MessageStatus `json:"status" bson:"status"`

	AppMetadata *AppMetadata `json:"app_metadata,omitempty" bson:"app_metadata"`

	MessageAttempts []MessageAttempt `json:"-" bson:"attempts"`

	CreatedAt primitive.DateTime `json:"created_at,omitempty" bson:"created_at,omitempty" swaggertype:"string"`
	UpdatedAt primitive.DateTime `json:"updated_at,omitempty" bson:"updated_at,omitempty" swaggertype:"string"`
	DeletedAt primitive.DateTime `json:"deleted_at,omitempty" bson:"deleted_at,omitempty" swaggertype:"string"`

	DocumentStatus DocumentStatus `json:"-" bson:"document_status"`
}

Message defines a payload to be sent to an application

type MessageAttempt added in v0.1.0 

type MessageAttempt struct {
	ID         primitive.ObjectID `json:"-" bson:"_id"`
	UID        string             `json:"uid" bson:"uid"`
	MsgID      string             `json:"msg_id" bson:"msg_id"`
	URL        string             `json:"url" bson:"url"`
	Method     string             `json:"method" bson:"method"`
	EndpointID string             `json:"endpoint_id" bson:"endpoint_id"`
	APIVersion string             `json:"api_version" bson:"api_version"`

	IPAddress        string        `json:"ip_address,omitempty" bson:"ip_address,omitempty"`
	RequestHeader    HttpHeader    `json:"request_http_header,omitempty" bson:"request_http_header,omitempty"`
	ResponseHeader   HttpHeader    `json:"response_http_header,omitempty" bson:"response_http_header,omitempty"`
	HttpResponseCode string        `json:"http_status,omitempty" bson:"http_status,omitempty"`
	ResponseData     string        `json:"response_data,omitempty" bson:"response_data,omitempty"`
	Error            string        `json:"error,omitempty" bson:"error,omitempty"`
	Status           MessageStatus `json:"status,omitempty" bson:"status,omitempty"`

	CreatedAt primitive.DateTime `json:"created_at,omitempty" bson:"created_at,omitempty" swaggertype:"string"`
	UpdatedAt primitive.DateTime `json:"updated_at,omitempty" bson:"updated_at,omitempty" swaggertype:"string"`
	DeletedAt primitive.DateTime `json:"deleted_at,omitempty" bson:"deleted_at,omitempty" swaggertype:"string"`
}

type MessageMetadata added in v0.1.0 

type MessageMetadata struct {
	Strategy config.StrategyProvider `json:"strategy" bson:"strategy"`
	// NextSendTime denotes the next time a message will be published in
	// case it failed the first time
	NextSendTime primitive.DateTime `json:"next_send_time" bson:"next_send_time"`

	// NumTrials: number of times we have tried to deliver this message to
	// an application
	NumTrials uint64 `json:"num_trials" bson:"num_trials"`

	IntervalSeconds uint64 `json:"interval_seconds" bson:"interval_seconds"`

	RetryLimit uint64 `json:"retry_limit" bson:"retry_limit"`
}

func (MessageMetadata) Value added in v0.1.0 

func (m MessageMetadata) Value() (driver.Value, error)

type MessageRepository added in v0.1.0 

type MessageRepository interface {
	CreateMessage(context.Context, *Message) error
	LoadMessageIntervals(context.Context, string, models.SearchParams, Period, int) ([]models.MessageInterval, error)
	LoadMessagesPagedByAppId(context.Context, string, models.SearchParams, models.Pageable) ([]Message, pager.PaginationData, error)
	FindMessageByID(ctx context.Context, id string) (*Message, error)
	LoadMessagesScheduledForPosting(context.Context) ([]Message, error)
	LoadMessagesForPostingRetry(context.Context) ([]Message, error)
	LoadAbandonedMessagesForPostingRetry(context.Context) ([]Message, error)
	UpdateStatusOfMessages(context.Context, []Message, MessageStatus) error
	UpdateMessageWithAttempt(ctx context.Context, m Message, attempt MessageAttempt) error
	LoadMessagesPaged(context.Context, string, string, models.SearchParams, models.Pageable) ([]Message, pager.PaginationData, error)
}

type MessageStatus added in v0.1.0 

type MessageStatus string
const (
	// ScheduledMessageStatus : when  a message has been scheduled for delivery
	ScheduledMessageStatus  MessageStatus = "Scheduled"
	ProcessingMessageStatus MessageStatus = "Processing"
	DiscardedMessageStatus  MessageStatus = "Discarded"
	FailureMessageStatus    MessageStatus = "Failure"
	SuccessMessageStatus    MessageStatus = "Success"
	RetryMessageStatus      MessageStatus = "Retry"
)

type Period added in v0.1.0 

type Period int
const (
	Daily Period = iota
	Weekly
	Monthly
	Yearly
)

type Plugin 

type Plugin interface {
	Apply(http.ResponseWriter, *http.Request) error
	Name() string
	IsEnabled() bool
}

type SentryHook 

type SentryHook struct {
	LogLevels []log.Level
}

func NewSentryHook 

func NewSentryHook(levels []log.Level) *SentryHook

func (*SentryHook) Fire 

func (s *SentryHook) Fire(entry *log.Entry) error

func (*SentryHook) Levels 

func (s *SentryHook) Levels() []log.Level

type TaskName 

type TaskName string
const (
	EventProcessor      TaskName = "EventProcessor"
	DeadLetterProcessor TaskName = "DeadLetterProcessor"
)

Source Files

View all Source files

Directories

Path Synopsis
algo
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.
redis
models
task

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.