postmand

package module

v1.8.1 Latest Latest Go to latest Published: Oct 16, 2024 License: MIT Imports: 7 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/allisson/postmand

Links

Open Source Insights

README ¶

postmand

Simple webhook delivery system powered by Golang and PostgreSQL.

Features

Simple rest api with only three endpoints (webhooks/deliveries/delivery-attempts).
Select the status codes that are considered valid for a delivery.
Control the maximum amount of delivery attempts and delay between these attempts (min and max backoff).
Locks control of worker deliveries using PostgreSQL SELECT FOR UPDATE SKIP LOCKED.
Sending the X-Hub-Signature header if the webhook is configured with a secret token.
Simplicity, it does the minimum necessary, it will not have authentication/permission scheme among other things, the idea is to use it internally in the cloud and not leave exposed.

Quickstart

Let's start with the basic concepts, we have three main entities that we must know to start:

Webhook: The configuration of the webhook.
Delivery: The content sent to a webhook.
Delivery Attempt: An attempt to deliver the content to the webhook.

Run the server

To run the server it is necessary to have a database available from postgresql, in this example we will consider that we have a database called postmand running in localhost with user and password equal to user.

docker run --name postgres --restart unless-stopped -e POSTGRES_USER=user -e POSTGRES_PASSWORD=pass -e POSTGRES_DB=postmand -p 5432:5432 -d postgres:12-alpine

Docker

docker run --rm --env POSTMAND_DATABASE_URL='postgres://user:password@host.docker.internal:5432/postmand?sslmode=disable' quay.io/allisson/postmand migrate # create database schema

docker run -p 8000:8000 -p 8001:8001 --env POSTMAND_DATABASE_URL='postgres://user:password@host.docker.internal:5432/postmand?sslmode=disable' quay.io/allisson/postmand server # run the server

Local

Install just command runner: https://github.com/casey/just?tab=readme-ov-file#installation

git clone https://github.com/allisson/postmand
cd postmand
cp local.env .env # and edit .env
just db-migrate # create database schema
just run-server # run the server
just run-worker

Run the worker

The worker is responsible to delivery content to the webhooks.

Docker

docker run --env POSTMAND_DATABASE_URL='postgres://user:pass@host.docker.internal:5432/postmand?sslmode=disable' quay.io/allisson/postmand worker

Local

just run-worker
go run cmd/postmand/main.go worker

Create a new webhook

The fields delivery_attempt_timeout/retry_min_backoff/retry_max_backoff are in seconds.

curl --location --request POST 'http://localhost:8000/v1/webhooks' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "Httpbin Post",
    "url": "https://httpbin.org/post",
    "content_type": "application/json",
    "valid_status_codes": [
        200,
        201
    ],
    "secret_token": "my-secret-token",
    "active": true,
    "max_delivery_attempts": 5,
    "delivery_attempt_timeout": 1,
    "retry_min_backoff": 10,
    "retry_max_backoff": 60
}'

{
  "id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "name":"Httpbin Post",
  "url":"https://httpbin.org/post",
  "content_type":"application/json",
  "valid_status_codes":[
    200,
    201
  ],
  "secret_token":"my-secret-token",
  "active":true,
  "max_delivery_attempts":5,
  "delivery_attempt_timeout":1,
  "retry_min_backoff":10,
  "retry_max_backoff":60,
  "created_at":"2021-03-08T20:41:25.433671Z",
  "updated_at":"2021-03-08T20:41:25.433671Z"
}

Create a new delivery

curl --location --request POST 'http://localhost:8000/v1/deliveries' \
--header 'Content-Type: application/json' \
--data-raw '{
    "webhook_id": "a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
    "payload": "{\"success\": true}"
}'

{
  "id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
  "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "payload":"{\"success\": true}",
  "scheduled_at":"2021-03-08T20:43:49.986771Z",
  "delivery_attempts":0,
  "status":"pending",
  "created_at":"2021-03-08T20:43:49.986771Z",
  "updated_at":"2021-03-08T20:43:49.986771Z"
}

Get deliveries

curl --location --request GET 'http://localhost:8000/v1/deliveries?webhook_id=a6e9a525-ac5a-488c-b118-bd7327ce6d8d'

{
  "deliveries":[
    {
      "id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
      "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
      "payload":"{\"success\": true}",
      "scheduled_at":"2021-03-08T20:43:49.986771Z",
      "delivery_attempts":1,
      "status":"succeeded",
      "created_at":"2021-03-08T20:43:49.986771Z",
      "updated_at":"2021-03-08T20:46:51.674623Z"
    }
  ],
  "limit":50,
  "offset":0
}

Get delivery

curl --location --request GET 'http://localhost:8000/v1/deliveries/bc76122c-e56b-45c7-8dc3-b80a861191d5'

{
  "id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
  "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "payload":"{\"success\": true}",
  "scheduled_at":"2021-03-08T20:43:49.986771Z",
  "delivery_attempts":1,
  "status":"succeeded",
  "created_at":"2021-03-08T20:43:49.986771Z",
  "updated_at":"2021-03-08T20:46:51.674623Z"
}

Get delivery attempts

curl --location --request GET 'http://localhost:8000/v1/delivery-attempts?delivery_id=bc76122c-e56b-45c7-8dc3-b80a861191d5'

{
  "delivery_attempts":[
    {
      "id":"d72719d6-5a79-4df7-a2c2-2029ab0e1848",
      "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
      "delivery_id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
      "raw_request":"POST /post HTTP/1.1\r\nHost: httpbin.org\r\nContent-Type: application/json\r\nX-Hub-Signature: 3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\r\n\r\n{\"success\": true}",
      "raw_response":"HTTP/2.0 200 OK\r\nContent-Length: 538\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Origin: *\r\nContent-Type: application/json\r\nDate: Mon, 08 Mar 2021 20:46:51 GMT\r\nServer: gunicorn/19.9.0\r\n\r\n{\n  \"args\": {}, \n  \"data\": \"{\\\"success\\\": true}\", \n  \"files\": {}, \n  \"form\": {}, \n  \"headers\": {\n    \"Accept-Encoding\": \"gzip\", \n    \"Content-Length\": \"17\", \n    \"Content-Type\": \"application/json\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\": \"Go-http-client/2.0\", \n    \"X-Amzn-Trace-Id\": \"Root=1-60468d3b-36d312777a03ec3e1c564e3b\", \n    \"X-Hub-Signature\": \"3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\"\n  }, \n  \"json\": {\n    \"success\": true\n  }, \n  \"origin\": \"191.35.122.74\", \n  \"url\": \"https://httpbin.org/post\"\n}\n",
      "response_status_code":200,
      "execution_duration":547,
      "success":true,
      "error":"",
      "created_at":"2021-03-08T20:46:51.680846Z"
    }
  ],
  "limit":50,
  "offset":0
}

Get delivery attempt

curl --location --request GET 'http://localhost:8000/v1/delivery-attempts/d72719d6-5a79-4df7-a2c2-2029ab0e1848'

{
  "id":"d72719d6-5a79-4df7-a2c2-2029ab0e1848",
  "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "delivery_id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
  "raw_request":"POST /post HTTP/1.1\r\nHost: httpbin.org\r\nContent-Type: application/json\r\nX-Hub-Signature: 3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\r\n\r\n{\"success\": true}",
  "raw_response":"HTTP/2.0 200 OK\r\nContent-Length: 538\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Origin: *\r\nContent-Type: application/json\r\nDate: Mon, 08 Mar 2021 20:46:51 GMT\r\nServer: gunicorn/19.9.0\r\n\r\n{\n  \"args\": {}, \n  \"data\": \"{\\\"success\\\": true}\", \n  \"files\": {}, \n  \"form\": {}, \n  \"headers\": {\n    \"Accept-Encoding\": \"gzip\", \n    \"Content-Length\": \"17\", \n    \"Content-Type\": \"application/json\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\": \"Go-http-client/2.0\", \n    \"X-Amzn-Trace-Id\": \"Root=1-60468d3b-36d312777a03ec3e1c564e3b\", \n    \"X-Hub-Signature\": \"3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\"\n  }, \n  \"json\": {\n    \"success\": true\n  }, \n  \"origin\": \"191.35.122.74\", \n  \"url\": \"https://httpbin.org/post\"\n}\n",
  "response_status_code":200,
  "execution_duration":547,
  "success":true,
  "error":"",
  "created_at":"2021-03-08T20:46:51.680846Z"
}

Swagger docs

The swagger spec is available at http://localhost:8000/swagger/index.html.

Health check

The health check server is running on port defined by envvar POSTMAND_HEALTH_CHECK_HTTP_PORT (defaults to 8001).

curl --location --request GET 'http://localhost:8001/healthz'

{
  "success":true
}

Environment variables

All environment variables is defined on file local.env.

How to build docker image

docker build -f Dockerfile -t postmand .

Documentation ¶

Constants ¶

View Source

const (
	// DeliveryStatusPending represents the delivery pending status
	DeliveryStatusPending = "pending"
	// DeliveryStatusSucceeded represents the delivery succeeded status
	DeliveryStatusSucceeded = "succeeded"
	// DeliveryStatusFailed represents the delivery failed status
	DeliveryStatusFailed = "failed"
)

Variables ¶

View Source

var (
	// ErrWebhookNotFound is returned by any operation that can't load a webhook.
	ErrWebhookNotFound = errors.New("webhook_not_found")
	// ErrDeliveryNotFound is returned by any operation that can't load a delivery.
	ErrDeliveryNotFound = errors.New("delivery_not_found")
	// ErrDeliveryAttemptNotFound is returned by any operation that can't load a delivery attempt.
	ErrDeliveryAttemptNotFound = errors.New("delivery_attempt_not_found")
)

Functions ¶

This section is empty.

Types ¶

type Delivery ¶

type Delivery struct {
	ID               ID        `json:"id" db:"id"`
	WebhookID        ID        `json:"webhook_id" db:"webhook_id"`
	Payload          string    `json:"payload" db:"payload"`
	ScheduledAt      time.Time `json:"scheduled_at" db:"scheduled_at"`
	DeliveryAttempts int       `json:"delivery_attempts" db:"delivery_attempts"`
	Status           string    `json:"status" db:"status"`
	CreatedAt        time.Time `json:"created_at" db:"created_at"`
	UpdatedAt        time.Time `json:"updated_at" db:"updated_at"`

} //@name Delivery

Delivery represents a payload that must be delivery using webhook context.

func (Delivery) Validate ¶

func (d Delivery) Validate() error

Validate implements ozzo validation Validatable interface

type DeliveryAttempt ¶

type DeliveryAttempt struct {
	ID                 ID        `json:"id" db:"id"`
	WebhookID          ID        `json:"webhook_id" db:"webhook_id"`
	DeliveryID         ID        `json:"delivery_id" db:"delivery_id"`
	RawRequest         string    `json:"raw_request" db:"raw_request"`
	RawResponse        string    `json:"raw_response" db:"raw_response"`
	ResponseStatusCode int       `json:"response_status_code" db:"response_status_code"`
	ExecutionDuration  int       `json:"execution_duration" db:"execution_duration"`
	Success            bool      `json:"success" db:"success"`
	Error              string    `json:"error" db:"error"`
	CreatedAt          time.Time `json:"created_at" db:"created_at"`

} //@name DeliveryAttempt

DeliveryAttempt represents a delivery attempt.

type DeliveryAttemptRepository ¶

type DeliveryAttemptRepository interface {
	Get(ctx context.Context, getOptions RepositoryGetOptions) (*DeliveryAttempt, error)
	List(ctx context.Context, listOptions RepositoryListOptions) ([]*DeliveryAttempt, error)
	Create(ctx context.Context, deliveryAttempt *DeliveryAttempt) error
}

DeliveryAttemptRepository is the interface that will be used to iterate with the DeliveryAttempt data.

type DeliveryAttemptService ¶

type DeliveryAttemptService interface {
	Get(ctx context.Context, getOptions RepositoryGetOptions) (*DeliveryAttempt, error)
	List(ctx context.Context, listOptions RepositoryListOptions) ([]*DeliveryAttempt, error)
}

DeliveryAttemptService is the interface that will be used to perform operations with delivery attempt.

type DeliveryRepository ¶

type DeliveryRepository interface {
	Get(ctx context.Context, getOptions RepositoryGetOptions) (*Delivery, error)
	List(ctx context.Context, listOptions RepositoryListOptions) ([]*Delivery, error)
	Create(ctx context.Context, delivery *Delivery) error
	Update(ctx context.Context, delivery *Delivery) error
	Delete(ctx context.Context, id ID) error
	Dispatch(ctx context.Context) (*DeliveryAttempt, error)
}

DeliveryRepository is the interface that will be used to iterate with the Delivery data.

type DeliveryService ¶

type DeliveryService interface {
	Get(ctx context.Context, getOptions RepositoryGetOptions) (*Delivery, error)
	List(ctx context.Context, listOptions RepositoryListOptions) ([]*Delivery, error)
	Create(ctx context.Context, delivery *Delivery) error
	Update(ctx context.Context, delivery *Delivery) error
	Delete(ctx context.Context, id ID) error
}

DeliveryService is the interface that will be used to perform operations with deliveries.

type ID ¶

type ID = uuid.UUID

ID represents the primary key for all entities.

type MigrationRepository ¶

type MigrationRepository interface {
	Run(ctx context.Context) error
}

MigrationRepository is the interface that will be used to run database migrations.

type MigrationService ¶

type MigrationService interface {
	Run(ctx context.Context) error
}

MigrationService is the interface that will be used to execute database migrations.

type PingRepository ¶

type PingRepository interface {
	Run(ctx context.Context) error
}

PingRepository is the interface that will be used to run ping against database.

type PingService ¶

type PingService interface {
	Run(ctx context.Context) error
}

PingService is the interface that will be used to perform ping operation against database.

type RepositoryGetOptions ¶

type RepositoryGetOptions struct {
	Filters map[string]interface{}
}

RepositoryGetOptions contains options used in the Get methods.

type RepositoryListOptions ¶

type RepositoryListOptions struct {
	Filters map[string]interface{}
	Limit   int
	Offset  int
	OrderBy string
	Order   string
}

RepositoryListOptions contains options used in the List methods.

type Webhook ¶

type Webhook struct {
	ID                     ID            `json:"id" db:"id"`
	Name                   string        `json:"name" db:"name"`
	URL                    string        `json:"url" db:"url"`
	ContentType            string        `json:"content_type" db:"content_type"`
	ValidStatusCodes       pq.Int32Array `json:"valid_status_codes" db:"valid_status_codes"`
	SecretToken            string        `json:"secret_token" db:"secret_token"`
	Active                 bool          `json:"active" db:"active"`
	MaxDeliveryAttempts    int           `json:"max_delivery_attempts" db:"max_delivery_attempts"`
	DeliveryAttemptTimeout int           `json:"delivery_attempt_timeout" db:"delivery_attempt_timeout"`
	RetryMinBackoff        int           `json:"retry_min_backoff" db:"retry_min_backoff"`
	RetryMaxBackoff        int           `json:"retry_max_backoff" db:"retry_max_backoff"`
	CreatedAt              time.Time     `json:"created_at" db:"created_at"`
	UpdatedAt              time.Time     `json:"updated_at" db:"updated_at"`

} //@name Webhook

Webhook represents a webhook in the system.

func (Webhook) Validate ¶

func (w Webhook) Validate() error

Validate implements ozzo validation Validatable interface

type WebhookRepository ¶

type WebhookRepository interface {
	Get(ctx context.Context, getOptions RepositoryGetOptions) (*Webhook, error)
	List(ctx context.Context, listOptions RepositoryListOptions) ([]*Webhook, error)
	Create(ctx context.Context, webhook *Webhook) error
	Update(ctx context.Context, webhook *Webhook) error
	Delete(ctx context.Context, id ID) error
}

WebhookRepository is the interface that will be used to iterate with the Webhook data.

type WebhookService ¶

type WebhookService interface {
	Get(ctx context.Context, getOptions RepositoryGetOptions) (*Webhook, error)
	List(ctx context.Context, listOptions RepositoryListOptions) ([]*Webhook, error)
	Create(ctx context.Context, webhook *Webhook) error
	Update(ctx context.Context, webhook *Webhook) error
	Delete(ctx context.Context, id ID) error
}

WebhookService is the interface that will be used to perform operations with webhooks.

type WorkerService ¶

type WorkerService interface {
	Run(ctx context.Context)
	Shutdown(ctx context.Context)
}

WorkerService is the interface that will be used on workers to dispatch webhooks.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
cmd
postmand
docs
http
handler
middleware
mocks
repository
service

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

README ¶

postmand

Features

Quickstart

Run the server

Docker

Local

Run the worker

Docker

Local

Create a new webhook

Create a new delivery

Get deliveries

Get delivery

Get delivery attempts

Get delivery attempt

Swagger docs

Health check

Environment variables

How to build docker image

Documentation ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

Types ¶

type Delivery ¶

func (Delivery) Validate ¶

type DeliveryAttempt ¶

type DeliveryAttemptRepository ¶

type DeliveryAttemptService ¶

type DeliveryRepository ¶

type DeliveryService ¶

type ID ¶

type MigrationRepository ¶

type MigrationService ¶

type PingRepository ¶

type PingService ¶

type RepositoryGetOptions ¶

type RepositoryListOptions ¶

type Webhook ¶

func (Webhook) Validate ¶

type WebhookRepository ¶

type WebhookService ¶

type WorkerService ¶

Source Files ¶

Directories ¶