flare

README

Flare is a service that listen to changes on HTTP endpoints and notify subscripted clients about the changes. It help reduce the pressure on APIs by avoiding the clients to do pooling requests to search for new/changed content and the need of the APIs to develop workers to notify the clients about the.

There is no need to the the service know anything about who is consuming it's updates, this is abstracted and lead to a simpler design on APIs. Problems like scaling the workers to notify the changes if the number of subscriptions increase, need to control the delivery success of the messages, include/update/delete the clients on your subscription list and so on are just solved with Flare.

How to run

go get github.com/diegobernardes/flare
cd $GOPATH/src/github.com/diegobernardes/flare/service/flare/cmd
go run flare.go start

There is a example at misc/example, it's a docker-compose that starts a Flare server, a producer and a consumer. From times to times the producer create/update/delete a given document and the consumer receives this changes from Flare. You must have docker-compose and docker to run this example.

go get github.com/diegobernardes/flare
cd $GOPATH/src/github.com/diegobernardes/flare/misc/example
make run

There is also a Docker image:

docker run --rm -p 8080:8080 diegobernardes/flare:v0.1.0-alpha

How it works

Flare has 3 basic entities: Resource, Subscription and Document. The origin of content is responsible for Resource and Document entities and the clients are responsible for Subscription.

Resource

Resource represents the entity you want to track. It cannot be updated, only deleted, and to delete, first you need to remove all the associated subscriptions.

Field	Description
endpoint	Is the actual document that gonna be tracked. wildcards are required to track the collection and they can be later used at subscriptions.
change.field	The field that is used to track changes on a document. It can be a string containing a date or a auto incremented integer.
change.format	If the field is a date, this fields has the format to parse the document date. More info about the format here.

Endpoint: POST http://flare.com/resources

{
	"endpoint": "http://api.company.com/users/{id}",
	"change": {
		"field": "updatedAt",
		"format": "2006-01-02T15:04:05Z07:00"
	}
}

Subscription

Subscription is the responsible to notify the clients when a document from a resource changes.

Field	Description
endpoint.url	The address of the client that gonna receive the notification.
endpoint.method	The method used on the notification request.
endpoint.headers	A list of headers to sent within the request.
endpoint.actions.(create,update,delete).(url,method,headers)	Override of attributes per action.
delivery.success	List of success status code. This is used to mark the notification as delivered for the respective client.
delivery.discard	List of status code to discard the notification.
content.document	Send the document.
content.envelope	Send the document, if marked, inside a envelope with some metadata.
data	Can only be set if content.envelope is true. Can be used to provide aditional information to the client that gonna receive the notification. It also can interpolate wildcards used at resource endpoint definition.

Endpoint: POST http://flare.com/resources/:resource-id/subscriptions

{
	"endpoint": {
		"url": "http://api.company.com/wishlist/{id}",
		"method": "post",
		"headers": {
			"Authorization": [
				"Basic YWxhZGRpbjpvcGVuc2VzYW1l"
			]
		},
		"actions": {
			"delete": {
				"method": "delete"
			}
		}
	},
	"delivery": {
		"success": [200],
		"discard": [500]
	},
	"content": {
		"envelope": true,
		"document": true
	},
	"data": {
		"service": "user",
		"id": "{id}"
	}
}

Document

To update a document, a PUT should be done at http://flare.com/documents/{endpoint}, where the {endpoint} is the real document endpoint and it should match the information inserted at the resource creation. The body should contain the document. If the origin send the same document or older documents more then one time, the service don't gonna notify the clients again because it know the document version each client has. The notification only happens when is really needed.

The delete should be sent with the delete method and no body.

Documentation

Index

• Constants
• type Document
  • func (doc *Document) Newer(reference *Document) bool
• type DocumentRepositorier
• type DocumentRepositoryError
• type Pagination
  • func (p *Pagination) Valid() error
• type Resource
• type ResourceChange
  • func (rc *ResourceChange) Valid() error
• type ResourceRepositorier
• type ResourceRepositoryError
• type Subscription
• type SubscriptionContent
• type SubscriptionDelivery
• type SubscriptionEndpoint
• type SubscriptionRepositorier
• type SubscriptionRepositoryError
• type SubscriptionTrigger

Constants

const (
	SubscriptionTriggerCreate = "create"
	SubscriptionTriggerUpdate = "update"
	SubscriptionTriggerDelete = "delete"
)

All kinds of actions a subscription trigger has.

Types

type Document

type Document struct {
	ID        url.URL
	Revision  int64
	Resource  Resource
	Content   map[string]interface{}
	UpdatedAt time.Time
}

Document represents the documents from a resource.

func (*Document) Newer

func (doc *Document) Newer(reference *Document) bool

Newer indicates if the current document is newer then the one passed as parameter.

type DocumentRepositorier

type DocumentRepositorier interface {
	FindByID(ctx context.Context, id url.URL) (*Document, error)
	Update(context.Context, *Document) error
	Delete(ctx context.Context, id url.URL) error
}

DocumentRepositorier used to interact with document data storage.

type DocumentRepositoryError

type DocumentRepositoryError interface {
	error
	NotFound() bool
}

DocumentRepositoryError implements all the errrors the repository can return.

type Pagination

type Pagination struct {
	Limit  int
	Offset int
	Total  int
}

Pagination used to fetch a slice of a given entity.

func (*Pagination) Valid

func (p *Pagination) Valid() error

Valid indicates if the current pagination is valid.

type Resource

type Resource struct {
	ID        string
	Endpoint  url.URL
	Change    ResourceChange
	CreatedAt time.Time
}

Resource represents the apis Flare track and the info to detect changes on documents.

type ResourceChange

type ResourceChange struct {
	Field  string
	Format string
}

ResourceChange holds the information to detect document change.

func (*ResourceChange) Valid

func (rc *ResourceChange) Valid() error

Valid indicates if the current resourceChange is valid.

type ResourceRepositorier

type ResourceRepositorier interface {
	Find(context.Context, *Pagination) ([]Resource, *Pagination, error)
	FindByID(context.Context, string) (*Resource, error)
	FindByURI(context.Context, url.URL) (*Resource, error)
	Partitions(ctx context.Context, id string) (partitions []string, err error)
	Create(context.Context, *Resource) error
	Delete(context.Context, string) error
}

ResourceRepositorier is used to interact with Resource repository.

type ResourceRepositoryError

type ResourceRepositoryError interface {
	error
	AlreadyExists() bool
	NotFound() bool
}

ResourceRepositoryError represents all the errors the repository can return.

type Subscription

type Subscription struct {
	ID        string
	Endpoint  SubscriptionEndpoint
	Delivery  SubscriptionDelivery
	Resource  Resource
	Partition string
	Data      map[string]interface{}
	Content   SubscriptionContent
	CreatedAt time.Time
}

Subscription has all the information needed to notify the clients from changes on documents.

type SubscriptionContent

type SubscriptionContent struct {
	Document bool
	Envelope bool
}

SubscriptionContent configure the content delived by the subscription.

type SubscriptionDelivery

type SubscriptionDelivery struct {
	Success []int
	Discard []int
}

SubscriptionDelivery is used to control whenever the notification can be considered successful or not.

type SubscriptionEndpoint

type SubscriptionEndpoint struct {
	URL     *url.URL
	Method  string
	Headers http.Header
	Action  map[string]SubscriptionEndpoint
}

SubscriptionEndpoint has the address information to notify the clients.

type SubscriptionRepositorier

type SubscriptionRepositorier interface {
	Find(context.Context, *Pagination, string) ([]Subscription, *Pagination, error)
	FindByID(ctx context.Context, resourceID, id string) (*Subscription, error)
	FindByPartition(
		ctx context.Context, resourceID, partition string,
	) (<-chan Subscription, <-chan error, error)
	Create(context.Context, *Subscription) error
	Delete(ctx context.Context, resourceID, id string) error
	Trigger(
		ctx context.Context,
		action string,
		document *Document,
		subscription *Subscription,
		fn func(context.Context, *Document, *Subscription, string) error,
	) error
}

SubscriptionRepositorier is used to interact with the subscription data storage.

type SubscriptionRepositoryError

type SubscriptionRepositoryError interface {
	error
	NotFound() bool
	AlreadyExists() bool
}

SubscriptionRepositoryError represents all the errors the repository can return.

type SubscriptionTrigger

type SubscriptionTrigger interface {
	Push(ctx context.Context, document *Document, action string) error
}

SubscriptionTrigger is used to trigger the change on documents.