websub

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 30, 2022 License: MIT Imports: 19 Imported by: 5

README

go-websub

go-websub is a websub subscriber, publisher, and hub library written in go. It passes all of the websub.rocks tests for publishers and hubs, and almost all of them for subscribers.

Inspired by https://github.com/tystuyfzand/websub-client and https://github.com/tystuyfzand/websub-server.

Examples

See the ./examples/ directory for examples of using a subscriber, publisher, and hub.

Importing

go get github.com/notnotquinn/go-websub

Features

  • BIG: Does not (currently) persist state between restarts.
  • Can run subscriber, publisher, and hub from one server, on one port, if needed. (see examples/single-port)
Subscriber features
  • BIG: Does not (currently) persist state between restarts.
  • Not completely spec compliant. (see Subscriber conformance)
  • Functions independently from publisher and hub.
  • Subscribe & unsubscribe to topics.
  • Simple function callback system, with access to subscription meta-data.
  • Request specific lease duration.
  • Provide hub.secret and verify hub signature.
  • Automatically refresh expiring subscriptions.
Subscriber methods

Interact with the subscriber via a Go API:

  • Subscribe to topic URLs with callback.
    • Example:
    subscription, err := s.Subscribe(
	// Topic URL that exposes Link headers for "hub" and "self".
	"https://example.com/topic1",
	// for authenticated content distribution (maximum of 200 characters)
	"random bytes",
	// Callback function is called when the subscriber receives a valid
	// request from the hub, not on invalid ones
	// (for example ones with a missing or invalid hub signature)
	func(sub *websub.SubscriberSubscription, contentType string, body io.Reader) {
		fmt.Println("Received content!")
		// do something...
	},
)

if err != nil {
	// handle
}
  • Unsubscribe from topics.
    • Example:
    err = s.Unsubscribe(subscription)

if err != nil {
	// handle
}
Publisher features
  • BIG: Does not (currently) persist state between restarts.
  • Completely spec compliant.
  • Functions independently from subscriber and hub.
  • Advertise topic and hub URLs for previously published topics.
  • Send publish requests for topic URLs that arent under the publishers base URL.
  • Send both hub.topic and hub.url on publish requests.
  • Treat https://example.com/baseUrl/topic/ equal to https://example.com/baseUrl/topic for incoming requests.
  • Optionally advertise topic and hub URLs for unpublished topics.
  • Optionally post content in publish request. (see publisher publishing methods #2)
Publisher methods

Interact with the publisher via a Go API:

  • Publish content with content-type.
    • Example:
    err = p.Publish(
	// Topic URL
	p.BaseUrl()+"/topic1",
	// Content Type
	"text/plain",
	// Content
	[]byte("Hello, WebSub!"),
)

if err != nil {
	// handle
}
Hub features
  • BIG: Does not (currently) persist state between restarts.
  • Completely spec compliant.
  • Functions independently from subscriber and publiser.
  • Configurable retry limits for distribution requests.
  • Configurable lease length limits and default lease length.
  • Configurable User-Agent for HTTP requests on behaf of the hub.
  • Configure one of "sha1", "sha256", "sha384", and "sha512" for signing publish requests.
  • Optionally expose all known topic URLs as a JSON array to /topics and provide websub updates for new topics.
  • Optionally accept publish requests with the body as the content. (see Hub accepted publishing methods #2)
Hub methods

Interact with the hub via a Go API:

  • Custom subscription validation.
    • Example:
    // Deny any subscription where the callback URL is not under "example.com"
h.AddValidator(func(sub *websub.HubSubscription) (ok bool, reason string) {
	parsed, err := url.Parse(sub.Callback)
	if err != nil {
		return false, "invalid callback url"
	}

	if parsed.Host != "example.com" {
		return false, "callback host not allowed"
	}

	return true, ""
})
  • Sniff on published topics. (to all or one topic, as if you were subscribed)
    • Example:
    // List the topic url as an empty string to listen to all publishes.
h.AddSniffer("https://example.com/topic1",
    func(topic, contentType string, content []byte) {
		fmt.Printf("New publish on \"https://example.com/topic1\" !")
	},
)
  • Publish content via method call.
    • Example:
    // no return value
h.Publish("https://example.com/topic1", "Content-Type", []byte("Content"))
  • Get all topic URLs programmatically.
    • Example:
    fmt.Printf("%#v", h.GetTopics())
// []string{"https://example.com/topic1", https://example.com/topic2", ...}

Spec conformance classes

As per the websub spec.

Subscriber conformance

The included subscriber technically does not follow the websub spec, because it does not support discovering links from HTML tags, Atom feeds, or RSS feeds.

The subscriber still discovers topics form their Link headers, so this does not impact the subscriber's interaction with the hub or publisher implemented here, but it may end up being a problem if you are planning on subscribing to other publisher implementations that don't provide Link headers for their topics.

A conforming subscriber:

  • [2/5] MUST support each discovery mechanism in the specified order to discover the topic and hub URLs as described in Discovery
    • HTTP header discovery
    • HTML tag discovery
    • Atom feed discovery
    • RSS feed discovery
    • Discovery priority
  • MUST send a subscription request as described in Subscriber Sends Subscription Request .
  • MUST acknowledge a content distribution request with an HTTP 2xx status code.
  • MAY request a specific lease duration
  • MAY request that a subscription is deactivated using the "unsubscribe" mechanism.
  • MAY include a secret in the subscription request, and if it does, then MUST use the secret to verify the signature in the content distribution request.
Publisher conformance

A conforming publisher:

  • MUST advertise topic and hub URLs for a given resource URL as described in Discovery.
Publisher publishing methods

There are 2 options to publish, which are both supported by the hub:

  1. (default) Send a POST request (as Content-Type: application/x-www-form-urlencoded) with the keys hub.mode set to publish, and hub.url & hub.topic both set to the updated URL in the body as a form.

  2. (opt-in) Send a POST request with the same keys as above in the query string parameters, and the key hub.content equal to body. The hub will not make any request to the topic URL, and instead will distribute the body of the POST request, and associated Content-Type to subscribers. This method is disabled by default for the hub.

The updated URL is duplicated because of possible hub implementation variations.

Hub conformance

A conforming hub:

  • MUST accept a subscription request with the parameters hub.callback, hub.mode and hub.topic.
  • MUST accept a subscription request with a hub.secret parameter.
  • MAY respect the requested lease duration in subscription requests.
    • Respects requested lease duration if it is within a configurable allowed range, otherwise it is pinned to the maximum or minimum limit.
  • MUST allow subscribers to re-request already active subscriptions.
  • MUST support unsubscription requests.
  • MUST send content distribution requests with a matching content type of the topic URL. (See Content Negotiation )
  • MUST send a X-Hub-Signature header if the subscription was made with a hub.secret as described in Authenticated Content Distribution .
  • MAY reduce the payload of the content distribution to a diff of the contents for supported formats as described in Content Distribution .
Hub accepted publishing methods

There are two ways for publishers to publish to the hub, which match the ones availible for the provided publisher:

  1. Send a POST request (as Content-Type: application/x-www-form-urlencoded) with the keys hub.mode equal to publish, one or both of hub.topic and hub.url equal to the topic URL that was updated in the body as a form. If both are provided hub.topic is used. The hub makes a GET request to the topic URL and distributes the content to subscribers, with the correct Content-Type.

  2. (disabled by default) Send a POST request with the same keys as above in the query string parameters, and the key hub.content equal to body. The hub will not make any request to the topic URL, and instead will distribute the body of the POST request, and associated Content-Type to subscribers.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// a non 2xx status code was returned when getting topic content
	ErrNon2xxGettingContent = errors.New(
		"a non 2xx status code was returned when getting topic content")
	// a non 2xx status code was returned when posting content to a subscriber
	ErrNon2xxPostingContent = errors.New(
		"a non 2xx status code was returned when posting content to a subscriber")
)
View Source 
var (
	// topic not discoverable
	ErrTopicNotDiscoverable = errors.New("topic not discoverable")
	// hub returned an invalid status code on subscription or unsubscription request
	ErrNon2xxOnSubReq = errors.New("hub returned an invalid status code on subscription or unsubscription request")
)
View Source 
var (
	// hub returned a non 2xx status code on publish request
	ErrNon2xxOnPubReq = errors.New("hub returned a non 2xx status code on publish request")
)

Functions

func Logger 

func Logger() zerolog.Logger

Logger returns the logger the websub package uses

Types

type Hub 

type Hub struct {
	// contains filtered or unexported fields
}

A Hub is a websub hub.

func NewHub 

func NewHub(hubUrl string, options ...HubOption) *Hub

NewHub creates a new hub with the specified options and starts background goroutines.

func (*Hub) AddSniffer 

func (h *Hub) AddSniffer(topic string, sniffer HubTopicSnifferFunc)

AddSniffer allows one to "sniff" publishes, receiving events as if they were subscribers.

If an emptry string is provided as the topic, all publishes are sniffed.

func (*Hub) AddValidator 

func (h *Hub) AddValidator(validator HubSubValidatorFunc)

AddValidator adds a validator for subscription requests. Multiple validators can exist on one hub.

All subscriptions are accepted by default.

func (*Hub) GetTopics 

func (h *Hub) GetTopics() (topics []string)

Returns an array of all topics.

Includes topics with no subscribers, or no publishes.

func (Hub) HubUrl 

func (h Hub) HubUrl() string

func (*Hub) Publish 

func (h *Hub) Publish(topic, contentType string, content []byte)

Publish publishes a topic with the specified content and content-type.

func (*Hub) ServeHTTP 

func (h *Hub) ServeHTTP(w http.ResponseWriter, r *http.Request)

Handles incoming HTTP requests

type HubOption 

type HubOption func(*Hub)

A HubOption specifies an option for a hub.

func HubAllowPostBodyAsContent added in v0.2.0 

func HubAllowPostBodyAsContent(enable bool) HubOption

HubAllowPostBodyAsContent allows publishers to post content as the body of the POST request if they provide hub.content = "body" and hub.mode = "publish". In this case, the Content-Type of the post request is used when distributing publish events.

NOTE: Because of the lack of authentication for publishers, this allows any machine with internet access to the hub to publish any content under any topic. Use with caution.

func HubExposeTopics added in v0.2.0 

func HubExposeTopics(enable bool) HubOption

HubExposeTopics enables a /topics endpoint that lists all available/active topics.

func HubWithCleanupInterval added in v0.2.0 

func HubWithCleanupInterval(interval time.Duration) HubOption

HubWithCleanupInterval sets the interval expired subscriptions are removed from the memory

func HubWithHashFunction added in v0.2.0 

func HubWithHashFunction(hashFunction string) HubOption

HubWithHashFunction sets the hash function used to compute hub signatures for subscriptions with a secret.

One of "sha1", "sha256", "sha384", "sha512"

Default is "sha1" for compatability, however it is insecure.

func HubWithLeaseSettings added in v0.2.0 

func HubWithLeaseSettings(minLease, maxLease, defaultLease time.Duration) HubOption

HubWithLeaseSettings sets the minimum, maximum, and default lease for a subscription on a hub

When a requested lease is outside of the allowed range, the lease becomes pinned to the minimum or maximum value. If a subscriber doesn't provide a lease length, the default lease length is used.

- Default minimum lease is 5 minutes

- Default maximum lease is 720 hours (30 days)

- Default default lease is 240 hours (10 days)

func HubWithRetryLimits added in v0.2.0 

func HubWithRetryLimits(retryLimit int, retryInterval time.Duration) HubOption

HubWithRetryLimits sets the retry limits for a hub.

Defaults to 5 retries, and a one minute interval.

func HubWithUserAgent added in v0.2.0 

func HubWithUserAgent(userAgent string) HubOption

HubWithUserAgent sets the user-agent for a hub

Default user agent is "go-websub-hub"

type HubSubValidatorFunc added in v0.2.0 

type HubSubValidatorFunc func(sub *HubSubscription) (ok bool, reason string)

HubSubValidatorFunc validates subscription requests.

The validation stops as soon as one validator returns ok=false. The provided reason is sent to the subscriber telling them their subscription request was denied.

The expiry date will not be set by the time the validators are called.

type HubSubscription added in v0.2.0 

type HubSubscription struct {
	// The HTTP callback to the subscriber.
	Callback string
	// The topic URL the subscriber is subscribing to.
	Topic string
	// The number of seconds the subscription will be active.
	LeaseLength int
	// The date/time the subscription will expire.
	// Is not set at validation time.
	Expires time.Time
	// The secret provided by the subscriber. Empty string means none.
	Secret string
}

an HubSubscription is a subscription used in the context of a Hub.

type HubTopicSnifferFunc added in v0.2.0 

type HubTopicSnifferFunc func(topic string, contentType string, body io.Reader)

A topic sniffer sniffs on topics as if it was a subscriber.

type Publisher 

type Publisher struct {
	// contains filtered or unexported fields
}

func NewPublisher 

func NewPublisher(baseUrl, hubUrl string, options ...PublisherOption) *Publisher

func (Publisher) BaseUrl 

func (p Publisher) BaseUrl() string

BaseUrl returns the base URL of this publisher (with any trailing slash trimmed)

func (*Publisher) Publish 

func (p *Publisher) Publish(topic string, contentType string, content []byte) error

Publish will send a publish request to the hub.

If the topic URL starts with this publisher's base URL, the publisher will return the content on HTTP GET requests to that url.

func (*Publisher) ServeHTTP 

func (p *Publisher) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP serves the content that has been published to this publisher, and advertises topic and hub urls in Link headers.

Only topics published with a URL that starts with the base URL are advertised.

type PublisherOption 

type PublisherOption func(p *Publisher)

func PublisherAdvertiseInvalidTopics added in v0.2.0 

func PublisherAdvertiseInvalidTopics(enabled bool) PublisherOption

PublisherAdvertiseInvalidTopics will advertise all topics with Link headers and return a 200 OK status as if they have already been published to with blank content.

func PublisherWithPostBodyAsContent added in v0.2.0 

func PublisherWithPostBodyAsContent(enabled bool) PublisherOption

PublisherWithPostBodyAsContent sends what is normally the body as the query parameters, and sends the content as the body. Also adds hub.content="body" in the query parameters.

Important: If the hub does not have this enabled, you will be unable to publish.

type SubscribeCallback 

type SubscribeCallback func(sub *SubscriberSubscription, contentType string, body io.Reader)

a SubscribeCallback is called when a subscriber receives a publish to the related topic.

type Subscriber 

type Subscriber struct {
	// contains filtered or unexported fields
}

func NewSubscriber 

func NewSubscriber(baseUrl string, options ...SubscriberOption) *Subscriber

NewSubscriber creates a new subscriber with the specified options.

func (Subscriber) BaseUrl 

func (s Subscriber) BaseUrl() string

Returns the base URL for this subscribers callback URLs.

func (*Subscriber) ServeHTTP 

func (s *Subscriber) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP handles all incoming HTTP requests, following websub spec.

func (*Subscriber) Subscribe 

func (s *Subscriber) Subscribe(topicUrl, secret string, callback SubscribeCallback) (*SubscriberSubscription, error)

subscribes to updates to the topicUrl, verifying using the secret

If the secret is an empty string, it is omitted.

When updates happen, the callback is called.

func (*Subscriber) Unsubscribe 

func (s *Subscriber) Unsubscribe(sub *SubscriberSubscription) error

Unsubscribe requests the hub to stop sending updates. Already pending unsubscriptions are ignored.

All events received in the meantime will not be fulfilled.

type SubscriberOption 

type SubscriberOption func(*Subscriber)

func SubscriberWithBaseUrl added in v0.2.0 

func SubscriberWithBaseUrl(baseUrl string) SubscriberOption

SubscriberWithBaseUrl sets the baseUrl for a subscriber

func SubscriberWithLeaseLength added in v0.2.0 

func SubscriberWithLeaseLength(LeaseLength time.Duration) SubscriberOption

SubscriberWithLeaseLength sets the LeaseLength for a subscriber

Default lease length is 10 days

type SubscriberSubscription added in v0.2.0 

type SubscriberSubscription struct {
	// Topic URL for this subscription.
	// Not always equal to the passed topic url.
	Topic string
	// Hub URL this subscription is from.
	Hub string
	// Secret string used for verifying the hub is the
	// sender of the subscription.
	Secret string
	// The date/time this subscription expires.
	Expires time.Time
	// Internal ID for this subscription. Part of the callback URL.
	Id string
	// contains filtered or unexported fields
}

a SubscriberSubscription is a subscription in the context of a Subscriber.

Source Files

View all Source files

Directories

Path Synopsis
examples
hub
multi-port
publisher
Interactive example of publishing content using go-websub.
 Interactive example of publishing content using go-websub.
single-port
sniffer
Expanded upon single-port, added sniffer
 Expanded upon single-port, added sniffer
subscriber
topic-announcement
Adapted from sniffer and added topic announcements
 Adapted from sniffer and added topic announcements

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.