webmention

README

Webmention library and service implementation in Go

This package can be used either as a standalone application or included as a library in your own projects.

go get github.com/cvanloo/gowebmention

Import it:

import webmention "github.com/cvanloo/gowebmention"

Send webmentions

go build cmd/mentioner/main.go -o mentioner
./mentioner <source> <target> [<targets>...]

Use as a library

Sending webmentions can be done through a WebMentionSender.

sender := webmention.NewSender()
sender.Update(source, pastMentions, currentMentions)
// source: the url for which you want to send mentions
// pastMentions: if you have sent mentions for the same url before, this list should include all targets mentioned the last time
//               otherwise you can leave the list empty or nil
// currentMentions: all targets that the source is currently mentioning

If you are sending updates for a now deleted source, it is your responsibility to ensure that the source is returning 410 Gone, optionally returning a tombstone representation of the old source as the response body.

Also note that the library does not persist anything. It is on you to remember pastMentions.

To receive webmentions setup an http endpoint and get the processing goroutine going. Also register one or more notifiers, with your custom logic describing how to react to a mention.

receiver := webmention.NewReceiver(
  webmention.WithNotifier(
    // your custom handlers
    LogMentions,
    SaveMentionsToDB,
    NotifyOnMatrix,
    NotifyByEMail,
  ),
)

// goroutine asynchronously validates and processes received webmentions
// webmentions that pass validation are passed on to the listeners
go receiver.ProcessMentions()

http.Handle("/api/webmention", receiver) // register webmention endpoint
http.ListenAndServe(":8080", nil)

For a more comprehensive example, including how to cleanly shutdown the receiver, look at the example implementation.

Notifiers need to implement the Notifier interface, which defines a single Receive method.

type MentionLogger struct{}
func (MentionLogger) Receive(mention webmention.Mention) {
  slog.Info("received mention", "mention", mention)
}
var LogMentions MentionLogger

Run as a service

Sending Webmentions

Mentioner can be run as a daemon to listen for commands on a socket.

cd cmd/mentioner
go build .
sudo cp mentioner /usr/local/bin/
sudo cp mentioner.service mentioner.socket /etc/systemd/system/
sudo systemctl start mentioner.socket

Something managing a source, eg., a blogging software, can send a command through the socket, instructing the Sender to send out webmentions.

socat - UNIX-CONNECT:/var/run/mentioner.socket
{"mentions":[{"source":"https://example.com/blog.html","past_targets":[],"current_targets":["https://example.com/some_other_blog.html"]}]}

A command has the following JSON structure:

{
  "mentions": [
    {
      "source": "<source 1 url>",
      "past_targets": [
        "<target 1 url>",
        "<target 2 url>",
        "<target ... url>"
      ],
      "current_targets": [
        "<target 1 url>",
        "<target 2 url>",
        "<target ... url>"
      ]
    },
    {
      "source": "<source 2 url>",
      "past_targets": [],
      "current_targets": []
    }
  ]
}

For each of the sources, the past and current targets will be mentioned.

The daemon responds for each mention with whether it was successful or not:

{
  "statuses": [
    {
      "source": "<source 1 url>",
      "error": ""
    }
  ],
  "error": ""
}

An empty error string indicates success.

Receiving Webmentions

Mentionee is a daemon that listens to incoming Webmentions.

cd cmd/mentionee
go build .
sudo cp mentionee /usr/local/bin/
sudo cp mentionee.service /etc/systemd/system/
sudo systemctl start mentionee.service

Since it listens on a local port (per default :8080), you can configure your web server to forward requests to it.

location = /api/webmention {
	proxy_pass http://localhost:8080;
	proxy_set_header Host $host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

Don't forget to advertise your Webmention endpoint!

One way is by sending Link headers:

location ~* \.html$ {
	expires 30d;
	add_header Cache-Control public;
	add_header Link "</api/webmention>; rel=webmention";
}

Another options is to add a <link> to your blog posts:

<html lang="en">
    <head>
        <meta charset="utf-8">
        <link rel="webmention" href="/api/webmention"> <!-- << advertise webmention endpoint here << -->
  </head>
  <body>
    <!-- Some super exciting blog post... -->
  </body>
</html>

Documentation

Index

• Constants
• Variables
• func BadRequest(msg string) error
• func MethodNotAllowed() error
• func TooManyRequests() error
• type ErrBadRequest
  • func (e ErrBadRequest) Error() string
  • func (e ErrBadRequest) RespondError(w http.ResponseWriter, r *http.Request) bool
• type ErrMethodNotAllowed
  • func (e ErrMethodNotAllowed) Error() string
  • func (e ErrMethodNotAllowed) RespondError(w http.ResponseWriter, r *http.Request) bool
• type ErrTooManyRequests
  • func (e ErrTooManyRequests) Error() string
  • func (e ErrTooManyRequests) RespondError(w http.ResponseWriter, r *http.Request) bool
• type ErrorResponder
• type MediaHandler
• type Mention
• type Notifier
• type NotifierFunc
  • func (f NotifierFunc) Receive(mention Mention)
• type Receiver
  • func NewReceiver(opts ...ReceiverOption) *Receiver
  • func (receiver *Receiver) ProcessMentions()
  • func (receiver *Receiver) ServeHTTP(w http.ResponseWriter, r *http.Request)
  • func (receiver *Receiver) Shutdown(ctx context.Context)
• type ReceiverOption
  • func WithAcceptsFunc(accepts TargetAcceptsFunc) ReceiverOption
  • func WithCacheTimeout(d time.Duration) ReceiverOption
  • func WithFetchUserAgent(agent string) ReceiverOption
  • func WithMediaHandler(mime string, qweight float64, handler MediaHandler) ReceiverOption
  • func WithNotifier(notifiers ...Notifier) ReceiverOption
  • func WithQueueSize(size int) ReceiverOption
• type Sender
  • func NewSender(opts ...SenderOption) *Sender
  • func (sender *Sender) DiscoverEndpoint(target URL) (endpoint URL, err error)
  • func (sender *Sender) Mention(source, target URL) error
  • func (sender *Sender) MentionMany(source URL, targets []URL) (err error)
  • func (sender *Sender) Update(source URL, pastTargets, currentTargets []URL) error
• type SenderOption
  • func WithUserAgent(agent string) SenderOption
• type Status
  • func HtmlHandler(content io.Reader, target URL) (status Status, err error)
  • func PlainHandler(content io.Reader, target URL) (status Status, err error)
• type TargetAcceptsFunc
• type URL
• type WebMentionSender

Examples

• Receiver

Constants

const (
	StatusLink    Status = "source links to target"
	StatusNoLink         = "source does not link to target"
	StatusDeleted        = "source itself got deleted"
)

Variables

var (
	ErrNotImplemented            = errors.New("not implemented")
	ErrNoEndpointFound           = errors.New("no webmention endpoint found")
	ErrNoRelWebmention           = errors.New("no webmention relationship found")
	ErrInvalidRelWebmention      = errors.New("target has invalid webmention url")
	ErrSourceDeleted             = errors.New("source got deleted")
	ErrSourceNotFound            = errors.New("source not found")
	ErrSourceDoesNotLinkToTarget = errors.New("source does not link to target")
)

var Report = func(err error, mention Mention) {
}

Report may be reassigned to handle 'unhandled' errors related to mention.

Functions

func BadRequest

func BadRequest(msg string) error

func MethodNotAllowed

func MethodNotAllowed() error

func TooManyRequests

func TooManyRequests() error

Types

type ErrBadRequest

type ErrBadRequest struct {
	Message string
}

func (ErrBadRequest) Error

func (e ErrBadRequest) Error() string

func (ErrBadRequest) RespondError

func (e ErrBadRequest) RespondError(w http.ResponseWriter, r *http.Request) bool

type ErrMethodNotAllowed

type ErrMethodNotAllowed struct{}

func (ErrMethodNotAllowed) Error

func (e ErrMethodNotAllowed) Error() string

func (ErrMethodNotAllowed) RespondError

func (e ErrMethodNotAllowed) RespondError(w http.ResponseWriter, r *http.Request) bool

type ErrTooManyRequests

type ErrTooManyRequests struct{}

func (ErrTooManyRequests) Error

func (e ErrTooManyRequests) Error() string

func (ErrTooManyRequests) RespondError

func (e ErrTooManyRequests) RespondError(w http.ResponseWriter, r *http.Request) bool

type ErrorResponder

type ErrorResponder interface {
	RespondError(w http.ResponseWriter, r *http.Request) bool
}

type MediaHandler

type MediaHandler func(sourceData io.Reader, target URL) (Status, error)

A MediaHandler searches sourceData for the target link. Only if an exact match is found a status of StatusLink and a nil error must be returned. If no (exact) match is found, a status of StatusNoLink and a nil error must be returned. If error is non-nil, it is treated as an internal error and the value of status is ignored. On error, no listeners will be invoked.

type Mention

type Mention struct {
	Source, Target URL
	Status         Status
}

type Notifier

type Notifier interface {
	Receive(mention Mention)
}

A registered Notifier is informed of any valid webmentions. This can be used to implement your own notifiers, e.g., to send a message on Discord, or XMPP. The Status field of the mention needs to be checked:

• StatusLink: the source (still, or newly) links to target
• StatusNoLink: the source does not (anymore) link to target
• StatusDeleted: the source got deleted

type NotifierFunc

type NotifierFunc func(mention Mention)

NotifierFunc adapts a function to an object that implements the Notifier interface.

func (NotifierFunc) Receive

func (f NotifierFunc) Receive(mention Mention)

type Receiver

type Receiver struct {
	// contains filtered or unexported fields
}

Receiver is a http.Handler that takes care of processing webmentions.

Example

acceptForTargetUrl := must(url.Parse("https://example.com"))
webmentionee := webmention.NewReceiver(
	webmention.WithAcceptsFunc(func(source, target *url.URL) bool {
		return acceptForTargetUrl.Scheme == target.Scheme && acceptForTargetUrl.Host == target.Host
	}),
	webmention.WithNotifier(webmention.NotifierFunc(func(mention webmention.Mention) {
		fmt.Printf("received webmention from %s for %s, status %s", mention.Source, mention.Target, mention.Status)
	})),
)
mux := &http.ServeMux{}
mux.Handle("/api/webmention", webmentionee)
srv := http.Server{
	Addr:    ":8080",
	Handler: mux,
}

// [!] should be started before http handler starts receiving
// [!] You can start as many processing goroutines as you'd like
go webmentionee.ProcessMentions()
go func() {
	if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
		log.Printf("http server error: %v", err)
	}
}()

// [!] Once it's time for shutdown...
shutdownCtx, release := context.WithTimeout(context.Background(), 20*time.Second)
defer release()
srv.SetKeepAlivesEnabled(false)
if err := srv.Shutdown(shutdownCtx); err != nil {
	log.Printf("http shutdown error: %v", err)
}
// [!] shut down the receiver only after http endpoint stopped
webmentionee.Shutdown(shutdownCtx)

func NewReceiver

func NewReceiver(opts ...ReceiverOption) *Receiver

func (*Receiver) ProcessMentions

func (receiver *Receiver) ProcessMentions()

ProcessMentions does not return until stopped by calling Shutdown. It is intended to run this function in its own goroutine. You may start multiple goroutines all running this function.

func (*Receiver) ServeHTTP

func (receiver *Receiver) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*Receiver) Shutdown

func (receiver *Receiver) Shutdown(ctx context.Context)

Shutdown causes the webmention service to stop accepting any new mentions. Mentions currently waiting in the request queue will still be processed, until ctx expires. The http server must be stopped first, ServeHTTP will panic otherwise.

type ReceiverOption

type ReceiverOption func(*Receiver)

func WithAcceptsFunc

func WithAcceptsFunc(accepts TargetAcceptsFunc) ReceiverOption

func WithCacheTimeout

func WithCacheTimeout(d time.Duration) ReceiverOption

WithCacheTimeout configures the time period which must pass between receiving updates on a mention. If a mention is sent again within this period, it is answered with http.StatusTooManyRequests.

func WithFetchUserAgent

func WithFetchUserAgent(agent string) ReceiverOption

WithFetchUserAgent configures the user agent to be used when fetching a mention's source.

func WithMediaHandler

func WithMediaHandler(mime string, qweight float64, handler MediaHandler) ReceiverOption

Register a handler for a certain media type. If multiple handlers for the same type are registered, only the last handler will be considered. The default handlers are:

• text/plain;q=1.0: PlainHandler
• text/html;q=0.1: HtmlHandler

To remove any of the default handlers, pass a nil handler.

func WithNotifier

func WithNotifier(notifiers ...Notifier) ReceiverOption

func WithQueueSize

func WithQueueSize(size int) ReceiverOption

Configure size of the request queue. The server will start returning http.StatusTooManyRequests when the request queue is full.

type Sender

type Sender struct {
	UserAgent  string
	HttpClient *http.Client
}

func NewSender

func NewSender(opts ...SenderOption) *Sender

func (*Sender) DiscoverEndpoint

func (sender *Sender) DiscoverEndpoint(target URL) (endpoint URL, err error)

DiscoverEndpoint searches the target for a webmention endpoint. Search stops at the first link that defines a webmention relationship. If that link is not a valid url, ErrInvalidRelWebmention is returned (check with errors.Is). If no link with a webmention relationship is found, ErrNoEndpointFound is returned. Any other error type indicates that we made a mistake, and not the target.

func (*Sender) Mention

func (sender *Sender) Mention(source, target URL) error

func (*Sender) MentionMany

func (sender *Sender) MentionMany(source URL, targets []URL) (err error)

func (*Sender) Update

func (sender *Sender) Update(source URL, pastTargets, currentTargets []URL) error

type SenderOption

type SenderOption func(*Sender)

func WithUserAgent

func WithUserAgent(agent string) SenderOption

Use a custom user agent when sending web mentions. Should (but doesn't have to) include the string "Webmention" to give the receiver an indication as to the purpose of requests.

type Status

type Status string

func HtmlHandler

func HtmlHandler(content io.Reader, target URL) (status Status, err error)

func PlainHandler

func PlainHandler(content io.Reader, target URL) (status Status, err error)

type TargetAcceptsFunc

type TargetAcceptsFunc func(source, target URL) bool

type URL

type URL = *url.URL

type WebMentionSender

type WebMentionSender interface {
	// Mention notifies the target url that it is being linked to by the source url.
	// Precondition: the source url must actually contain an exact match of the target url.
	Mention(source, target URL) error

	// Calls Mention for each of the target urls.
	// All mentions are made from the same source.
	// Continues on on errors with the next target.
	// The returned error is a composite consisting of all encountered errors.
	MentionMany(source URL, targets []URL) error

	// Update resends any previously sent webmentions for the source url.
	// pastTargets are all targets mentioned by the source in its last version.
	// currentTargets are all targets mentioned by the source in its current version.
	// If the source url has been deleted, it is expected (of the user) to
	// have it setup to return 410 Gone and return a tombstone
	// representation in the body.
	// Update can also be called if its the first time mentioning a post,
	// in which case an empty or nil pastTargets should be passed.
	Update(source URL, pastTargets, currentTargets []URL) error
}