golongpoll

README

golongpoll

golang HTTP longpolling library, making web pub-sub easy!

Table of contents

• Basic Usage
  • HTTP Subscription Handler
• What is longpolling?
• Included examples
  • Basic
  • Advanced
• More advanced use
  • Events with JSON payloads
  • Wrapping subscriptions
  • Publishing events via the web

Basic usage

To use, create a LongpollManager and then use it to publish events and expose an HTTP handler to subscribe to events.

import	"github.com/jcuga/golongpoll"

// This launches a goroutine and creates channels for all the plumbing
manager, err := golongpoll.CreateManager()

// Expose events to browsers
// See subsection on how to interact with the subscription handler
http.HandleFunc("/events", manager.SubscriptionHandler)
http.ListenAndServe("127.0.0.1:8081", nil)

// Pass the manager around or create closures and publish:
manager.Publish("subscription-category", "Some data.  Can be string or any obj convertable to JSON")
manager.Publish("different-category", "More data")

Note that you can add extra access-control, validation, or other behavior on top of the manager's SubscriptionHandler. See the advanced example. This example also shows how to publish a more complicated payload JSON object.

You can also configure the LongpollManager by using

golongpoll.CreateCustomManager(...)

HTTP Subscription Handler

The LongpollManager has a field called SubscriptionHandler that you can attach as an http.HandleFunc.

This HTTP handler has the following URL query params as input.

• timeout number of seconds the server should wait until issuing a timeout response in the event there are no new events during the client's longpoll. The default manager created via CreateManager has a max timeout of 180 seconds, but you can customize this by using CreateCustomManager
• category the subscription category to subscribe to. When you publish an event, you publish it on a specific category.
• since_time optional. the number of milliseconds since epoch. If not provided, defaults to current time. This tells the longpoll server to only give you events that have occurred since this time.

The response from this HTTP handler is one of the following application/json responses:

• error response: {"error": "error message as to why request failed."}
  • Perhaps you forgot to include a query param? Or an invalid timeout?
• timeout response: {"timeout": "no events before timeout"}
  • This means no events occurred within the timeout window. (given your since_time param)
• event(s) response: {"events":[{"timestamp":1447218359843,"category":"farm","data":"Pig went 'Oink! Oink!'"}]}
  • includes one or more event object. If no events occurred, you should get a timeout instead.

To receive a continuous stream of chronological events, you should keep hitting the http handler after each response, but with an updated since_time value equal to that of the last event's timestamp.

You can see how to make these longpoll requests using jquery by viewing the example programs' code.

What is longpolling

Longpolling is a way to get events/data "pushed" to the browser as soon as they occur* (with a usually very small delay). Longpolling is an option to consider when you want updates to be sent from the webserver to a webapp as they occur. This is a one-way communication path. If you need full-duplex communication, consider an alternative like websockets.

To better understand longpolling, let's consider what it improves upon. A naive way to get updates as soon as possible from a webserver is to continuously make AJAX requests from a webpage asking the server if there is anything new.

The problem with this approach is that when there are no updates, you are continuously spamming the webserver with new requests. An alternative approach is to have the webserver wait until there is actually data before responding to your request.

This is an improvement since both the client and the server aren't setting up and tearing down connections so quickly. But you can't just wait forever to hear from the server. So longpolling has the concept of a timeout. If the server waits too long and there are no new events, the server responds to the client that there's nothing new. The client can then initiate a new longpoll request.

Essentially, longpolling is a much more sane version of spamming the server with a bunch of requests asking for new data.

Why not just use websockets instead? Websockets are great if you need to push data in both directions. But if you're really interested in pushing data from the server to the client and not vice-versa, then longpolling may be a viable option for a number of reasons.

• longpolling is just simple, plain old HTTP. The server is just... slow to respond at times.
  • This means much wider range of browser support, especially the older ones
  • Will work over infrastructure that uses proxies that only allow port 80/443
  • Also works well through VPN webclient type products that do "magic" to web traffic
    • As a general rule, the closer to traditional HTTP you are, the wider support you have.

Why does everyone run to websockets even when they only need server-to-client pushing? Probably because it's difficult to get longpolling right. By this I mean handling the subtleties on the server end to make sure that any events that occur in the small window between the time that a client gets a response and before they make a new request, as well as buffering older events in case clients went offline. There is a plethora of posts on the internet to make a half-baked longpoll server, but few if any posts outline how to make a robust one. (that's why you should use golongpoll--it will do this for you!).

Also, depending on what language you're writing the webserver in, longpolling might be more difficult. Think python running in a WSGI container. Without the flexibility of golang and it's channels, such implementations could be quite the headache.

Included examples

There are two fully-functional example programs provided. Basic

This program creates a default LongpollManager, shows how a goroutine can generate some events, and how to subscribe from a webpage. See basic.go

To run this example

go build examples/basic/basic.go
./basic
OR: ./basic.exe

Then visit:

http://127.0.0.1:8081/basic

And observe the events appearing every 0-5 seconds.

Advanced

This program creates a custom LongpollManager, shows how an http handler can publish events, and how to subscribe from a webpage. See advanced.go

To run this example

go build examples/advanced/advanced.go
./advanced
OR: ./advanced.exe

Then visit:

http://127.0.0.1:8081/advanced

Try clicking around and notice the events showing up in the tables. Try opening multiple windows as different users and observe events. Toggle whether user's events are public or private.

More advanced use

All of the below topics are demonstrated in the advanced example: Events with JSON payloads

Try passing any type that is convertable to JSON to Publish(). If the type can be passed to encoding/json.Marshal(), it will work.

Wrapping subscriptions

You can create your own HTTP handler that calls LongpollManager.SubscriptionHandler to add your own layer of logic on top of the subscription handler. Uses include: user authentication/access-control and limiting subscriptions to a known set of categories.

Publishing events via the web

You can create a closure that captures the LongpollManager and attach it as an http handler function. Within that function, simply call Publish().

Documentation

Index

• type LongpollManager
  • func CreateCustomManager(maxTimeoutSeconds, eventBufferSize int, loggingEnabled bool) (*LongpollManager, error)
  • func CreateManager() (*LongpollManager, error)
  • func (m *LongpollManager) Publish(category string, data interface{}) error
  • func (m *LongpollManager) Shutdown()

Types

type LongpollManager

type LongpollManager struct {
	SubscriptionHandler func(w http.ResponseWriter, r *http.Request)
	// contains filtered or unexported fields
}

LongpollManager provides an interface to interact with the internal longpolling pup-sub goroutine.

This allows you to publish events via Publish() If for some reason you want to stop the pub-sub goroutine at any time you can call Shutdown() and all longpolling will be disabled. Note that the pub-sub goroutine will exit on program exit, so for most simple programs, calling Shutdown() is not necessary.

A LongpollManager is created with each subscriptionManager that gets created by CreateLongpollManager() This interface also exposes the HTTP handler that client code can attach to a URL like so:

mux := http.NewServeMux()
mux.HandleFunc("/custom/path/to/events", manager.SubscriptionHandler)

Note, this http handler can be wrapped by another function (try capturing the manager in a closure) to add additional validation, access control, or other functionality on top of the subscription handler.

You can have another http handler publish events by capturing the manager in a closure and calling manager.Publish() from inside a http handler. See the advanced example (examples/advanced/advanced.go)

If for some reason you want multiple goroutines handling different pub-sub channels, you can simply create multiple LongpollManagers.

func CreateCustomManager

func CreateCustomManager(maxTimeoutSeconds, eventBufferSize int, loggingEnabled bool) (*LongpollManager, error)

Creates a custom LongpollManager and pub-sub goroutine connected via channels that are exposed via LongpollManager's Publish() function and SubscriptionHandler field which can get used as an http handler function.

The options are as follows. maxTimeoutSeconds, the max number of seconds a longpoll web request can wait before returning a timeout response in the event of no events on that subscription category.

eventBufferSize, the number of events that get buffered per subscription category before we start throwing the oldest events out. These buffers are used to support longpolling for events in the past, and giving a better guarantee that any events that occurred between client's longpoll requests can still be seen by their next request.

loggingEnabled, whether or not log statements are printed out.

func CreateManager

func CreateManager() (*LongpollManager, error)

Creates a basic LongpollManager and pub-sub goroutine connected via channels that are exposed via LongpollManager's Publish() function and SubscriptionHandler field which can get used as an http handler function. This basic LongpollManager's default options should cover most client's longpolling needs without having to worry about details. This uses the following options: maxTimeoutSeconds of 180. eventBufferSize size 250 (buffers this many most recent events per subscription category) and loggingEnabled as true.

Creates a basic LongpollManager with the default settings. This manager is an interface on top of an internal goroutine that uses channels to support event pub-sub.

The default settings should handle most use cases, unless you expect to have a large number of events on a given subscription category within a short amount of time. Perhaps having more than 50 events a second on a single subscription category would be the time to start considering tweaking the settings via CreateCustomManager.

The default settings are: a max longpoll timeout window of 180 seconds, the per-subscription-category event buffer size of 250 events, and logging enabled.

func (*LongpollManager) Publish

func (m *LongpollManager) Publish(category string, data interface{}) error

Publish an event for a given subscription category. This event can have any arbitrary data that is convert-able to JSON via the standard's json.Marshal() the category param must be a non-empty string no longer than 1024, otherwise you get an error.

func (*LongpollManager) Shutdown

func (m *LongpollManager) Shutdown()

Shutdown allows the internal goroutine that handles the longpull pup-sub to be stopped. This may be useful if you want to turn off longpolling without terminating your program. After a shutdown, you can't call Publish() or get any new results from the SubscriptionHandler. Multiple calls to this function on the same manager will result in a panic.