incus

package module
v0.0.0-...-825a8c7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 11, 2015 License: MIT Imports: 23 Imported by: 3

README

Incus Build Status Coverage Status

incus

Middleware for distributing messages via websockets, long polling, and push notifications

Features

  • Websocket authentication and management
  • Long Poll fall back
  • iOS push notifications support through APNS
  • Android push notifications support through GCM
  • Routing messages to specific phone, authenticated user, or webpage url
  • Configurable option for allowing users to send messages to other users
  • Redis pub/sub and Redis List support for sending messages from an application
  • SSL support
  • Stats logging

Usage

diagram

Front-end bindings
JavaScript: incus.js

The incus.js front-end npm browserified module is provided for consuming WebSocket events in the Browser or server-side. Self-contained, minified downloads are also available.

Application to a web browser

UID is a unique identifier for the user. It can be anywhere from an auto incremented ID to something more private such as a session id or OAuth token. If messages to users should be private from other users, then you should consider the UID a shared secret between the user and the server.

To send events to Incus from your webapp you need to publish a json formated string to a Redis pub/sub channel that Incus is listening on. This channel key can be configured but defaults to Incus. The json format is as follows:

{
    "command" : {
        "command" : string (message|setpage),
        "user"    : (optional) string -- Unique User ID,
        "page"    : (optional) string -- page identifier
    },
    "message" : {
        "event" : string,
        "data"  : object,
        "time"  : int
    }
}

the command is used to route the message to the correct user.

  • if user and page are both unset the message object will be sent to all users
  • if both user and page are set the message object will be sent to that user on that page
  • if just user is set, the message object will be sent to all sockets owned by the user identified by UID
  • if just page is set, the message object will be sent to all sockets whose page matches the page identifier
Push notifications

To send push notifications from your app, you need to push a json formated string to a Redis list. The list key is configurable but defaults to Incus_Queue

Android and iOS have slightly different schemas for sending push notifications.

iOS:
{
    "command" : {
        "command"      : "push",
        "push_type"    : "ios",
        "device_token" : string -- device token registered with APNS,
        "build"        : string -- build environment (store|beta|enterprise|development)
    },
    "message" : {
        "event" : string,
        "data"  : {
            "badge_count": optional int,
            "message_text": string,
            ...,
        },
        "time"  : int
    }
}

Notes:

  • At this time, dictionary APNS alerts are not supported.
Android:

Multiple registration ids can be listed in the same command

{
    "command" : {
        "command"          : "push",
        "push_type"        : "android",
        "registration_ids" : string -- one or more registration ids separated by commas
    },
    "message" : {
        "event" : string,
        "data"  : object,
        "time"  : int
    }
}
Presence-based message routing
{
    "command" : {
        "command": "pushormessage",
        "user": string -- Unique User ID,
        "device_token": string -- device token registered with APNS,
        "build": string -- build environment (store|beta|enterprise|development)
        "registration_ids": string -- one or more registration ids separated by commas
    },
    "message" : {
        "push": {
            "ios": {
                "badge_count": optional int,
                "message_text": string,
                ...
            },
            "android": {
                ...
            }
        },
        "websocket": {
            ...
        }
        "time": int
    }
}
APNS and GCM errors

Incus does not interact with the APNS Feedback Service. You should follow the APNS' guidelines on failed push attempts. They require querying their feedback service daily to find bad device tokens.

The GCM service does not offer a feedback service. When a push fails, Incus will add all relevant information to an error list in Redis (defaults to Incus_Android_Error_Queue). This should be used to remove bad registration ids from your app.

Installation

Method 1: Docker

  • Install Docker

  • Create a folder to share with Docker container:

mkdir -p ~/incus
  • Download configuration file and edit it at will:
wget https://raw.githubusercontent.com/Imgur/incus/master/config.yml -O ~/incus/config.yml
  • Start an instance of Redis (optional), skip it if you run your own Redis instance:
docker run -d --name incusredis redis

Start Incus:

docker run -i --name incus -p 4000:4000 --net="host" -v ~/incus:/etc/incus imgur/incus

To stop Incus run:

docker stop incus

To run it again simply:

docker run incus
Method 2: Source

Install GO: https://golang.org/doc/install

Clone the repo:

mkdir $GOPATH/src/github.com/Imgur/
cd $GOPATH/src/github.com/Imgur/
git clone git@github.com:Imgur/incus.git

To Install:

cd $GOPATH/src/github.com/Imgur/incus
go get -v ./...
go install -v ./incus
cp $GOPATH/bin/incus /usr/sbin/incus
cp scripts/initd.sh /etc/init.d/incus
mkdir /etc/incus
cp config.yml /etc/incus/config.yml
touch /var/log/incus.log

Starting, Stopping, Restarting incus:

sudo /etc/init.d/incus start
sudo /etc/init.d/incus stop
sudo /etc/init.d/incus restart

Configuration

Incus needs to be restarted after any configuration change.

CLIENT_BROADCASTS

true

Clients may send messages to other clients

false

Clients may not send messages to other clients.

Default: true

LISTENING_PORT

This value controls the port that Incus binds to (TCP).

Default: 4000

CONNECTION_TIMEOUT (unstable)

This value controls how long TCP connections are held open for.

0

Connections are held open forever.

anything else

Connections are held open for this many seconds.

Default: 0

LOG_LEVEL

debug

All messages, including errors and debug are printed to standard output.

error

Only errors are printed to standard output.

Default: debug

REDIS_PORT_6379_TCP_ADDR

This value controls the TCP address (or hostname) to connect to Redis.

Note: The variable name is always REDIS_PORT_6379_TCP_ADDR even if the port is not 6379.

Default: 127.0.0.1

REDIS_PORT_6379_TCP_PORT

This value controls the TCP port to connect to Redis.

Note: The variable name is always REDIS_PORT_6379_TCP_PORT even if the port is not 6379.

Default: 6379

REDIS_MESSAGE_CHANNEL

This value controls the Redis PubSub channel to use.

Default: Incus

TLS_ENABLED

This value controls whether the server will also listen on a TLS-enabled port.

false

TLS is disabled, so the server will only listen over its insecure port.

true

TLS is enabled, so the socket will listen over both its insecure port and its TLS-enabled secure port.

Default: false

TLS_PORT

This value controls what TCP port is exposed when using TLS

Default: 443

CERT_FILE

This value controls what X.509 certificate is offered to clients connecting on the TLS port. The certificate is expected in PEM format. The value is a path name resolved relative to the working directory of Incus.

Default: cert.pem

KEY_FILE

This value controls what X.509 private key is used for decrypting the TLS traffic. The key is expected in PEM format.

Default: key.pem

APNS_ENABLED

This value controls whether the server will listen for and send iOS push notifications

false

APSN is disabled, the server will not listen for iOS push notifications

true

APNS is enabled, the server will listen for and send iOS push notifications

Default: false

APNS_[BUILD]_CERT

Where [BUILD] is one of: DEVELOPMENT, STORE, ENTERPRISE, or BETA

This value controls what APNS granted cert the server will use when calling the APNS API. Each build environment has its own instance of this configuration variable.

Default: myapnsappcert.pem

APNS_[BUILD]_PRIVATE_KEY

Where [BUILD] is one of: DEVELOPMENT, STORE, ENTERPRISE, or BETA

This value controls what APNS granted private key the server will use when calling the APNS API. Each build environment has its own instance of this configuration variable.

Default: myapnsappprivatekey.pem

APNS_[BUILD]_URL

Where [BUILD] is one of: DEVELOPMENT, STORE, ENTERPRISE, or BETA

This value controls what APNS url is used when calling the APNS API. Each build environment has its own instance of this configuration variable.

Default: gateway.push.apple.com:2195

APNS_DEVELOPMENT_URL defaults to gateway.sandbox.push.apple.com:2195

IOS_PUSH_SOUND

This value controls what sound plays on push notification receive.

Default: bingbong.aiff

GCM_ENABLED

This value controls whether the server will listen for and send Android push notifications

false

GCM is disabled, the server will not listen for Android push notifications

true

GCM is enabled, the server will listen for and send Android push notifications

Default: false

GCM_API_KEY

This is the GCM granted api key used for calling the GCM API.

Default: foobar

ANDROID_ERROR_QUEUE

This value controls where Android push errors are stored for later retrieval.

Default: Incus_Android_Error_Queue

Documentation

Index

Constants

View Source 
const ClientsKey = "SocketClients"
View Source 
const PageKey = "PageClients"
View Source 
const PresenceKeyPrefix = "ClientPresence"

Variables

View Source 
var (
	DEBUG        bool = false
	CLIENT_BROAD bool = false
)

Functions

func ConfigOption 

func ConfigOption(key string, default_value interface{}) string

func NewConfig 

func NewConfig(configFilePath string)

Types

type CommandMsg 

type CommandMsg struct {
	Command map[string]string      `json:"command"`
	Message map[string]interface{} `json:"message,omitempty"`
}

func (*CommandMsg) FromRedis 

func (this *CommandMsg) FromRedis(server *Server)

func (*CommandMsg) FromSocket 

func (this *CommandMsg) FromSocket(sock *Socket)

type DatadogStats 

type DatadogStats struct {
	// contains filtered or unexported fields
}

func NewDatadogStats 

func NewDatadogStats(datadogHost string) (*DatadogStats, error)

func (*DatadogStats) LogAPNSError 

func (d *DatadogStats) LogAPNSError()

func (*DatadogStats) LogAPNSPush 

func (d *DatadogStats) LogAPNSPush()

func (*DatadogStats) LogBroadcastMessage 

func (d *DatadogStats) LogBroadcastMessage()

func (*DatadogStats) LogClientCount 

func (d *DatadogStats) LogClientCount(clients int64)

func (*DatadogStats) LogCommand 

func (d *DatadogStats) LogCommand(from, cmdType string)

func (*DatadogStats) LogGCMError 

func (d *DatadogStats) LogGCMError()

func (*DatadogStats) LogGCMFailure 

func (d *DatadogStats) LogGCMFailure()

func (*DatadogStats) LogGCMPush 

func (d *DatadogStats) LogGCMPush()

func (*DatadogStats) LogGoroutines 

func (d *DatadogStats) LogGoroutines(goroutines int)

func (*DatadogStats) LogInvalidJSON 

func (d *DatadogStats) LogInvalidJSON()

func (*DatadogStats) LogLongpollConnect 

func (d *DatadogStats) LogLongpollConnect()

func (*DatadogStats) LogLongpollDisconnect 

func (d *DatadogStats) LogLongpollDisconnect()

func (*DatadogStats) LogPageMessage 

func (d *DatadogStats) LogPageMessage()

func (*DatadogStats) LogPendingRedisActivityCommandsListLength 

func (d *DatadogStats) LogPendingRedisActivityCommandsListLength(length int)

func (*DatadogStats) LogReadMessage 

func (d *DatadogStats) LogReadMessage()

func (*DatadogStats) LogStartup 

func (d *DatadogStats) LogStartup()

func (*DatadogStats) LogUserMessage 

func (d *DatadogStats) LogUserMessage()

func (*DatadogStats) LogWebsocketConnection 

func (d *DatadogStats) LogWebsocketConnection()

func (*DatadogStats) LogWebsocketDisconnection 

func (d *DatadogStats) LogWebsocketDisconnection()

func (*DatadogStats) LogWriteMessage 

func (d *DatadogStats) LogWriteMessage()

type DiscardStats 

type DiscardStats struct{}

func (*DiscardStats) LogAPNSError 

func (d *DiscardStats) LogAPNSError()

func (*DiscardStats) LogAPNSPush 

func (d *DiscardStats) LogAPNSPush()

func (*DiscardStats) LogBroadcastMessage 

func (d *DiscardStats) LogBroadcastMessage()

func (*DiscardStats) LogClientCount 

func (d *DiscardStats) LogClientCount(int64)

func (*DiscardStats) LogCommand 

func (d *DiscardStats) LogCommand(from, cmdType string)

func (*DiscardStats) LogGCMError 

func (d *DiscardStats) LogGCMError()

func (*DiscardStats) LogGCMFailure 

func (d *DiscardStats) LogGCMFailure()

func (*DiscardStats) LogGCMPush 

func (d *DiscardStats) LogGCMPush()

func (*DiscardStats) LogGoroutines 

func (d *DiscardStats) LogGoroutines(int)

func (*DiscardStats) LogInvalidJSON 

func (d *DiscardStats) LogInvalidJSON()

func (*DiscardStats) LogLongpollConnect 

func (d *DiscardStats) LogLongpollConnect()

func (*DiscardStats) LogLongpollDisconnect 

func (d *DiscardStats) LogLongpollDisconnect()

func (*DiscardStats) LogPageMessage 

func (d *DiscardStats) LogPageMessage()

func (*DiscardStats) LogPendingRedisActivityCommandsListLength 

func (d *DiscardStats) LogPendingRedisActivityCommandsListLength(int)

func (*DiscardStats) LogReadMessage 

func (d *DiscardStats) LogReadMessage()

func (*DiscardStats) LogStartup 

func (d *DiscardStats) LogStartup()

func (*DiscardStats) LogUserMessage 

func (d *DiscardStats) LogUserMessage()

func (*DiscardStats) LogWebsocketConnection 

func (d *DiscardStats) LogWebsocketConnection()

func (*DiscardStats) LogWebsocketDisconnection 

func (d *DiscardStats) LogWebsocketDisconnection()

func (*DiscardStats) LogWriteMessage 

func (d *DiscardStats) LogWriteMessage()

type GCMClient 

type GCMClient interface {
	Send(*gcm.Message, int) (*gcm.Response, error)
}

type MemoryStore 

type MemoryStore struct {
	// contains filtered or unexported fields
}

func (*MemoryStore) Client 

func (this *MemoryStore) Client(UID string) (map[string]*Socket, error)

func (*MemoryStore) Clients 

func (this *MemoryStore) Clients() map[string]map[string]*Socket

func (*MemoryStore) Count 

func (this *MemoryStore) Count() (int64, error)

func (*MemoryStore) Remove 

func (this *MemoryStore) Remove(sock *Socket) error

func (*MemoryStore) Save 

func (this *MemoryStore) Save(sock *Socket) error

func (*MemoryStore) SetPage 

func (this *MemoryStore) SetPage(sock *Socket) error

func (*MemoryStore) UnsetPage 

func (this *MemoryStore) UnsetPage(sock *Socket) error

type Message 

type Message struct {
	Event string                 `json:"event"`
	Data  map[string]interface{} `json:"data"`
	Time  int64                  `json:"time"`
	Url   string                 `json:"internal_url,omitempty"`
}

type RedisCallback 

type RedisCallback (func(redis.Conn) (interface{}, error))

type RedisCommand 

type RedisCommand struct {
	Callback RedisCallback
	Result   chan RedisCommandResult
}

type RedisCommandResult 

type RedisCommandResult struct {
	Error error
	Value interface{}
}

type RedisQueue 

type RedisQueue struct {
	// contains filtered or unexported fields
}

A queue of redis commands

func NewRedisQueue 

func NewRedisQueue(consumers int, stats RuntimeStats, pool *redisPool) *RedisQueue

func (*RedisQueue) DispatchForever 

func (r *RedisQueue) DispatchForever()

Dispatch a command to a goroutine that is blocking on receiving a command.

func (*RedisQueue) ReceiveForever 

func (r *RedisQueue) ReceiveForever()

Receive a command, save/enqueue it, and wake the dispatching goroutine

func (*RedisQueue) RunAsyncTimeout 

func (r *RedisQueue) RunAsyncTimeout(timeout time.Duration, callback RedisCallback) RedisCommandResult

type RedisQueueConsumer 

type RedisQueueConsumer struct {
	// contains filtered or unexported fields
}

func NewRedisQueueConsumer 

func NewRedisQueueConsumer(commands <-chan RedisCommand, pool *redisPool) *RedisQueueConsumer

func (*RedisQueueConsumer) ConsumeForever 

func (r *RedisQueueConsumer) ConsumeForever()

type RedisStore 

type RedisStore struct {
	// contains filtered or unexported fields
}

func (*RedisStore) ActivateLongpollKillswitch 

func (this *RedisStore) ActivateLongpollKillswitch(seconds int64) error

func (*RedisStore) Clients 

func (this *RedisStore) Clients() ([]string, error)

func (*RedisStore) CloseConn 

func (this *RedisStore) CloseConn(conn redis.Conn)

func (*RedisStore) Count 

func (this *RedisStore) Count() (int64, error)

func (*RedisStore) DeactivateLongpollKillswitch 

func (this *RedisStore) DeactivateLongpollKillswitch() error

func (*RedisStore) GetConn 

func (this *RedisStore) GetConn() (redis.Conn, error)

func (*RedisStore) GetIsLongpollKillswitchActive 

func (this *RedisStore) GetIsLongpollKillswitchActive() (bool, error)

func (*RedisStore) MarkActive 

func (this *RedisStore) MarkActive(user, socket_id string, timestamp int64) error

func (*RedisStore) MarkInactive 

func (this *RedisStore) MarkInactive(user, socket_id string) error

func (*RedisStore) Poll 

func (this *RedisStore) Poll(c chan []byte, queue string) error

func (*RedisStore) Publish 

func (this *RedisStore) Publish(channel string, message string)

func (*RedisStore) Push 

func (this *RedisStore) Push(queue string, message string)

func (*RedisStore) QueryIsUserActive 

func (this *RedisStore) QueryIsUserActive(user string, nowTimestamp int64) (bool, error)

func (*RedisStore) Remove 

func (this *RedisStore) Remove(sock *Socket) error

func (*RedisStore) Save 

func (this *RedisStore) Save(sock *Socket) error

func (*RedisStore) SetPage 

func (this *RedisStore) SetPage(sock *Socket) error

func (*RedisStore) Subscribe 

func (this *RedisStore) Subscribe(c chan []byte, channel string) (redis.Conn, error)

func (*RedisStore) UnsetPage 

func (this *RedisStore) UnsetPage(sock *Socket) error

type RuntimeStats 

type RuntimeStats interface {
	LogStartup()

	LogClientCount(int64)
	LogGoroutines(int)

	LogCommand(from, cmdType string)
	LogPageMessage()
	LogUserMessage()
	LogBroadcastMessage()
	LogReadMessage()
	LogWriteMessage()
	LogInvalidJSON()

	LogWebsocketConnection()
	LogWebsocketDisconnection()

	LogLongpollConnect()
	LogLongpollDisconnect()

	LogAPNSPush()
	LogAPNSError()

	LogGCMPush()
	LogGCMError()
	LogGCMFailure()

	LogPendingRedisActivityCommandsListLength(int)
}

type Server 

type Server struct {
	ID    string
	Store *Storage
	Stats RuntimeStats
	// contains filtered or unexported fields
}

func NewServer 

func NewServer(store *Storage, stats RuntimeStats) *Server

func (*Server) GetAPNSClient 

func (this *Server) GetAPNSClient(build string) apns.APNSClient

func (*Server) GetGCMClient 

func (this *Server) GetGCMClient() GCMClient

func (*Server) ListenForHTTPPings 

func (this *Server) ListenForHTTPPings()

func (*Server) ListenFromLongpoll 

func (this *Server) ListenFromLongpoll()

func (*Server) ListenFromRedis 

func (this *Server) ListenFromRedis()

func (*Server) ListenFromSockets 

func (this *Server) ListenFromSockets()

func (*Server) LogConnectedClientsPeriodically 

func (this *Server) LogConnectedClientsPeriodically(period time.Duration)

func (*Server) MonitorLongpollKillswitch 

func (this *Server) MonitorLongpollKillswitch()

func (*Server) RecordStats 

func (this *Server) RecordStats(period time.Duration)

func (*Server) SendHeartbeatsPeriodically 

func (this *Server) SendHeartbeatsPeriodically(period time.Duration)

type Socket 

type Socket struct {
	SID  string // socket ID, randomly generated
	UID  string // User ID, passed in via client
	Page string // Current page, if set.

	Server *Server
	// contains filtered or unexported fields
}

func (*Socket) Authenticate 

func (this *Socket) Authenticate(UID string) error

func (*Socket) Close 

func (this *Socket) Close() error

type Storage 

type Storage struct {
	StorageType string
	// contains filtered or unexported fields
}

func NewStore 

func NewStore(stats RuntimeStats) *Storage

func (*Storage) Client 

func (this *Storage) Client(UID string) (map[string]*Socket, error)

func (*Storage) ClientList 

func (this *Storage) ClientList() ([]string, error)

func (*Storage) Clients 

func (this *Storage) Clients() map[string]map[string]*Socket

func (*Storage) Count 

func (this *Storage) Count() (int64, error)

func (*Storage) Remove 

func (this *Storage) Remove(sock *Socket) error

func (*Storage) Save 

func (this *Storage) Save(sock *Socket) error

func (*Storage) SetPage 

func (this *Storage) SetPage(sock *Socket) error

func (*Storage) UnsetPage 

func (this *Storage) UnsetPage(sock *Socket) error

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.