bigbro

README

Big Brother

bigbro is a website interaction logging service. Logging can currently be written directly to a file (i.e. CSV) or ingested to Elasticsearch.

Client

To use bigbro on a website, include the following Javascript snippet (this repository hosts bigbro.js in the js folder):

<script type="text/javascript" src="bigbro.js"></script>
<script type="text/javascript">
    BigBro.init("username", "localhost:1984");
</script>

This will allow the page to capture most events that occur by interacting with the page.

The arguments to BigBro.init are as follows:

• actor: A unique identifier of the current user.
• server: The address bigbro is running on (please omit protocol; this is determined automatically).
• (optional) events: A list of events that will be listened on globally (at the window level); e.g. "click", "mousemove".

Additional events can also be loaded in at initiation:

<script type="text/javascript" src="bigbro.js"></script>
<script type="text/javascript">
    BigBro.init("username", "localhost:1984", ["mousemove", "onload"]);
</script>

Custom logging events can also be added, for example:

<script type="text/javascript" src="bigbro.js"></script>
<script type="text/javascript">
    let bb = BigBro.init("username", "localhost:1984");
    window.addEventListener("click", function (e) {
        bb.log(e, "custom_event");
    })
</script>

Server

bigbro is written in Go. To install locally, please use:

$ go install github.com/hscells/bigbro/cmd/bigbro

Alternatively, download a prebuilt binary.

For help with the server, run:

$ bigbro --help

Usage: bigbro [--filename FILENAME] [--index INDEX] [--v V] [--url URL] FORMAT

Positional arguments:
  FORMAT                 how events should be formatted and written

Options:
  --filename FILENAME    filename to output logs to
  --index INDEX          index for Elasticsearch to use
  --v V                  version for Elasticsearch event type
  --url URL              URL for Elasticsearch
  --help, -h             display this help and exit
  --version              display version and exit

To output logs to a CSV file, use:

$ bigbro --filename my_application.log csv

To output logs to Elasticsearch, use:

$ bigbro --index bigbro --v 1 --url http://localhost:9200 elasticsearch

Tools

To perform analysis, one may create heatmaps with the bbheat tool. To install this tool, use:

$ go install github.com/hscells/bigbro/cmd/bbheat

For help with this tool, run:

$ bbheat --help

Usage: bbheat --log LOG --heatmap HEATMAP [--image IMAGE] [--start START] [--end END] [--interval INTERVAL] [--location LOCATION] [--actor ACTOR] [--method METHOD] [--width WIDTH] [--height HEIGHT]

Options:
  --log LOG              path to log file produced by bigbro
  --heatmap HEATMAP      path to output heatmap to
  --image IMAGE          path to image file for analysis
  --start START          time to start analysis
  --end END              time to end analysis
  --interval INTERVAL    create an animation every n seconds
  --location LOCATION    URL to limit analysis (regex)
  --actor ACTOR          actor to limit analysis
  --method METHOD        method to limit analysis
  --width WIDTH          width to limit analysis
  --height HEIGHT        height to limit analysis
  --help, -h             display this help and exit
  --version              display version and exit

bbheat can create both static heatmaps and animations. The image format accepted for input is png. The output format of static heatmaps is png. The output format of animations is gif. To make a gif use the interval argument to specify how many seconds must pass the log for a new heatmap to be made. The start time, end time, actor, location, and method can be specified to limit aspects of the log used in the heatmap. The location accepts a regular expression. To overlay the heatmap over an image, use the image option. This image must be a png. To create a heatmap, the screen width and height must be the same. If there are multiple screen sizes in the log, limit this with the width and height arguments.

Documentation

Index

• Variables
• func WriteCapture(capture Capture) error
• func WsEvent(ws *websocket.Conn, l Logger)
• func WsRecord(ws *websocket.Conn)
• type CSVFormatter
  • func (l CSVFormatter) Format(e Event) string
  • func (l CSVFormatter) Write(e Event) error
• type Capture
• type ElasticsearchFormatter
  • func NewElasticsearchFormatter(index, version, url string) (ElasticsearchFormatter, error)
  • func (f ElasticsearchFormatter) Format(e Event) string
  • func (f ElasticsearchFormatter) Write(e Event) error
• type Event
• type Formatter
• type Logger
  • func NewCSVLogger(name string) (Logger, error)
  • func NewElasticsearchLogger(index, version, url string) (Logger, error)
  • func (l Logger) GinEndpoint(c *gin.Context)
  • func (l Logger) GorillaEndpoint(w http.ResponseWriter, r *http.Request)
  • func (l Logger) Log(e Event) error
• type LogstashEvent

Variables

var (
	CapturePath = "captures"
)

var Upgrader = websocket.Upgrader{
	ReadBufferSize: 1024,
	CheckOrigin: func(r *http.Request) bool {
		return true
	},
}

Upgrader upgrades a web socket.

Functions

func WriteCapture

func WriteCapture(capture Capture) error

func WsEvent

func WsEvent(ws *websocket.Conn, l Logger)

WsEvent handles reading events from the web socket.

func WsRecord

func WsRecord(ws *websocket.Conn)

WsRecord handles reading screen captures from the web socket.

Types

type CSVFormatter

type CSVFormatter struct {
	// contains filtered or unexported fields
}

CSVFormatter is a formatter that formats in comma separated format.

func (CSVFormatter) Format

func (l CSVFormatter) Format(e Event) string

Format in comma separated file.

func (CSVFormatter) Write

func (l CSVFormatter) Write(e Event) error

Write the csv line to file.

type Capture

type Capture struct {
	Actor string    `json:"actor"`
	Data  string    `json:"data"`
	Time  time.Time `json:"time"`
}

type ElasticsearchFormatter

type ElasticsearchFormatter struct {
	// contains filtered or unexported fields
}

ElasticsearchFormatter is a log formatter that can output to Elasticsearch.

func NewElasticsearchFormatter

func NewElasticsearchFormatter(index, version, url string) (ElasticsearchFormatter, error)

NewElasticsearchFormatter creates a new formatter for Elasticsearch.

func (ElasticsearchFormatter) Format

func (f ElasticsearchFormatter) Format(e Event) string

func (ElasticsearchFormatter) Write

func (f ElasticsearchFormatter) Write(e Event) error

type Event

type Event struct {
	// The element which has been triggered.
	Target string `json:"target",csv:"target"`
	// The name attribute of the element.
	Name string `json:"name",csv:"name"`
	// The id attribute of the element.
	ID string `json:"id",csv:"id"`
	// The method which triggered the event.
	Method string `json:"method",csv:"method"`
	// The web page location on the server.
	Location string `json:"location",csv:"location"`
	// Any additional information that can be useful.
	Comment string `json:"comment",csv:"comment"`
	// X position of the event.
	X int `json:"x",csv:"x"`
	// Y position of the event.
	Y int `json:"y",csv:"y"`
	// Width of the actors screen.
	ScreenWidth int `json:"screenWidth",csv:"screenWidth"`
	// Height of the actors screen.
	ScreenHeight int `json:"screenHeight",csv:"screenHeight"`
	// The time the Event happened.
	Time time.Time `json:"time",csv:"time"`
	// The actor that caused the Event.
	Actor string `json:"actor",csv:"actor"`
}

Event is something which has happened on a web page.

type Formatter

type Formatter interface {
	Format(e Event) string
	Write(e Event) error
}

Formatter is a way of specifying how to format an event for logging.

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger is the way in which logs are written to a file.

func NewCSVLogger

func NewCSVLogger(name string) (Logger, error)

NewCSVLogger creates a new CSV logger.

func NewElasticsearchLogger

func NewElasticsearchLogger(index, version, url string) (Logger, error)

NewElasticsearchLogger creates a new Elasticsearch logger.

func (Logger) GinEndpoint

func (l Logger) GinEndpoint(c *gin.Context)

func (Logger) GorillaEndpoint

func (l Logger) GorillaEndpoint(w http.ResponseWriter, r *http.Request)

func (Logger) Log

func (l Logger) Log(e Event) error

Log writes an event to the log file using the specified formatter.

type LogstashEvent

type LogstashEvent struct {
	Message   string    `json:"message"`
	Version   string    `json:"@version"`
	Timestamp time.Time `json:"@timestamp"`
	Type      string    `json:"type"`
	Host      string    `json:"host"`
	Event     Event     `json:"event"`
}

LogstashEvent is an event that is recognised by Elasticsearch.