demoinfocs

README

demoinfocs-golang - A CSGO Demo Parser

Is a Go library for super fast parsing and analysing of Counter Strike: Global Offensive (CSGO) demos (aka replays). It is based on Valve's demoinfogo and SatsHelix's demoinfo.

Discussions / Chat

You can use gitter to ask questions and discuss ideas about this project.

Go Get

go get -u github.com/markus-wa/demoinfocs-golang

Example

This is a simple example on how to use the library. It collects all positions where weapons were fired from (using events.WeaponFire) and creates a heatmap using go-heatmap.

Check out the examples folder for more examples and the godoc of the events package for some information about the other available events and their purpose.

package main

import (
	"fmt"
	"image"
	"image/draw"
	"image/jpeg"
	"log"
	"os"

	heatmap "github.com/dustin/go-heatmap"
	schemes "github.com/dustin/go-heatmap/schemes"

	dem "github.com/markus-wa/demoinfocs-golang"
	events "github.com/markus-wa/demoinfocs-golang/events"
	metadata "github.com/markus-wa/demoinfocs-golang/metadata"
)

// Run like this: go run heatmap.go > out.jpg
func main() {
	f, err := os.Open("/path/to/demo.dem")
	checkError(err)
	defer f.Close()

	p := dem.NewParser(f)

	// Parse header (contains map-name etc.)
	header, err := p.ParseHeader()
	checkError(err)

	// Get metadata for the map that's being played
	m := metadata.MapNameToMap[header.MapName]

	// Register handler for WeaponFire, triggered every time a shot is fired
	points := []heatmap.DataPoint{}
	var bounds image.Rectangle
	p.RegisterEventHandler(func(e events.WeaponFire) {
		// Translate positions from in-game coordinates to radar overview
		x, y := m.TranslateScale(e.Shooter.Position.X, e.Shooter.Position.Y)

		// Track bounds to get around the normalization done by the heatmap library
		bounds = updatedBounds(bounds, int(x), int(y))

		// Add shooter's position as datapoint
		// Invert Y since it expects data to be ordered from bottom to top
		points = append(points, heatmap.P(x, y*-1))
	})

	// Parse to end
	err = p.ParseToEnd()
	checkError(err)

	// Get map overview as base image
	fMap, err := os.Open(fmt.Sprintf("../../metadata/maps/%s.jpg", header.MapName))
	checkError(err)

	imgMap, _, err := image.Decode(fMap)
	checkError(err)

	// Create output canvas
	img := image.NewRGBA(imgMap.Bounds())

	draw.Draw(img, imgMap.Bounds(), imgMap, image.ZP, draw.Over)

	// Generate heatmap
	width := bounds.Max.X - bounds.Min.X
	height := bounds.Max.Y - bounds.Min.Y
	imgHeatmap := heatmap.Heatmap(image.Rect(0, 0, width, height), points, 15, 128, schemes.AlphaFire)

	// Draw it on top of the overview
	draw.Draw(img, bounds, imgHeatmap, image.ZP, draw.Over)

	// Write to stdout
	jpeg.Encode(os.Stdout, img, &jpeg.Options{
		Quality: 90,
	})
}

func updatedBounds(b image.Rectangle, x, y int) image.Rectangle {
	if b.Min.X > x || b.Min.X == 0 {
		b.Min.X = x
	} else if b.Max.X < x {
		b.Max.X = x
	}

	if b.Min.Y > y || b.Min.Y == 0 {
		b.Min.Y = y
	} else if b.Max.Y < y {
		b.Max.Y = y
	}

	return b
}

func checkError(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

Result

Running the above code (go run heatmap.go > heatmap.png) will create a JPEG of a radar overview with dots on all the locations where shots were fired.

Features

• Game events (kills, shots, round starts/ends, footsteps etc.) - docs / example
• Tracking of game-state (players, teams, grenades etc.) - docs
• Grenade projectiles / trajectories - docs / example
• Access to entities, server-classes & data-tables - docs / example
• Access to all net-messages - docs / example
• Chat & console messages ¹ - docs / example
• Easy debugging via build-flags
• Built with performance & concurrency in mind

1. Only for some demos; in MM demos the chat is encrypted for example.

Performance / Benchmarks

One of the top priorities of this parser is performance and concurrency.

Here are some benchmark results from a system with a Intel i7 2600k CPU and SSD disk running Windows 10 and a demo with 85'000 frames.

Overview

Benchmark	Description	Average Duration	Speed
BenchmarkConcurrent	Read and parse 8 demos concurrently	2.90 s (per 8 demos)	~234'000 ticks / s
BenchmarkDemoInfoCs	Read demo from drive and parse	1.39 s	~61'000 ticks / s
BenchmarkInMemory	Read demo from memory and parse	1.38 s	~61'000 ticks / s

Raw output

$ go test -run _NONE_ -bench . -benchtime 30s -benchmem -concurrentdemos 8
goos: windows
goarch: amd64
pkg: github.com/markus-wa/demoinfocs-golang
BenchmarkDemoInfoCs-8                 30        1397398190 ns/op        162254528 B/op    839779 allocs/op
BenchmarkInMemory-8                   30        1384877250 ns/op        162109924 B/op    839628 allocs/op
BenchmarkConcurrent-8                 20        2902574295 ns/op        1297042534 B/op  6717163 allocs/op
--- BENCH: BenchmarkConcurrent-8
        demoinfocs_test.go:425: Running concurrency benchmark with 8 demos
        demoinfocs_test.go:425: Running concurrency benchmark with 8 demos
PASS
ok      github.com/markus-wa/demoinfocs-golang  147.800s

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository. There is one caveat however: Beta features - which are marked as such via comments and in release notes - may change in minor releases.

It's recommended to use some kind of dependency management system such as dep to ensure reproducible builds.

Development

Running tests

To run tests Git LFS is required.

git submodule init
git submodule update
pushd cs-demos && git lfs pull -I '*' && popd
go test

Debugging

You can use the build tag debugdemoinfocs (i.e. go test -tags debugdemoinfocs -v) to print out debugging information - such as game events or unhandled demo-messages - during the parsing process.
Side-note: The tag isn't called debug to avoid naming conflicts with other libs (and underscores in tags don't work, apparently).

To change the default debugging behavior Go's ldflags paramter can be used. Example for additionally printing out all server-classes with their properties: -ldflags '-X github.com/markus-wa/demoinfocs-golang.debugServerClasses=YES'

Check out debug_on.go for any other settings that can be changed.

Generating protobuf code

Should you need to re-generate the protobuf generated code in the msg package, you will need the following tools:

• The latest protobuf generator (protoc) from your package manager or https://github.com/google/protobuf/releases

• And protoc-gen-gogofaster from gogoprotobuf to generate code for go.

    go get -u github.com/gogo/protobuf/protoc-gen-gogofaster
  

Make sure both are inside your PATH variable.

After installing these use go generate ./msg to generate the protobuf code.

Documentation

Overview

Package demoinfocs provides a demo parser for the game Counter-Strike: Global Offensive. It is based on the official demoinfogo tool by Valve as well as Stats Helix's demoinfo.

A good entry point to using the library is the Parser type.

Demo events are documented in the events package.

Index

• Variables
• type EventEmitter
• type GameState
  • func (gs GameState) Bomb() *common.Bomb
  • func (gs GameState) Entities() map[int]*st.Entity
  • func (gs GameState) GrenadeProjectiles() map[int]*common.GrenadeProjectile
  • func (gs GameState) Infernos() map[int]*common.Inferno
  • func (gs GameState) IngameTick() int
  • func (gs GameState) Participants() Participants
  • func (gs *GameState) TeamCounterTerrorists() *TeamState
  • func (gs *GameState) TeamTerrorists() *TeamState
• type NetMessageCreator
• type Parser
  • func NewParser(demostream io.Reader) *Parser
  • func NewParserWithConfig(demostream io.Reader, config ParserConfig) *Parser
  • func (p *Parser) Cancel()
  • func (p *Parser) CurrentFrame() int
  • func (p *Parser) CurrentTime() time.Duration
  • func (p *Parser) GameState() *GameState
  • func (p *Parser) Header() common.DemoHeader
  • func (p *Parser) ParseHeader() (common.DemoHeader, error)
  • func (p *Parser) ParseNextFrame() (b bool, err error)
  • func (p *Parser) ParseToEnd() (err error)
  • func (p *Parser) Progress() float32
  • func (p *Parser) RegisterEventHandler(handler interface{}) dp.HandlerIdentifier
  • func (p *Parser) RegisterNetMessageHandler(handler interface{}) dp.HandlerIdentifier
  • func (p *Parser) ServerClasses() st.ServerClasses
  • func (p *Parser) UnregisterEventHandler(identifier dp.HandlerIdentifier)
  • func (p *Parser) UnregisterNetMessageHandler(identifier dp.HandlerIdentifier)
• type ParserConfig
• type Participants
  • func (ptcp Participants) All() []*common.Player
  • func (ptcp Participants) ByEntityID() map[int]*common.Player
  • func (ptcp Participants) ByUserID() map[int]*common.Player
  • func (ptcp Participants) FindByHandle(handle int) *common.Player
  • func (ptcp Participants) Playing() []*common.Player
  • func (ptcp Participants) TeamMembers(team common.Team) []*common.Player
• type TeamState
  • func (ts TeamState) ClanName() string
  • func (ts TeamState) Flag() string
  • func (ts TeamState) ID() int
  • func (ts TeamState) Score() int

Variables

var (
	// ErrCancelled signals that parsing was cancelled via Parser.Cancel()
	ErrCancelled = errors.New("Parsing was cancelled before it finished (ErrCancelled)")

	// ErrUnexpectedEndOfDemo signals that the demo is incomplete / corrupt -
	// these demos may still be useful, check how far the parser got.
	ErrUnexpectedEndOfDemo = errors.New("Demo stream ended unexpectedly (ErrUnexpectedEndOfDemo)")

	// ErrInvalidFileType signals that the input isn't a valid CS:GO demo.
	ErrInvalidFileType = errors.New("Invalid File-Type; expecting HL2DEMO in the first 8 bytes")

	// ErrHeaderNotParsed signals that the header hasn't been parsed before attempting to parse a tick.
	ErrHeaderNotParsed = errors.New("Header must be parsed before trying to parse a tick. See Parser.ParseHeader()")
)

Parsing errors

var DefaultParserConfig = ParserConfig{
	MsgQueueBufferSize: -1,
}

DefaultParserConfig is the default Parser configuration used by NewParser().

Types

type EventEmitter

type EventEmitter interface {
	Register(parser *Parser, eventDispatcher func(event interface{}))
}

EventEmitter is the interface to define additional event-emitters. The emitters may fire additional events by calling the eventDispatcher function received during registration of the emitter.

See also: package fuzzy for existing emitters with fuzzy-logic that depends on the demo-type.

type GameState

type GameState struct {
	// contains filtered or unexported fields
}

GameState contains all game-state relevant information.

func (GameState) Bomb

func (gs GameState) Bomb() *common.Bomb

Bomb returns the current bomb state.

func (GameState) Entities

func (gs GameState) Entities() map[int]*st.Entity

Entities returns all currently existing entities. (Almost?) everything in the game is an entity, such as weapons, players, fire etc.

func (GameState) GrenadeProjectiles

func (gs GameState) GrenadeProjectiles() map[int]*common.GrenadeProjectile

GrenadeProjectiles returns a map from entity-IDs to all live grenade projectiles.

Only constains projectiles currently in-flight or still active (smokes etc.), i.e. have been thrown but have yet to detonate.

func (GameState) Infernos

func (gs GameState) Infernos() map[int]*common.Inferno

Infernos returns a map from entity-IDs to all currently burning infernos (fires from incendiaries and Molotovs).

func (GameState) IngameTick

func (gs GameState) IngameTick() int

IngameTick returns the latest actual tick number of the server during the game.

Watch out, I've seen this return wonky negative numbers at the start of demos.

func (GameState) Participants

func (gs GameState) Participants() Participants

Participants returns a struct with all currently connected players & spectators and utility functions. The struct contains references to the original maps so it's always up-to-date.

func (*GameState) TeamCounterTerrorists

func (gs *GameState) TeamCounterTerrorists() *TeamState

TeamCounterTerrorists returns the TeamState of the CT team.

Make sure to handle swapping sides properly if you keep the reference.

func (*GameState) TeamTerrorists

func (gs *GameState) TeamTerrorists() *TeamState

TeamTerrorists returns the TeamState of the T team.

Make sure to handle swapping sides properly if you keep the reference.

type NetMessageCreator

type NetMessageCreator func() proto.Message

NetMessageCreator creates additional net-messages to be dispatched to net-message handlers.

See also: ParserConfig.AdditionalNetMessageCreators & Parser.RegisterNetMessageHandler()

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser can parse a CS:GO demo. Creating a Parser is done via NewParser().

To start off use Parser.ParseHeader() to parse the demo header. After parsing the header Parser.ParseNextFrame() and Parser.ParseToEnd() can be used to parse the demo.

Use Parser.RegisterEventHandler() to receive notifications about events.

Example (without error handling):

f, _ := os.Open("/path/to/demo.dem")
p := dem.NewParser(f)
header := p.ParseHeader()
fmt.Println("Map:", header.MapName)
p.RegisterEventHandler(func(e events.BombExplode) {
	fmt.Printf(e.Site, "went BOOM!")
})
p.ParseToEnd()

Prints out '{A/B} site went BOOM!' when a bomb explodes.

func NewParser

func NewParser(demostream io.Reader) *Parser

NewParser creates a new Parser with the default configuration. The demostream io.Reader (e.g. os.File or bytes.Reader) must provide demo data in the '.DEM' format.

See also: NewCustomParser() & DefaultParserConfig

func NewParserWithConfig

func NewParserWithConfig(demostream io.Reader, config ParserConfig) *Parser

NewParserWithConfig returns a new Parser with a custom configuration.

See also: NewParser() & ParserConfig

func (*Parser) Cancel

func (p *Parser) Cancel()

Cancel aborts ParseToEnd(). All information that was already read up to this point may still be used (and new events may still be sent out).

func (*Parser) CurrentFrame

func (p *Parser) CurrentFrame() int

CurrentFrame return the number of the current frame, aka. 'demo-tick' (Since demos often have a different tick-rate than the game). Starts with frame 0, should go up to DemoHeader.PlaybackFrames but might not be the case (usually it's just close to it).

func (*Parser) CurrentTime

func (p *Parser) CurrentTime() time.Duration

CurrentTime returns the time elapsed since the start of the demo

func (*Parser) GameState

func (p *Parser) GameState() *GameState

GameState returns the current game-state. It contains most of the relevant information about the game such as players, teams, scores, grenades etc.

func (*Parser) Header

func (p *Parser) Header() common.DemoHeader

Header returns the DemoHeader which contains the demo's metadata. Only possible after ParserHeader() has been called.

func (*Parser) ParseHeader

func (p *Parser) ParseHeader() (common.DemoHeader, error)

ParseHeader attempts to parse the header of the demo.

Returns ErrInvalidFileType if the filestamp (first 8 bytes) doesn't match HL2DEMO.

func (*Parser) ParseNextFrame

func (p *Parser) ParseNextFrame() (b bool, err error)

ParseNextFrame attempts to parse the next frame / demo-tick (not ingame tick).

Returns true unless the demo command 'stop' or an error was encountered. Returns an error if the header hasn't been parsed yet - see Parser.ParseHeader().

May return ErrUnexpectedEndOfDemo for incomplete / corrupt demos. May panic if the demo is corrupt in some way.

See also: ParseToEnd() for parsing the complete demo in one go (faster).

func (*Parser) ParseToEnd

func (p *Parser) ParseToEnd() (err error)

ParseToEnd attempts to parse the demo until the end. Aborts and returns ErrCancelled if Cancel() is called before the end.

See also: ParseNextFrame() for other possible errors.

func (*Parser) Progress

func (p *Parser) Progress() float32

Progress returns the parsing progress from 0 to 1. Where 0 means nothing has been parsed yet and 1 means the demo has been parsed to the end.

Might not be 100% correct since it's just based on the reported tick count of the header.

func (*Parser) RegisterEventHandler

func (p *Parser) RegisterEventHandler(handler interface{}) dp.HandlerIdentifier

RegisterEventHandler registers a handler for game events.

The handler must be of type func(<EventType>) where EventType is the kind of event to be handled. To catch all events func(interface{}) can be used.

Example:

parser.RegisterEventHandler(func(e events.WeaponFired) {
	fmt.Printf("%s fired his %s\n", e.Shooter.Name, e.Weapon.Weapon)
})

Parameter handler has to be of type interface{} because lolnogenerics.

Returns a identifier with which the handler can be removed via UnregisterEventHandler().

func (*Parser) RegisterNetMessageHandler

func (p *Parser) RegisterNetMessageHandler(handler interface{}) dp.HandlerIdentifier

RegisterNetMessageHandler registers a handler for net-messages.

The handler must be of type func(*<MessageType>) where MessageType is the kind of net-message to be handled.

Returns a identifier with which the handler can be removed via UnregisterNetMessageHandler().

See also: RegisterEventHandler()

func (*Parser) ServerClasses

func (p *Parser) ServerClasses() st.ServerClasses

ServerClasses returns the server-classes of this demo. These are available after events.DataTablesParsed has been fired.

func (*Parser) UnregisterEventHandler

func (p *Parser) UnregisterEventHandler(identifier dp.HandlerIdentifier)

UnregisterEventHandler removes a game event handler via identifier.

The identifier is returned at registration by RegisterEventHandler().

func (*Parser) UnregisterNetMessageHandler

func (p *Parser) UnregisterNetMessageHandler(identifier dp.HandlerIdentifier)

UnregisterNetMessageHandler removes a net-message handler via identifier.

The identifier is returned at registration by RegisterNetMessageHandler().

type ParserConfig

type ParserConfig struct {
	// MsgQueueBufferSize defines the size of the internal net-message queue.
	// For large demos, fast i/o and slow CPUs higher numbers are suggested and vice versa.
	// The buffer size can easily be in the hundred-thousands to low millions for the best performance.
	// A negative value will make the Parser automatically decide the buffer size during ParseHeader()
	// based on the number of ticks in the demo (nubmer of ticks = buffer size);
	// this is the default behavior for DefaultParserConfig.
	// Zero enforces sequential parsing.
	MsgQueueBufferSize int

	// AdditionalNetMessageCreators maps net-message-IDs to creators (instantiators).
	// The creators should return a new instance of the correct protobuf-message type (from the msg package).
	// Interesting net-message-IDs can easily be discovered with the build-tag 'debugdemoinfocs'; when looking for 'UnhandledMessage'.
	// Check out demopacket.go to see which net-messages are already being parsed by default.
	// This is a beta feature and may be changed or replaced without notice.
	AdditionalNetMessageCreators map[int]NetMessageCreator

	// AdditionalEventEmitters contains additional event emitters - either from the fuzzy package or custom ones.
	// This is mainly used to add logic specifically for one type of demo (e.g. Matchmaking, FaceIt etc.).
	// This is a beta feature and may be changed or replaced without notice.
	// See also: package fuzzy for existing emitters with fuzzy-logic that depends on the demo-type.
	AdditionalEventEmitters []EventEmitter
}

ParserConfig contains the configuration for creating a new Parser.

type Participants

type Participants struct {
	// contains filtered or unexported fields
}

Participants provides helper functions on top of the currently connected players. E.g. ByUserID(), ByEntityID(), TeamMembers(), etc.

See GameState.Participants()

func (Participants) All

func (ptcp Participants) All() []*common.Player

All returns all currently connected players & spectators.

func (Participants) ByEntityID

func (ptcp Participants) ByEntityID() map[int]*common.Player

ByEntityID returns all currently connected players in a map where the key is the entity-ID. The map is a snapshot and is not updated (not a reference to the actual, underlying map). Includes spectators.

func (Participants) ByUserID

func (ptcp Participants) ByUserID() map[int]*common.Player

ByUserID returns all currently connected players in a map where the key is the user-ID. The map is a snapshot and is not updated (not a reference to the actual, underlying map). Includes spectators.

func (Participants) FindByHandle

func (ptcp Participants) FindByHandle(handle int) *common.Player

FindByHandle attempts to find a player by his entity-handle. The entity-handle is often used in entity-properties when referencing other entities such as a weapon's owner.

Returns nil if not found or if handle == invalidEntityHandle (used when referencing no entity).

func (Participants) Playing

func (ptcp Participants) Playing() []*common.Player

Playing returns all players that aren't spectating or unassigned.

func (Participants) TeamMembers

func (ptcp Participants) TeamMembers(team common.Team) []*common.Player

TeamMembers returns all players belonging to the requested team at this time.

type TeamState

type TeamState struct {
	// contains filtered or unexported fields
}

TeamState contains a team's ID, score, clan name & country flag.

func (TeamState) ClanName

func (ts TeamState) ClanName() string

ClanName returns the team's clan name.

func (TeamState) Flag

func (ts TeamState) Flag() string

Flag returns the team's country flag. E.g. DE, FR, etc.

Watch out, in some demos this is upper-case and in some lower-case.

func (TeamState) ID

func (ts TeamState) ID() int

ID returns the team-ID.

This stays the same even after switching sides.

func (TeamState) Score

func (ts TeamState) Score() int

Score returns the team's number of rounds won.