demoinfocs

README

demoinfocs-golang

Is a high performance demo parser for the game Counter Strike: Global Offensive (CS:GO) written in Go and based on Valve's demoinfogo and SatsHelix's demoinfo.

Discussions / Chat

You can use gitter to ask questions and discuss ideas about this project.
There are also some other rooms available around the topic of CS:GO demos.

Go Get

go get -u github.com/markus-wa/demoinfocs-golang

Example

This is a simple example on how to use the library. It collects all positions where weapons were fired from (using events.WeaponFiredEvent) and creates a heatmap using go-heatmap.

Check out the examples folder for more examples and the godoc of the events package for some information about the other available events and their purpose.

package main

import (
	"image"
	"image/png"
	"log"
	"os"

	heatmap "github.com/dustin/go-heatmap"
	schemes "github.com/dustin/go-heatmap/schemes"

	dem "github.com/markus-wa/demoinfocs-golang"
	events "github.com/markus-wa/demoinfocs-golang/events"
)

// Run like this: go run heatmap.go > out.png
func main() {
	f, err := os.Open("/path/to/demo.dem")
	checkErr(err)
	defer f.Close()

	p := dem.NewParser(f)

	// Parse header (contains map-name etc.)
	_, err = p.ParseHeader()
	checkErr(err)

	// Register handler for WeaponFiredEvent, triggered every time a shot is fired
	points := []heatmap.DataPoint{}
	p.RegisterEventHandler(func(e events.WeaponFiredEvent) {
		// Add shooter's position as datapoint
		points = append(points, heatmap.P(e.Shooter.Position.X, e.Shooter.Position.Y))
	})

	// Parse to end
	err = p.ParseToEnd()
	checkErr(err)

	// Generate heatmap and write to standard output
	img := heatmap.Heatmap(image.Rect(0, 0, 1024, 1024), points, 15, 128, schemes.AlphaFire)
	png.Encode(os.Stdout, img)
}

func checkErr(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

Result

Running the above code (go run heatmap.go > heatmap.png) will create a PNG with dots on all the locations where shots were fired (the heatmap 'overlay').

This doesn't look too interesting on it's own but that can be helped by quickly mapping it to the map overview in an image editing tool (2 min tops, no skills required).

Features

• Game events (kills, shots, round starts/ends, footsteps etc.) - docs / example
• Tracking of game-state (players, teams, grenades etc.) - docs
• Access to entities, server-classes & data-tables
• Access to all net-messages - docs / example
• Chat & console messages ¹ - docs / example
• Easy debugging via build-flags
• Built with performance & concurrency in mind

1. Only for some demos; in MM demos the chat is encrypted for example.

Performance / Benchmarks

One of the top priorities of this parser is performance and concurrency.

Here are some benchmark results from a system with a Intel i7 2600k CPU and SSD disk running Windows 10 and a demo with 85'000 frames.

Overview

Benchmark	Description	Average Duration	Speed
BenchmarkConcurrent	Read and parse 8 demos concurrently	2.90 s (per 8 demos)	~234'000 ticks / s
BenchmarkDemoInfoCs	Read demo from drive and parse	1.39 s	~61'000 ticks / s
BenchmarkInMemory	Read demo from memory and parse	1.38 s	~61'000 ticks / s

Raw output

$ go test -run _NONE_ -bench . -benchtime 30s -benchmem -concurrentdemos 8
goos: windows
goarch: amd64
pkg: github.com/markus-wa/demoinfocs-golang
BenchmarkDemoInfoCs-8                 30        1397398190 ns/op        162254528 B/op    839779 allocs/op
BenchmarkInMemory-8                   30        1384877250 ns/op        162109924 B/op    839628 allocs/op
BenchmarkConcurrent-8                 20        2902574295 ns/op        1297042534 B/op  6717163 allocs/op
--- BENCH: BenchmarkConcurrent-8
        demoinfocs_test.go:425: Running concurrency benchmark with 8 demos
        demoinfocs_test.go:425: Running concurrency benchmark with 8 demos
PASS
ok      github.com/markus-wa/demoinfocs-golang  147.800s

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository. There is one caveat however: Beta features - which are marked as such via comments and in release notes - may change in minor releases.

It's recommended to use some kind of dependency management system such as dep to ensure reproducible builds.

Development

Running tests

To run tests Git LFS is required.

git submodule init
git submodule update
pushd cs-demos && git lfs pull -I '*' && popd
go test

Here's a cool gist of a pre-commit hook to run tests before each commit. You can put this inside the .git/hooks directory to avoid commiting/pushing code with build errors or failing tests.

Debugging

You can use the build tag debugdemoinfocs (i.e. go test -tags debugdemoinfocs -v) to print out debugging information - such as game events or unhandled demo-messages - during the parsing process.
Side-note: The tag isn't called debug to avoid naming conflicts with other libs (and underscores in tags don't work, apparently).

To change the default debugging behavior Go's ldflags paramter can be used. Example for additionally printing out the ingame-tick-numbers: -ldflags '-X github.com/markus-wa/demoinfocs-golang.debugIngameTicks=YES'

Check out debug_on.go for any other settings that can be changed.

Generating protobuf code

Should you need to re-generate the protobuf generated code in the msg package, you will need the following tools:

• The latest protobuf generator (protoc) from your package manager or https://github.com/google/protobuf/releases

• And protoc-gen-gogofaster from gogoprotobuf to generate code for go.

    go get -u github.com/gogo/protobuf/protoc-gen-gogofaster
  

Make sure both are inside your PATH variable.

After installing these use go generate ./msg to generate the protobuf code.

Documentation

Overview

Package demoinfocs provides a demo parser for the game Counter-Strike: Global Offensive. It is based on the official demoinfogo tool by Valve as well as Stats Helix's demoinfo.

A good entry point to using the library is the Parser type.

Demo events are documented in the events package.

Index

• Variables
• type EventEmitter
• type GameState
  • func (gs *GameState) CTState() *TeamState
  • func (gs GameState) GrenadeProjectiles() map[int]*common.GrenadeProjectile
  • func (gs GameState) IngameTick() int
  • func (gs GameState) Participants() []*common.Player
  • func (gs GameState) PlayingParticipants() []*common.Player
  • func (gs *GameState) TState() *TeamState
  • func (gs GameState) TeamMembers(team common.Team) []*common.Player
• type NetMessageCreator
• type Parser
  • func NewParser(demostream io.Reader) *Parser
  • func NewParserWithConfig(demostream io.Reader, config ParserConfig) *Parser
  • func (p *Parser) Cancel()
  • func (p *Parser) CurrentFrame() int
  • func (p *Parser) CurrentTime() float32
  • func (p *Parser) Entities() map[int]*st.Entity
  • func (p *Parser) GameState() *GameState
  • func (p *Parser) Header() common.DemoHeader
  • func (p *Parser) ParseHeader() (common.DemoHeader, error)
  • func (p *Parser) ParseNextFrame() (b bool, err error)
  • func (p *Parser) ParseToEnd() (err error)
  • func (p *Parser) Progress() float32
  • func (p *Parser) RegisterEventHandler(handler interface{}) dp.HandlerIdentifier
  • func (p *Parser) RegisterNetMessageHandler(handler interface{}) dp.HandlerIdentifier
  • func (p *Parser) SendTableParser() *st.SendTableParser
  • func (p *Parser) UnregisterEventHandler(identifier dp.HandlerIdentifier)
  • func (p *Parser) UnregisterNetMessageHandler(identifier dp.HandlerIdentifier)
• type ParserConfig
• type TeamState
  • func (ts TeamState) ClanName() string
  • func (ts TeamState) Flag() string
  • func (ts TeamState) ID() int
  • func (ts TeamState) Score() int

Variables

var (
	// ErrCancelled signals that parsing was cancelled via Parser.Cancel()
	ErrCancelled = errors.New("Parsing was cancelled before it finished (ErrCancelled)")

	// ErrUnexpectedEndOfDemo signals that the demo is incomplete / corrupt -
	// these demos may still be useful, check how far the parser got.
	ErrUnexpectedEndOfDemo = errors.New("Demo stream ended unexpectedly (ErrUnexpectedEndOfDemo)")

	// ErrInvalidFileType signals that the input isn't a valid CS:GO demo.
	ErrInvalidFileType = errors.New("Invalid File-Type; expecting HL2DEMO in the first 8 bytes")

	// ErrHeaderNotParsed signals that the header hasn't been parsed before attempting to parse a tick.
	ErrHeaderNotParsed = errors.New("Header must be parsed before trying to parse a tick. See Parser.ParseHeader()")
)

Parsing errors

var DefaultParserConfig = ParserConfig{
	MsgQueueBufferSize: -1,
}

DefaultParserConfig is the default Parser configuration used by NewParser().

Types

type EventEmitter

type EventEmitter interface {
	Register(parser *Parser, eventDispatcher func(event interface{}))
}

EventEmitter is the interface to define additional event-emitters. The emitters may fire additional events by calling the eventDispatcher function received during registration of the emitter.

See also: package fuzzy for existing emitters with fuzzy-logic that depends on the demo-type.

type GameState

type GameState struct {
	// contains filtered or unexported fields
}

GameState contains all game-state relevant information.

func (*GameState) CTState

func (gs *GameState) CTState() *TeamState

CTState returns the TeamState of the CT team.

Make sure to handle swapping sides properly if you keep the reference.

func (GameState) GrenadeProjectiles

func (gs GameState) GrenadeProjectiles() map[int]*common.GrenadeProjectile

GrenadeProjectiles returns a map from entity-IDs to all live grenade projectiles.

Only constains projectiles currently in-flight or still active (smokes etc.), i.e. have been thrown but have yet to detonate.

func (GameState) IngameTick

func (gs GameState) IngameTick() int

IngameTick returns the latest actual tick number of the server during the game.

Watch out, I've seen this return wonky negative numbers at the start of demos.

func (GameState) Participants

func (gs GameState) Participants() []*common.Player

Participants returns all connected players. This includes spectators.

func (GameState) PlayingParticipants

func (gs GameState) PlayingParticipants() []*common.Player

PlayingParticipants returns all players that aren't spectating or unassigned.

func (*GameState) TState

func (gs *GameState) TState() *TeamState

TState returns the TeamState of the T team.

Make sure to handle swapping sides properly if you keep the reference.

func (GameState) TeamMembers

func (gs GameState) TeamMembers(team common.Team) []*common.Player

TeamMembers returns all players belonging to the requested team.

type NetMessageCreator

type NetMessageCreator func() proto.Message

NetMessageCreator creates additional net-messages to be dispatched to net-message handlers.

See also: ParserConfig.AdditionalNetMessageCreators & Parser.RegisterNetMessageHandler()

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser can parse a CS:GO demo. Creating a Parser is done via NewParser().

To start off use Parser.ParseHeader() to parse the demo header. After parsing the header Parser.ParseNextFrame() and Parser.ParseToEnd() can be used to parse the demo.

Use Parser.RegisterEventHandler() to receive notifications about events.

Example (without error handling):

f, _ := os.Open("/path/to/demo.dem")
p := dem.NewParser(f)
header := p.ParseHeader()
fmt.Println("Map:", header.MapName)
p.RegisterEventHandler(func(e events.BombExplodedEvent) {
	fmt.Printf(e.Site, "went BOOM!")
})
p.ParseToEnd()

Prints out '{A/B} site went BOOM!' when a bomb explodes.

func NewParser

func NewParser(demostream io.Reader) *Parser

NewParser creates a new Parser with the default configuration. The demostream io.Reader (e.g. os.File or bytes.Reader) must provide demo data in the '.DEM' format.

See also: NewCustomParser() & DefaultParserConfig

func NewParserWithConfig

func NewParserWithConfig(demostream io.Reader, config ParserConfig) *Parser

NewParserWithConfig returns a new Parser with a custom configuration.

See also: NewParser() & ParserConfig

func (*Parser) Cancel

func (p *Parser) Cancel()

Cancel aborts ParseToEnd(). All information that was already read up to this point may still be used (and new events may still be sent out).

func (*Parser) CurrentFrame

func (p *Parser) CurrentFrame() int

CurrentFrame return the number of the current frame, aka. 'demo-tick' (Since demos often have a different tick-rate than the game). Starts with frame 0, should go up to DemoHeader.PlaybackFrames but might not be the case (usually it's just close to it).

func (*Parser) CurrentTime

func (p *Parser) CurrentTime() float32

CurrentTime returns the ingame time in seconds since the start of the demo.

func (*Parser) Entities

func (p *Parser) Entities() map[int]*st.Entity

Entities returns the available entities.

func (*Parser) GameState

func (p *Parser) GameState() *GameState

GameState returns the current game-state.

func (*Parser) Header

func (p *Parser) Header() common.DemoHeader

Header returns the DemoHeader which contains the demo's metadata.

func (*Parser) ParseHeader

func (p *Parser) ParseHeader() (common.DemoHeader, error)

ParseHeader attempts to parse the header of the demo.

Returns ErrInvalidFileType if the filestamp (first 8 bytes) doesn't match HL2DEMO.

func (*Parser) ParseNextFrame

func (p *Parser) ParseNextFrame() (b bool, err error)

ParseNextFrame attempts to parse the next frame / demo-tick (not ingame tick).

Returns true unless the demo command 'stop' or an error was encountered. Returns an error if the header hasn't been parsed yet - see Parser.ParseHeader().

May return ErrUnexpectedEndOfDemo for incomplete / corrupt demos. May panic if the demo is corrupt in some way.

See also: ParseToEnd() for parsing the complete demo in one go (faster).

func (*Parser) ParseToEnd

func (p *Parser) ParseToEnd() (err error)

ParseToEnd attempts to parse the demo until the end. Aborts and returns ErrCancelled if Cancel() is called before the end.

See also: ParseNextFrame() for other possible errors.

func (*Parser) Progress

func (p *Parser) Progress() float32

Progress returns the parsing progress from 0 to 1. Where 0 means nothing has been parsed yet and 1 means the demo has been parsed to the end.

Might not be 100% correct since it's just based on the reported tick count of the header.

func (*Parser) RegisterEventHandler

func (p *Parser) RegisterEventHandler(handler interface{}) dp.HandlerIdentifier

RegisterEventHandler registers a handler for game events.

The handler must be of type func(<EventType>) where EventType is the kind of event to be handled. To catch all events func(interface{}) can be used.

Example:

parser.RegisterEventHandler(func(e events.WeaponFiredEvent) {
	fmt.Printf("%s fired his %s\n", e.Shooter.Name, e.Weapon.Weapon)
})

Parameter handler has to be of type interface{} because lolnogenerics.

Returns a identifier with which the handler can be removed via UnregisterEventHandler().

func (*Parser) RegisterNetMessageHandler

func (p *Parser) RegisterNetMessageHandler(handler interface{}) dp.HandlerIdentifier

RegisterNetMessageHandler registers a handler for net-messages.

The handler must be of type func(*<MessageType>) where MessageType is the kind of net-message to be handled.

Returns a identifier with which the handler can be removed via UnregisterNetMessageHandler().

See also: RegisterEventHandler()

This is a beta feature and may be changed or replaced without notice.

func (*Parser) SendTableParser

func (p *Parser) SendTableParser() *st.SendTableParser

SendTableParser returns the sendtable parser.

This is a beta feature and may be changed or replaced without notice.

func (*Parser) UnregisterEventHandler

func (p *Parser) UnregisterEventHandler(identifier dp.HandlerIdentifier)

UnregisterEventHandler removes a game event handler via identifier.

The identifier is returned at registration by RegisterEventHandler().

func (*Parser) UnregisterNetMessageHandler

func (p *Parser) UnregisterNetMessageHandler(identifier dp.HandlerIdentifier)

UnregisterNetMessageHandler removes a net-message handler via identifier.

The identifier is returned at registration by RegisterNetMessageHandler().

type ParserConfig

type ParserConfig struct {
	// MsgQueueBufferSize defines the size of the internal net-message queue.
	// For large demos, fast i/o and slow CPUs higher numbers are suggested and vice versa.
	// The buffer size can easily be in the hundred-thousands to low millions for the best performance.
	// A negative value will make the Parser automatically decide the buffer size during ParseHeader()
	// based on the number of ticks in the demo (nubmer of ticks = buffer size);
	// this is the default behavior for DefaultParserConfig.
	// Zero enforces sequential parsing.
	MsgQueueBufferSize int

	// AdditionalNetMessageCreators maps net-message-IDs to creators (instantiators).
	// The creators should return a new instance of the correct protobuf-message type (from the msg package).
	// Interesting net-message-IDs can easily be discovered with the build-tag 'debugdemoinfocs'; when looking for 'UnhandledMessage'.
	// Check out demopacket.go to see which net-messages are already being parsed by default.
	// This is a beta feature and may be changed or replaced without notice.
	AdditionalNetMessageCreators map[int]NetMessageCreator

	// AdditionalEventEmitters contains additional event emitters - either from the fuzzy package or custom ones.
	// This is mainly used to add logic specifically for one type of demo (e.g. Matchmaking, FaceIt etc.).
	// This is a beta feature and may be changed or replaced without notice.
	// See also: package fuzzy for existing emitters with fuzzy-logic that depends on the demo-type.
	AdditionalEventEmitters []EventEmitter
}

ParserConfig contains the configuration for creating a new Parser.

type TeamState

type TeamState struct {
	// contains filtered or unexported fields
}

TeamState contains a team's ID, score, clan name & country flag.

func (TeamState) ClanName

func (ts TeamState) ClanName() string

ClanName returns the team's clan name.

func (TeamState) Flag

func (ts TeamState) Flag() string

Flag returns the team's country flag.

Watch out, in some demos this is upper-case and in some lower-case.

func (TeamState) ID

func (ts TeamState) ID() int

ID returns the team-ID.

This stays the same even after switching sides.

func (TeamState) Score

func (ts TeamState) Score() int

Score returns the team's number of rounds won.