keyhole

README

Keyhole - Survey Your Mongo Land

Keyhole is a performance analytics tool, written in GO (Golang), to collect stats and to analyze performance for MongoDB clusters. Golang was chosen to eliminate the needs to install an interpreter or software modules. Use Maobi to create HTML reports from data collected by Keyhole.

Blogs

Peek at your MongoDB Clusters like a Pro with Keyhole:

• Part 1
• Part 2
• Part 3

Updated post is available at Survey Your Mongo Land.

Use Cases

With Keyhole, experienced users should be able to spot performance issues and to determine whether upgrades are needed quickly from a few minutes of testing and analyzing the results. Keyhole supports TLS/SSL connections.

Several features are available, and they are

• Write Throughputs Test measures the MongoDB performance by writing documents at a high rate to a MongoDB cluster.
• Load test extends the Write throughputs test by issuing different ops against a MongoDB cluster. Stats analytic is also provided
  • Memory: resident, virtual, and page faults
  • Executor and ops
  • Latency: read, write, and command
  • Metrics: index keys examined, collection scan, in-memory sort, and ops
  • WiredTiger analytic
• Customized load test with a sample document. Uses can load test using their own document format (see simulation.md for details).
• Cluster Info and Sanity Check to display information of a cluster including stats to help determine working set data size.
• Display all indexes and their usages
• Duplicate all indexes to another MongoDB cluster.
• Seed data for demo and educational purposes as a trainer.
• Display ops time and query patterns by parsing logs.
• Display indexes scores of a query shape.
• Monitor WiredTiger Cache in near real time.
• View FTDC Data and scores with a friendly interface.
• MongoDB Atlas API integration.

Refer to wiki for user's guide.

Build and Download

Build and download instructions are available at Build and Download Keyhole.

Usage

keyhole --help

Unit Tests

./test.sh load

License

Apache-2.0 License

Disclaimer

This software is not supported by MongoDB, Inc. under any of their commercial support subscriptions or otherwise. Any usage of keyhole is at your own risk. Bug reports, feature requests and questions can be posted in the Issues section on GitHub.

Documentation

Index

• func AnalyzeMongoLogs(loginfo *mdb.LogInfo, filenames []string) error
• func CollectCluserDetails(stats *mdb.ClusterStats, client *mongo.Client, ...) error
• func CollectIndexStats(index *mdb.IndexStats, client *mongo.Client) error
• func DuplicateIndexesFromFile(index *mdb.IndexStats, client *mongo.Client, filename string) error
• func GetClusterSummary(version string, client *mongo.Client) string
• func MonitorWiredTigerCache(version string, client *mongo.Client)
• func Run(fullVersion string)
• func StartSimulation(runner *sim.Runner) error
• type Comparison
  • func NewComparison(keyholeVersion string) *Comparison
  • func (p *Comparison) Compare(source string, target string) error
  • func (p *Comparison) OutputBSON() error
  • func (p *Comparison) Run() error
  • func (p *Comparison) SetNoColor(nocolor bool)
  • func (p *Comparison) SetVerbose(verbose bool)
• type Model
• type Robot
• type Seed
  • func NewSeed() *Seed
  • func (f *Seed) SeedAllDemoData(client *mongo.Client) error
  • func (f *Seed) SeedData(client *mongo.Client) error
  • func (f *Seed) SeedFavorites(client *mongo.Client) error
  • func (f *Seed) SeedVehicles(client *mongo.Client) error
  • func (f *Seed) SetCollection(collection string)
  • func (f *Seed) SetDatabase(database string)
  • func (f *Seed) SetFile(file string)
  • func (f *Seed) SetIsDrop(isDrop bool)
  • func (f *Seed) SetNumberConnections(conns int)
  • func (f *Seed) SetShowProgress(showProgress bool)
  • func (f *Seed) SetTotal(total int)
• type Task

Functions

func AnalyzeMongoLogs

func AnalyzeMongoLogs(loginfo *mdb.LogInfo, filenames []string) error

AnalyzeMongoLogs a helper function to analyze logs

func CollectCluserDetails

func CollectCluserDetails(stats *mdb.ClusterStats, client *mongo.Client, connString connstring.ConnString) error

CollectCluserDetails collects cluster details

func CollectIndexStats

func CollectIndexStats(index *mdb.IndexStats, client *mongo.Client) error

CollectIndexStats collects all indexes stats

func DuplicateIndexesFromFile

func DuplicateIndexesFromFile(index *mdb.IndexStats, client *mongo.Client, filename string) error

DuplicateIndexesFromFile duplicate indexes from definitions defined in a file

func GetClusterSummary

func GetClusterSummary(version string, client *mongo.Client) string

GetClusterSummary returns MongoDB cluster summary

func MonitorWiredTigerCache

func MonitorWiredTigerCache(version string, client *mongo.Client)

MonitorWiredTigerCache monitor wiredTiger cache

func Run

func Run(fullVersion string)

Run executes main()

func StartSimulation

func StartSimulation(runner *sim.Runner) error

StartSimulation kicks off simulation

Types

type Comparison

type Comparison struct {
	Logger      *gox.Logger       `bson:"keyhole"`
	SourceStats *mdb.ClusterStats `bson:"source"`
	TargetStats *mdb.ClusterStats `bson:"target"`
	// contains filtered or unexported fields
}

Comparison contains parameters of comparison parameters

func NewComparison

func NewComparison(keyholeVersion string) *Comparison

NewComparison returns *Comparison

func (*Comparison) Compare

func (p *Comparison) Compare(source string, target string) error

Compare executes compare commands

func (*Comparison) OutputBSON

func (p *Comparison) OutputBSON() error

OutputBSON writes bson data to a file

func (*Comparison) Run

func (p *Comparison) Run() error

Run executes compare commands

func (*Comparison) SetNoColor

func (p *Comparison) SetNoColor(nocolor bool)

SetNoColor set nocolor flag

func (*Comparison) SetVerbose

func (p *Comparison) SetVerbose(verbose bool)

SetVerbose sets verbose

type Model

type Model struct {
	ID          string `json:"_id" bson:"_id"`
	Name        string
	Description string
	Year        int
}

Model - robot model

type Robot

type Robot struct {
	ID         string  `json:"_id" bson:"_id"`
	ModelID    string  `json:"modelId,omitempty" bson:"modelId,omitempty"`
	Notes      string  `json:"notes" bson:"notes"`
	BatteryPct float32 `json:"batteryPct,omitempty" bson:"batteryPct,omitempty"`
	Tasks      []Task  `json:"tasks" bson:"tasks"`
}

Robot -

type Seed

type Seed struct {
	// contains filtered or unexported fields
}

Seed seeds feeder

func NewSeed

func NewSeed() *Seed

NewSeed establish seeding parameters

func (*Seed) SeedAllDemoData

func (f *Seed) SeedAllDemoData(client *mongo.Client) error

SeedAllDemoData - seed data for demo

 models: {
   "_id": string
  "name": string,
  "description": string
  "year": integer
}

robots: {
  "_id": string
  "modelId": string
  "notes": string,
  "batteryPct": float,
  "tasks": [{"for": string, "minutesUsed": integer}]
}

func (*Seed) SeedData

func (f *Seed) SeedData(client *mongo.Client) error

SeedData seeds all demo data

func (*Seed) SeedFavorites

func (f *Seed) SeedFavorites(client *mongo.Client) error

SeedFavorites seeds demo data of collection favorites

func (*Seed) SeedVehicles

func (f *Seed) SeedVehicles(client *mongo.Client) error

SeedVehicles seeds vehicles collection

func (*Seed) SetCollection

func (f *Seed) SetCollection(collection string)

SetCollection set collection

func (*Seed) SetDatabase

func (f *Seed) SetDatabase(database string)

SetDatabase set database

func (*Seed) SetFile

func (f *Seed) SetFile(file string)

SetFile set file

func (*Seed) SetIsDrop

func (f *Seed) SetIsDrop(isDrop bool)

SetIsDrop set isDrop

func (*Seed) SetNumberConnections

func (f *Seed) SetNumberConnections(conns int)

SetNumberConnections set conns

func (*Seed) SetShowProgress

func (f *Seed) SetShowProgress(showProgress bool)

SetShowProgress set showProgress

func (*Seed) SetTotal

func (f *Seed) SetTotal(total int)

SetTotal set total

type Task

type Task struct {
	For         string `json:"for" bson:"for"`
	MinutesUsed int    `json:"minutesUsed" bson:"minutesUsed"`
}

Task - robot task