moira

package module
v2.5.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 12, 2019 License: MIT Imports: 10 Imported by: 124

README

Moira 2.0 Documentation Status Telegram Gitter Go Report Card

Moira is a real-time alerting tool, based on Graphite data.

Build status

branch status
master Build Status Coverage Status
develop Build Status Coverage Status

Installation

Docker Compose is the easiest way to try:

git clone https://github.com/moira-alert/docker-compose.git
cd docker-compose
docker-compose pull
docker-compose up

Feed data in Graphite format to localhost:2003:

echo "local.random.diceroll 4 `date +%s`" | nc localhost 2003

Configure triggers at localhost:8080 using your browser.

Other installation methods are available, see documentation.

Getting Started

See our user guide that is based on a number of real-life scenarios, from simple and universal to complicated and specific.

Why 2.0?

Moira 2.0 is different from the first version in two important ways:

  1. We got rid of Python, because it was slow. Checker and API services are now written in Go, based on carbonapi implementation of Graphite functions.
  2. We got rid of Angular, because our main stack is React now. We just don't know how to do Angular anymore. We also revamped the UI.

What is in the other repositories?

Code in this repository is the backend part of Moira monitoring application.

Contact us

If you have any questions, you can ask us on Telegram.

Thanks

SKB Kontur

Moira was originally developed and is supported by SKB Kontur, a B2G company based in Ekaterinburg, Russia. We express gratitude to our company for encouraging us to opensource Moira and for giving back to the community that created Graphite and many other useful DevOps tools.

Documentation

Index

Constants

View Source
const (
	// VariableContactID is used to render template with contact.ID
	VariableContactID = "${contact_id}"
	// VariableContactValue is used to render template with contact.Value
	VariableContactValue = "${contact_value}"
	// VariableContactType is used to render template with contact.Type
	VariableContactType = "${contact_type}"
	// VariableTriggerID is used to render template with trigger.ID
	VariableTriggerID = "${trigger_id}"
	// VariableTriggerName is used to render template with trigger.Name
	VariableTriggerName = "${trigger_name}"
)
View Source
const (
	// FallingTrigger represents falling trigger type, in which OK > WARN > ERROR
	FallingTrigger = "falling"
	// RisingTrigger represents rising trigger type, in which OK < WARN < ERROR
	RisingTrigger = "rising"
	// ExpressionTrigger represents trigger type with custom user expression
	ExpressionTrigger = "expression"
)
View Source
const (
	SelfStateOK    = "OK"    // OK means notifier is healthy
	SelfStateERROR = "ERROR" // ERROR means notifier is stopped, admin intervention is required
)

Moira notifier self-states

Variables

This section is empty.

Functions

func ChunkSlice

func ChunkSlice(original []string, chunkSize int) (divided [][]string)

ChunkSlice gets slice of strings and chunks it to a given size. It returns a batch of chunked lists

func GetStringListsDiff

func GetStringListsDiff(stringLists ...[]string) []string

GetStringListsDiff returns the members of the set resulting from the difference between the first set and all the successive lists.

func GetStringListsUnion

func GetStringListsUnion(stringLists ...[]string) []string

GetStringListsUnion returns the union set of stringLists

func Int64ToTime

func Int64ToTime(timeStamp int64) time.Time

Int64ToTime returns time.Time from int64

func IsValidFloat64

func IsValidFloat64(val float64) bool

IsValidFloat64 checks float64 for Inf and NaN. If it is then float64 is not valid

func SetMaintenanceUserAndTime

func SetMaintenanceUserAndTime(maintenanceCheck MaintenanceCheck, maintenance int64, user string, callMaintenance int64)

SetMaintenanceUserAndTime set startuser and starttime or stopuser and stoptime for MaintenanceInfo

func Subset

func Subset(first, second []string) bool

Subset return whether first is a subset of second

func UnsafeBytesToString

func UnsafeBytesToString(b []byte) string

UnsafeBytesToString converts source to string without copying

func UnsafeStringToBytes

func UnsafeStringToBytes(s string) []byte

UnsafeStringToBytes converts string to source without copying

func UseFloat64

func UseFloat64(f *float64) float64

UseFloat64 gets pointer value of float64 or default float64 if pointer is nil

func UseString

func UseString(str *string) string

UseString gets pointer value of string or default string if pointer is nil

Types

type BytesScanner

type BytesScanner struct {
	// contains filtered or unexported fields
}

BytesScanner allows to scan for subslices separated by separator

func NewBytesScanner

func NewBytesScanner(bytes []byte, separator byte) *BytesScanner

NewBytesScanner slices bytes into all subslices separated by separator and returns a scanner which allows to scan for these subslices

func (*BytesScanner) HasNext

func (it *BytesScanner) HasNext() bool

HasNext checks if next subslice available or not

func (*BytesScanner) Next

func (it *BytesScanner) Next() (result []byte)

Next returns available subslice and advances the scanner to next slice

type CheckData

type CheckData struct {
	Metrics                      map[string]MetricState `json:"metrics"`
	Score                        int64                  `json:"score"`
	State                        State                  `json:"state"`
	Maintenance                  int64                  `json:"maintenance,omitempty"`
	MaintenanceInfo              MaintenanceInfo        `json:"maintenance_info"`
	Timestamp                    int64                  `json:"timestamp,omitempty"`
	EventTimestamp               int64                  `json:"event_timestamp,omitempty"`
	LastSuccessfulCheckTimestamp int64                  `json:"last_successful_check_timestamp"`
	Suppressed                   bool                   `json:"suppressed,omitempty"`
	SuppressedState              State                  `json:"suppressed_state,omitempty"`
	Message                      string                 `json:"msg,omitempty"`
}

CheckData represents last trigger check data

func (CheckData) GetEventTimestamp

func (checkData CheckData) GetEventTimestamp() int64

GetEventTimestamp gets event timestamp for given check

func (*CheckData) GetMaintenance

func (checkData *CheckData) GetMaintenance() (MaintenanceInfo, int64)

GetMaintenance return metricState MaintenanceInfo

func (*CheckData) GetOrCreateMetricState

func (checkData *CheckData) GetOrCreateMetricState(metric string, emptyTimestampValue int64, muteNewMetric bool) MetricState

GetOrCreateMetricState gets metric state from check data or create new if CheckData has no state for given metric

func (*CheckData) SetMaintenance

func (checkData *CheckData) SetMaintenance(maintenanceInfo *MaintenanceInfo, maintenance int64)

SetMaintenance set maintenance user, time for CheckData

func (*CheckData) UpdateScore

func (checkData *CheckData) UpdateScore() int64

UpdateScore update and return checkData score, based on metric states and checkData state

type ContactData

type ContactData struct {
	Type  string `json:"type"`
	Value string `json:"value"`
	ID    string `json:"id"`
	User  string `json:"user"`
}

ContactData represents contact object

type Database

type Database interface {
	// SelfState
	UpdateMetricsHeartbeat() error
	GetMetricsUpdatesCount() (int64, error)
	GetChecksUpdatesCount() (int64, error)
	GetRemoteChecksUpdatesCount() (int64, error)
	GetNotifierState() (string, error)
	SetNotifierState(string) error

	// Tag storing
	GetTagNames() ([]string, error)
	RemoveTag(tagName string) error
	GetTagTriggerIDs(tagName string) ([]string, error)

	// LastCheck storing
	GetTriggerLastCheck(triggerID string) (CheckData, error)
	SetTriggerLastCheck(triggerID string, checkData *CheckData, isRemote bool) error
	RemoveTriggerLastCheck(triggerID string) error
	SetTriggerCheckMaintenance(triggerID string, metrics map[string]int64, triggerMaintenance *int64, userLogin string, timeCallMaintenance int64) error

	// Trigger storing
	GetLocalTriggerIDs() ([]string, error)
	GetAllTriggerIDs() ([]string, error)
	GetRemoteTriggerIDs() ([]string, error)
	GetTrigger(triggerID string) (Trigger, error)
	GetTriggers(triggerIDs []string) ([]*Trigger, error)
	GetTriggerChecks(triggerIDs []string) ([]*TriggerCheck, error)
	SaveTrigger(triggerID string, trigger *Trigger) error
	RemoveTrigger(triggerID string) error
	GetPatternTriggerIDs(pattern string) ([]string, error)
	RemovePatternTriggerIDs(pattern string) error

	// Throttling
	GetTriggerThrottling(triggerID string) (time.Time, time.Time)
	SetTriggerThrottling(triggerID string, next time.Time) error
	DeleteTriggerThrottling(triggerID string) error

	// NotificationEvent storing
	GetNotificationEvents(triggerID string, start, size int64) ([]*NotificationEvent, error)
	PushNotificationEvent(event *NotificationEvent, ui bool) error
	GetNotificationEventCount(triggerID string, from int64) int64
	FetchNotificationEvent() (NotificationEvent, error)
	RemoveAllNotificationEvents() error

	// ContactData storing
	GetContact(contactID string) (ContactData, error)
	GetContacts(contactIDs []string) ([]*ContactData, error)
	GetAllContacts() ([]*ContactData, error)
	RemoveContact(contactID string) error
	SaveContact(contact *ContactData) error
	GetUserContactIDs(userLogin string) ([]string, error)

	// SubscriptionData storing
	GetSubscription(id string) (SubscriptionData, error)
	GetSubscriptions(subscriptionIDs []string) ([]*SubscriptionData, error)
	SaveSubscription(subscription *SubscriptionData) error
	SaveSubscriptions(subscriptions []*SubscriptionData) error
	RemoveSubscription(subscriptionID string) error
	GetUserSubscriptionIDs(userLogin string) ([]string, error)
	GetTagsSubscriptions(tags []string) ([]*SubscriptionData, error)

	// ScheduledNotification storing
	GetNotifications(start, end int64) ([]*ScheduledNotification, int64, error)
	RemoveNotification(notificationKey string) (int64, error)
	RemoveAllNotifications() error
	FetchNotifications(to int64) ([]*ScheduledNotification, error)
	AddNotification(notification *ScheduledNotification) error
	AddNotifications(notification []*ScheduledNotification, timestamp int64) error

	// Patterns and metrics storing
	GetPatterns() ([]string, error)
	AddPatternMetric(pattern, metric string) error
	GetPatternMetrics(pattern string) ([]string, error)
	RemovePattern(pattern string) error
	RemovePatternsMetrics(pattern []string) error
	RemovePatternWithMetrics(pattern string) error

	SubscribeMetricEvents(tomb *tomb.Tomb) (<-chan *MetricEvent, error)
	SaveMetrics(buffer map[string]*MatchedMetric) error
	GetMetricRetention(metric string) (int64, error)
	GetMetricsValues(metrics []string, from int64, until int64) (map[string][]*MetricValue, error)
	RemoveMetricValues(metric string, toTime int64) error
	RemoveMetricsValues(metrics []string, toTime int64) error

	AddLocalTriggersToCheck(triggerIDs []string) error
	GetLocalTriggersToCheck(count int) ([]string, error)
	GetLocalTriggersToCheckCount() (int64, error)

	AddRemoteTriggersToCheck(triggerIDs []string) error
	GetRemoteTriggersToCheck(count int) ([]string, error)
	GetRemoteTriggersToCheckCount() (int64, error)

	// TriggerCheckLock storing
	AcquireTriggerCheckLock(triggerID string, timeout int) error
	DeleteTriggerCheckLock(triggerID string) error
	SetTriggerCheckLock(triggerID string) (bool, error)

	// Bot data storing
	GetIDByUsername(messenger, username string) (string, error)
	SetUsernameID(messenger, username, id string) error
	RemoveUser(messenger, username string) error

	// Triggers without subscription manipulation
	MarkTriggersAsUnused(triggerIDs ...string) error
	GetUnusedTriggerIDs() ([]string, error)
	MarkTriggersAsUsed(triggerIDs ...string) error

	// Triggers to reindex in full-text search index
	FetchTriggersToReindex(from int64) ([]string, error)
	RemoveTriggersToReindex(to int64) error

	// Creates Lock
	NewLock(name string, ttl time.Duration) Lock
}

Database implements DB functionality

type Lock

type Lock interface {
	Acquire(stop <-chan struct{}) (lost <-chan struct{}, error error)
	Release()
}

Lock implements lock abstraction

type Logger

type Logger interface {
	Debug(args ...interface{})
	Debugf(format string, args ...interface{})
	Info(args ...interface{})
	Infof(format string, args ...interface{})
	Error(args ...interface{})
	Errorf(format string, args ...interface{})
	Fatal(args ...interface{})
	Fatalf(format string, args ...interface{})
	Warning(args ...interface{})
	Warningf(format string, args ...interface{})
}

Logger implements logger abstraction

type MaintenanceCheck

type MaintenanceCheck interface {
	SetMaintenance(maintenanceInfo *MaintenanceInfo, maintenance int64)
	GetMaintenance() (MaintenanceInfo, int64)
}

MaintenanceCheck set maintenance user, time

type MaintenanceInfo

type MaintenanceInfo struct {
	StartUser *string `json:"setup_user"`
	StartTime *int64  `json:"setup_time"`
	StopUser  *string `json:"remove_user"`
	StopTime  *int64  `json:"remove_time"`
}

MaintenanceInfo represents user and time set/unset maintenance

func (*MaintenanceInfo) Set

func (maintenanceInfo *MaintenanceInfo) Set(startUser *string, startTime *int64, stopUser *string, stopTime *int64)

Set maintanace start and stop users and times

type MatchedMetric

type MatchedMetric struct {
	Metric             string
	Patterns           []string
	Value              float64
	Timestamp          int64
	RetentionTimestamp int64
	Retention          int
}

MatchedMetric represents parsed and matched metric data

type MetricEvent

type MetricEvent struct {
	Metric  string `json:"metric"`
	Pattern string `json:"pattern"`
}

MetricEvent represents filter metric event

type MetricState

type MetricState struct {
	EventTimestamp  int64           `json:"event_timestamp"`
	State           State           `json:"state"`
	Suppressed      bool            `json:"suppressed"`
	SuppressedState State           `json:"suppressed_state,omitempty"`
	Timestamp       int64           `json:"timestamp"`
	Value           *float64        `json:"value,omitempty"`
	Maintenance     int64           `json:"maintenance,omitempty"`
	MaintenanceInfo MaintenanceInfo `json:"maintenance_info"`
}

MetricState represents metric state data for given timestamp

func (*MetricState) GetCheckPoint

func (metricState *MetricState) GetCheckPoint(checkPointGap int64) int64

GetCheckPoint gets check point for given MetricState CheckPoint is the timestamp from which to start checking the current state of the metric

func (MetricState) GetEventTimestamp

func (metricState MetricState) GetEventTimestamp() int64

GetEventTimestamp gets event timestamp for given metric

func (*MetricState) GetMaintenance

func (metricState *MetricState) GetMaintenance() (MaintenanceInfo, int64)

GetMaintenance return metricState MaintenanceInfo

func (*MetricState) SetMaintenance

func (metricState *MetricState) SetMaintenance(maintenanceInfo *MaintenanceInfo, maintenance int64)

SetMaintenance set maintenance user, time for MetricState

type MetricValue

type MetricValue struct {
	RetentionTimestamp int64   `json:"step,omitempty"`
	Timestamp          int64   `json:"ts"`
	Value              float64 `json:"value"`
}

MetricValue represents metric data

type NotificationEvent

type NotificationEvent struct {
	IsTriggerEvent bool     `json:"trigger_event,omitempty"`
	Timestamp      int64    `json:"timestamp"`
	Metric         string   `json:"metric"`
	Value          *float64 `json:"value,omitempty"`
	State          State    `json:"state"`
	TriggerID      string   `json:"trigger_id"`
	SubscriptionID *string  `json:"sub_id,omitempty"`
	ContactID      string   `json:"contactId,omitempty"`
	OldState       State    `json:"old_state"`
	Message        *string  `json:"msg,omitempty"`
}

NotificationEvent represents trigger state changes event

func (NotificationEvent) FormatTimestamp

func (event NotificationEvent) FormatTimestamp(location *time.Location) string

FormatTimestamp gets event timestamp and format it using given location to human readable presentation

func (NotificationEvent) GetMetricValue

func (event NotificationEvent) GetMetricValue() string

GetMetricValue gets event metric value and format it to human readable presentation

func (NotificationEvent) String

func (event NotificationEvent) String() string

type NotificationEvents

type NotificationEvents []NotificationEvent

NotificationEvents represents slice of NotificationEvent

func (NotificationEvents) GetSubjectState

func (events NotificationEvents) GetSubjectState() State

GetSubjectState returns the most critical state of events

type PlotTheme

type PlotTheme interface {
	GetTitleStyle() chart.Style
	GetGridStyle() chart.Style
	GetCanvasStyle() chart.Style
	GetBackgroundStyle(maxMarkLen int) chart.Style
	GetThresholdStyle(thresholdType string) chart.Style
	GetAnnotationStyle(thresholdType string) chart.Style
	GetSerieStyles(curveInd int) (curveStyle, pointStyle chart.Style)
	GetLegendStyle() chart.Style
	GetXAxisStyle() chart.Style
	GetYAxisStyle() chart.Style
}

PlotTheme is an interface to access plot theme styles

type PlottingData

type PlottingData struct {
	Enabled bool   `json:"enabled"`
	Theme   string `json:"theme"`
}

PlottingData represents plotting settings

type ScheduleData

type ScheduleData struct {
	Days           []ScheduleDataDay `json:"days"`
	TimezoneOffset int64             `json:"tzOffset"`
	StartOffset    int64             `json:"startOffset"`
	EndOffset      int64             `json:"endOffset"`
}

ScheduleData represents subscription schedule

func (*ScheduleData) IsScheduleAllows

func (schedule *ScheduleData) IsScheduleAllows(ts int64) bool

IsScheduleAllows check if the time is in the allowed schedule interval

type ScheduleDataDay

type ScheduleDataDay struct {
	Enabled bool   `json:"enabled"`
	Name    string `json:"name,omitempty"`
}

ScheduleDataDay represents week day of schedule

type ScheduledNotification

type ScheduledNotification struct {
	Event     NotificationEvent `json:"event"`
	Trigger   TriggerData       `json:"trigger"`
	Contact   ContactData       `json:"contact"`
	Plotting  PlottingData      `json:"plotting"`
	Throttled bool              `json:"throttled"`
	SendFail  int               `json:"send_fail"`
	Timestamp int64             `json:"timestamp"`
}

ScheduledNotification represent notification object

func (*ScheduledNotification) GetKey

func (notification *ScheduledNotification) GetKey() string

GetKey return notification key to prevent duplication to the same contact

type SearchHighlight

type SearchHighlight struct {
	Field, Value string
}

SearchHighlight represents highlight

type SearchResult

type SearchResult struct {
	ObjectID   string
	Highlights []SearchHighlight
}

SearchResult represents fulltext search result

type Searcher

type Searcher interface {
	Start() error
	Stop() error
	IsReady() bool
	SearchTriggers(filterTags []string, searchString string, onlyErrors bool,
		page int64, size int64) (searchResults []*SearchResult, total int64, err error)
}

Searcher interface implements full-text search index functionality

type Sender

type Sender interface {
	SendEvents(events NotificationEvents, contact ContactData, trigger TriggerData, plot []byte, throttled bool) error
	Init(senderSettings map[string]string, logger Logger, location *time.Location, dateTimeFormat string) error
}

Sender interface for implementing specified contact type sender

type State

type State string

State type describe all default moira triggers or metrics states

var (
	StateOK        State = "OK"
	StateWARN      State = "WARN"
	StateERROR     State = "ERROR"
	StateNODATA    State = "NODATA"
	StateEXCEPTION State = "EXCEPTION" // Use this for trigger check unexpected errors
	StateTEST      State = "TEST"      // Use this only for test notifications
)

Moira trigger and metric states

type SubscriptionData

type SubscriptionData struct {
	Contacts          []string     `json:"contacts"`
	Tags              []string     `json:"tags"`
	Schedule          ScheduleData `json:"sched"`
	Plotting          PlottingData `json:"plotting"`
	ID                string       `json:"id"`
	Enabled           bool         `json:"enabled"`
	IgnoreWarnings    bool         `json:"ignore_warnings,omitempty"`
	IgnoreRecoverings bool         `json:"ignore_recoverings,omitempty"`
	ThrottlingEnabled bool         `json:"throttling"`
	User              string       `json:"user"`
}

SubscriptionData represents user subscription

func (*SubscriptionData) MustIgnore

func (subscription *SubscriptionData) MustIgnore(eventData *NotificationEvent) bool

MustIgnore returns true if given state transition must be ignored

type TTLState

type TTLState string

TTLState declares all ttl (NODATA) states, used if metric has no values for given interval (ttl)

var (
	TTLStateOK     TTLState = "OK"
	TTLStateWARN   TTLState = "WARN"
	TTLStateERROR  TTLState = "ERROR"
	TTLStateNODATA TTLState = "NODATA"
	TTLStateDEL    TTLState = "DEL"
)

Moira ttl states

func (TTLState) ToMetricState

func (state TTLState) ToMetricState() State

ToMetricState is an auxiliary function to handle metric state properly.

func (TTLState) ToTriggerState

func (state TTLState) ToTriggerState() State

ToTriggerState is an auxiliary function to handle trigger state properly.

type Trigger

type Trigger struct {
	ID               string        `json:"id"`
	Name             string        `json:"name"`
	Desc             *string       `json:"desc,omitempty"`
	Targets          []string      `json:"targets"`
	WarnValue        *float64      `json:"warn_value"`
	ErrorValue       *float64      `json:"error_value"`
	TriggerType      string        `json:"trigger_type"`
	Tags             []string      `json:"tags"`
	TTLState         *TTLState     `json:"ttl_state,omitempty"`
	TTL              int64         `json:"ttl,omitempty"`
	Schedule         *ScheduleData `json:"sched,omitempty"`
	Expression       *string       `json:"expression,omitempty"`
	PythonExpression *string       `json:"python_expression,omitempty"`
	Patterns         []string      `json:"patterns"`
	IsRemote         bool          `json:"is_remote"`
	MuteNewMetrics   bool          `json:"mute_new_metrics"`
}

Trigger represents trigger data object

func GetTriggerListsDiff

func GetTriggerListsDiff(triggerLists ...[]*Trigger) []*Trigger

GetTriggerListsDiff returns the members of the set resulting from the difference between the first set and all the successive lists.

func (*Trigger) IsSimple

func (trigger *Trigger) IsSimple() bool

IsSimple checks triggers patterns If patterns more than one or it contains standard graphite wildcard symbols, when this target can contain more then one metrics, and is it not simple trigger

type TriggerCheck

type TriggerCheck struct {
	Trigger
	Throttling int64             `json:"throttling"`
	LastCheck  CheckData         `json:"last_check"`
	Highlights map[string]string `json:"highlights"`
}

TriggerCheck represents trigger data with last check data and check timestamp

type TriggerData

type TriggerData struct {
	ID         string   `json:"id"`
	Name       string   `json:"name"`
	Desc       string   `json:"desc"`
	Targets    []string `json:"targets"`
	WarnValue  float64  `json:"warn_value"`
	ErrorValue float64  `json:"error_value"`
	IsRemote   bool     `json:"is_remote"`
	Tags       []string `json:"__notifier_trigger_tags"`
}

TriggerData represents trigger object

func (*TriggerData) GetTags

func (trigger *TriggerData) GetTags() string

GetTags returns "[tag1][tag2]...[tagN]" string

func (TriggerData) GetTriggerURI

func (trigger TriggerData) GetTriggerURI(frontURI string) string

GetTriggerURI gets frontUri and returns triggerUrl, returns empty string on selfcheck and test notifications

Directories

Path Synopsis
api
dto
nolint nolint nolint nolint nolint nolint nolint nolint nolint
nolint nolint nolint nolint nolint nolint nolint nolint nolint
cmd
api
cli
logging
metrics
graphite/go-metrics
nolint nolint nolint nolint nolint nolint
nolint nolint nolint nolint nolint nolint
mock
metric_source
Package mock_metric_source is a generated GoMock package.
Package mock_metric_source is a generated GoMock package.
moira-alert
Package mock_moira_alert is a generated GoMock package.
Package mock_moira_alert is a generated GoMock package.
notifier
Package mock_notifier is a generated GoMock package.
Package mock_notifier is a generated GoMock package.
scheduler
Package mock_scheduler is a generated GoMock package.
Package mock_scheduler is a generated GoMock package.
senders

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL