prepalert

package module
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 12, 2024 License: MIT Imports: 45 Imported by: 0

README

prepalert

Latest GitHub release Github Actions test License

Toil reduction tool to prepare before responding to Mackerel alerts

preplert consists of two parts: a webhook server that receives Mackerel webhooks and sends the payload to Amazon SQS, and a worker that queries various data based on the webhooks and pastes information for alert response as a GraphAnnotation.

Install

Homebrew (macOS and Linux)
$ brew install mashiike/tap/prepalert
Binary packages

Releases

QuickStart

Set your Mackerel API key to the environment variable MACKEREL_APIKEY.
and Execute the following command:

$ prepalert init 

Or the following command:

$ prepalert --coinfig <output config path> init

Usage

Usage: prepalert <command>

A webhook server for prepare alert memo

Flags:
  -h, --help                      Show context-sensitive help.
      --log-level="info"          output log-level ($PREPALERT_LOG_LEVEL)
      --mackerel-apikey=STRING    for access mackerel API ($MACKEREL_APIKEY)
      --config="."                config path ($PREPALERT_CONFIG)

Commands:
  run
    run server (default command)

  init
    create initial config

  validate
    validate the configuration

  exec <alert-id>
    Generate a virtual webhook from past alert to execute the rule

  version
    Show version

Run "prepalert <command> --help" for more information on a command.

If the command is omitted, the run command is executed.

Configurations

Configuration file is HCL (HashiCorp Configuration Language) format. prepalert init can generate a initial configuration file.

The most small configuration file is as follows:

prepalert {
    required_version = ">=v0.12.0"
    sqs_queue_name   = "prepalert"
}

locals {
    default_message =  <<EOF
How do you respond to alerts?
Describe information about your alert response here.
EOF
}

rule "simple" {
    // rule execute when org_name is "Macker..." and alert id is "4gx..."
    when = [
        webhook.org_name == "Macker...",
        get_monitor(webhook.alert).id == "4gx...",
    ]
    update_alert {
        memo = local.default_message
    }
}

Usage with AWS Lambda (serverless)

prepalert works with AWS Lambda and Amazon SQS.

Lambda Function requires a webhook and a worker

sequenceDiagram
  autonumber
  Mackerel->>+http lambda function : POST /
  http lambda function ->>+Amazon SQS: SendMessage
  Amazon SQS-->- http lambda function: 200 Ok
  http lambda function-->- Mackerel: 200 Ok
  Amazon SQS ->>+ worker lambda function: trigger by AWS Lambda
  worker lambda function ->>+ Data Source: query
  Data Source -->- worker lambda function: query results
  worker lambda function  ->>+ Mackerel: Create Graph Annotation
  Mackerel-->- worker lambda function : 200 Ok
  worker lambda function ->>-  Amazon SQS: Success Delete message

Let's solidify the Lambda package with the following zip arcive (runtime provided.al2)

lambda.zip
├── bootstrap    # build binary
└── config.hcl   # configuration file

A related document is https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html

for example.

deploy lambda function, prepalert in lambda directory
The example of lambda directory uses lambroll for deployment.

Advanced Usage

Plugin System

prepalert has a plugin system. you can add custom provider. plugin is gRPC Server program. protocol buffer definition is here

in Go language, following code is example plugin.

package main

import (
	"context"

	"github.com/mashiike/prepalert/plugin"
	"github.com/mashiike/prepalert/provider"
)

type Provider struct {
	// ...
}

func (p *Provider) ValidateProviderParameter(ctx context.Context, pp *provider.ProviderParameter) error {
	/*
		Your custom provider parameter validation
	*/
	return nil
}

func (p *Provider) GetQuerySchema(ctx context.Context) (*plugin.Schema, error) {
	/*
		Your custom provider's query paramete schema
	*/
	return &plugin.Schema{}, nil
}

func (p *Provider) RunQuery(ctx context.Context, req *plugin.RunQueryRequest) (*plugin.RunQueryResponse, error) {
	/*
		Your custom provider's query execution
	*/
	return &plugin.RunQueryResponse{
		Name:    req.QueryName,
		Query:   "<your query string>",
		Columns: []string{"<column name>", "<column name>", "<column name>"},
		Rows: [][]string{
			{"<row value>", "<row value>", "<row value>"},
			{"<row value>", "<row value>", "<row value>"},
			{"<row value>", "<row value>", "<row value>"},
		},
	}, nil
}

func main() {
	plugin.ServePlugin(plugin.WithProviderPlugin(&Provider{}))
}

example plugin is cmd/example-http-csv-plugin

configuration is as follows:

prepalert {
  required_version = ">=v0.12.0"
  sqs_queue_name   = "prepalert"

  plugins {
    http = {
      cmd         = "go run github.com/mashiike/prepalert/cmd/example-http-csv-plugin@latest" // your plugin execution command
      sync_output = true  // sync plugin output to prepalert log
    }
  }
}

provider "http" {
  endpoint = "<your csv server endpoint>"
}

query "http" "csv" {}

rule "always" {
  when = true
  update_alert {
    memo = "${query.http.csv.result.query}\n${result_to_table(query.http.csv)}"
  }
}

Local Development

$ PREPALERT_CANYON_ENV=development go run cmd/prepalert/main.go --config testdata/config/simple.hcl
time=2023-10-01T14:55:47.267+09:00 level=INFO msg="create temporary file backend, canyon request body upload to temporary directory" path=/var/folders/rn/jj26k6s93x9c5yblnjq7_dw80000gp/T/canon-240945814 version=v0.12.0 app=prepalert
time=2023-10-01T14:55:47.268+09:00 level=INFO msg="running canyon" env=development version=v0.12.0 app=prepalert
time=2023-10-01T14:55:47.268+09:00 level=INFO msg="enable in memory queue" visibility_timeout=30s max_receive_count=3 version=v0.12.0 app=prepalert
time=2023-10-01T14:55:47.268+09:00 level=INFO msg="starting up with local httpd" address=:8080 version=v0.12.0 app=prepalert
time=2023-10-01T14:55:47.268+09:00 level=INFO msg="staring polling sqs queue" queue=prepalert on_memory_queue_mode=true version=v0.12.0 app=prepalert

prepalert is using canyon. if you want to use local development, you can use PREPALERT_CANYON_ENV=development environment variable. this variable is enable to use local file backend and sqs simulated in memory queue.

LICENSE

MIT License

Copyright (c) 2022 IKEDA Masashi

Documentation

Index

Constants

View Source
const (
	FindGraphAnnotationOffset = int64(15 * time.Minute / time.Second)
)
View Source
const (
	HeaderRequestID = "Prepalert-Request-ID"
)

Variables

View Source
var (
	GraphAnnotationDescriptionMaxSize = 1024
	AlertMemoMaxSize                  = 80 * 1000
	CacheDuration                     = 1 * time.Minute
)
View Source
var Version = "v1.0.2"

Functions

func ExpressionToString added in v0.12.0

func ExpressionToString(expr hcl.Expression, evalCtx *hcl.EvalContext) (string, error)

func GenerateInitialConfig added in v0.12.0

func GenerateInitialConfig(sqsQueueName string, orgName string) ([]byte, error)

func RunCLI added in v0.10.0

func RunCLI(ctx context.Context, args []string, setLogLevel func(string)) error

Types

type Alert

type Alert struct {
	OpenedAt          int64    `json:"openedAt" cty:"opened_at"`
	ClosedAt          *int64   `json:"closedAt" cty:"closed_at"`
	CreatedAt         int64    `json:"createdAt" cty:"created_at"`
	CriticalThreshold *float64 `json:"criticalThreshold,omitempty" cty:"critical_threshold,omitempty"`
	Duration          int64    `json:"duration" cty:"duration"`
	IsOpen            bool     `json:"isOpen" cty:"is_open"`
	MetricLabel       string   `json:"metricLabel" cty:"metric_label"`
	MetricValue       float64  `json:"metricValue" cty:"metric_value"`
	MonitorName       string   `json:"monitorName" cty:"monitor_name"`
	MonitorOperator   string   `json:"monitorOperator" cty:"monitor_operator"`
	Status            string   `json:"status" cty:"status"`
	Trigger           string   `json:"trigger" cty:"trigger"`
	ID                string   `json:"id" cty:"id"`
	URL               string   `json:"url" cty:"url"`
	WarningThreshold  *float64 `json:"warningThreshold,omitempty" cty:"warning_threshold,omitempty"`
}

type App

type App struct {
	// contains filtered or unexported fields
}

func New

func New(apikey string) *App

func (*App) Backend added in v0.12.0

func (app *App) Backend() Backend

func (*App) CheckBasicAuth

func (app *App) CheckBasicAuth(r *http.Request) bool

func (*App) Close added in v0.12.0

func (app *App) Close() error

func (*App) DecodeBody added in v0.12.0

func (app *App) DecodeBody(body hcl.Body, evalCtx *hcl.EvalContext) hcl.Diagnostics

func (*App) EnableBasicAuth

func (app *App) EnableBasicAuth() bool

func (*App) EnableWebhookServer added in v0.12.0

func (app *App) EnableWebhookServer() bool

func (*App) Exec added in v0.5.0

func (app *App) Exec(ctx context.Context, opts *ExecOptions) error

func (*App) ExecuteRules added in v0.12.0

func (app *App) ExecuteRules(ctx context.Context, body *WebhookBody) error

func (*App) Init added in v0.12.0

func (app *App) Init(ctx context.Context, outputPath string) error

func (*App) LoadConfig added in v0.12.0

func (app *App) LoadConfig(dir string, optFns ...func(*LoadConfigOptions)) error

func (*App) LoadPlugin added in v0.12.0

func (app *App) LoadPlugin(ctx context.Context, cfg *LoadPluginConfig) error

func (*App) MackerelService added in v0.12.0

func (app *App) MackerelService() *MackerelService

func (*App) NewEvalContext added in v0.12.0

func (app *App) NewEvalContext(body *WebhookBody) (*hcl.EvalContext, error)

func (*App) NewRule added in v0.12.0

func (app *App) NewRule(ruleName string) *Rule

func (*App) ProviderList added in v0.12.0

func (app *App) ProviderList() []string

func (*App) QueryList added in v0.12.0

func (app *App) QueryList() []string

func (*App) RetryPolicy added in v1.0.0

func (app *App) RetryPolicy() *RetryPolicy

func (*App) Rules added in v0.12.0

func (app *App) Rules() []*Rule

func (*App) Run

func (app *App) Run(ctx context.Context, opts *RunOptions) error

func (*App) SQSQueueName added in v0.12.0

func (app *App) SQSQueueName() string

func (*App) ServeHTTP

func (app *App) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*App) SetMackerelClient added in v0.12.0

func (app *App) SetMackerelClient(client MackerelClient) *App

func (*App) SetupS3Buckend added in v0.12.0

func (app *App) SetupS3Buckend(body hcl.Body) hcl.Diagnostics

func (*App) UnwrapAndDumpDiagnoctics added in v0.12.0

func (app *App) UnwrapAndDumpDiagnoctics(err error) error

func (*App) WebhookServerIsReady added in v0.12.0

func (app *App) WebhookServerIsReady() bool

func (*App) WithPrepalertFunctions added in v0.12.0

func (app *App) WithPrepalertFunctions(evalCtx *hcl.EvalContext) *hcl.EvalContext

func (*App) WorkerIsReady added in v0.12.0

func (app *App) WorkerIsReady() bool

type Backend added in v0.12.0

type Backend interface {
	http.Handler
	fmt.Stringer
	Upload(ctx context.Context, evalCtx *hcl.EvalContext, name string, body io.Reader) (string, bool, error)
}

type CLI added in v0.10.0

type CLI struct {
	LogLevel       string        `help:"output log-level" env:"PREPALERT_LOG_LEVEL" default:"info"`
	MackerelAPIKey string        `name:"mackerel-apikey" help:"for access mackerel API" env:"MACKEREL_APIKEY"`
	ErrorHandling  ErrorHandling `help:"error handling" env:"PREPALERT_ERROR_HANDLING" default:"continue" enum:"continue,return"`
	Config         string        `help:"config path" env:"PREPALERT_CONFIG" default:"."`
	Run            *RunOptions   `cmd:"" help:"run server (default command)" default:""`
	Init           struct{}      `cmd:"" help:"create initial config"`
	Validate       struct{}      `cmd:"" help:"validate the configuration"`
	Exec           *ExecOptions  `cmd:"" help:"Generate a virtual webhook from past alert to execute the rule"`
	Version        struct{}      `cmd:"" help:"Show version"`
}

func ParseCLI added in v0.10.0

func ParseCLI(ctx context.Context, args []string, opts ...kong.Option) (string, *CLI, error)

type DiscardBackend added in v0.12.0

type DiscardBackend struct{}

func NewDiscardBackend added in v0.12.0

func NewDiscardBackend() *DiscardBackend

func (*DiscardBackend) ServeHTTP added in v0.12.0

func (b *DiscardBackend) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*DiscardBackend) String added in v0.12.0

func (b *DiscardBackend) String() string

func (*DiscardBackend) Upload added in v0.12.0

func (b *DiscardBackend) Upload(ctx context.Context, evalCtx *hcl.EvalContext, name string, body io.Reader) (string, bool, error)

type ErrorHandling added in v0.12.0

type ErrorHandling int
const (
	ContinueOnError ErrorHandling = iota // if load config on error, continue run
	ReturnOnError                        // if load config on error, return error
)

func (ErrorHandling) String added in v0.12.0

func (e ErrorHandling) String() string

func (*ErrorHandling) UnmarshalText added in v0.12.0

func (e *ErrorHandling) UnmarshalText(text []byte) error

type ExecOptions added in v0.10.0

type ExecOptions struct {
	AlertID string `arg:"" name:"alert-id" help:"Mackerel AlertID" required:""`
}

type Host

type Host struct {
	ID        string  `json:"id" cty:"id"`
	Name      string  `json:"name" cty:"name"`
	URL       string  `json:"url" cty:"url"`
	Type      string  `json:"type,omitempty" cty:"type"`
	Status    string  `json:"status" cty:"status"`
	Memo      string  `json:"memo" cty:"memo"`
	IsRetired bool    `json:"isRetired" cty:"is_retired"`
	Roles     []*Role `json:"roles" cty:"roles,omitempty"`
}

type LoadConfigOptions added in v0.12.0

type LoadConfigOptions struct {
	DiagnosticDestination io.Writer
	Color                 *bool
	Width                 *uint
}

type LoadPluginConfig added in v0.12.0

type LoadPluginConfig struct {
	PluginName string `cty:"-"`
	Command    string `cty:"cmd"`
	SyncOutput bool   `cty:"sync_output"`
}

type MackerelClient added in v0.12.0

type MackerelClient interface {
	UpdateAlert(alertID string, param mackerel.UpdateAlertParam) (*mackerel.UpdateAlertResponse, error)
	FindGraphAnnotations(service string, from int64, to int64) ([]*mackerel.GraphAnnotation, error)
	UpdateGraphAnnotation(annotationID string, annotation *mackerel.GraphAnnotation) (*mackerel.GraphAnnotation, error)
	CreateGraphAnnotation(annotation *mackerel.GraphAnnotation) (*mackerel.GraphAnnotation, error)
	GetOrg() (*mackerel.Org, error)
	GetAlert(string) (*mackerel.Alert, error)
	GetMonitor(string) (mackerel.Monitor, error)
	FindHost(id string) (*mackerel.Host, error)
}

type MackerelService added in v0.12.0

type MackerelService struct {
	// contains filtered or unexported fields
}

func NewMackerelService added in v0.12.0

func NewMackerelService(client MackerelClient) *MackerelService

func (*MackerelService) GetAlertWithCache added in v0.12.0

func (svc *MackerelService) GetAlertWithCache(ctx context.Context, alertID string) (*mackerel.Alert, error)

func (*MackerelService) GetMonitorByAlertID added in v0.12.0

func (svc *MackerelService) GetMonitorByAlertID(ctx context.Context, alertID string) (mackerel.Monitor, error)

func (*MackerelService) GetMonitorWithCache added in v0.12.0

func (svc *MackerelService) GetMonitorWithCache(ctx context.Context, monitorID string) (mackerel.Monitor, error)

func (*MackerelService) NewEmulatedWebhookBody added in v0.12.0

func (svc *MackerelService) NewEmulatedWebhookBody(ctx context.Context, alertID string) (*WebhookBody, error)

func (*MackerelService) NewExampleWebhookBody added in v0.12.0

func (svc *MackerelService) NewExampleWebhookBody() *WebhookBody

func (*MackerelService) NewMackerelUpdater added in v0.12.0

func (svc *MackerelService) NewMackerelUpdater(body *WebhookBody, backend Backend) *MackerelUpdater

func (*MackerelService) PostGraphAnnotation added in v0.12.0

func (svc *MackerelService) PostGraphAnnotation(ctx context.Context, params *mackerel.GraphAnnotation) error

func (*MackerelService) UpdateAlertMemo added in v0.12.0

func (svc *MackerelService) UpdateAlertMemo(ctx context.Context, alertID string, memo string) error

type MackerelUpdater added in v0.12.0

type MackerelUpdater struct {
	// contains filtered or unexported fields
}

func (*MackerelUpdater) AddAdditionalDescription added in v0.12.0

func (u *MackerelUpdater) AddAdditionalDescription(service string, text string)

func (*MackerelUpdater) AddMemoSectionText added in v0.12.0

func (u *MackerelUpdater) AddMemoSectionText(sectionName string, text string, sizeLimit *int)

func (*MackerelUpdater) AddService added in v0.12.0

func (u *MackerelUpdater) AddService(service string)

func (*MackerelUpdater) Flush added in v0.12.0

func (u *MackerelUpdater) Flush(ctx context.Context, evalCtx *hcl.EvalContext) error

type PostGraphAnnotationAction added in v0.12.0

type PostGraphAnnotationAction struct {
	// contains filtered or unexported fields
}

func (*PostGraphAnnotationAction) DecodeBody added in v0.12.0

func (action *PostGraphAnnotationAction) DecodeBody(body hcl.Body, evalCtx *hcl.EvalContext) hcl.Diagnostics

func (*PostGraphAnnotationAction) DependsOnQueries added in v0.12.0

func (action *PostGraphAnnotationAction) DependsOnQueries() []string

func (*PostGraphAnnotationAction) Enable added in v0.12.0

func (action *PostGraphAnnotationAction) Enable() bool

func (*PostGraphAnnotationAction) Execute added in v0.12.0

func (action *PostGraphAnnotationAction) Execute(ctx context.Context, evalCtx *hcl.EvalContext, u *MackerelUpdater) error

type RequestIDGenerator added in v0.12.0

type RequestIDGenerator interface {
	NextID() (uint64, error)
}
var DefaultRequestIDGeneartor RequestIDGenerator = must(katsubushi.NewGenerator(1))

type RetryPolicy added in v1.0.0

type RetryPolicy struct {
	Interval      float64 `hcl:"interval,optional"`
	Jitter        float64 `hcl:"jitter,optional"`
	MaxInterval   float64 `hcl:"max_interval,optional"`
	BackoffFactor float64 `hcl:"backoff_factor,optional"`
	// contains filtered or unexported fields
}

func (*RetryPolicy) DecodeAttributes added in v1.0.0

func (rp *RetryPolicy) DecodeAttributes(attrs hcl.Attributes, evalCtx *hcl.EvalContext) hcl.Diagnostics

func (*RetryPolicy) SetRetryAfter added in v1.0.0

func (rp *RetryPolicy) SetRetryAfter(w http.ResponseWriter, r *http.Request)

func (*RetryPolicy) String added in v1.0.0

func (rp *RetryPolicy) String() string

type Role

type Role struct {
	Fullname    string `json:"fullname" cty:"fullname"`
	ServiceName string `json:"serviceName" cty:"service_name"`
	ServiceURL  string `json:"serviceUrl" cty:"service_url"`
	RoleName    string `json:"roleName" cty:"role_name"`
	RoleURL     string `json:"roleUrl" cty:"role_url"`
}

type Rule

type Rule struct {
	// contains filtered or unexported fields
}

func (*Rule) DecodeBody added in v0.12.0

func (rule *Rule) DecodeBody(body hcl.Body, evalCtx *hcl.EvalContext) hcl.Diagnostics

func (*Rule) DependsOnQueries added in v0.12.0

func (rule *Rule) DependsOnQueries() []string

func (*Rule) Execute added in v0.12.0

func (rule *Rule) Execute(ctx context.Context, evalCtx *hcl.EvalContext, u *MackerelUpdater) error

func (*Rule) Match

func (rule *Rule) Match(evalCtx *hcl.EvalContext) bool

func (*Rule) Name added in v0.5.0

func (rule *Rule) Name() string

func (*Rule) PostGraphAnnotationAction added in v0.12.0

func (rule *Rule) PostGraphAnnotationAction() *PostGraphAnnotationAction

func (*Rule) Priority added in v0.12.0

func (rule *Rule) Priority() int

func (*Rule) UpdateAlertAction added in v0.12.0

func (rule *Rule) UpdateAlertAction() *UpdateAlertAction

type RunOptions

type RunOptions struct {
	Mode      string `help:"run mode" env:"PREPALERT_MODE" default:"all" enum:"all,http,worker,webhook"`
	Address   string `help:"run local address" env:"PREPALERT_ADDRESS" default:":8080"`
	Prefix    string `help:"run server prefix" env:"PREPALERT_PREFIX" default:"/"`
	BatchSize int    `help:"run local sqs batch size" env:"PREPALERT_BATCH_SIZE" default:"1"`
}

type S3Backend added in v0.12.0

type S3Backend struct {
	BucketName                    string
	ObjectKeyPrefix               *string
	ObjectKeyTemplate             *hcl.Expression
	ViewerBaseURLString           string
	ViewerGoogleClientID          *string
	ViewerGoogleClientSecret      *string
	ViewerSessionEncryptKeyString *string
	Allowed                       []string
	Denied                        []string

	ViewerBaseURL           *url.URL
	ViewerSessionEncryptKey []byte
	// contains filtered or unexported fields
}

func (*S3Backend) EnableGoogleAuth added in v0.12.0

func (b *S3Backend) EnableGoogleAuth() bool

func (*S3Backend) IsEmpty added in v0.12.0

func (b *S3Backend) IsEmpty() bool

func (*S3Backend) ServeHTTP added in v0.12.0

func (b *S3Backend) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*S3Backend) String added in v0.12.0

func (b *S3Backend) String() string

func (*S3Backend) Upload added in v0.12.0

func (b *S3Backend) Upload(ctx context.Context, evalCtx *hcl.EvalContext, name string, body io.Reader) (string, bool, error)

type S3Client added in v0.12.0

type S3Client interface {
	manager.UploadAPIClient
	ls3viewer.S3Client
}
var GlobalS3Client S3Client

type Service

type Service struct {
	ID    string  `json:"id" cty:"id"`
	Memo  string  `json:"memo" cty:"memo"`
	Name  string  `json:"name" cty:"name"`
	OrgID string  `json:"orgId" cty:"org_id"`
	Roles []*Role `json:"roles" cty:"roles,omitempty"`
}

type UpdateAlertAction added in v0.12.0

type UpdateAlertAction struct {
	// contains filtered or unexported fields
}

func (*UpdateAlertAction) DecodeBody added in v0.12.0

func (action *UpdateAlertAction) DecodeBody(body hcl.Body, evalCtx *hcl.EvalContext) hcl.Diagnostics

func (*UpdateAlertAction) DependsOnQueries added in v0.12.0

func (action *UpdateAlertAction) DependsOnQueries() []string

func (*UpdateAlertAction) Enable added in v0.12.0

func (action *UpdateAlertAction) Enable() bool

func (*UpdateAlertAction) Execute added in v0.12.0

func (action *UpdateAlertAction) Execute(ctx context.Context, evalCtx *hcl.EvalContext, u *MackerelUpdater) error

type WebhookBody

type WebhookBody struct {
	OrgName  string   `json:"orgName" cty:"org_name"`
	Text     string   `json:"text" cty:"-"`
	Event    string   `json:"event" cty:"event"`
	ImageURL *string  `json:"imageUrl" cty:"image_url"`
	Memo     string   `json:"memo" cty:"memo"`
	Host     *Host    `json:"host,omitempty" cty:"host,omitempty"`
	Service  *Service `json:"service,omitempty" cty:"service,omitempty"`
	Alert    *Alert   `json:"alert" cty:"alert,omitempty"`
}

func WebhookFromEvalContext added in v0.12.0

func WebhookFromEvalContext(evalCtx *hcl.EvalContext) (*WebhookBody, error)

Directories

Path Synopsis
cmd
Package mock is a generated GoMock package.
Package mock is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL