go_splunk_rest

README

go-splunk-rest

golang library to interact with splunk Rest API.

Supports basic, session key and authentication token methods of authentication.

Provides functions to deal with Search jobs.

Please open an issue if you want any specific Splunk API endpoints included.

Install

go get github.com/pvik/go-splunk-rest 

Examples

Splunk2DB uses this library, for additional code on how to use this library.

---

To make a simple blocking search

import "github.com/pvik/go-splunk-rest"

// ... 

	splunkConn := splunk.Connection{
		Host: "https://abc.splunk.com:8089",
		AuthType: "authentication-token",
		AuthenticationToken: "abcdef111",

		// or
		//AuthType: "authorization-token" // will use session keys or set this to "basic" for basic auth
		//Username: "api-user",
		//Password: "secure-password"
	}
	
	recs, err := splunkConn.Search("| from my_datamodel | fields - _raw | head 100", splunk.SearchOptions{})

---

The API provides an easy way to automatically shrink the search time window if the API result return is limited to the max_count (typically defaults to 10000)

Example To use this feature

	splunkConn := splunk.Connection{
		Host: "https://abc.splunk.com:8089",
		AuthType: "authentication-token",
		AuthenticationToken: "abcdef111",

		// or
		//AuthType: "authorization-token" // will use session keys or set this to "basic" for basic auth
		//Username: "api-user",
		//Password: "secure-password"
	}
	
	searchOptions := splunk.SearchOptions{
		MaxCount: 100,
		
		UseEarliestTime: true,
		EarliestTime: time.Now().Sub(30*24*time.Hour),
		UseLatestTime: true,
		LatestTime: time.Now(),

		AllowPartition: true,
	}
	
	recs, err := splunkConn.Search("| from my_datamodel | fields - _raw | head 100", searchOptions)

---

The library provides an easy way to search in an async fashion

	splunkConn := splunk.Connection{
		Host: "https://abc.splunk.com:8089",
		AuthType: "authentication-token",
		AuthenticationToken: "abcdef111",

		// or
		//AuthType: "authorization-token" // will use session keys or set this to "basic" for basic auth
		//Username: "api-user",
		//Password: "secure-password"
	}
	
	go splunkConn.SearchAndExec("| from my_datamodel | fields - _raw | head 100",  splunk.SearchOptions{},
		func(results []map[string]interface{}) error {
			// do something with results 
			// this will be called once the search completes 
			
			// ... 
			
			return nil 
		}, 
		func(e error) {
			// handle search error 
			log.Errorf("search failed: %s", e)
		}
	)

Documentation

Index

• Constants
• type AuthenticationType
  • func GetAllAuthenticationTypes() []AuthenticationType
  • func ParseAuthenticationType(s string) (c AuthenticationType, err error)
• type Connection
  • func (c Connection) Search(searchQuery string, searchOptions SearchOptions) ([]map[string]interface{}, error)
  • func (c Connection) SearchAndExec(searchQuery string, searchOptions SearchOptions, ...)
  • func (c Connection) SearchJobCreate(searchQuery string, searchOptions SearchOptions) (string, error)
  • func (c Connection) SearchJobResults(jobID string) ([]map[string]interface{}, error)
  • func (c Connection) SearchJobStatus(jobID string) (SearchJobStatus, error)
• type SearchJobStatus
  • func (s SearchJobStatus) IsDone() (bool, error)
• type SearchOptions

Constants

const DEFAULT_MAX_COUNT = 10000

const PARTITION_COUNT = 5

const SEARCH_WAIT = 5

const SPLUNK_TIME_FORMAT = "%m/%d/%Y:%H:%M:%S"

const TIME_FORMAT = "01/02/2006:15:04:05"

Types

type AuthenticationType

type AuthenticationType string

const AuthenticationTokenAuth AuthenticationType = "authentication-token"

const AuthorizationTokenAuth AuthenticationType = "authorization-token"

const BasicAuth AuthenticationType = "basic"

func GetAllAuthenticationTypes

func GetAllAuthenticationTypes() []AuthenticationType

func ParseAuthenticationType

func ParseAuthenticationType(s string) (c AuthenticationType, err error)

type Connection

type Connection struct {
	Host                string             `toml:"host"`
	AuthType            AuthenticationType `toml:"auth-type"` // basic, authorization-token, authentication-token
	Username            string             `toml:"username"`
	Password            string             `toml:"password"`
	AuthenticationToken string             `toml:"authentication-token"`
	MaxCount            int                `toml:"max-count"`
	InsecureSkipVerify  bool               `toml:"insecure-skip-verify"`
	// contains filtered or unexported fields
}

func (Connection) Search

func (c Connection) Search(searchQuery string, searchOptions SearchOptions) ([]map[string]interface{}, error)

Blocking Search function this will queue a search job, and wait in SEARCH_WAIT increments to check search-job status, and then return the result records

func (Connection) SearchAndExec

func (c Connection) SearchAndExec(searchQuery string, searchOptions SearchOptions,
	onSuccess func([]map[string]interface{}) error,
	onError func(error),
)

Stub function making it easier to search in an Async fashion as a goroutine

func (Connection) SearchJobCreate

func (c Connection) SearchJobCreate(searchQuery string, searchOptions SearchOptions) (string, error)

func (Connection) SearchJobResults

func (c Connection) SearchJobResults(jobID string) ([]map[string]interface{}, error)

func (Connection) SearchJobStatus

func (c Connection) SearchJobStatus(jobID string) (SearchJobStatus, error)

type SearchJobStatus

type SearchJobStatus struct {
	Messages []struct {
		Type    string `json:"type"`
		Message string `json:"text"`
	}
	Entry []struct {
		Content struct {
			IsDone   bool `json:"isDone"`
			IsFailed bool `json:"isFailed"`
		} `json:"content"`
	} `json:"entry"`
}

func (SearchJobStatus) IsDone

func (s SearchJobStatus) IsDone() (bool, error)

type SearchOptions

type SearchOptions struct {
	// max records, defaults to DEFAULT_MAX_COUNT
	MaxCount int

	// Sets the earliest (inclusive), respectively, time bounds for the search.
	// use time format %m/%d/%Y:%H:%M:%S
	UseEarliestTime bool
	EarliestTime    time.Time
	// Sets the latest (exclusive), respectively, time bounds for the search.
	// use time format %m/%d/%Y:%H:%M:%S
	UseLatestTime bool
	LatestTime    time.Time

	// In the Search function ; for searches which hit the maxCount,
	// to recursively create new searches on reduced time ranges
	// (by using shrinking earliest and latest time fields)
	// and combine the results at the end
	AllowPartition bool

	// AdhocSearchLevel can be fast, smart or verbose according to splunk documentation
	AdhocSearchLevel string
}

hold options that can be passed to a search job more details can be found here: https://docs.splunk.com/Documentation/Splunk/9.1.0/RESTREF/RESTsearch#search.2Fjobs