osquery

package module
v0.0.0-...-629f995 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 31, 2025 License: MIT Imports: 9 Imported by: 40

README

osquery-go

GoDoc

osquery exposes an operating system as a high-performance relational database. This allows you to write SQL-based queries to explore operating system data. With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.

If you're interested in learning more about osquery, visit the GitHub project, the website, and the users guide.

What is osquery-go?

In osquery, SQL tables, configuration retrieval, log handling, etc. are implemented via a robust plugin and extensions API. This project contains Go bindings for creating osquery extensions in Go. To create an extension, you must create an executable binary which instantiates an ExtensionManagerServer and registers the plugins that you would like to be added to osquery. You can then have osquery load the extension in your desired context (ie: in a long running instance of osqueryd or during an interactive query session with osqueryi). For more information about how this process works at a lower level, see the osquery wiki.

Install

This library is compatible with Go Modules. To install:

go get github.com/osquery/osquery-go

Using the library

Creating a new osquery table

If you want to create a custom osquery table in Go, you'll need to write an extension which registers the implementation of your table. Consider the following Go program:

package main

import (
	"context"
	"log"
	"os"
	"flag"

	"github.com/osquery/osquery-go"
	"github.com/osquery/osquery-go/plugin/table"
)

func main() {
	socket := flag.String("socket", "", "Path to osquery socket file")
	flag.Parse()
	if *socket == "" {
		log.Fatalf(`Usage: %s --socket SOCKET_PATH`, os.Args[0])
	}

	server, err := osquery.NewExtensionManagerServer("foobar", *socket)
	if err != nil {
		log.Fatalf("Error creating extension: %s\n", err)
	}

	// Create and register a new table plugin with the server.
	// table.NewPlugin requires the table plugin name,
	// a slice of Columns and a Generate function.
	server.RegisterPlugin(table.NewPlugin("foobar", FoobarColumns(), FoobarGenerate))
	if err := server.Run(); err != nil {
		log.Fatalln(err)
	}
}

// FoobarColumns returns the columns that our table will return.
func FoobarColumns() []table.ColumnDefinition {
	return []table.ColumnDefinition{
		table.TextColumn("foo"),
		table.TextColumn("baz"),
	}
}

// FoobarGenerate will be called whenever the table is queried. It should return
// a full table scan.
func FoobarGenerate(ctx context.Context, queryContext table.QueryContext) ([]map[string]string, error) {
	return []map[string]string{
		{
			"foo": "bar",
			"baz": "baz",
		},
		{
			"foo": "bar",
			"baz": "baz",
		},
	}, nil
}

To test this code, start an osquery shell and find the path of the osquery extension socket:

osqueryi --nodisable_extensions
osquery> select value from osquery_flags where name = 'extensions_socket';
+-----------------------------------+
| value                             |
+-----------------------------------+
| /Users/USERNAME/.osquery/shell.em |
+-----------------------------------+

Then start the Go extension and have it communicate with osqueryi via the extension socket that you retrieved above:

go run ./my_table_plugin.go --socket /Users/USERNAME/.osquery/shell.em

Alternatively, you can also autoload your extension when starting an osquery shell:

go build -o my_table_plugin my_table_plugin.go
osqueryi --extension /path/to/my_table_plugin

This will register a table called "foobar". As you can see, the table will return two rows:

osquery> select * from foobar;
+-----+-----+
| foo | baz |
+-----+-----+
| bar | baz |
| bar | baz |
+-----+-----+
osquery>

This is obviously a contrived example, but it's easy to imagine the possibilities.

Using the instructions found on the wiki, you can deploy your extension with an existing osquery deployment.

Creating logger and config plugins

The process required to create a config and/or logger plugin is very similar to the process outlined above for creating an osquery table. Specifically, you would create an ExtensionManagerServer instance in func main(), register your plugin and launch the extension as described above. The only difference is that the implementation of your plugin would be different. Each plugin package has a NewPlugin function which takes the plugin name as the first argument, followed by a list of required arguments to implement the plugin. For example, consider the implementation of an example logger plugin:

func main() {
    // create and register the plugin
	server.RegisterPlugin(logger.NewPlugin("example_logger", LogString))
}

func LogString(ctx context.Context, typ logger.LogType, logText string) error {
	log.Printf("%s: %s\n", typ, logText)
	return nil
}

Additionally, consider the implementation of an example config plugin:

func main() {
    // create and register the plugin
	server.RegisterPlugin(config.NewPlugin("example", GenerateConfigs))
}

func GenerateConfigs(ctx context.Context) (map[string]string, error) {
	return map[string]string{
		"config1": `
{
  "options": {
    "host_identifier": "hostname",
    "schedule_splay_percent": 10
  },
  "schedule": {
    "macos_kextstat": {
      "query": "SELECT * FROM kernel_extensions;",
      "interval": 10
    },
    "foobar": {
      "query": "SELECT foo, bar, pid FROM foobar_table;",
      "interval": 600
    }
  }
}
`,
	}, nil
}

All of these examples and more can be found in the examples subdirectory of this repository.

Execute queries in Go

This library can also be used to create a Go client for the osqueryd or osqueryi's extension socket. You can use this to add the ability to performantly execute osquery queries to your Go program. Consider the following example:

package main

import (
	"fmt"
	"os"
	"time"

	"github.com/osquery/osquery-go"
)

func main() {
	if len(os.Args) != 3 {
		log.Fatalf("Usage: %s SOCKET_PATH QUERY", os.Args[0])
	}

	client, err := osquery.NewClient(os.Args[1], 10*time.Second)
	if err != nil {
		log.Fatalf("Error creating Thrift client: %v", err)
	}
	defer client.Close()

	resp, err := client.Query(os.Args[2])
	if err != nil {
		log.Fatalf("Error communicating with osqueryd: %v",err)
	}
	if resp.Status.Code != 0 {
		log.Fatalf("osqueryd returned error: %s", resp.Status.Message)
	}

	fmt.Printf("Got results:\n%#v\n", resp.Response)
}
Loading extensions with osqueryd

If you write an extension with a logger or config plugin, you'll likely want to autoload the extensions when osqueryd starts. osqueryd has a few requirements for autoloading extensions, documented on the wiki. Here's a quick example using a logging plugin to get you started:

  1. Build the plugin. Make sure to add .ext as the file extension. It is required by osqueryd. go build -o /usr/local/osquery_extensions/my_logger.ext

  2. Set the correct permissions on the file and directory. If osqueryd runs as root, the directory for the extension must only be writable by root.

sudo chown -R root /usr/local/osquery_extensions/
  1. Create an extensions.load file with the path of your extension.
echo "/usr/local/osquery_extensions/my_logger.ext" > /tmp/extensions.load
  1. Start osqueryd with the --extensions_autoload flag.
sudo osqueryd --extensions_autoload=/tmp/extensions.load --logger-plugin=my_logger -verbose

Contributing

For more information on contributing to this project, see CONTRIBUTING.md.

Vulnerabilities

If you find a vulnerability in this software, please email security@kolide.co.

Documentation

Index

Constants

View Source 
const MaxSocketPathCharacters = 97

MaxSocketPathCharacters is set to 97 because a ".12345" uuid is added to the socket down stream if the provided socket is greater than 97 we may exceed the limit of 103 (104 causes an error) why 103 limit? https://unix.stackexchange.com/questions/367008/why-is-socket-path-length-limited-to-a-hundred-chars

Variables

This section is empty.

Functions

func NewLocker 

func NewLocker(defaultTimeout time.Duration, maxWait time.Duration) *locker

Types

type CallFunc 

type CallFunc func(registry string, item string, req osquery.ExtensionPluginRequest) (*osquery.ExtensionResponse, error)

type ClientOption 

type ClientOption func(*ExtensionManagerClient)

func DefaultWaitTime 

func DefaultWaitTime(d time.Duration) ClientOption

WaitTime sets the default amount of wait time for the osquery socket to free up. You can override this on a per call basis by setting a context deadline

func MaxWaitTime 

func MaxWaitTime(d time.Duration) ClientOption

MaxWaitTime is the maximum amount of time something is allowed to wait for the osquery socket. This takes precedence over the context deadline.

type CloseFunc 

type CloseFunc func()

type DeregisterExtensionFunc 

type DeregisterExtensionFunc func(uuid osquery.ExtensionRouteUUID) (*osquery.ExtensionStatus, error)

type ExtensionManager 

type ExtensionManager interface {
	Close()
	Ping() (*osquery.ExtensionStatus, error)
	Call(registry, item string, req osquery.ExtensionPluginRequest) (*osquery.ExtensionResponse, error)
	Extensions() (osquery.InternalExtensionList, error)
	RegisterExtension(info *osquery.InternalExtensionInfo, registry osquery.ExtensionRegistry) (*osquery.ExtensionStatus, error)
	DeregisterExtension(uuid osquery.ExtensionRouteUUID) (*osquery.ExtensionStatus, error)
	Options() (osquery.InternalOptionList, error)
	Query(sql string) (*osquery.ExtensionResponse, error)
	GetQueryColumns(sql string) (*osquery.ExtensionResponse, error)
}

type ExtensionManagerClient 

type ExtensionManagerClient struct {
	// contains filtered or unexported fields
}

ExtensionManagerClient is a wrapper for the osquery Thrift extensions API.

func NewClient 

func NewClient(path string, socketOpenTimeout time.Duration, opts ...ClientOption) (*ExtensionManagerClient, error)

NewClient creates a new client communicating to osquery over the socket at the provided path. If resolving the address or connecting to the socket fails, this function will error.

func (*ExtensionManagerClient) Call 

func (c *ExtensionManagerClient) Call(registry, item string, request osquery.ExtensionPluginRequest) (*osquery.ExtensionResponse, error)

Call requests a call to an extension (or core) registry plugin, using a new background context

func (*ExtensionManagerClient) CallContext 

func (c *ExtensionManagerClient) CallContext(ctx context.Context, registry, item string, request osquery.ExtensionPluginRequest) (*osquery.ExtensionResponse, error)

CallContext requests a call to an extension (or core) registry plugin.

func (*ExtensionManagerClient) Close 

func (c *ExtensionManagerClient) Close()

Close should be called to close the transport when use of the client is completed.

func (*ExtensionManagerClient) DeregisterExtension 

func (c *ExtensionManagerClient) DeregisterExtension(uuid osquery.ExtensionRouteUUID) (*osquery.ExtensionStatus, error)

DeregisterExtension de-registers the extension plugins with the osquery process, using a new background context

func (*ExtensionManagerClient) DeregisterExtensionContext 

func (c *ExtensionManagerClient) DeregisterExtensionContext(ctx context.Context, uuid osquery.ExtensionRouteUUID) (*osquery.ExtensionStatus, error)

DeregisterExtensionContext de-registers the extension plugins with the osquery process.

func (*ExtensionManagerClient) Extensions 

func (c *ExtensionManagerClient) Extensions() (osquery.InternalExtensionList, error)

Extensions requests the list of active registered extensions, using a new background context

func (*ExtensionManagerClient) ExtensionsContext 

func (c *ExtensionManagerClient) ExtensionsContext(ctx context.Context) (osquery.InternalExtensionList, error)

ExtensionsContext requests the list of active registered extensions.

func (*ExtensionManagerClient) GetQueryColumns 

func (c *ExtensionManagerClient) GetQueryColumns(sql string) (*osquery.ExtensionResponse, error)

GetQueryColumns requests the columns returned by the parsed query, using a new background context.

func (*ExtensionManagerClient) GetQueryColumnsContext 

func (c *ExtensionManagerClient) GetQueryColumnsContext(ctx context.Context, sql string) (*osquery.ExtensionResponse, error)

GetQueryColumnsContext requests the columns returned by the parsed query.

func (*ExtensionManagerClient) Options 

func (c *ExtensionManagerClient) Options() (osquery.InternalOptionList, error)

Options requests the list of bootstrap or configuration options, using a new background context.

func (*ExtensionManagerClient) OptionsContext 

func (c *ExtensionManagerClient) OptionsContext(ctx context.Context) (osquery.InternalOptionList, error)

OptionsContext requests the list of bootstrap or configuration options.

func (*ExtensionManagerClient) Ping 

func (c *ExtensionManagerClient) Ping() (*osquery.ExtensionStatus, error)

Ping requests metadata from the extension manager, using a new background context

func (*ExtensionManagerClient) PingContext 

func (c *ExtensionManagerClient) PingContext(ctx context.Context) (*osquery.ExtensionStatus, error)

PingContext requests metadata from the extension manager.

func (*ExtensionManagerClient) Query 

func (c *ExtensionManagerClient) Query(sql string) (*osquery.ExtensionResponse, error)

Query requests a query to be run and returns the extension response, using a new background context. Consider using the QueryRow or QueryRows helpers for a more friendly interface.

func (*ExtensionManagerClient) QueryContext 

func (c *ExtensionManagerClient) QueryContext(ctx context.Context, sql string) (*osquery.ExtensionResponse, error)

QueryContext requests a query to be run and returns the extension response. Consider using the QueryRow or QueryRows helpers for a more friendly interface.

func (*ExtensionManagerClient) QueryRow 

func (c *ExtensionManagerClient) QueryRow(sql string) (map[string]string, error)

QueryRow behaves similarly to QueryRows, but it returns an error if the query does not return exactly one row.

func (*ExtensionManagerClient) QueryRowContext 

func (c *ExtensionManagerClient) QueryRowContext(ctx context.Context, sql string) (map[string]string, error)

QueryRowContext behaves similarly to QueryRows, but it returns an error if the query does not return exactly one row.

func (*ExtensionManagerClient) QueryRows 

func (c *ExtensionManagerClient) QueryRows(sql string) ([]map[string]string, error)

QueryRows is a helper that executes the requested query and returns the results. It handles checking both the transport level errors and the osquery internal errors by returning a normal Go error type.

func (*ExtensionManagerClient) QueryRowsContext 

func (c *ExtensionManagerClient) QueryRowsContext(ctx context.Context, sql string) ([]map[string]string, error)

QueryRowsContext is a helper that executes the requested query and returns the results. It handles checking both the transport level errors and the osquery internal errors by returning a normal Go error type.

func (*ExtensionManagerClient) RegisterExtension 

func (c *ExtensionManagerClient) RegisterExtension(info *osquery.InternalExtensionInfo, registry osquery.ExtensionRegistry) (*osquery.ExtensionStatus, error)

RegisterExtension registers the extension plugins with the osquery process, using a new background context

func (*ExtensionManagerClient) RegisterExtensionContext 

func (c *ExtensionManagerClient) RegisterExtensionContext(ctx context.Context, info *osquery.InternalExtensionInfo, registry osquery.ExtensionRegistry) (*osquery.ExtensionStatus, error)

RegisterExtensionContext registers the extension plugins with the osquery process.

type ExtensionManagerServer 

type ExtensionManagerServer struct {
	// contains filtered or unexported fields
}

ExtensionManagerServer is an implementation of the full ExtensionManager API. Plugins can register with an extension manager, which handles the communication with the osquery process.

func NewExtensionManagerServer 

func NewExtensionManagerServer(name string, sockPath string, opts ...ServerOption) (*ExtensionManagerServer, error)

NewExtensionManagerServer creates a new extension management server communicating with osquery over the socket at the provided path. If resolving the address or connecting to the socket fails, this function will error.

func (*ExtensionManagerServer) Call 

func (s *ExtensionManagerServer) Call(ctx context.Context, registry string, item string, request osquery.ExtensionPluginRequest) (*osquery.ExtensionResponse, error)

Call routes a call from the osquery process to the appropriate registered plugin.

func (*ExtensionManagerServer) Ping 

func (s *ExtensionManagerServer) Ping(ctx context.Context) (*osquery.ExtensionStatus, error)

Ping implements the basic health check.

func (*ExtensionManagerServer) RegisterPlugin 

func (s *ExtensionManagerServer) RegisterPlugin(plugins ...OsqueryPlugin)

RegisterPlugin adds one or more OsqueryPlugins to this extension manager.

func (*ExtensionManagerServer) Run 

func (s *ExtensionManagerServer) Run() error

Run starts the extension manager and runs until osquery calls for a shutdown or the osquery instance goes away.

func (*ExtensionManagerServer) Shutdown 

func (s *ExtensionManagerServer) Shutdown(ctx context.Context) (err error)

Shutdown deregisters the extension, stops the server and closes all sockets.

func (*ExtensionManagerServer) Start 

func (s *ExtensionManagerServer) Start() error

Start registers the extension plugins and begins listening on a unix socket for requests from the osquery process. All plugins should be registered with RegisterPlugin() before calling Start().

type ExtensionsFunc 

type ExtensionsFunc func() (osquery.InternalExtensionList, error)

type GetQueryColumnsFunc 

type GetQueryColumnsFunc func(sql string) (*osquery.ExtensionResponse, error)

type MockExtensionManager 

type MockExtensionManager struct {
	CloseFunc        CloseFunc
	CloseFuncInvoked bool

	PingFunc        PingFunc
	PingFuncInvoked bool

	CallFunc        CallFunc
	CallFuncInvoked bool

	ExtensionsFunc        ExtensionsFunc
	ExtensionsFuncInvoked bool

	RegisterExtensionFunc        RegisterExtensionFunc
	RegisterExtensionFuncInvoked bool

	DeRegisterExtensionFunc        DeregisterExtensionFunc
	DeRegisterExtensionFuncInvoked bool

	OptionsFunc        OptionsFunc
	OptionsFuncInvoked bool

	QueryFunc        QueryFunc
	QueryFuncInvoked bool

	GetQueryColumnsFunc        GetQueryColumnsFunc
	GetQueryColumnsFuncInvoked bool
}

func (*MockExtensionManager) Call 

func (m *MockExtensionManager) Call(registry string, item string, req osquery.ExtensionPluginRequest) (*osquery.ExtensionResponse, error)

func (*MockExtensionManager) Close 

func (m *MockExtensionManager) Close()

func (*MockExtensionManager) DeregisterExtension 

func (m *MockExtensionManager) DeregisterExtension(uuid osquery.ExtensionRouteUUID) (*osquery.ExtensionStatus, error)

func (*MockExtensionManager) Extensions 

func (m *MockExtensionManager) Extensions() (osquery.InternalExtensionList, error)

func (*MockExtensionManager) GetQueryColumns 

func (m *MockExtensionManager) GetQueryColumns(sql string) (*osquery.ExtensionResponse, error)

func (*MockExtensionManager) Options 

func (m *MockExtensionManager) Options() (osquery.InternalOptionList, error)

func (*MockExtensionManager) Ping 

func (m *MockExtensionManager) Ping() (*osquery.ExtensionStatus, error)

func (*MockExtensionManager) Query 

func (m *MockExtensionManager) Query(sql string) (*osquery.ExtensionResponse, error)

func (*MockExtensionManager) RegisterExtension 

func (m *MockExtensionManager) RegisterExtension(info *osquery.InternalExtensionInfo, registry osquery.ExtensionRegistry) (*osquery.ExtensionStatus, error)

type OptionsFunc 

type OptionsFunc func() (osquery.InternalOptionList, error)

type OsqueryPlugin 

type OsqueryPlugin interface {
	// Name is the name used to refer to the plugin (eg. the name of the
	// table the plugin implements).
	Name() string
	// RegistryName is which "registry" the plugin should be added to.
	// Valid names are ["config", "logger", "table"].
	RegistryName() string
	// Routes returns the detailed information about the interface exposed
	// by the plugin. See the example plugins for samples.
	Routes() osquery.ExtensionPluginResponse
	// Ping implements a health check for the plugin. If the plugin is in a
	// healthy state, StatusOK should be returned.
	Ping() osquery.ExtensionStatus
	// Call requests the plugin to perform its defined behavior, returning
	// a response containing the result.
	Call(context.Context, osquery.ExtensionPluginRequest) osquery.ExtensionResponse
	// Shutdown alerts the plugin to stop.
	Shutdown()
}

type PingFunc 

type PingFunc func() (*osquery.ExtensionStatus, error)

type QueryFunc 

type QueryFunc func(sql string) (*osquery.ExtensionResponse, error)

type RegisterExtensionFunc 

type RegisterExtensionFunc func(info *osquery.InternalExtensionInfo, registry osquery.ExtensionRegistry) (*osquery.ExtensionStatus, error)

type ServerOption 

type ServerOption func(*ExtensionManagerServer)

func ExtensionVersion 

func ExtensionVersion(version string) ServerOption

func ServerConnectivityCheckInterval 

func ServerConnectivityCheckInterval(interval time.Duration) ServerOption

ServerSideConnectivityCheckInterval Sets a thrift package variable for the ticker interval used by connectivity check in thrift compiled TProcessorFunc implementations. See the thrift docs for more information

func ServerPingInterval 

func ServerPingInterval(interval time.Duration) ServerOption

func ServerTimeout 

func ServerTimeout(timeout time.Duration) ServerOption

func WithClient 

func WithClient(client ExtensionManager) ServerOption

WithClient sets the server to use an existing ExtensionManagerClient instead of creating a new one.

Source Files

View all Source files

Directories

Path Synopsis
examples
call
config
distributed
logger
query
table
gen
osquery
plugin
config
Package config creates an osquery configuration plugin.
 Package config creates an osquery configuration plugin.
distributed
Package distributed creates an osquery distributed query plugin.
 Package distributed creates an osquery distributed query plugin.
logger
Package logger creates an osquery logging plugin.
 Package logger creates an osquery logging plugin.
table
Package table creates an osquery table plugin.
 Package table creates an osquery table plugin.
Package traces allows for instrumenting osquery-go with OpenTelemetry traces.
 Package traces allows for instrumenting osquery-go with OpenTelemetry traces.
Package transport provides Thrift TTransport and TServerTransport implementations for use on mac/linux (TSocket/TServerSocket) and Windows (custom named pipe implementation).
 Package transport provides Thrift TTransport and TServerTransport implementations for use on mac/linux (TSocket/TServerSocket) and Windows (custom named pipe implementation).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.