kusto

Documentation

Overview

Package kusto provides a client for accessing Azure Data Explorer, also known as Kusto.

For details on the Azure Kusto service, see: https://azure.microsoft.com/en-us/services/data-explorer/

For general documentation on APIs and the Kusto query language, see: https://docs.microsoft.com/en-us/azure/data-explorer/

## Examples

Examples for various scenarios can be found on [pkg.go.dev](https://pkg.go.dev/github.com/Azure/azure-kusto-go#readme-examples) or in the example*_test.go files in our GitHub repo for [azure-kusto-go](https://github.com/Azure/azure-kusto-go/tree/master/kusto).

### Create the connection string

Azure Data Explorer (Kusto) connection strings are created using a connection string builder for an existing Azure Data Explorer (Kusto) cluster endpoint of the form `https://<cluster name>.<location>.kusto.windows.net`.

```go kustoConnectionStringBuilder := kusto.NewConnectionStringBuilder(endpoint) ```

### Create and authenticate the client

Azure Data Explorer (Kusto) clients are created from a connection string and authenticated using a credential from the [Azure Identity package][azure_identity_pkg], like [DefaultAzureCredential][default_azure_credential]. You can also authenticate a client using a system- or user-assigned managed identity with Azure Active Directory (AAD) credentials.

#### Using the `DefaultAzureCredential`

```go // kusto package is: github.com/Azure/azure-kusto-go/kusto

// Initialize a new kusto client using the default Azure credential kustoConnectionString := kustoConnectionStringBuilder.WithDefaultAzureCredential() client, err = kusto.New(kustoConnectionString)

if err != nil {
	panic("add error handling")
}

// Be sure to close the client when you're done. (Error handling omitted for brevity.) defer client.Close() ```

#### Using the `az cli`

```go kustoConnectionString := kustoConnectionStringBuilder.WithAzCli() client, err = kusto.New(kustoConnectionString) ```

#### Using a system-assigned managed identity

```go kustoConnectionString := kustoConnectionStringBuilder.WithSystemManagedIdentity() client, err = kusto.New(kustoConnectionString) ```

#### Using a user-assigned managed identity

```go kustoConnectionString := kustoConnectionStringBuilder.WithUserManagedIdentity(clientID) client, err = kusto.New(kustoConnectionString) ```

#### Using a bearer token

```go kustoConnectionString := kustoConnectionStringBuilder.WithApplicationToken(appId, token) client, err = kusto.New(kustoConnectionString) ```

#### Using an app id and secret

```go kustoConnectionString := kustoConnectionStringBuilder.WithAadAppKey(clientID, clientSecret, tenantID) client, err = kusto.New(kustoConnectionString) ```

#### Using an application certificate

```go kustoConnectionString := kustoConnectionStringBuilder.WithAppCertificate(appId, certificate, thumbprint, sendCertChain, authorityID) client, err = kusto.New(kustoConnectionString) ```

### Querying

#### Simple queries

* Works for queries and management commands. * Limited to queries that can be built using a string literal known at compile time.

The simplest queries can be built using `kql.New`:

```go query := kql.New("systemNodes | project CollectionTime, NodeId") ```

Queries can only be built using a string literals known at compile time, and special methods for specific parts of the query. The reason for this is to discourage the use of string concatenation to build queries, which can lead to security vulnerabilities.

#### Queries with parameters

* Can re-use the same query with different parameters. * Only work for queries, management commands are not supported.

It is recommended to use parameters for queries that contain user input. Management commands can not use parameters, and therefore should be built using the builder (see next section).

Parameters can be implicitly referenced in a query:

```go query := kql.New("systemNodes | project CollectionTime, NodeId | where CollectionTime > startTime and NodeId == nodeIdValue") ```

Here, `startTime` and `nodeIdValue` are parameters that can be passed to the query.

To Pass the parameters values to the query, create `kql.Parameters`:

``` params := kql.NewParameters().AddDateTime("startTime", dt).AddInt("nodeIdValue", 1) ```

And then pass it to the `Query` method, as an option: ```go results, err := client.Query(ctx, database, query, QueryParameters(params))

if err != nil {
    panic("add error handling")
}

// You can see the generated parameters using the ToDeclarationString() method: fmt.Println(params.ToDeclarationString()) // declare query_parameters(startTime:datetime, nodeIdValue:int); ```

#### Queries with inline parameters * Works for queries and management commands. * More involved building of queries, but allows for more flexibility.

Queries with runtime data can be built using `kql.New`. The builder will only accept the correct types for each part of the query, and will escape any special characters in the data.

For example, here is a query that dynamically accepts values for the table name, and the comparison parameters for the columns:

```go dt, _ := time.Parse(time.RFC3339Nano, "2020-03-04T14:05:01.3109965Z") tableName := "system nodes" value := 1

query := kql.New("")

.AddTable(tableName)
.AddLiteral(" | where CollectionTime == ").AddDateTime(dt)
.AddLiteral(" and ")
.AddLiteral("NodeId == ").AddInt(value)

// To view the query string, use the String() method: fmt.Println(query.String()) // Output: ['system nodes'] | where CollectionTime == datetime(2020-03-04T14:05:01.3109965Z) and NodeId == int(1) ```

Building queries like this is useful for queries that are built from user input, or for queries that are built from a template, and are valid for management commands too.

#### Query For Rows

The kusto `table` package queries data into a ***table.Row** which can be printed or have the column data extracted.

```go // table package is: github.com/Azure/azure-kusto-go/kusto/data/table

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds. iter, err := client.Query(ctx, "database", query)

if err != nil {
	panic("add error handling")
}

defer iter.Stop()

// .Do() will call the function for every row in the table. err = iter.DoOnRowOrError(

    func(row *table.Row, e *kustoErrors.Error) error {
        if e != nil {
            return e
        }
        if row.Replace {
            fmt.Println("---") // Replace flag indicates that the query result should be cleared and replaced with this row
        }
        fmt.Println(row) // As a convenience, printing a *table.Row will output csv
        return nil
	},

)

if err != nil {
	panic("add error handling")
}

```

#### Query Into Structs

Users will often want to turn the returned data into Go structs that are easier to work with. The ***table.Row** object that is returned supports this via the `.ToStruct()` method.

```go // NodeRec represents our Kusto data that will be returned.

type NodeRec struct {
	// ID is the table's NodeId. We use the field tag here to instruct our client to convert NodeId to ID.
	ID int64 `kusto:"NodeId"`
	// CollectionTime is Go representation of the Kusto datetime type.
	CollectionTime time.Time
}

iter, err := client.Query(ctx, "database", query)

if err != nil {
	panic("add error handling")
}

defer iter.Stop()

recs := []NodeRec{} err = iter.DoOnRowOrError(

    func(row *table.Row, e *kustoErrors.Error) error {
        if e != nil {
        return e
        }
		rec := NodeRec{}
		if err := row.ToStruct(&rec); err != nil {
			return err
		}
		if row.Replace {
			recs = recs[:0] // Replace flag indicates that the query result should be cleared and replaced with this row
		}
		recs = append(recs, rec)
		return nil
	},

)

if err != nil {
	panic("add error handling")
}

```

### Ingestion

The `ingest` package provides access to Kusto's ingestion service for importing data into Kusto. This requires some prerequisite knowledge of acceptable data formats, mapping references, etc.

That documentation can be found [here](https://docs.microsoft.com/en-us/azure/kusto/management/data-ingestion/)

If ingesting data from memory, it is suggested that you stream the data in via `FromReader()` passing in the reader from an `io.Pipe()`. The data will not begin ingestion until the writer closes.

#### Creating a queued ingestion client

Setup is quite simple, simply pass a `*kusto.Client`, the name of the database and table you wish to ingest into.

```go in, err := ingest.New(kustoClient, "database", "table")

if err != nil {
	panic("add error handling")
}

// Be sure to close the ingestor when you're done. (Error handling omitted for brevity.) defer in.Close() ```

#### Other Ingestion Clients

There are other ingestion clients that can be used for different ingestion scenarios. The `ingest` package provides the following clients:

• Queued Ingest - `ingest.New()` - the default client, uses queues and batching to ingest data. Most reliable.
• Streaming Ingest - `ingest.NewStreaming()` - Directly streams data into the engine. Fast, but is limited with size and can fail.
• Managed Streaming Ingest - `ingest.NewManaged()` - Combines a streaming ingest client with a queued ingest client to provide a reliable ingestion method that is fast and can ingest large amounts of data. Managed Streaming will try to stream the data, and if it fails multiple times, it will fall back to a queued ingestion.

#### Ingestion From a File

Ingesting a local file requires simply passing the path to the file to be ingested:

```go

if _, err := in.FromFile(ctx, "/path/to/a/local/file"); err != nil {
	panic("add error handling")
}

```

`FromFile()` will accept Unix path names on Unix platforms and Windows path names on Windows platforms. The file will not be deleted after upload (there is an option that will allow that though).

#### From a Blob Storage File

This package will also accept ingestion from an Azure Blob Storage file:

```go

if _, err := in.FromFile(ctx, "https://myaccount.blob.core.windows.net/$root/myblob"); err != nil {
	panic("add error handling")
}

```

This will ingest a file from Azure Blob Storage. We only support `https://` paths and your domain name may differ than what is here.

#### Ingestion from an io.Reader

Sometimes you want to ingest a stream of data that you have in memory without writing to disk. You can do this simply by chunking the data via an `io.Reader`.

```go r, w := io.Pipe()

enc := json.NewEncoder(w)

go func() {
	defer w.Close()
	for _, data := range dataSet {
		if err := enc.Encode(data); err != nil {
			panic("add error handling")
		}
	}
}()

if _, err := in.FromReader(ctx, r); err != nil {
	panic("add error handling")
}

```

It is important to remember that `FromReader()` will terminate when it receives an `io.EOF` from the `io.Reader`. Use `io.Readers` that won't return `io.EOF` until the `io.Writer` is closed (such as `io.Pipe`).

Mocking

To support mocking for this client in your code for hermetic testing purposes, this client supports mocking the data returned by our RowIterator object. Please see the MockRows documentation for code examples.

Package Examples

Below you will find a simple and complex example of doing Query() the represent compiled code:

Example (Complex)

// This example sets up a Query where we want to query for nodes that have a NodeId (a Kusto Long type) that has a
// particular NodeId. The will require inserting a value where ParamNodeId is in the query.
// We will used a parameterized query to do this.
query := kql.New("systemNodes | project CollectionTime, NodeId | where NodeId == ParamNodeId")
params := kql.NewParameters()

// NodeRec represents our Kusto data that will be returned.
type NodeRec struct {
	// ID is the table's NodeId. We use the field tag here to instruct our client to convert NodeId in the Kusto
	// table to ID in our struct.
	ID int64 `kusto:"NodeId"`
	// CollectionTime is Go representation of the Kusto datetime type.
	CollectionTime time.Time
}

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}
// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for our specific node. We are only doing a single query here as an example,
// normally you would take in requests of some type for different NodeIds.
iter, err := client.Query(ctx, "database", query, QueryParameters(params))
if err != nil {
	panic("add error handling")
}
defer iter.Stop()

rec := NodeRec{} // We are assuming unique NodeId, so we will only get 1 row.
err = iter.DoOnRowOrError(
	func(row *table.Row, e *kustoErrors.Error) error {
		if e != nil {
			return e
		}
		return row.ToStruct(&rec)
	},
)

if err != nil {
	panic("add error handling")
}

fmt.Println(rec.ID)

Example (Simple)

// Query and capture the values and put them in a slice of structs representing the row.

// NodeRec represents our Kusto data that will be returned.
type NodeRec struct {
	// ID is the table's NodeId. We use the field tag here to instruct our client to convert NodeId to ID.
	ID int64 `kusto:"NodeId"`
	// CollectionTime is Go representation of the Kusto datetime type.
	CollectionTime time.Time
}

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")
client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}

// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds.
iter, err := client.Query(ctx, "database", kql.New("systemNodes | project CollectionTime, NodeId"))
if err != nil {
	panic("add error handling")
}
defer iter.Stop()

var recs []NodeRec

err = iter.DoOnRowOrError(
	func(row *table.Row, e *kustoErrors.Error) error {
		if e != nil {
			return e
		}
		rec := NodeRec{}
		if err := row.ToStruct(&rec); err != nil {
			return err
		}
		if row.Replace {
			recs = recs[:0]
		}
		recs = append(recs, rec)
		return nil
	},
)

if err != nil {
	panic("add error handling")
}

for _, rec := range recs {
	fmt.Println(rec.ID)
}

Index

• Constants
• func CalculateTimeout(ctx context.Context, opt *queryOptions, queryType int)
• type Authorization
• type Client
  • func New(kcsb *ConnectionStringBuilder, options ...Option) (*Client, error)
  • func NewMockClient() *Client
  • func (c *Client) Auth() Authorization
  • func (c *Client) ClientDetails() *ClientDetails
  • func (c *Client) Close() error
  • func (c *Client) Endpoint() string
  • func (c *Client) HttpClient() *http.Client
  • func (c *Client) Mgmt(ctx context.Context, db string, query Statement, options ...QueryOption) (*RowIterator, error)
  • func (c *Client) Query(ctx context.Context, db string, query Statement, options ...QueryOption) (*RowIterator, error)
  • func (c *Client) QueryToJson(ctx context.Context, db string, query Statement, options ...QueryOption) (string, error)
• type ClientDetails
  • func NewClientDetails(applicationForTracing string, userNameForTracing string) *ClientDetails
  • func (c *ClientDetails) ApplicationForTracing() string
  • func (c *ClientDetails) ClientVersionForTracing() string
  • func (c *ClientDetails) UserNameForTracing() string
• type CloudInfo
  • func GetMetadata(kustoUri string, httpClient *http.Client) (CloudInfo, error)
• type Conn
  • func NewConn(endpoint string, auth Authorization, client *http.Client, ...) (*Conn, error)
  • func (c *Conn) Close() error
  • func (c *Conn) StreamIngest(ctx context.Context, db, table string, payload io.Reader, ...) error
• type ConnectionStringBuilder
  • func NewConnectionStringBuilder(connStr string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) AttachPolicyClientOptions(options *azcore.ClientOptions) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) SetConnectorDetails(name, version, appName, appVersion string, sendUser bool, overrideUser string, ...)
  • func (kcsb *ConnectionStringBuilder) WitAadUserToken(usertoken string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithAadAppKey(appId string, appKey string, authorityID string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithAadUserPassAuth(uname string, pswrd string, authorityID string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithAppCertificate(appId string, certificate string, thumprint string, sendCertChain bool, ...) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithApplicationToken(appId string, appToken string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithAzCli() *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithDefaultAzureCredential() *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithInteractiveLogin(authorityID string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithKubernetesWorkloadIdentity(appId, tokenFilePath, authorityID string) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithSystemManagedIdentity() *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithTokenCredential(tokenCredential azcore.TokenCredential) *ConnectionStringBuilder
  • func (kcsb *ConnectionStringBuilder) WithUserManagedIdentity(clientID string) *ConnectionStringBuilder
• type DataFormatForStreaming
• type DataScope
• type Definitions
  • func NewDefinitions() Definitions
  • func (p Definitions) IsZero() bool
  • func (p Definitions) Must(types ParamTypes) Definitions
  • func (p Definitions) String() string
  • func (p Definitions) With(types ParamTypes) (Definitions, error)
• type MgmtOption
• type MockRows
  • func NewMockRows(columns table.Columns) (*MockRows, error)
  • func (m *MockRows) Error(err error) error
  • func (m *MockRows) Row(row value.Values) error
  • func (m *MockRows) Struct(p interface{}) error
• type Option
  • func WithHttpClient(client *http.Client) Option
• type ParamType
• type ParamTypes
• type Parameters
  • func NewParameters() Parameters
  • func (q Parameters) IsZero() bool
  • func (q Parameters) Must(values QueryValues) Parameters
  • func (q Parameters) With(values QueryValues) (Parameters, error)
• type QueryOption
  • func Application(appName string) QueryOption
  • func ClientMaxRedirectCount(i int64) QueryOption
  • func ClientRequestID(clientRequestID string) QueryOption
  • func CustomQueryOption(paramName string, i interface{}) QueryOption
  • func DeferPartialQueryFailures() QueryOption
  • func IngestionEndpoint() QueryOption
  • func MaterializedViewShuffle(s string) QueryOption
  • func MaxMemoryConsumptionPerIterator(i uint64) QueryOption
  • func MaxMemoryConsumptionPerQueryPerNode(i uint64) QueryOption
  • func MaxOutputColumns(i int) QueryOption
  • func NoRequestTimeout() QueryOption
  • func NoTruncation() QueryOption
  • func PushSelectionThroughAggregation() QueryOption
  • func QueryBinAutoAt(s string) QueryOption
  • func QueryBinAutoSize(s string) QueryOption
  • func QueryConsistency(c string) QueryOption
  • func QueryCursorAfterDefault(s string) QueryOption
  • func QueryCursorBeforeOrAtDefault(s string) QueryOption
  • func QueryCursorCurrent(s string) QueryOption
  • func QueryCursorDisabled(s string) QueryOption
  • func QueryCursorScopedTables(l []string) QueryOption
  • func QueryDataScope(ds DataScope) QueryOption
  • func QueryDateTimeScopeColumn(s string) QueryOption
  • func QueryDateTimeScopeFrom(t time.Time) QueryOption
  • func QueryDateTimeScopeTo(t time.Time) QueryOption
  • func QueryDistributionNodesSpan(i int64) QueryOption
  • func QueryFanoutNodesPercent(i int) QueryOption
  • func QueryFanoutThreadsPercent(i int) QueryOption
  • func QueryForceRowLevelSecurity() QueryOption
  • func QueryLanguage(s string) QueryOption
  • func QueryLogQueryParameters() QueryOption
  • func QueryMaxEntitiesInUnion(i int64) QueryOption
  • func QueryNow(t time.Time) QueryOption
  • func QueryParameters(queryParameters *kql.Parameters) QueryOption
  • func QueryPythonDebug(i int) QueryOption
  • func QueryResultsApplyGetschema() QueryOption
  • func QueryResultsCacheMaxAge(d time.Duration) QueryOption
  • func QueryResultsCachePerShard() QueryOption
  • func QueryResultsProgressiveRowCount(i int64) QueryOption
  • func QueryResultsProgressiveUpdatePeriod(i int32) QueryOption
  • func QueryTakeMaxRecords(i int64) QueryOption
  • func RequestAppName(s string) QueryOption
  • func RequestBlockRowLevelSecurity() QueryOption
  • func RequestCalloutDisabled() QueryOption
  • func RequestDescription(s string) QueryOption
  • func RequestExternalTableDisabled() QueryOption
  • func RequestImpersonationDisabled() QueryOption
  • func RequestReadonly() QueryOption
  • func RequestRemoteEntitiesDisabled() QueryOption
  • func RequestSandboxedExecutionDisabled() QueryOption
  • func RequestUser(s string) QueryOption
  • func ResultsProgressiveDisable() QueryOption
  • func ServerTimeout(d time.Duration) QueryOption
  • func TruncationMaxRecords(i int64) QueryOption
  • func TruncationMaxSize(i int64) QueryOption
  • func User(userName string) QueryOption
  • func ValidatePermissions() QueryOption
• type QueryValues
• type Row
• type RowIterator
  • func (r *RowIterator) Do(f func(r *table.Row) error) error
  • func (r *RowIterator) DoOnRowOrError(f func(r *table.Row, e *errors.Error) error) error
  • func (r *RowIterator) GetExtendedProperties() (v2.DataTable, error)
  • func (r *RowIterator) GetNonPrimary(tableKind, tableName frames.TableKind) (v2.DataTable, error)
  • func (r *RowIterator) GetQueryCompletionInformation() (v2.DataTable, error)
  • func (r *RowIterator) Mock(m *MockRows) error
  • func (r *RowIterator) Next() (row *table.Row, finalError error)
  • func (r *RowIterator) NextRowOrError() (row *table.Row, inlineError *errors.Error, finalError error)
  • func (r *RowIterator) Progress() float64
  • func (r *RowIterator) Progressive() bool
  • func (r *RowIterator) Stop()
• type Statement
• type Stmt
  • func NewStmt(query stringConstant, options ...StmtOption) Stmt
  • func (s Stmt) Add(query stringConstant) Stmt
  • func (s Stmt) GetParameters() (map[string]string, error)
  • func (s Stmt) MustDefinitions(defs Definitions) Stmt
  • func (s Stmt) MustParameters(params Parameters) Stmt
  • func (s Stmt) String() string
  • func (s Stmt) SupportsInlineParameters() bool
  • func (s Stmt) UnsafeAdd(query string) Stmt
  • func (s Stmt) ValuesJSON() (string, error)
  • func (s Stmt) WithDefinitions(defs Definitions) (Stmt, error)
  • func (s Stmt) WithParameters(params Parameters) (Stmt, error)
• type StmtOption
  • func UnsafeStmt(options unsafe.Stmt) StmtOption
• type StringPair
• type TableOfContents
• type TokenProvider
  • func (tkp *TokenProvider) AcquireToken(ctx context.Context) (string, string, error)
  • func (tkp *TokenProvider) AuthorizationRequired() bool
  • func (tkp *TokenProvider) SetHttp(http *http.Client)

Examples

• Package (Complex)
• Package (Simple)
• Authorization (Config)
• Authorization (Msi)
• Client.Query (Do)
• Client.Query (Rows)
• Client.Query (Struct)
• Statement
• Stmt

Constants

const (
	// DSDefault is used to set a query's datascope to default.
	DSDefault dataScope = "default"
	// DSAll is used to set a query's datascope to all.
	DSAll dataScope = "all"
	// DSHotCache is used to set a query's datascope to hotcache.
	DSHotCache dataScope = "hotcache"
)

const ApplicationHeader = "x-ms-app"

const (
	BEARER_TYPE = "Bearer"
)

const ClientMaxRedirectCountValue = "client_max_redirect_count"

const ClientRequestIdHeader = "x-ms-client-request-id"

const ClientVersionHeader = "x-ms-client-version"

const DeferPartialQueryFailuresValue = "deferpartialqueryfailures"

const MaterializedViewShuffleValue = "materialized_view_shuffle"

const MaxMemoryConsumptionPerIteratorValue = "maxmemoryconsumptionperiterator"

const MaxMemoryConsumptionPerQueryPerNodeValue = "max_memory_consumption_per_query_per_node"

const MaxOutputColumnsValue = "maxoutputcolumns"

const NONE = "[none]"

const NoRequestTimeoutValue = "norequesttimeout"

const NoTruncationValue = "notruncation"

const PushSelectionThroughAggregationValue = "push_selection_through_aggregation"

const QueryBinAutoAtValue = "query_bin_auto_at"

const QueryBinAutoSizeValue = "query_bin_auto_size"

const QueryConsistencyValue = "queryconsistency"

const QueryCursorAfterDefaultValue = "query_cursor_after_default"

const QueryCursorBeforeOrAtDefaultValue = "query_cursor_before_or_at_default"

const QueryCursorCurrentValue = "query_cursor_current"

const QueryCursorDisabledValue = "query_cursor_disabled"

const QueryCursorScopedTablesValue = "query_cursor_scoped_tables"

const QueryDatascopeValue = "query_datascope"

const QueryDateTimeScopeColumnValue = "query_datetimescope_column"

const QueryDateTimeScopeFromValue = "query_datetimescope_from"

const QueryDateTimeScopeToValue = "query_datetimescope_to"

const QueryDistributionNodesSpanValue = "query_distribution_nodes_span"

const QueryFanoutNodesPercentValue = "query_fanout_nodes_percent"

const QueryFanoutThreadsPercentValue = "query_fanout_threads_percent"

const QueryForceRowLevelSecurityValue = "query_force_row_level_security"

const QueryLanguageValue = "query_language"

const QueryLogQueryParametersValue = "query_log_query_parameters"

const QueryMaxEntitiesInUnionValue = "query_max_entities_in_union"

const QueryNowValue = "query_now"

const QueryPythonDebugValue = "query_python_debug"

const QueryResultsApplyGetschemaValue = "query_results_apply_getschema"

const QueryResultsCacheMaxAgeValue = "query_results_cache_max_age"

const QueryResultsCachePerShardValue = "query_results_cache_per_shard"

const QueryResultsProgressiveRowCountValue = "query_results_progressive_row_count"

const QueryResultsProgressiveUpdatePeriodValue = "query_results_progressive_update_period"

const QueryTakeMaxRecordsValue = "query_take_max_records"

const RequestAppNameValue = "request_app_name"

const RequestBlockRowLevelSecurityValue = "request_block_row_level_security"

const RequestCalloutDisabledValue = "request_callout_disabled"

const RequestDescriptionValue = "request_description"

const RequestExternalTableDisabledValue = "request_external_table_disabled"

const RequestImpersonationDisabledValue = "request_impersonation_disabled"

const RequestProgressiveEnabledValue = "results_progressive_enabled"

const RequestReadonlyValue = "request_readonly"

const RequestRemoteEntitiesDisabledValue = "request_remote_entities_disabled"

const RequestSandboxedExecutionDisabledValue = "request_sandboxed_execution_disabled"

const RequestUserValue = "request_user"

const ServerTimeoutValue = "servertimeout"

const TruncationMaxRecordsValue = "truncation_max_records"

const TruncationMaxSizeValue = "truncation_max_size"

const UserHeader = "x-ms-user"

const ValidatePermissionsValue = "validate_permissions"

Functions

func CalculateTimeout

func CalculateTimeout(ctx context.Context, opt *queryOptions, queryType int)

Types

type Authorization

type Authorization struct {
	// Token provider that can be used to get the access token.
	TokenProvider *TokenProvider
}

Authorization provides the TokenProvider needed to acquire the auth token.

Example (Config)

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

// Normally here you take a client.
_, err := New(kcsb)
if err != nil {
	panic("add error handling")
}

Example (Msi)

kcsb := NewConnectionStringBuilder("endpoint").WithUserManagedIdentity("clientID")

// Normally here you take a client.
_, err := New(kcsb)
if err != nil {
	panic("add error handling")
}

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a client to a Kusto instance.

func New

func New(kcsb *ConnectionStringBuilder, options ...Option) (*Client, error)

New returns a new Client.

func NewMockClient

func NewMockClient() *Client

func (*Client) Auth

func (c *Client) Auth() Authorization

Auth returns the Authorization passed to New().

func (*Client) ClientDetails

func (c *Client) ClientDetails() *ClientDetails

func (*Client) Close

func (c *Client) Close() error

func (*Client) Endpoint

func (c *Client) Endpoint() string

Endpoint returns the endpoint passed to New().

func (*Client) HttpClient

func (c *Client) HttpClient() *http.Client

func (*Client) Mgmt

func (c *Client) Mgmt(ctx context.Context, db string, query Statement, options ...QueryOption) (*RowIterator, error)

Mgmt is used to do management queries to Kusto. Details can be found at: https://docs.microsoft.com/en-us/azure/kusto/management/ Mgmt accepts a Stmt, but that Stmt cannot have any query parameters attached at this time. Note that the server has a timeout of 10 minutes for a management call by default unless the context deadline is set. There is a maximum of 1 hour.

func (*Client) Query

func (c *Client) Query(ctx context.Context, db string, query Statement, options ...QueryOption) (*RowIterator, error)

Query queries Kusto for data. context can set a timeout or cancel the query. query is a injection safe Stmt object. Queries cannot take longer than 5 minutes by default and have row/size limitations. Note that the server has a timeout of 4 minutes for a query by default unless the context deadline is set. Queries can take a maximum of 1 hour.

Example (Do)

// This is similar to our (Row) example. In this one though, we use the RowIterator.Do() method instead of
// manually iterating over the row. This makes for shorter code while maintaining readability.

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}
// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds.
iter, err := client.Query(ctx, "database", kql.New("systemNodes | project CollectionTime, NodeId"))
if err != nil {
	panic("add error handling")
}
defer iter.Stop()

// Iterate through the returned rows until we get an error or receive an io.EOF, indicating the end of
// the data being returned.

err = iter.DoOnRowOrError(
	func(row *table.Row, e *kustoErrors.Error) error {
		if e != nil {
			return e
		}
		if row.Replace {
			fmt.Println("---") // Replace flag indicates that the query result should be cleared and replaced with this row
		}
		for _, v := range row.Values {
			fmt.Printf("%s,", v)
		}
		fmt.Println("") // Add a carriage return
		return nil
	},
)
if err != nil {
	panic("add error handling")
}

Example (Rows)

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}
// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds.
iter, err := client.Query(ctx, "database", kql.New("systemNodes | project CollectionTime, NodeId"))
if err != nil {
	panic("add error handling")
}
defer iter.Stop()

// Iterate through the returned rows until we get an error or receive an io.EOF, indicating the end of
// the data being returned.
for {
	row, inlineErr, err := iter.NextRowOrError()
	if inlineErr != nil {
		panic("add error handling")
	}
	if err != nil {
		if err == io.EOF {
			break
		}
		if err != nil {
			panic("add error handling")
		}
	}

	// Print out the row values
	for _, v := range row.Values {
		fmt.Printf("%s,", v)
	}
	fmt.Println("") // Add a carriage return
}

Example (Struct)

// Capture our values into a struct and sends those values into a channel. Normally this would be done between
// a couple of functions representing a sender and a receiver.

// NodeRec represents our Kusto data that will be returned.
type NodeRec struct {
	// ID is the table's NodeId. We use the field tag here to instruct our client to convert NodeId to ID.
	ID int64 `kusto:"NodeId"`
	// CollectionTime is Go representation of the Kusto datetime type.
	CollectionTime time.Time

	// err is used internally to signal downstream that we encounter an error.
	err error
}

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}
// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds.
iter, err := client.Query(ctx, "database", kql.New("systemNodes | project CollectionTime, NodeId"))
if err != nil {
	panic("add error handling")
}
defer iter.Stop()

// printCh is used to receive NodeRecs for printing.
printCh := make(chan NodeRec, 1)

// Iterate through the returned rows, convert them to NodeRecs and send them on printCh to be printed.
go func() {
	// Note: we ignore the error here because we send it on a channel and an error will automatically
	// end the iteration.
	_ = iter.DoOnRowOrError(
		func(row *table.Row, e *kustoErrors.Error) error {
			if e != nil {
				return e
			}
			rec := NodeRec{}
			rec.err = row.ToStruct(&rec)
			printCh <- rec
			return rec.err
		},
	)
}()

// Receive the NodeRecs on printCh and print them to the screen.
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
	defer wg.Done()
	for rec := range printCh {
		if rec.err != nil {
			fmt.Println("Got error: ", err)
			return
		}
		fmt.Printf("NodeID: %d, CollectionTime: %s\n", rec.ID, rec.CollectionTime)
	}
}()

wg.Wait()

func (*Client) QueryToJson

func (c *Client) QueryToJson(ctx context.Context, db string, query Statement, options ...QueryOption) (string, error)

type ClientDetails

type ClientDetails struct {
	// contains filtered or unexported fields
}

func NewClientDetails

func NewClientDetails(applicationForTracing string, userNameForTracing string) *ClientDetails

func (*ClientDetails) ApplicationForTracing

func (c *ClientDetails) ApplicationForTracing() string

func (*ClientDetails) ClientVersionForTracing

func (c *ClientDetails) ClientVersionForTracing() string

func (*ClientDetails) UserNameForTracing

func (c *ClientDetails) UserNameForTracing() string

type CloudInfo

type CloudInfo struct {
	LoginEndpoint          string `json:"LoginEndpoint"`
	LoginMfaRequired       bool   `json:"LoginMfaRequired"`
	KustoClientAppID       string `json:"KustoClientAppId"`
	KustoClientRedirectURI string `json:"KustoClientRedirectUri"`
	KustoServiceResourceID string `json:"KustoServiceResourceId"`
	FirstPartyAuthorityURL string `json:"FirstPartyAuthorityUrl"`
}

func GetMetadata

func GetMetadata(kustoUri string, httpClient *http.Client) (CloudInfo, error)

type Conn

type Conn struct {
	// contains filtered or unexported fields
}

Conn provides connectivity to a Kusto instance.

func NewConn

func NewConn(endpoint string, auth Authorization, client *http.Client, clientDetails *ClientDetails) (*Conn, error)

NewConn returns a new Conn object with an injected http.Client

func (*Conn) Close

func (c *Conn) Close() error

func (*Conn) StreamIngest

func (c *Conn) StreamIngest(ctx context.Context, db, table string, payload io.Reader, format DataFormatForStreaming, mappingName string, clientRequestId string, isBlobUri bool) error

type ConnectionStringBuilder

type ConnectionStringBuilder struct {
	DataSource                       string
	AadUserID                        string
	Password                         string
	UserToken                        string
	ApplicationClientId              string
	ApplicationKey                   string
	AuthorityId                      string
	ApplicationCertificate           string
	ApplicationCertificateThumbprint string
	SendCertificateChain             bool
	ApplicationToken                 string
	AzCli                            bool
	MsiAuthentication                bool
	WorkloadAuthentication           bool
	FederationTokenFilePath          string
	ManagedServiceIdentity           string
	InteractiveLogin                 bool
	RedirectURL                      string
	DefaultAuth                      bool
	ClientOptions                    *azcore.ClientOptions
	ApplicationForTracing            string
	UserForTracing                   string
	TokenCredential                  azcore.TokenCredential
}

func NewConnectionStringBuilder

func NewConnectionStringBuilder(connStr string) *ConnectionStringBuilder

NewConnectionStringBuilder Creates new Kusto ConnectionStringBuilder. Params takes kusto connection string connStr: string. Kusto connection string should be of the format: https://<clusterName>.<location>.kusto.windows.net;AAD User ID="user@microsoft.com";Password=P@ssWord For more information please look at: https://docs.microsoft.com/azure/data-explorer/kusto/api/connection-strings/kusto

func (*ConnectionStringBuilder) AttachPolicyClientOptions

func (kcsb *ConnectionStringBuilder) AttachPolicyClientOptions(options *azcore.ClientOptions) *ConnectionStringBuilder

AttachPolicyClientOptions Assigns ClientOptions to string builder that contains configuration settings like Logging and Retry configs for a client's pipeline. Read more at https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azcore@v1.2.0/policy#ClientOptions

func (*ConnectionStringBuilder) SetConnectorDetails

func (kcsb *ConnectionStringBuilder) SetConnectorDetails(name, version, appName, appVersion string, sendUser bool, overrideUser string, additionalFields ...StringPair)

func (*ConnectionStringBuilder) WitAadUserToken

func (kcsb *ConnectionStringBuilder) WitAadUserToken(usertoken string) *ConnectionStringBuilder

WitAadUserToken Creates a Kusto Connection string builder that will authenticate with AAD user token

func (*ConnectionStringBuilder) WithAadAppKey

func (kcsb *ConnectionStringBuilder) WithAadAppKey(appId string, appKey string, authorityID string) *ConnectionStringBuilder

WithAadAppKey Creates a Kusto Connection string builder that will authenticate with AAD application and key.

func (*ConnectionStringBuilder) WithAadUserPassAuth

func (kcsb *ConnectionStringBuilder) WithAadUserPassAuth(uname string, pswrd string, authorityID string) *ConnectionStringBuilder

WithAadUserPassAuth Creates a Kusto Connection string builder that will authenticate with AAD user name and password.

func (*ConnectionStringBuilder) WithAppCertificate

func (kcsb *ConnectionStringBuilder) WithAppCertificate(appId string, certificate string, thumprint string, sendCertChain bool, authorityID string) *ConnectionStringBuilder

WithAppCertificate Creates a Kusto Connection string builder that will authenticate with AAD application using a certificate.

func (*ConnectionStringBuilder) WithApplicationToken

func (kcsb *ConnectionStringBuilder) WithApplicationToken(appId string, appToken string) *ConnectionStringBuilder

WithApplicationToken Creates a Kusto Connection string builder that will authenticate with AAD application and an application token.

func (*ConnectionStringBuilder) WithAzCli

func (kcsb *ConnectionStringBuilder) WithAzCli() *ConnectionStringBuilder

WithAzCli Creates a Kusto Connection string builder that will use existing authenticated az cli profile password.

func (*ConnectionStringBuilder) WithDefaultAzureCredential

func (kcsb *ConnectionStringBuilder) WithDefaultAzureCredential() *ConnectionStringBuilder

WithDefaultAzureCredential Create Kusto Conntection String that will be used for default auth mode. The order of auth will be via environment variables, managed identity and Azure CLI . Read more at https://learn.microsoft.com/azure/developer/go/azure-sdk-authentication?tabs=bash#2-authenticate-with-azure

func (*ConnectionStringBuilder) WithInteractiveLogin

func (kcsb *ConnectionStringBuilder) WithInteractiveLogin(authorityID string) *ConnectionStringBuilder

WithInteractiveLogin Creates a Kusto Connection string builder that will authenticate by launching the system default browser to interactively authenticate a user, and obtain an access token

func (*ConnectionStringBuilder) WithKubernetesWorkloadIdentity

func (kcsb *ConnectionStringBuilder) WithKubernetesWorkloadIdentity(appId, tokenFilePath, authorityID string) *ConnectionStringBuilder

WithKubernetesWorkloadIdentity Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using Kubernetes workload identity.

func (*ConnectionStringBuilder) WithSystemManagedIdentity

func (kcsb *ConnectionStringBuilder) WithSystemManagedIdentity() *ConnectionStringBuilder

WithSystemManagedIdentity Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using system assigned id.

func (*ConnectionStringBuilder) WithTokenCredential

func (kcsb *ConnectionStringBuilder) WithTokenCredential(tokenCredential azcore.TokenCredential) *ConnectionStringBuilder

func (*ConnectionStringBuilder) WithUserManagedIdentity

func (kcsb *ConnectionStringBuilder) WithUserManagedIdentity(clientID string) *ConnectionStringBuilder

WithUserManagedIdentity Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using user assigned id.

type DataFormatForStreaming

type DataFormatForStreaming interface {
	CamelCase() string
	KnownOrDefault() DataFormatForStreaming
}

type DataScope

type DataScope interface {
	// contains filtered or unexported methods
}

DataScope is used with QueryDataScope() to control a query's datascope.

type Definitions

type Definitions struct {
	// contains filtered or unexported fields
}

Definitions represents definitions of parameters that are substituted for variables in a Kusto Query. This provides both variable substitution in a Stmt and provides protection against SQL-like injection attacks. See https://docs.microsoft.com/en-us/azure/kusto/query/queryparametersstatement?pivots=azuredataexplorer for internals. This object is not thread-safe and passing it as an argument to a function will create a copy that will share the internal state with the original.

func NewDefinitions

func NewDefinitions() Definitions

NewDefinitions is the constructor for Definitions.

func (Definitions) IsZero

func (p Definitions) IsZero() bool

IsZero indicates if the Definitions object is the zero type.

func (Definitions) Must

func (p Definitions) Must(types ParamTypes) Definitions

Must is the same as With(), but it must succeed or it panics.

func (Definitions) String

func (p Definitions) String() string

String implements fmt.Stringer.

func (Definitions) With

func (p Definitions) With(types ParamTypes) (Definitions, error)

With returns a copy of the Definitions object with the parameters names and types defined in "types".

type MgmtOption

type MgmtOption = QueryOption

Deprecated: MgmtOption will be removed in a future release. Use QueryOption instead.

type MockRows

type MockRows struct {
	// contains filtered or unexported fields
}

MockRows provides the abilty to provide mocked Row data that can be played back from a RowIterator. This allows for creating hermetic tests from mock data or creating mock data from a real data fetch.

func NewMockRows

func NewMockRows(columns table.Columns) (*MockRows, error)

NewMockRows is the constructor for MockRows.

func (*MockRows) Error

func (m *MockRows) Error(err error) error

Error adds an error into the result stream. Nothing else added to this stream will matter once this is called.

func (*MockRows) Row

func (m *MockRows) Row(row value.Values) error

Row adds Row data that will be replayed in a RowIterator.

func (*MockRows) Struct

func (m *MockRows) Struct(p interface{}) error

Struct adds Row data that will be replayed in a RowIterator by parsing the passed *struct into value.Values.

type Option

type Option func(c *Client)

Option is an optional argument type for New().

func WithHttpClient

func WithHttpClient(client *http.Client) Option

type ParamType

type ParamType struct {
	// Type is the type of Column type this QueryParam will represent.
	Type types.Column
	// Default is a default value to use if the query doesn't provide this value.
	// The value that can be set is defined by the Type:
	// CTBool must be a bool
	// CTDateTime must be a time.Time
	// CTDynamic cannot have a default value
	// CTGuid must be an uuid.UUID
	// CTInt must be an int32
	// CTLong must be an int64
	// CTReal must be an float64
	// CTString must be a string
	// CTTimespan must be a time.Duration
	// CTDecimal must be a string or *big.Float representing a decimal value
	Default interface{}
	// contains filtered or unexported fields
}

ParamType provides type and default value information about the query parameter

type ParamTypes

type ParamTypes map[string]ParamType

ParamTypes is a list of parameter types and corresponding type data.

type Parameters

type Parameters struct {
	// contains filtered or unexported fields
}

Parameters represents values that will be substituted for a Stmt's Parameter. Keys are the names of corresponding Parameters, values are the value to be used. Keys must exist in the Parameter and value must be a Go type that corresponds to the ParamType.

func NewParameters

func NewParameters() Parameters

NewParameters is the construtor for Parameters.

func (Parameters) IsZero

func (q Parameters) IsZero() bool

IsZero returns if Parameters is the zero value.

func (Parameters) Must

func (q Parameters) Must(values QueryValues) Parameters

Must is the same as With() except any error is a panic.

func (Parameters) With

func (q Parameters) With(values QueryValues) (Parameters, error)

With returns a Parameters set to "values". values' keys represents Definitions names that will substituted for and the values to be subsituted.

type QueryOption

type QueryOption func(q *queryOptions) error

QueryOption is an option type for a call to Query().

func Application

func Application(appName string) QueryOption

Application sets the x-ms-app header, and can be used to identify the application making the request in the `.show queries` output.

func ClientMaxRedirectCount

func ClientMaxRedirectCount(i int64) QueryOption

ClientMaxRedirectCount If set and positive, indicates the maximum number of HTTP redirects that the client will process.

func ClientRequestID

func ClientRequestID(clientRequestID string) QueryOption

ClientRequestID sets the x-ms-client-request-id header, and can be used to identify the request in the `.show queries` output.

func CustomQueryOption

func CustomQueryOption(paramName string, i interface{}) QueryOption

CustomQueryOption exists to allow a QueryOption that is not defined in the Go SDK, as all options are not defined. Please Note: you should always use the type safe options provided below when available. Also note that Kusto does not error on non-existent parameter names or bad values, it simply doesn't work as expected.

func DeferPartialQueryFailures

func DeferPartialQueryFailures() QueryOption

DeferPartialQueryFailures disables reporting partial query failures as part of the result set.

func IngestionEndpoint

func IngestionEndpoint() QueryOption

IngestionEndpoint will instruct the Mgmt call to connect to the ingest-[endpoint] instead of [endpoint]. This is not often used by end users and can only be used with a Mgmt() call.

func MaterializedViewShuffle

func MaterializedViewShuffle(s string) QueryOption

MaterializedViewShuffle A hint to use shuffle strategy for materialized views that are referenced in the query. The property is an array of materialized views names and the shuffle keys to use. Examples: 'dynamic([ { "Name": "V1", "Keys" : [ "K1", "K2" ] } ])' (shuffle view V1 by K1, K2) or 'dynamic([ { "Name": "V1" } ])' (shuffle view V1 by all keys)

func MaxMemoryConsumptionPerIterator

func MaxMemoryConsumptionPerIterator(i uint64) QueryOption

MaxMemoryConsumptionPerIterator overrides the default maximum amount of memory a query operator may allocate.

func MaxMemoryConsumptionPerQueryPerNode

func MaxMemoryConsumptionPerQueryPerNode(i uint64) QueryOption

MaxMemoryConsumptionPerQueryPerNode overrides the default maximum amount of memory a whole query may allocate per node.

func MaxOutputColumns

func MaxOutputColumns(i int) QueryOption

MaxOutputColumns overrides the default maximum number of columns a query is allowed to produce.

func NoRequestTimeout

func NoRequestTimeout() QueryOption

NoRequestTimeout enables setting the request timeout to its maximum value.

func NoTruncation

func NoTruncation() QueryOption

NoTruncation enables suppressing truncation of the query results returned to the caller.

func PushSelectionThroughAggregation

func PushSelectionThroughAggregation() QueryOption

PushSelectionThroughAggregation will push simple selection through aggregation .

func QueryBinAutoAt

func QueryBinAutoAt(s string) QueryOption

QueryBinAutoAt When evaluating the bin_auto() function, the start value to use.

func QueryBinAutoSize

func QueryBinAutoSize(s string) QueryOption

QueryBinAutoSize When evaluating the bin_auto() function, the bin size value to use.

func QueryConsistency

func QueryConsistency(c string) QueryOption

QueryConsistency Controls query consistency

func QueryCursorAfterDefault

func QueryCursorAfterDefault(s string) QueryOption

QueryCursorAfterDefault sets the default parameter value of the cursor_after() function when called without parameters.

func QueryCursorBeforeOrAtDefault

func QueryCursorBeforeOrAtDefault(s string) QueryOption

QueryCursorBeforeOrAtDefault sets the default parameter value of the cursor_before_or_at() function when called without parameters.

func QueryCursorCurrent

func QueryCursorCurrent(s string) QueryOption

QueryCursorCurrent overrides the cursor value returned by the cursor_current() or current_cursor() functions.

func QueryCursorDisabled

func QueryCursorDisabled(s string) QueryOption

QueryCursorDisabled overrides the cursor value returned by the cursor_current() or current_cursor() functions.

func QueryCursorScopedTables

func QueryCursorScopedTables(l []string) QueryOption

QueryCursorScopedTables is a list of table names that should be scoped to cursor_after_default .. cursor_before_or_at_default (upper bound is optional).

func QueryDataScope

func QueryDataScope(ds DataScope) QueryOption

QueryDataScope controls the query's datascope -- whether the query applies to all data or just part of it. ['default', 'all', or 'hotcache']

func QueryDateTimeScopeColumn

func QueryDateTimeScopeColumn(s string) QueryOption

QueryDateTimeScopeColumn controls the column name for the query's datetime scope (query_datetimescope_to / query_datetimescope_from)

func QueryDateTimeScopeFrom

func QueryDateTimeScopeFrom(t time.Time) QueryOption

QueryDateTimeScopeFrom controls the query's datetime scope (earliest) -- used as auto-applied filter on query_datetimescope_column only (if defined).

func QueryDateTimeScopeTo

func QueryDateTimeScopeTo(t time.Time) QueryOption

QueryDateTimeScopeTo controls the query's datetime scope (latest) -- used as auto-applied filter on query_datetimescope_column only (if defined).

func QueryDistributionNodesSpan

func QueryDistributionNodesSpan(i int64) QueryOption

QueryDistributionNodesSpan If set, controls the way the subquery merge behaves: the executing node will introduce an additional level in the query hierarchy for each subgroup of nodes; the size of the subgroup is set by this option.

func QueryFanoutNodesPercent

func QueryFanoutNodesPercent(i int) QueryOption

QueryFanoutNodesPercent The percentage of nodes to fan out execution to.

func QueryFanoutThreadsPercent

func QueryFanoutThreadsPercent(i int) QueryOption

QueryFanoutThreadsPercent The percentage of threads to fan out execution to.

func QueryForceRowLevelSecurity

func QueryForceRowLevelSecurity() QueryOption

QueryForceRowLevelSecurity If specified, forces Row Level Security rules, even if row_level_security policy is disabled

func QueryLanguage

func QueryLanguage(s string) QueryOption

QueryLanguage Controls how the query text is to be interpreted (Kql or Sql).

func QueryLogQueryParameters

func QueryLogQueryParameters() QueryOption

QueryLogQueryParameters Enables logging of the query parameters, so that they can be viewed later in the .show queries journal.

func QueryMaxEntitiesInUnion

func QueryMaxEntitiesInUnion(i int64) QueryOption

QueryMaxEntitiesInUnion Overrides the default maximum number of entities in a union.

func QueryNow

func QueryNow(t time.Time) QueryOption

QueryNow Overrides the datetime value returned by the now(0s) function.

func QueryParameters

func QueryParameters(queryParameters *kql.Parameters) QueryOption

QueryParameters sets the parameters to be used in the query.

func QueryPythonDebug

func QueryPythonDebug(i int) QueryOption

QueryPythonDebug If set, generate python debug query for the enumerated python node (default first).

func QueryResultsApplyGetschema

func QueryResultsApplyGetschema() QueryOption

QueryResultsApplyGetschema If set, retrieves the schema of each tabular data in the results of the query instead of the data itself.

func QueryResultsCacheMaxAge

func QueryResultsCacheMaxAge(d time.Duration) QueryOption

QueryResultsCacheMaxAge If positive, controls the maximum age of the cached query results the service is allowed to return

func QueryResultsCachePerShard

func QueryResultsCachePerShard() QueryOption

QueryResultsCachePerShard If set, enables per-shard query cache.

func QueryResultsProgressiveRowCount

func QueryResultsProgressiveRowCount(i int64) QueryOption

QueryResultsProgressiveRowCount Hint for Kusto as to how many records to send in each update (takes effect only if OptionResultsProgressiveEnabled is set)

func QueryResultsProgressiveUpdatePeriod

func QueryResultsProgressiveUpdatePeriod(i int32) QueryOption

QueryResultsProgressiveUpdatePeriod Hint for Kusto as to how often to send progress frames (takes effect only if OptionResultsProgressiveEnabled is set)

func QueryTakeMaxRecords

func QueryTakeMaxRecords(i int64) QueryOption

QueryTakeMaxRecords Enables limiting query results to this number of records.

func RequestAppName

func RequestAppName(s string) QueryOption

RequestAppName Request application name to be used in the reporting (e.g. show queries). Does not set the `Application` property in `.show queries`, see `Application` for that.

func RequestBlockRowLevelSecurity

func RequestBlockRowLevelSecurity() QueryOption

RequestBlockRowLevelSecurity If specified, blocks access to tables for which row_level_security policy is enabled.

func RequestCalloutDisabled

func RequestCalloutDisabled() QueryOption

RequestCalloutDisabled If specified, indicates that the request can't call-out to a user-provided service.

func RequestDescription

func RequestDescription(s string) QueryOption

RequestDescription Arbitrary text that the author of the request wants to include as the request description.

func RequestExternalTableDisabled

func RequestExternalTableDisabled() QueryOption

RequestExternalTableDisabled If specified, indicates that the request can't invoke code in the ExternalTable.

func RequestImpersonationDisabled

func RequestImpersonationDisabled() QueryOption

RequestImpersonationDisabled If specified, indicates that the service should not impersonate the caller's identity.

func RequestReadonly

func RequestReadonly() QueryOption

RequestReadonly If specified, indicates that the request can't write anything.

func RequestRemoteEntitiesDisabled

func RequestRemoteEntitiesDisabled() QueryOption

RequestRemoteEntitiesDisabled If specified, indicates that the request can't access remote databases and clusters.

func RequestSandboxedExecutionDisabled

func RequestSandboxedExecutionDisabled() QueryOption

RequestSandboxedExecutionDisabled If specified, indicates that the request can't invoke code in the sandbox.

func RequestUser

func RequestUser(s string) QueryOption

RequestUser Request user to be used in the reporting (e.g. show queries). Does not set the `User` property in `.show queries`, see `User` for that.

func ResultsProgressiveDisable

func ResultsProgressiveDisable() QueryOption

ResultsProgressiveDisable disables the progressive query stream.

func ServerTimeout

func ServerTimeout(d time.Duration) QueryOption

ServerTimeout overrides the default request timeout.

func TruncationMaxRecords

func TruncationMaxRecords(i int64) QueryOption

TruncationMaxRecords Overrides the default maximum number of records a query is allowed to return to the caller (truncation).

func TruncationMaxSize

func TruncationMaxSize(i int64) QueryOption

TruncationMaxSize Overrides the default maximum data size a query is allowed to return to the caller (truncation).

func User

func User(userName string) QueryOption

User sets the x-ms-user header, and can be used to identify the user making the request in the `.show queries` output.

func ValidatePermissions

func ValidatePermissions() QueryOption

ValidatePermissions Validates user's permissions to perform the query and doesn't run the query itself.

type QueryValues

type QueryValues map[string]interface{}

QueryValues represents a set of values that are substituted in Parameters. Every QueryValue key must have a corresponding Parameter name. All values must be compatible with the Kusto Column type it will go into (int64 for a long, int32 for int, time.Time for datetime, ...)

type Row

type Row struct {
	Values  value.Values
	Error   *errors.Error
	Replace bool
}

Row is a row of data from Kusto, or an error. Replace indicates whether the existing result set should be cleared and replaced with this row.

type RowIterator

type RowIterator struct {

	// RequestHeader is the http.Header sent in the request to the server.
	RequestHeader http.Header
	// ResponseHeader is the http.header sent in the response from the server.
	ResponseHeader http.Header
	// contains filtered or unexported fields
}

RowIterator is used to iterate over the returned Row objects returned by Kusto.

func (*RowIterator) Do

func (r *RowIterator) Do(f func(r *table.Row) error) error

Deprecated: Use DoOnRowOrError() instead for more robust error handling. In a future version, this will be removed, and NextRowOrError will replace it. Do calls f for every row returned by the query. If f returns a non-nil error, iteration stops. This method will fail on errors inline within the rows, even though they could potentially be recovered and more data might be available. This behavior is to keep the interface compatible.

func (*RowIterator) DoOnRowOrError

func (r *RowIterator) DoOnRowOrError(f func(r *table.Row, e *errors.Error) error) error

DoOnRowOrError calls f for every row returned by the query. If errors occur inline within the rows, they are passed to f. Other errors will stop the iteration and be returned. If f returns a non-nil error, iteration stops.

func (*RowIterator) GetExtendedProperties

func (r *RowIterator) GetExtendedProperties() (v2.DataTable, error)

GetExtendedProperties will return the extended properties' table from the iterator, if it exists. Returns io.ErrUnexpectedEOF if not found. May not have all tables until RowIterator has reached io.EOF.

func (*RowIterator) GetNonPrimary

func (r *RowIterator) GetNonPrimary(tableKind, tableName frames.TableKind) (v2.DataTable, error)

GetNonPrimary will return a non-primary dataTable if it exists from the last query. The non-primary table and common names are defined under the frames.TableKind enum. Returns io.ErrUnexpectedEOF if not found. May not have all tables until RowIterator has reached io.EOF.

func (*RowIterator) GetQueryCompletionInformation

func (r *RowIterator) GetQueryCompletionInformation() (v2.DataTable, error)

GetQueryCompletionInformation will return the query completion information table from the iterator, if it exists. Returns io.ErrUnexpectedEOF if not found. May not have all tables until RowIterator has reached io.EOF.

func (*RowIterator) Mock

func (r *RowIterator) Mock(m *MockRows) error

Mock is used to tell the RowIterator to return specific data for tests. This is useful when building fakes of the client's Query() call for hermetic tests. This can only be called in a test or it will panic.

func (*RowIterator) Next

func (r *RowIterator) Next() (row *table.Row, finalError error)

Deprecated: Use NextRowOrError() instead for more robust error handling. In a future version, this will be removed, and NextRowOrError will replace it. Next gets the next Row from the query. io.EOF is returned if there are no more entries in the output. This method will fail on errors inline within the rows, even though they could potentially be recovered and more data might be available. Once Next() returns an error, all subsequent calls will return the same error.

func (*RowIterator) NextRowOrError

func (r *RowIterator) NextRowOrError() (row *table.Row, inlineError *errors.Error, finalError error)

NextRowOrError gets the next Row or service-side error from the query. On partial success, inlineError will be set. Once finalError returns non-nil, all subsequent calls will return the same error. finalError will be set to io.EOF is when frame parsing completed with success or partial success (data + errors). if finalError is not io.EOF, reading the frame has resulted in a failure state (no data is expected).

func (*RowIterator) Progress

func (r *RowIterator) Progress() float64

Progress returns the progress of the query, 0-100%. This is only valid on Progressive data returns.

func (*RowIterator) Progressive

func (r *RowIterator) Progressive() bool

Progressive indicates if the RowIterator is unpacking progressive (streaming) frames.

func (*RowIterator) Stop

func (r *RowIterator) Stop()

Stop is called to stop any further iteration. Always defer a Stop() call after receiving a RowIterator.

type Statement

type Statement interface {
	fmt.Stringer
	GetParameters() (map[string]string, error)
	SupportsInlineParameters() bool
}

Statement is an interface designated to generalize query/management objects - both Stmt, and kql.StatementBuilder

Example

package main

import (
	"fmt"
	"github.com/Azure/azure-kusto-go/kusto/kql"
)

var (
	// rootStatement represents our root statementBuilder object in which we can derive other statementBuilders.
	rootStatement = kql.New("").AddTable("systemNodes")
	// singleBasicStatement is derived from the rootStatement but includes a where clause to limit the query to a wanted result.
	singleBasicStatement = rootStatement.AddLiteral(" | where ").
				AddColumn("NodeId").AddLiteral(" == ").AddInt(1)

	// We will also define a similar Statement, but this time with a Parameters object as well to define the "NodeId" word in the
	// query as an int (aka, using KQL query parameters).
	singleParameterStatement = kql.New("systemNodes").AddLiteral(" | where NodeId == id")
	singleQueryParameter     = kql.NewParameters().AddInt("id", 1)
)

func main() {

	// If we wanted to build a query , we could build it from singleBasicStatement like so :
	fmt.Println("Basic Builder:\n", singleBasicStatement.String())
	// and send it to querying: client.Query(ctx, "database", singleBasicStatement)

	// Or we can use the query parameters option:
	fmt.Println("Basic Builder with parameters:\n", singleParameterStatement)
	for k, v := range singleQueryParameter.ToParameterCollection() {
		fmt.Printf("Query parameters:\n{%s: %s}\n", k, v)
	}

	// and send it to querying: client.Query(ctx, "database", singleParameterStatement,
	//	[]kusto.QueryOption{kusto.QueryParameters(*singleQueryParameter)})
	// Where the query will be:
	fmt.Printf("Actual query:\n%s\n%s\n", singleQueryParameter.ToDeclarationString(), singleParameterStatement)

}

Output:

Basic Builder:
 systemNodes | where NodeId == int(1)
Basic Builder with parameters:
 systemNodes | where NodeId == id
Query parameters:
{id: int(1)}
Actual query:
declare query_parameters(id:int);
systemNodes | where NodeId == id

type Stmt

type Stmt struct {
	// contains filtered or unexported fields
}

Stmt is a Kusto Query statement. A Stmt is thread-safe, but methods on the Stmt are not. All methods on a Stmt do not alter the statement, they return a new Stmt object with the changes. This includes a copy of the Definitions and Parameters objects, if provided. This allows a root Stmt object that can be built upon. You should not pass *Stmt objects.

Example

package main

import (
	"fmt"
	"github.com/Azure/azure-kusto-go/kusto/kql"
)

const (
	// qRoot is a a root query text that will be used in a Stmt. It contains everything that any
	// other Stmt objects will need.
	qRoot = "set notruncation;dcmInventoryComponentSystem\n| project NodeId, Version"
	// singleNode includes syntax that will be included only when we want to grab a single node.
	// The word "Node" here is substituted for variable.
	singleNode = "\n| where NodeId == Node\n"
)

var (
	// rootStmt represents our root Stmt object in which we can derive other Stmts. This definition
	// will always include NodeId and Version fields.
	rootStmt = kql.New(qRoot)
	// singleStmt is derived from the rootStmt but includes a where clause to limit the query to a
	// single node. Node is an implicit parameter that will be substituted for a value when we run the query.
	singleStmt = kql.FromBuilder(rootStmt).AddLiteral(singleNode)
)

func main() {
	// This will print the rootStmt that could be used to list all nodes in the table.
	fmt.Println("All Nodes Builder:")
	fmt.Println(rootStmt.String())

	// This will build the parameters for the singleStmt. We can use these parameters to run the query.
	params := kql.NewParameters().AddString("Node", "my_id")

	fmt.Println("Single Builder:")
	fmt.Println(singleStmt.String())
	fmt.Println("Single Builder Parameter declaration:")
	fmt.Println(params.ToDeclarationString())

	// Alternatively, we can build the statement with the value in it.
	stmt := kql.New(qRoot).AddLiteral("\n| where NodeId == ").AddString("my_id")

	fmt.Println("Single Builder(Built):")
	fmt.Println(stmt.String())

}

Output:

All Nodes Builder:
set notruncation;dcmInventoryComponentSystem
| project NodeId, Version
Single Builder:
set notruncation;dcmInventoryComponentSystem
| project NodeId, Version
| where NodeId == Node

Single Builder Parameter declaration:
declare query_parameters(Node:string);
Single Builder(Built):
set notruncation;dcmInventoryComponentSystem
| project NodeId, Version
| where NodeId == "my_id"

func NewStmt

func NewStmt(query stringConstant, options ...StmtOption) Stmt

Deprecated: Use kql.New and kql.NewParameters instead. NewStmt creates a Stmt from a string constant.

func (Stmt) Add

func (s Stmt) Add(query stringConstant) Stmt

Add will add more text to the Stmt. This is similar to the + operator on two strings, except it only can be done with string constants. This allows dynamically building of a query from a root Stmt.

func (Stmt) GetParameters

func (s Stmt) GetParameters() (map[string]string, error)

func (Stmt) MustDefinitions

func (s Stmt) MustDefinitions(defs Definitions) Stmt

MustDefinitions is the same as WithDefinitions with the exceptions that an error causes a panic.

func (Stmt) MustParameters

func (s Stmt) MustParameters(params Parameters) Stmt

MustParameters is the same as WithParameters with the exceptions that an error causes a panic.

func (Stmt) String

func (s Stmt) String() string

String implements fmt.Stringer. This can be used to see what the query statement to the server will be for debugging purposes.

func (Stmt) SupportsInlineParameters

func (s Stmt) SupportsInlineParameters() bool

func (Stmt) UnsafeAdd

func (s Stmt) UnsafeAdd(query string) Stmt

UnsafeAdd provides a method to add strings that are not injection protected to the Stmt. To utilize this method, you must create the Stmt with the UnsafeStmt() option and pass the unsafe.Stmt with .Add set to true. If not set, THIS WILL PANIC!

func (Stmt) ValuesJSON

func (s Stmt) ValuesJSON() (string, error)

ValuesJSON returns a string in JSON format representing the Kusto QueryOptions.Parameters value that will be passed to the server. These values are substitued for Definitions in the Stmt and are represented by the Parameters that was passed.

func (Stmt) WithDefinitions

func (s Stmt) WithDefinitions(defs Definitions) (Stmt, error)

WithDefinitions will return a Stmt that can be used in a Query() with Kusto Parameters to protect against SQL-like injection attacks. These Parameters must align with the placeholders in the statement. The new Stmt object will have a copy of the Parameters passed, not the original.

func (Stmt) WithParameters

func (s Stmt) WithParameters(params Parameters) (Stmt, error)

WithParameters returns a Stmt that has the Parameters that will be substituted for Definitions in the query. Must have supplied the appropriate Definitions using WithQueryParamaters().

type StmtOption

type StmtOption func(s *Stmt)

StmtOption is an optional argument to NewStmt().

func UnsafeStmt

func UnsafeStmt(options unsafe.Stmt) StmtOption

UnsafeStmt enables unsafe actions on a Stmt and all Stmts derived from that Stmt. This turns off safety features that could allow a service client to compromise your data store. USE AT YOUR OWN RISK!

type StringPair

type StringPair struct {
	Key   string
	Value string
}

type TableOfContents

type TableOfContents struct {
	Ordinal    int64
	Kind       string
	Name       string
	Id         string
	PrettyName string
}

type TokenProvider

type TokenProvider struct {
	// contains filtered or unexported fields
}

func (*TokenProvider) AcquireToken

func (tkp *TokenProvider) AcquireToken(ctx context.Context) (string, string, error)

tokenProvider need to be received as reference, to reflect updations to the structs

func (*TokenProvider) AuthorizationRequired

func (tkp *TokenProvider) AuthorizationRequired() bool

func (*TokenProvider) SetHttp

func (tkp *TokenProvider) SetHttp(http *http.Client)