azkustodata

package module
v1.0.0-preview-5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2024 License: MIT Imports: 33 Imported by: 6

Documentation

Overview

Package azkustodata provides a client for querying and managing data in Azure Data Explorer (Kusto) clusters.

The package supports running both Kusto Query Language (KQL) queries and management commands, with built-in support for streaming responses and mapping results to Go structs.

To start using this package, create an instance of the Client, passing in a connection string built using the NewConnectionStringBuilder() function. The Client can be authenticated using various methods from the Azure Identity package.

Example For Querying the Kusto cluster: 

kcsb := azkustodata.NewConnectionStringBuilder("https://help.kusto.windows.net/").WithDefaultAzureCredential()
client, err := azkustodata.New(kcsb)

if err != nil {
	panic(err)
}

defer client.Close() // Always close the client when done.

ctx := context.Background()
dataset, err := client.IterativeQuery(ctx, "Samples", kql.New("PopulationData"))

// Don't forget to close the dataset when you're done.
defer dataset.Close()

primaryResult := <-dataset.Tables() // The first table in the dataset will be the primary results.

// Make sure to check for errors.
if primaryResult.Err() != nil {
	panic("add error handling")
}

for rowResult := range primaryResult.Table().Rows() {
	if rowResult.Err() != nil {
		panic("add error handling")
	}
	row := rowResult.Row()

	fmt.Println(row) // As a convenience, printing a *table.Row will output csv
	// or Access the columns directly
	fmt.Println(row.IntByName("EventId"))
	fmt.Println(row.StringByIndex(1))
}

Example for Management/Administration commands: 

kcsb := azkustodata.NewConnectionStringBuilder("https://help.kusto.windows.net/").WithDefaultAzureCredential()
client, err := azkustodata.New(kcsb)

if err != nil {
	panic(err)
}

defer client.Close() // Always close the client when done.

ctx := context.Background()
dataset, err := client.Mgmt(ctx, "Samples", kql.New(".show tables"))

table := dataset.Tables()[0]

// convert the table to a struct
structs, err := query.ToStructs[myStruct](table)

To handle results, the package provides utilities to directly stream rows, fetch tables into memory, and map results to structs.

For complete documentation, please visit: https://github.com/Azure/azure-kusto-go https://pkg.go.dev/github.com/Azure/azure-kusto-go/azkustodata

Example (Complex) 
// This example sets up a Query where we want to query for nodes that have a NodeId (a Kusto Long type) that has a
// particular NodeId. The will require inserting a value where ParamNodeId is in the query.
// We will used a parameterized query to do this.
q := kql.New("systemNodes | project CollectionTime, NodeId | where NodeId == ParamNodeId")
params := kql.NewParameters()

// NodeRec represents our Kusto data that will be returned.
type NodeRec struct {
	// ID is the table's NodeId. We use the field tag here to instruct our client to convert NodeId in the Kusto
	// table to ID in our struct.
	ID int64 `kusto:"NodeId"`
	// CollectionTime is Go representation of the Kusto datetime type.
	CollectionTime time.Time
}

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}
// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for our specific node. We are only doing a single query here as an example,
// normally you would take in requests of some type for different NodeIds.
data, err := client.Query(ctx, "database", q, QueryParameters(params))
if err != nil {
	panic("add error handling")
}

primary := data.Tables()[0]

recs, err := query.ToStructs[NodeRec](primary)

if err != nil {
	panic("add error handling")
}

for _, rec := range recs {
	fmt.Println(rec.ID)
}

Output:
Example (Simple) 
// Query and capture the values and put them in a slice of structs representing the row.

// NodeRec represents our Kusto data that will be returned.
type NodeRec struct {
	// ID is the table's NodeId. We use the field tag here to instruct our client to convert NodeId to ID.
	ID int64 `kusto:"NodeId"`
	// CollectionTime is Go representation of the Kusto datetime type.
	CollectionTime time.Time
}

kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")
client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}

// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds.
data, err := client.Query(ctx, "database", kql.New("systemNodes | project CollectionTime, NodeId"))
if err != nil {
	panic("add error handling")
}

primary := data.Tables()[0]

recs, err := query.ToStructs[NodeRec](primary)

if err != nil {
	panic("add error handling")
}

for _, rec := range recs {
	fmt.Println(rec.ID)
}

Output:

Index

Examples

Constants

View Source 
const (
	// DSDefault is used to set a query's datascope to default.
	DSDefault dataScope = "default"
	// DSAll is used to set a query's datascope to all.
	DSAll dataScope = "all"
	// DSHotCache is used to set a query's datascope to hotcache.
	DSHotCache dataScope = "hotcache"
)
View Source 
const ApplicationHeader = "x-ms-app"
View Source 
const (
	BEARER_TYPE = "Bearer"
)
View Source 
const ClientMaxRedirectCountValue = "client_max_redirect_count"
View Source 
const ClientRequestIdHeader = "x-ms-client-request-id"
View Source 
const ClientVersionHeader = "x-ms-client-version"
View Source 
const DeferPartialQueryFailuresValue = "deferpartialqueryfailures"
View Source 
const MaterializedViewShuffleValue = "materialized_view_shuffle"
View Source 
const MaxMemoryConsumptionPerIteratorValue = "maxmemoryconsumptionperiterator"
View Source 
const MaxMemoryConsumptionPerQueryPerNodeValue = "max_memory_consumption_per_query_per_node"
View Source 
const MaxOutputColumnsValue = "maxoutputcolumns"
View Source 
const NONE = "[none]"
View Source 
const NoRequestTimeoutValue = "norequesttimeout"
View Source 
const NoTruncationValue = "notruncation"
View Source 
const PushSelectionThroughAggregationValue = "push_selection_through_aggregation"
View Source 
const QueryBinAutoAtValue = "query_bin_auto_at"
View Source 
const QueryBinAutoSizeValue = "query_bin_auto_size"
View Source 
const QueryConsistencyValue = "queryconsistency"
View Source 
const QueryCursorAfterDefaultValue = "query_cursor_after_default"
View Source 
const QueryCursorBeforeOrAtDefaultValue = "query_cursor_before_or_at_default"
View Source 
const QueryCursorCurrentValue = "query_cursor_current"
View Source 
const QueryCursorDisabledValue = "query_cursor_disabled"
View Source 
const QueryCursorScopedTablesValue = "query_cursor_scoped_tables"
View Source 
const QueryDatascopeValue = "query_datascope"
View Source 
const QueryDateTimeScopeColumnValue = "query_datetimescope_column"
View Source 
const QueryDateTimeScopeFromValue = "query_datetimescope_from"
View Source 
const QueryDateTimeScopeToValue = "query_datetimescope_to"
View Source 
const QueryDistributionNodesSpanValue = "query_distribution_nodes_span"
View Source 
const QueryFanoutNodesPercentValue = "query_fanout_nodes_percent"
View Source 
const QueryFanoutThreadsPercentValue = "query_fanout_threads_percent"
View Source 
const QueryForceRowLevelSecurityValue = "query_force_row_level_security"
View Source 
const QueryLanguageValue = "query_language"
View Source 
const QueryLogQueryParametersValue = "query_log_query_parameters"
View Source 
const QueryMaxEntitiesInUnionValue = "query_max_entities_in_union"
View Source 
const QueryNowValue = "query_now"
View Source 
const QueryPythonDebugValue = "query_python_debug"
View Source 
const QueryResultsApplyGetschemaValue = "query_results_apply_getschema"
View Source 
const QueryResultsCacheMaxAgeValue = "query_results_cache_max_age"
View Source 
const QueryResultsCachePerShardValue = "query_results_cache_per_shard"
View Source 
const QueryResultsProgressiveRowCountValue = "query_results_progressive_row_count"
View Source 
const QueryResultsProgressiveUpdatePeriodValue = "query_results_progressive_update_period"
View Source 
const QueryTakeMaxRecordsValue = "query_take_max_records"
View Source 
const RequestAppNameValue = "request_app_name"
View Source 
const RequestBlockRowLevelSecurityValue = "request_block_row_level_security"
View Source 
const RequestCalloutDisabledValue = "request_callout_disabled"
View Source 
const RequestDescriptionValue = "request_description"
View Source 
const RequestExternalTableDisabledValue = "request_external_table_disabled"
View Source 
const RequestImpersonationDisabledValue = "request_impersonation_disabled"
View Source 
const RequestReadonlyValue = "request_readonly"
View Source 
const RequestRemoteEntitiesDisabledValue = "request_remote_entities_disabled"
View Source 
const RequestSandboxedExecutionDisabledValue = "request_sandboxed_execution_disabled"
View Source 
const RequestUserValue = "request_user"
View Source 
const ResultsErrorReportingPlacementEndOfDataset = "end_of_dataset"
View Source 
const ResultsErrorReportingPlacementEndOfTable = "end_of_table"
View Source 
const ResultsErrorReportingPlacementInData = "in_data"
View Source 
const ResultsErrorReportingPlacementValue = "results_error_reporting_placement"
View Source 
const ResultsProgressiveEnabledValue = "results_progressive_enabled"
View Source 
const ServerTimeoutValue = "servertimeout"
View Source 
const TruncationMaxRecordsValue = "truncationmaxrecords"
View Source 
const TruncationMaxSizeValue = "truncationmaxsize"
View Source 
const UserHeader = "x-ms-user"
View Source 
const V2FragmentPrimaryTablesValue = "results_v2_fragment_primary_tables"
View Source 
const V2NewlinesBetweenFramesValue = "results_v2_newlines_between_frames"
View Source 
const ValidatePermissionsValue = "validate_permissions"

Variables

This section is empty.

Functions

func CalculateTimeout 

func CalculateTimeout(ctx context.Context, opt *queryOptions, queryType int)

Types

type Authorization 

type Authorization struct {
	// Token provider that can be used to get the access token.
	TokenProvider *TokenProvider
}

Authorization provides the TokenProvider needed to acquire the auth token.

Example (Config) 
kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

// Normally here you take a client.
_, err := New(kcsb)
if err != nil {
	panic("add error handling")
}

Output:
Example (Msi) 
kcsb := NewConnectionStringBuilder("endpoint").WithUserManagedIdentity("clientID")

// Normally here you take a client.
_, err := New(kcsb)
if err != nil {
	panic("add error handling")
}

Output:

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a client to a Kusto instance.

func New 

func New(kcsb *ConnectionStringBuilder, options ...Option) (*Client, error)

New returns a new Client.

func (*Client) Auth 

func (c *Client) Auth() Authorization

Auth returns the Authorization passed to New().

func (*Client) ClientDetails 

func (c *Client) ClientDetails() *ClientDetails

func (*Client) Close 

func (c *Client) Close() error

func (*Client) Endpoint 

func (c *Client) Endpoint() string

Endpoint returns the endpoint passed to New().

func (*Client) HttpClient 

func (c *Client) HttpClient() *http.Client

func (*Client) IterativeQuery 

func (c *Client) IterativeQuery(ctx context.Context, db string, kqlQuery Statement, options ...QueryOption) (query.IterativeDataset, error)

func (*Client) Mgmt 

func (c *Client) Mgmt(ctx context.Context, db string, kqlQuery Statement, options ...QueryOption) (v1.Dataset, error)

func (*Client) Query 

func (c *Client) Query(ctx context.Context, db string, kqlQuery Statement, options ...QueryOption) (query.Dataset, error)
Example (Rows) 
kcsb := NewConnectionStringBuilder("endpoint").WithAadAppKey("clientID", "clientSecret", "tenentID")

client, err := New(kcsb)
if err != nil {
	panic("add error handling")
}
// Be sure to close the client when you're done. (Error handling omitted for brevity.)
defer client.Close()

ctx := context.Background()

// Query our database table "systemNodes" for the CollectionTimes and the NodeIds.
iter, err := client.IterativeQuery(ctx, "database", kql.New("systemNodes | project CollectionTime, NodeId"))
if err != nil {
	panic("add error handling")
}
defer iter.Close()

for res := range iter.Tables() {
	if res.Err() != nil {
		panic("add error handling")
	}
	var tb = res.Table()
	for rowResult := range tb.Rows() {
		if rowResult.Err() != nil {
			panic("add error handling")
		}
		var row = rowResult.Row()
		for _, v := range row.Values() {
			fmt.Printf("%s,", v)
		}
		fmt.Println("") // Add a carriage return
	}
}

Output:

func (*Client) QueryToJson 

func (c *Client) QueryToJson(ctx context.Context, db string, query Statement, options ...QueryOption) (string, error)

func (*Client) RawV2 

func (c *Client) RawV2(ctx context.Context, db string, kqlQuery Statement, options []QueryOption) (io.ReadCloser, error)

type ClientDetails 

type ClientDetails struct {
	// contains filtered or unexported fields
}

func NewClientDetails 

func NewClientDetails(applicationForTracing string, userNameForTracing string) *ClientDetails

func (*ClientDetails) ApplicationForTracing 

func (c *ClientDetails) ApplicationForTracing() string

func (*ClientDetails) ClientVersionForTracing 

func (c *ClientDetails) ClientVersionForTracing() string

func (*ClientDetails) UserNameForTracing 

func (c *ClientDetails) UserNameForTracing() string

type CloudInfo 

type CloudInfo struct {
	LoginEndpoint          string `json:"LoginEndpoint"`
	LoginMfaRequired       bool   `json:"LoginMfaRequired"`
	KustoClientAppID       string `json:"KustoClientAppId"`
	KustoClientRedirectURI string `json:"KustoClientRedirectUri"`
	KustoServiceResourceID string `json:"KustoServiceResourceId"`
	FirstPartyAuthorityURL string `json:"FirstPartyAuthorityUrl"`
}

func GetMetadata 

func GetMetadata(kustoUri string, httpClient *http.Client) (CloudInfo, error)

type Conn 

type Conn struct {
	// contains filtered or unexported fields
}

Conn provides connectivity to a Kusto instance.

func NewConn 

func NewConn(endpoint string, auth Authorization, client *http.Client, clientDetails *ClientDetails) (*Conn, error)

NewConn returns a new Conn object with an injected http.Client

func (*Conn) Close 

func (c *Conn) Close() error

func (*Conn) StreamIngest 

func (c *Conn) StreamIngest(ctx context.Context, db, table string, payload io.Reader, format DataFormatForStreaming, mappingName string, clientRequestId string, isBlobUri bool) error

type ConnectionStringBuilder 

type ConnectionStringBuilder struct {
	DataSource                     string
	AadUserID                      string
	Password                       string
	UserToken                      string
	ApplicationClientId            string
	ApplicationKey                 string
	AuthorityId                    string
	ApplicationCertificatePath     string
	ApplicationCertificateBytes    []byte
	ApplicationCertificatePassword []byte
	SendCertificateChain           bool
	ApplicationToken               string
	AzCli                          bool
	MsiAuthentication              bool
	WorkloadAuthentication         bool
	FederationTokenFilePath        string
	// Deprecated: Use ManagedServiceIdentityClientId or ManagedServiceIdentityResourceId instead
	ManagedServiceIdentity           string
	ManagedServiceIdentityClientId   string
	ManagedServiceIdentityResourceId string
	InteractiveLogin                 bool
	RedirectURL                      string
	DefaultAuth                      bool
	ClientOptions                    *azcore.ClientOptions
	ApplicationForTracing            string
	UserForTracing                   string
	TokenCredential                  azcore.TokenCredential
}

func NewConnectionStringBuilder 

func NewConnectionStringBuilder(connStr string) *ConnectionStringBuilder

NewConnectionStringBuilder Creates new Kusto ConnectionStringBuilder. Params takes kusto connection string connStr: string. Kusto connection string should be of the format: https://<clusterName>.<location>.kusto.windows.net;AAD User ID="user@microsoft.com";Password=P@ssWord For more information please look at: https://docs.microsoft.com/azure/data-explorer/kusto/api/connection-strings/kusto

func (*ConnectionStringBuilder) AttachPolicyClientOptions 

func (kcsb *ConnectionStringBuilder) AttachPolicyClientOptions(options *azcore.ClientOptions) *ConnectionStringBuilder

AttachPolicyClientOptions Assigns ClientOptions to string builder that contains configuration settings like Logging and Retry configs for a client's pipeline. Read more at https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azcore@v1.2.0/policy#ClientOptions

func (*ConnectionStringBuilder) SetConnectorDetails 

func (kcsb *ConnectionStringBuilder) SetConnectorDetails(name, version, appName, appVersion string, sendUser bool, overrideUser string, additionalFields ...StringPair)

func (*ConnectionStringBuilder) WithAadAppKey 

func (kcsb *ConnectionStringBuilder) WithAadAppKey(appId string, appKey string, authorityID string) *ConnectionStringBuilder

WithAadAppKey Creates a Kusto Connection string builder that will authenticate with AAD application and key.

func (*ConnectionStringBuilder) WithAadUserPassAuth 

func (kcsb *ConnectionStringBuilder) WithAadUserPassAuth(uname string, pswrd string, authorityID string) *ConnectionStringBuilder

WithAadUserPassAuth Creates a Kusto Connection string builder that will authenticate with AAD user name and password.

func (*ConnectionStringBuilder) WithAadUserToken 

func (kcsb *ConnectionStringBuilder) WithAadUserToken(usertoken string) *ConnectionStringBuilder

WithAadUserToken Creates a Kusto Connection string builder that will authenticate with AAD user token

func (*ConnectionStringBuilder) WithAppCertificateBytes 

func (kcsb *ConnectionStringBuilder) WithAppCertificateBytes(appId string, certificateBytes []byte, password []byte, sendCertChain bool, authorityID string) *ConnectionStringBuilder

WithAppCertificateBytes Creates a Kusto Connection string builder that will authenticate with AAD application using a certificate.

func (*ConnectionStringBuilder) WithAppCertificatePath 

func (kcsb *ConnectionStringBuilder) WithAppCertificatePath(appId string, certificatePath string, password []byte, sendCertChain bool, authorityID string) *ConnectionStringBuilder

WithAppCertificatePath Creates a Kusto Connection string builder that will authenticate with AAD application using a certificate.

func (*ConnectionStringBuilder) WithApplicationToken 

func (kcsb *ConnectionStringBuilder) WithApplicationToken(appId string, appToken string) *ConnectionStringBuilder

WithApplicationToken Creates a Kusto Connection string builder that will authenticate with AAD application and an application token.

func (*ConnectionStringBuilder) WithAzCli 

func (kcsb *ConnectionStringBuilder) WithAzCli() *ConnectionStringBuilder

WithAzCli Creates a Kusto Connection string builder that will use existing authenticated az cli profile password.

func (*ConnectionStringBuilder) WithDefaultAzureCredential 

func (kcsb *ConnectionStringBuilder) WithDefaultAzureCredential() *ConnectionStringBuilder

WithDefaultAzureCredential Create Kusto Conntection String that will be used for default auth mode. The order of auth will be via environment variables, managed identity and Azure CLI . Read more at https://learn.microsoft.com/azure/developer/go/azure-sdk-authentication?tabs=bash#2-authenticate-with-azure

func (*ConnectionStringBuilder) WithInteractiveLogin 

func (kcsb *ConnectionStringBuilder) WithInteractiveLogin(authorityID string) *ConnectionStringBuilder

WithInteractiveLogin Creates a Kusto Connection string builder that will authenticate by launching the system default browser to interactively authenticate a user, and obtain an access token

func (*ConnectionStringBuilder) WithKubernetesWorkloadIdentity 

func (kcsb *ConnectionStringBuilder) WithKubernetesWorkloadIdentity(appId, tokenFilePath, authorityID string) *ConnectionStringBuilder

WithKubernetesWorkloadIdentity Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using Kubernetes workload identity.

func (*ConnectionStringBuilder) WithSystemManagedIdentity 

func (kcsb *ConnectionStringBuilder) WithSystemManagedIdentity() *ConnectionStringBuilder

WithSystemManagedIdentity Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using system assigned id.

func (*ConnectionStringBuilder) WithTokenCredential 

func (kcsb *ConnectionStringBuilder) WithTokenCredential(tokenCredential azcore.TokenCredential) *ConnectionStringBuilder

func (*ConnectionStringBuilder) WithUserAssignedIdentityClientId 

func (kcsb *ConnectionStringBuilder) WithUserAssignedIdentityClientId(clientID string) *ConnectionStringBuilder

WithUserAssignedIdentityClientId Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using user assigned id.

func (*ConnectionStringBuilder) WithUserAssignedIdentityResourceId 

func (kcsb *ConnectionStringBuilder) WithUserAssignedIdentityResourceId(resourceID string) *ConnectionStringBuilder

WithUserAssignedIdentityResourceId Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using an MSI's resourceID.

func (*ConnectionStringBuilder) WithUserManagedIdentity deprecated 

func (kcsb *ConnectionStringBuilder) WithUserManagedIdentity(clientID string) *ConnectionStringBuilder

Deprecated: use WithUserManagedIdentityClientId or WithUserManagedIdentityResourceId instead WithUserManagedIdentity Creates a Kusto Connection string builder that will authenticate with AAD application, using an application token obtained from a Microsoft Service Identity endpoint using user assigned id.

type DataFormatForStreaming 

type DataFormatForStreaming interface {
	CamelCase() string
	KnownOrDefault() DataFormatForStreaming
}

type DataScope 

type DataScope interface {
	// contains filtered or unexported methods
}

DataScope is used with QueryDataScope() to control a query's datascope.

type Option 

type Option func(c *Client)

Option is an optional argument type for New().

func WithHttpClient 

func WithHttpClient(client *http.Client) Option

type QueryOption 

type QueryOption func(q *queryOptions) error

QueryOption is an option type for a call to Query().

func Application 

func Application(appName string) QueryOption

Application sets the x-ms-app header, and can be used to identify the application making the request in the `.show queries` output.

func ClientMaxRedirectCount 

func ClientMaxRedirectCount(i int64) QueryOption

ClientMaxRedirectCount If set and positive, indicates the maximum number of HTTP redirects that the client will process.

func ClientRequestID 

func ClientRequestID(clientRequestID string) QueryOption

ClientRequestID sets the x-ms-client-request-id header, and can be used to identify the request in the `.show queries` output.

func CustomQueryOption 

func CustomQueryOption(paramName string, i interface{}) QueryOption

CustomQueryOption exists to allow a QueryOption that is not defined in the Go SDK, as all options are not defined. Please Note: you should always use the type safe options provided below when available. Also note that Kusto does not error on non-existent parameter names or bad values, it simply doesn't work as expected.

func DeferPartialQueryFailures 

func DeferPartialQueryFailures() QueryOption

DeferPartialQueryFailures disables reporting partial query failures as part of the result set.

func MaterializedViewShuffle 

func MaterializedViewShuffle(s string) QueryOption

MaterializedViewShuffle A hint to use shuffle strategy for materialized views that are referenced in the query. The property is an array of materialized views names and the shuffle keys to use. Examples: 'dynamic([ { "Name": "V1", "Keys" : [ "K1", "K2" ] } ])' (shuffle view V1 by K1, K2) or 'dynamic([ { "Name": "V1" } ])' (shuffle view V1 by all keys)

func MaxMemoryConsumptionPerIterator 

func MaxMemoryConsumptionPerIterator(i uint64) QueryOption

MaxMemoryConsumptionPerIterator overrides the default maximum amount of memory a query operator may allocate.

func MaxMemoryConsumptionPerQueryPerNode 

func MaxMemoryConsumptionPerQueryPerNode(i uint64) QueryOption

MaxMemoryConsumptionPerQueryPerNode overrides the default maximum amount of memory a whole query may allocate per node.

func MaxOutputColumns 

func MaxOutputColumns(i int) QueryOption

MaxOutputColumns overrides the default maximum number of columns a query is allowed to produce.

func NoRequestTimeout 

func NoRequestTimeout() QueryOption

NoRequestTimeout enables setting the request timeout to its maximum value.

func NoTruncation 

func NoTruncation() QueryOption

NoTruncation enables suppressing truncation of the query results returned to the caller.

func PushSelectionThroughAggregation 

func PushSelectionThroughAggregation() QueryOption

PushSelectionThroughAggregation will push simple selection through aggregation .

func QueryBinAutoAt 

func QueryBinAutoAt(s string) QueryOption

QueryBinAutoAt When evaluating the bin_auto() function, the start value to use.

func QueryBinAutoSize 

func QueryBinAutoSize(s string) QueryOption

QueryBinAutoSize When evaluating the bin_auto() function, the bin size value to use.

func QueryConsistency 

func QueryConsistency(c string) QueryOption

QueryConsistency Controls query consistency

func QueryCursorAfterDefault 

func QueryCursorAfterDefault(s string) QueryOption

QueryCursorAfterDefault sets the default parameter value of the cursor_after() function when called without parameters.

func QueryCursorBeforeOrAtDefault 

func QueryCursorBeforeOrAtDefault(s string) QueryOption

QueryCursorBeforeOrAtDefault sets the default parameter value of the cursor_before_or_at() function when called without parameters.

func QueryCursorCurrent 

func QueryCursorCurrent(s string) QueryOption

QueryCursorCurrent overrides the cursor value returned by the cursor_current() or current_cursor() functions.

func QueryCursorDisabled 

func QueryCursorDisabled(s string) QueryOption

QueryCursorDisabled overrides the cursor value returned by the cursor_current() or current_cursor() functions.

func QueryCursorScopedTables 

func QueryCursorScopedTables(l []string) QueryOption

QueryCursorScopedTables is a list of table names that should be scoped to cursor_after_default .. cursor_before_or_at_default (upper bound is optional).

func QueryDataScope 

func QueryDataScope(ds DataScope) QueryOption

QueryDataScope controls the query's datascope -- whether the query applies to all data or just part of it. ['default', 'all', or 'hotcache']

func QueryDateTimeScopeColumn 

func QueryDateTimeScopeColumn(s string) QueryOption

QueryDateTimeScopeColumn controls the column name for the query's datetime scope (query_datetimescope_to / query_datetimescope_from)

func QueryDateTimeScopeFrom 

func QueryDateTimeScopeFrom(t time.Time) QueryOption

QueryDateTimeScopeFrom controls the query's datetime scope (earliest) -- used as auto-applied filter on query_datetimescope_column only (if defined).

func QueryDateTimeScopeTo 

func QueryDateTimeScopeTo(t time.Time) QueryOption

QueryDateTimeScopeTo controls the query's datetime scope (latest) -- used as auto-applied filter on query_datetimescope_column only (if defined).

func QueryDistributionNodesSpan 

func QueryDistributionNodesSpan(i int64) QueryOption

QueryDistributionNodesSpan If set, controls the way the subquery merge behaves: the executing node will introduce an additional level in the query hierarchy for each subgroup of nodes; the size of the subgroup is set by this option.

func QueryFanoutNodesPercent 

func QueryFanoutNodesPercent(i int) QueryOption

QueryFanoutNodesPercent The percentage of nodes to fan out execution to.

func QueryFanoutThreadsPercent 

func QueryFanoutThreadsPercent(i int) QueryOption

QueryFanoutThreadsPercent The percentage of threads to fan out execution to.

func QueryForceRowLevelSecurity 

func QueryForceRowLevelSecurity() QueryOption

QueryForceRowLevelSecurity If specified, forces Row Level Security rules, even if row_level_security policy is disabled

func QueryLanguage 

func QueryLanguage(s string) QueryOption

QueryLanguage Controls how the query text is to be interpreted (Kql or Sql).

func QueryLogQueryParameters 

func QueryLogQueryParameters() QueryOption

QueryLogQueryParameters Enables logging of the query parameters, so that they can be viewed later in the .show queries journal.

func QueryMaxEntitiesInUnion 

func QueryMaxEntitiesInUnion(i int64) QueryOption

QueryMaxEntitiesInUnion Overrides the default maximum number of entities in a union.

func QueryNow 

func QueryNow(t time.Time) QueryOption

QueryNow Overrides the datetime value returned by the now(0s) function.

func QueryParameters 

func QueryParameters(queryParameters *kql.Parameters) QueryOption

QueryParameters sets the parameters to be used in the query.

func QueryPythonDebug 

func QueryPythonDebug(i int) QueryOption

QueryPythonDebug If set, generate python debug query for the enumerated python node (default first).

func QueryResultsApplyGetschema 

func QueryResultsApplyGetschema() QueryOption

QueryResultsApplyGetschema If set, retrieves the schema of each tabular data in the results of the query instead of the data itself.

func QueryResultsCacheMaxAge 

func QueryResultsCacheMaxAge(d time.Duration) QueryOption

QueryResultsCacheMaxAge If positive, controls the maximum age of the cached query results the service is allowed to return

func QueryResultsCachePerShard 

func QueryResultsCachePerShard() QueryOption

QueryResultsCachePerShard If set, enables per-shard query cache.

func QueryResultsProgressiveRowCount 

func QueryResultsProgressiveRowCount(i int64) QueryOption

QueryResultsProgressiveRowCount Hint for Kusto as to how many records to send in each update (takes effect only if OptionResultsProgressiveEnabled is set)

func QueryResultsProgressiveUpdatePeriod 

func QueryResultsProgressiveUpdatePeriod(i int32) QueryOption

QueryResultsProgressiveUpdatePeriod Hint for Kusto as to how often to send progress frames (takes effect only if OptionResultsProgressiveEnabled is set)

func QueryTakeMaxRecords 

func QueryTakeMaxRecords(i int64) QueryOption

QueryTakeMaxRecords Enables limiting query results to this number of records.

func RequestAppName 

func RequestAppName(s string) QueryOption

RequestAppName Request application name to be used in the reporting (e.g. show queries). Does not set the `Application` property in `.show queries`, see `Application` for that.

func RequestBlockRowLevelSecurity 

func RequestBlockRowLevelSecurity() QueryOption

RequestBlockRowLevelSecurity If specified, blocks access to tables for which row_level_security policy is enabled.

func RequestCalloutDisabled 

func RequestCalloutDisabled() QueryOption

RequestCalloutDisabled If specified, indicates that the request can't call-out to a user-provided service.

func RequestDescription 

func RequestDescription(s string) QueryOption

RequestDescription Arbitrary text that the author of the request wants to include as the request description.

func RequestExternalTableDisabled 

func RequestExternalTableDisabled() QueryOption

RequestExternalTableDisabled If specified, indicates that the request can't invoke code in the ExternalTable.

func RequestImpersonationDisabled 

func RequestImpersonationDisabled() QueryOption

RequestImpersonationDisabled If specified, indicates that the service should not impersonate the caller's identity.

func RequestReadonly 

func RequestReadonly() QueryOption

RequestReadonly If specified, indicates that the request can't write anything.

func RequestRemoteEntitiesDisabled 

func RequestRemoteEntitiesDisabled() QueryOption

RequestRemoteEntitiesDisabled If specified, indicates that the request can't access remote databases and clusters.

func RequestSandboxedExecutionDisabled 

func RequestSandboxedExecutionDisabled() QueryOption

RequestSandboxedExecutionDisabled If specified, indicates that the request can't invoke code in the sandbox.

func RequestUser 

func RequestUser(s string) QueryOption

RequestUser Request user to be used in the reporting (e.g. show queries). Does not set the `User` property in `.show queries`, see `User` for that.

func ResultsErrorReportingPlacement 

func ResultsErrorReportingPlacement(s string) QueryOption

ResultsErrorReportingPlacement Decides the placement of errors in the result set: 1. "in_data" (default) - errors are placed in the table or table fragment, within the array of data rows. 2. "end_of_table" - errors are placed in the table completion frame, after the array of data rows. Only applies to queries that are progressive or fragmented.

  1. "end_of_dataset" - errors are placed in the dataset completion frame.

func ResultsProgressiveEnabled 

func ResultsProgressiveEnabled() QueryOption

ResultsProgressiveEnabled enables the progressive query stream.

func ServerTimeout 

func ServerTimeout(d time.Duration) QueryOption

ServerTimeout overrides the default request timeout.

func TruncationMaxRecords 

func TruncationMaxRecords(i int64) QueryOption

TruncationMaxRecords Overrides the default maximum number of records a query is allowed to return to the caller (truncation).

func TruncationMaxSize 

func TruncationMaxSize(i int64) QueryOption

TruncationMaxSize Overrides the default maximum data size a query is allowed to return to the caller (truncation).

func User 

func User(userName string) QueryOption

User sets the x-ms-user header, and can be used to identify the user making the request in the `.show queries` output.

func V2FragmentPrimaryTables 

func V2FragmentPrimaryTables() QueryOption

V2FragmentPrimaryTables Causes primary tables to be sent in multiple fragments, each containing a subset of the rows.

func V2IoCapacity 

func V2IoCapacity(i int) QueryOption

V2IoCapacity sets the size of the buffer, in frames, when reading from the network.

func V2NewlinesBetweenFrames 

func V2NewlinesBetweenFrames() QueryOption

V2NewlinesBetweenFrames Adds new lines between frames in the results, in order to make it easier to parse them.

func V2RowCapacity 

func V2RowCapacity(i int) QueryOption

V2RowCapacity sets the capacity of the buffer of data rows per table.

func V2TableCapacity 

func V2TableCapacity(i int) QueryOption

V2TableCapacity sets the capacity of the buffer of data fragments in the result set.

func ValidatePermissions 

func ValidatePermissions() QueryOption

ValidatePermissions Validates user's permissions to perform the query and doesn't run the query itself.

type Statement 

type Statement = *kql.Builder
Example 
package main

import (
	"fmt"
	"github.com/Azure/azure-kusto-go/azkustodata/kql"
)

var (
	// rootStatement represents our root statementBuilder object in which we can derive other statementBuilders.
	rootStatement = kql.New("").AddTable("systemNodes")
	// singleBasicStatement is derived from the rootStatement but includes a where clause to limit the query to a wanted result.
	singleBasicStatement = rootStatement.AddLiteral(" | where ").
				AddColumn("NodeId").AddLiteral(" == ").AddInt(1)

	// We will also define a similar Statement, but this time with a Parameters object as well to define the "NodeId" word in the
	// query as an int (aka, using KQL query parameters).
	singleParameterStatement = kql.New("systemNodes").AddLiteral(" | where NodeId == id")
	singleQueryParameter     = kql.NewParameters().AddInt("id", 1)
)

func main() {

	// If we wanted to build a query , we could build it from singleBasicStatement like so :
	fmt.Println("Basic Builder:\n", singleBasicStatement.String())
	// and send it to querying: client.Query(ctx, "database", singleBasicStatement)

	// Or we can use the query parameters option:
	fmt.Println("Basic Builder with parameters:\n", singleParameterStatement)
	for k, v := range singleQueryParameter.ToParameterCollection() {
		fmt.Printf("Query parameters:\n{%s: %s}\n", k, v)
	}

	// and send it to querying: client.Query(ctx, "database", singleParameterStatement,
	//	[]kusto.QueryOption{kusto.QueryParameters(*singleQueryParameter)})
	// Where the query will be:
	fmt.Printf("Actual query:\n%s\n%s\n", singleQueryParameter.ToDeclarationString(), singleParameterStatement)

}

Output:

Basic Builder:
 systemNodes | where NodeId == int(1)
Basic Builder with parameters:
 systemNodes | where NodeId == id
Query parameters:
{id: int(1)}
Actual query:
declare query_parameters(id:int);
systemNodes | where NodeId == id

type StringPair 

type StringPair struct {
	Key   string
	Value string
}

type TokenProvider 

type TokenProvider struct {
	// contains filtered or unexported fields
}

func (*TokenProvider) AcquireToken 

func (tkp *TokenProvider) AcquireToken(ctx context.Context) (string, string, error)

tokenProvider need to be received as reference, to reflect updations to the structs

func (*TokenProvider) AuthorizationRequired 

func (tkp *TokenProvider) AuthorizationRequired() bool

func (*TokenProvider) SetHttp 

func (tkp *TokenProvider) SetHttp(http *http.Client)

Source Files

View all Source files

Directories

Path Synopsis
Package errors provides the error package for Kusto.
 Package errors provides the error package for Kusto.
internal
response
version
Package version keeps the internal version number of the client.
 Package version keeps the internal version number of the client.
v1
v2
test
etoe
Package types holds Kusto type information that is used to describe what type would be held in a cell based on the column's type setting.
 Package types holds Kusto type information that is used to describe what type would be held in a cell based on the column's type setting.
Package unsafe provides methods and types that loosen the native protections of the Kusto package.
 Package unsafe provides methods and types that loosen the native protections of the Kusto package.
Package value holds Kusto data value representations.
 Package value holds Kusto data value representations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.