ydb

package module
v3.18.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 7, 2022 License: Apache-2.0 Imports: 43 Imported by: 86

README

ydb

PkgGoDev GoDoc tests lint Go Report Card codecov

YDB API client written in Go.

godoc

Table of contents

  1. Overview
  2. About semantic versioning
  3. About expected go1.18 changes
  4. Prerequisites
  5. Installation
  6. Usage
  7. Credentials
  8. Environment variables
  9. Ecosystem of debug tools
  10. Examples

Overview

Currently package ydb provides scheme and table API client implementations for YDB.

About semantic versioning

We follow the SemVer 2.0.0. In particular, we provide backward compatibility in the MAJOR releases. New features without loss of backward compatibility appear on the MINOR release. In the minor version, the patch number starts from 0. Bug fixes and internal changes are released with the third digit (PATCH) in the version.

There are, however, some changes with the loss of backward compatibility that we consider to be MINOR:

  • extension or modification of internal ydb-go-sdk interfaces. We understand that this will break the compatibility of custom implementations of the ydb-go-sdk internal interfaces. But we believe that the internal interfaces of ydb-go-sdk are implemented well enough that they do not require custom implementation. We are working to ensure that all internal interfaces have limited access only inside ydb-go-sdk.
  • major changes to (including removal of) the public interfaces and types that have been previously exported by ydb-go-sdk. We understand that these changes will break the backward compatibility of early adopters of these interfaces. However, these changes are generally coordinated with early adopters and have the concise interfacing with ydb-go-sdk as a goal.

Internal interfaces outside from internal directory are marked with comment such as

// Warning: only for internal usage inside ydb-go-sdk

We publish the planned breaking MAJOR changes:

  • via the comment Deprecated in the code indicating what should be used instead
  • through the file NEXT_MAJOR_RELEASE.md

About expected go1.18 changes

Some changes from go1.18 are expected and will be allowed to ydb-go-sdk:

  • type set constraints instead abstract interface{}. Firstly, this changes will be applied to scanner API (Scan, ScanWithDefaults and ScanNamed)
  • improve log package with parametrized types.

That changes could break backward compatibility. We are tried to support go1.18 features seamlessly. But now we was tested go1.18-beta implementation and detected some strong issues:

  • go version in go.mod must be forced up to 1.18,
  • interfaces with methods are not supported in type sets and other.

Prerequisites

Requires go1.14 or later.

Installation 

go get -u github.com/ydb-platform/ydb-go-sdk/v3

Usage

The straightforward example of querying data may look similar to this:

   ctx := context.Background()

   // ydb.New() returns connection object which provide necessary clients for different ydb services
   // such as table.Client, scheme.Client, coordination.Client, etc.
   db, err := ydb.New(
      ctx,
      ydb.WithConnectionString(os.Getenv("YDB_CONNECTION_STRING")),
      ydb.WithDialTimeout(3 * time.Second),
      ydb.WithCertificatesFromFile("~/.ydb/CA.pem"),
      ydb.WithSessionPoolIdleThreshold(time.Second * 5),
      ydb.WithSessionPoolKeepAliveMinSize(-1),
      ydb.WithDiscoveryInterval(5 * time.Second),
      ydb.WithAccessTokenCredentials(os.GetEnv("YDB_ACCESS_TOKEN_CREDENTIALS")),
   )
   if err != nil {
      // handle error
   }
   defer func() { _ = db.Close(ctx) }()

   // Prepare transaction control for upcoming query execution.
   // NOTE: result of TxControl() may be reused.
   txc := table.TxControl(
      table.BeginTx(table.WithSerializableReadWrite()),
      table.CommitTx(),
   )

   var res resultset.Result

   // Do() provide the best effort for executing operation
   // Do implements internal busy loop until one of the following conditions occurs:
   // - deadline was cancelled or deadlined
   // - operation returned nil as error
   // Note that in case of prepared statements call to Prepare() must be made
   // inside the function body.
   err := db.Table().Do(
      ctx, 
      func(ctx context.Context, s table.Session) (err error) {
         // Execute text query without preparation and with given "autocommit"
         // transaction control. That is, transaction will be committed without
         // additional calls. Notice the "_" unused variable – it stands for created
         // transaction during execution, but as said above, transaction is committed
         // for us and `ydb-go-sdk` do not want to do anything with it.
         _, res, err := s.Execute(ctx, txc,
            `--!syntax_v1
             DECLARE $mystr AS Utf8?;
             SELECT 42 as id, $mystr as mystr
             `,
            table.NewQueryParameters(
               table.ValueParam("$mystr", types.OptionalValue(types.UTF8Value("test"))),
            ),
         )
         return err
      },
   )
   if err != nil {
       return err // handle error
   }
   defer func() {
      _ = res.Close()
   }()
   // Scan for received values within the result set(s).
   // res.Err() reports the reason of last unsuccessful one.
   var (
       id    int32
       myStr *string //optional value
   )
   for res.NextResultSet(ctx, "id", "mystr") {
       for res.NextRow() {
           // Suppose our "users" table has two rows: id and age.
           // Thus, current row will contain two appropriate items with
           // exactly the same order.
           err := res.Scan(&id, &myStr)
   
           // Error handling.
           if err != nil {
               return err
           }
           // do something with data
           fmt.Printf("got id %v, got mystr: %v\n", id, *myStr)
       }
   }
   if res.Err() != nil {
       return res.Err() // handle error
   }

YDB sessions may become staled and appropriate error will be returned. To reduce boilerplate overhead for such cases ydb-go-sdk provides generic retry logic

Credentials

There are different variants to get credentials.Credentials object to get authorized.

Package Type Description Link of example usage
ydb-go-yc credentials credentials provider for Yandex.Cloud yc.WithServiceAccountKeyFileCredentials yc.WithInternalCA yc.WithMetadataCredentials
ydb-go-yc-metadata credentials metadata credentials provider for Yandex.Cloud yc.WithInternalCA yc.WithCredentials
ydb-go-sdk-auth-environ credentials create credentials from environ ydbEnviron. WithEnvironCredentials

Usage examples can be found here.

Environment variables

Name Type Default Description
YDB_SSL_ROOT_CERTIFICATES_FILE string path to certificates file
YDB_LOG_SEVERITY_LEVEL string quiet severity logging level. Supported: trace, debug, info, warn, error, fatal, quiet
YDB_LOG_NO_COLOR bool false set any non empty value to disable colouring logs
GRPC_GO_LOG_VERBOSITY_LEVEL integer set to 99 to see grpc logs
GRPC_GO_LOG_SEVERITY_LEVEL string set to info to see grpc logs

Ecosystem of debug tools over ydb-go-sdk

Package ydb-go-sdk provide debugging over trace events in package trace. Now supports driver events in trace.Driver struct and table-service events in trace.Table struct. Next packages provide debug tooling:

Package Type Description Link of example usage
ydb-go-sdk-zap logging logging ydb-go-sdk events with zap package ydbZap.WithTraces
ydb-go-sdk-zerolog logging logging ydb-go-sdk events with zerolog package ydbZerolog.WithTraces
ydb-go-sdk-metrics metrics common metrics of ydb-go-sdk. Package declare interfaces such as Registry, GaugeVec and Gauge and use it for create trace.Driver and trace.Table traces
ydb-go-sdk-prometheus metrics prometheus wrapper over ydb-go-sdk-metrics ydbPrometheus.WithTraces
ydb-go-sdk-opentracing tracing opentracing plugin for trace internal ydb-go-sdk calls ydbOpentracing.WithTraces

Examples

More examples are listed in examples repository.

Documentation

Index

Constants

View Source 
const Version = meta.VersionMajor + "." + meta.VersionMinor + "." + meta.VersionPatch

Version alias for except cycle import

Variables

This section is empty.

Functions

func IsOperationError 

func IsOperationError(err error, codes ...Ydb.StatusIds_StatusCode) bool

func IsOperationErrorAlreadyExistsError added in v3.5.0 

func IsOperationErrorAlreadyExistsError(err error) bool

func IsOperationErrorNotFoundError added in v3.5.0 

func IsOperationErrorNotFoundError(err error) bool

func IsOperationErrorOverloaded added in v3.5.0 

func IsOperationErrorOverloaded(err error) bool

func IsOperationErrorSchemeError added in v3.5.0 

func IsOperationErrorSchemeError(err error) bool

func IsOperationErrorUnavailable added in v3.5.0 

func IsOperationErrorUnavailable(err error) bool

func IsRatelimiterAcquireError added in v3.11.0 

func IsRatelimiterAcquireError(err error) bool

func IsTimeoutError 

func IsTimeoutError(err error) bool

func IsTransportError 

func IsTransportError(err error, codes ...grpcCodes.Code) bool

func IsYdbError added in v3.4.2 

func IsYdbError(err error) bool

func IterateByIssues 

func IterateByIssues(err error, it func(message string, code Ydb.StatusIds_StatusCode, severity uint32))

func RegisterParser added in v3.9.0 

func RegisterParser(param string, parser func(value string) ([]config.Option, error)) (err error)

func ToRatelimiterAcquireError added in v3.11.0 

func ToRatelimiterAcquireError(err error) ratelimiter.AcquireError

func WithOperationCancelAfter added in v3.18.0 

func WithOperationCancelAfter(ctx context.Context, operationCancelAfter time.Duration) context.Context

WithOperationCancelAfter returns a copy of parent context in which YDB operation cancel after parameter is set to d. If parent context cancellation timeout is smaller than d, parent context is returned.

func WithOperationTimeout added in v3.18.0 

func WithOperationTimeout(ctx context.Context, operationTimeout time.Duration) context.Context

WithOperationTimeout returns a copy of parent context in which YDB operation timeout parameter is set to d. If parent context timeout is smaller than d, parent context is returned.

func WithRequestType added in v3.13.0 

func WithRequestType(ctx context.Context, requestType string) context.Context

WithRequestType returns a copy of parent context with custom request type

func WithTraceID added in v3.13.0 

func WithTraceID(ctx context.Context, traceID string) context.Context

WithTraceID returns a copy of parent context with traceID

Types

type Connection 

type Connection interface {
	closer.Closer
	database.Info
	grpc.ClientConnInterface

	// Table returns table client
	Table() table.Client

	// Scheme returns scheme client
	Scheme() scheme.Client

	// Coordination returns coordination client
	Coordination() coordination.Client

	// Ratelimiter returns rate limiter client
	Ratelimiter() ratelimiter.Client

	// Discovery returns discovery client
	Discovery() discovery.Client

	// Scripting returns scripting client
	Scripting() scripting.Client

	// With returns Connection specified with custom options
	// Options provide options replacement for all clients taked from new Connection
	With(ctx context.Context, opts ...Option) (Connection, error)
}

Connection interface provide access to YDB service clients Interface and list of clients may be changed in the future

func New 

func New(ctx context.Context, opts ...Option) (_ Connection, err error)

New connects to name and return name runtime holder

type Error added in v3.4.2 

type Error xerrors.Error

func OperationError added in v3.16.9 

func OperationError(err error) Error

func TransportError added in v3.16.9 

func TransportError(err error) Error

type LoggerOption added in v3.5.3 

type LoggerOption logger.Option

func WithErrWriter added in v3.5.0 

func WithErrWriter(err io.Writer) LoggerOption

func WithExternalLogger added in v3.5.0 

func WithExternalLogger(external log.Logger) LoggerOption

func WithMinLevel added in v3.3.0 

func WithMinLevel(minLevel log.Level) LoggerOption

func WithNamespace added in v3.3.0 

func WithNamespace(namespace string) LoggerOption

func WithNoColor added in v3.3.0 

func WithNoColor(b bool) LoggerOption

func WithOutWriter added in v3.5.0 

func WithOutWriter(out io.Writer) LoggerOption

type Option 

type Option func(ctx context.Context, c *connection) error

func MergeOptions added in v3.5.1 

func MergeOptions(opts ...Option) Option

func With 

func With(options ...config.Option) Option

func WithAccessTokenCredentials 

func WithAccessTokenCredentials(accessToken string) Option

func WithAnonymousCredentials 

func WithAnonymousCredentials() Option

func WithBalancer added in v3.6.0 

func WithBalancer(balancer balancer.Balancer) Option

func WithCertificate 

func WithCertificate(cert *x509.Certificate) Option

func WithCertificatesFromFile 

func WithCertificatesFromFile(caFile string) Option

func WithCertificatesFromPem 

func WithCertificatesFromPem(bytes []byte) Option

func WithConnectionString 

func WithConnectionString(connectionString string) Option

WithConnectionString accept connection string like 'grpc[s]://{endpoint}/?database={database}'

func WithConnectionTTL added in v3.7.0 

func WithConnectionTTL(ttl time.Duration) Option

WithConnectionTTL defines duration for parking idle connections Warning: if defined WithSessionPoolIdleThreshold - idleThreshold must be less than connectionTTL

func WithCreateCredentialsFunc 

func WithCreateCredentialsFunc(createCredentials func(ctx context.Context) (credentials.Credentials, error)) Option

func WithCredentials 

func WithCredentials(c credentials.Credentials) Option

WithCredentials in conjunction with Connection.With function prohibit reuse of conn pool. Thus, Connection.With will effectively create totally separate Connection.

func WithDatabase added in v3.2.1 

func WithDatabase(database string) Option

func WithDialTimeout 

func WithDialTimeout(timeout time.Duration) Option

func WithDiscoveryInterval 

func WithDiscoveryInterval(discoveryInterval time.Duration) Option

func WithEndpoint added in v3.2.1 

func WithEndpoint(endpoint string) Option

func WithInsecure added in v3.8.6 

func WithInsecure() Option

func WithLogger added in v3.3.0 

func WithLogger(details trace.Details, opts ...LoggerOption) Option

func WithMinTLSVersion added in v3.9.1 

func WithMinTLSVersion(minVersion uint16) Option

func WithPanicCallback added in v3.17.0 

func WithPanicCallback(panicCallback func(e interface{})) Option

WithPanicCallback specified behavior on panic Warning: WithPanicCallback must be defined on start of all options (before `WithTrace{Driver,Table,Scheme,Scripting,Coordination,Ratelimiter}` and other options) If not defined - panic would not intercept with driver

func WithRatelimiterOptions added in v3.11.0 

func WithRatelimiterOptions(opts ...ratelimiterConfig.Option) Option

func WithRequestsType added in v3.13.0 

func WithRequestsType(requestsType string) Option

func WithSecure added in v3.7.0 

func WithSecure(secure bool) Option

func WithSessionPoolCreateSessionTimeout 

func WithSessionPoolCreateSessionTimeout(createSessionTimeout time.Duration) Option

func WithSessionPoolDeleteTimeout 

func WithSessionPoolDeleteTimeout(deleteTimeout time.Duration) Option

func WithSessionPoolIdleThreshold 

func WithSessionPoolIdleThreshold(idleThreshold time.Duration) Option

WithSessionPoolIdleThreshold defines keep-alive interval for idle sessions Warning: if defined WithConnectionTTL - idleThreshold must be less than connectionTTL

func WithSessionPoolKeepAliveMinSize 

func WithSessionPoolKeepAliveMinSize(keepAliveMinSize int) Option

func WithSessionPoolKeepAliveTimeout 

func WithSessionPoolKeepAliveTimeout(keepAliveTimeout time.Duration) Option

func WithSessionPoolSizeLimit 

func WithSessionPoolSizeLimit(sizeLimit int) Option

func WithTLSSInsecureSkipVerify added in v3.11.0 

func WithTLSSInsecureSkipVerify() Option

func WithTableConfigOption 

func WithTableConfigOption(option tableConfig.Option) Option

func WithTraceCoordination added in v3.10.0 

func WithTraceCoordination(t trace.Coordination, opts ...trace.CoordinationComposeOption) Option

WithTraceCoordination returns coordination trace option

func WithTraceDiscovery added in v3.10.0 

func WithTraceDiscovery(t trace.Discovery, opts ...trace.DiscoveryComposeOption) Option

WithTraceDiscovery returns discovery trace option

func WithTraceDriver 

func WithTraceDriver(trace trace.Driver, opts ...trace.DriverComposeOption) Option

WithTraceDriver returns deadline which has associated Driver with it.

func WithTraceRatelimiter added in v3.10.0 

func WithTraceRatelimiter(t trace.Ratelimiter, opts ...trace.RatelimiterComposeOption) Option

WithTraceRatelimiter returns ratelimiter trace option

func WithTraceScheme added in v3.10.0 

func WithTraceScheme(t trace.Scheme, opts ...trace.SchemeComposeOption) Option

WithTraceScheme returns scheme trace option

func WithTraceScripting added in v3.10.0 

func WithTraceScripting(t trace.Scripting, opts ...trace.ScriptingComposeOption) Option

WithTraceScripting scripting trace option

func WithTraceTable 

func WithTraceTable(t trace.Table, opts ...trace.TableComposeOption) Option

WithTraceTable returns table trace option

func WithUserAgent added in v3.7.0 

func WithUserAgent(userAgent string) Option

Source Files

View all Source files

Directories

Path Synopsis
config
config
internal
balancer
balancer/list
balancer/multi
balancer/rr
balancer/single
balancer/stub
closer
cluster
cluster/entry
cmd/gtrace
conn
coordination
credentials
database
deadline
decimal
Package decimal provides tools for working with YDB's decimal types.
 Package decimal provides tools for working with YDB's decimal types.
discovery
dsn
endpoint
feature
lazy
logger
logger/level
meta
operation
ratelimiter
ratelimiter/errors
ratelimiter/options
repeater
response
scheme
scripting
table
table/scanner
timeutil
value
wrap
xerrors
xnet
xrand
xresolver
config
config
config
config
options
result
result/indexed
result/named
stats
types
timeutil
timeutil/timetest

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.