ydb

package module
v3.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 21, 2022 License: Apache-2.0 Imports: 38 Imported by: 86

README

ydb

PkgGoDev GoDoc tests lint Go Report Card codecov codecov

YDB API client written in Go.

godoc

Table of contents

  1. Overview
  2. About semantic versioning
  3. About expected go1.18 changes
  4. Prerequisites
  5. Installation
  6. Usage
  7. Credentials
  8. Environment variables
  9. Ecosystem of debug tools
  10. Examples

Overview

Currently package ydb provides scheme and table API client implementations for YDB.

About semantic versioning

We follow the SemVer 2.0.0. In particular, we provide backward compatibility in the MAJOR releases. New features without loss of backward compatibility appear on the MINOR release. In the minor version, the patch number starts from 0. Bug fixes and internal changes are released with the third digit (PATCH) in the version.

There are, however, some changes with the loss of backward compatibility that we consider to be MINOR:

  • extension or modification of internal ydb-go-sdk interfaces. We understand that this will break the compatibility of custom implementations of the ydb-go-sdk internal interfaces. But we believe that the internal interfaces of ydb-go-sdk are implemented well enough that they do not require custom implementation. We are working to ensure that all internal interfaces have limited access only inside ydb-go-sdk.
  • major changes to (including removal of) the public interfaces and types that have been previously exported by ydb-go-sdk. We understand that these changes will break the backward compatibility of early adopters of these interfaces. However, these changes are generally coordinated with early adopters and have the concise interfacing with ydb-go-sdk as a goal.

Internal interfaces outside from internal directory are marked with comment such as

// Warning: only for internal usage inside ydb-go-sdk

We publish the planned breaking MAJOR changes:

  • via the comment Deprecated in the code indicating what should be used instead
  • through the file NEXT_MAJOR_RELEASE.md

About expected go1.18 changes

Some changes from go1.18 are expected and will be allowed to ydb-go-sdk:

  • type set constraints instead abstract interface{}. Firstly, this changes will be applied to scanner API (Scan, ScanWithDefaults and ScanNamed)
  • improve log package with parametrized types.

That changes could break backward compatibility. We are tried to support go1.18 features seamlessly. But now we was tested go1.18-beta implementation and detected some strong issues:

  • go version in go.mod must be forced up to 1.18,
  • interfaces with methods are not supported in type sets and other.

Prerequisites

Requires go1.13 or later.

Installation 

go get -u github.com/ydb-platform/ydb-go-sdk/v3

Usage

The straightforward example of querying data may look similar to this:

   ctx := context.Background()

   // ydb.New() returns connection object which provide necessary clients for different ydb services
   // such as table.Client, scheme.Client, coordination.Client, etc.
   db, err := ydb.New(
      ctx,
      ydb.WithConnectionString(os.Getenv("YDB_CONNECTION_STRING")),
      ydb.WithDialTimeout(3 * time.Second),
      ydb.WithCertificatesFromFile("~/.ydb/CA.pem"),
      ydb.WithSessionPoolIdleThreshold(time.Second * 5),
      ydb.WithSessionPoolKeepAliveMinSize(-1),
      ydb.WithDiscoveryInterval(5 * time.Second),
      ydb.WithAccessTokenCredentials(os.GetEnv("YDB_ACCESS_TOKEN_CREDENTIALS")),
   )
   if err != nil {
      // handle error
   }
   defer func() { _ = db.Close(ctx) }()

   // Prepare transaction control for upcoming query execution.
   // NOTE: result of TxControl() may be reused.
   txc := table.TxControl(
      table.BeginTx(table.WithSerializableReadWrite()),
      table.CommitTx(),
   )

   var res resultset.Result

   // Do() provide the best effort for executing operation
   // Do implements internal busy loop until one of the following conditions occurs:
   // - deadline was cancelled or deadlined
   // - operation returned nil as error
   // Note that in case of prepared statements call to Prepare() must be made
   // inside the function body.
   err := db.Table().Do(
      ctx, 
      func(ctx context.Context, s table.Session) (err error) {
         // Execute text query without preparation and with given "autocommit"
         // transaction control. That is, transaction will be committed without
         // additional calls. Notice the "_" unused variable – it stands for created
         // transaction during execution, but as said above, transaction is committed
         // for us and `ydb-go-sdk` do not want to do anything with it.
         _, res, err := s.Execute(ctx, txc,
            `--!syntax_v1
             DECLARE $mystr AS Utf8?;
             SELECT 42 as id, $mystr as mystr
             `,
            table.NewQueryParameters(
               table.ValueParam("$mystr", types.OptionalValue(types.UTF8Value("test"))),
            ),
         )
         return err
      },
   )
   if err != nil {
       return err // handle error
   }
   defer func() {
      _ = res.Close()
   }()
   // Scan for received values within the result set(s).
   // res.Err() reports the reason of last unsuccessful one.
   var (
       id    int32
       myStr *string //optional value
   )
   for res.NextResultSet(ctx, "id", "mystr") {
       for res.NextRow() {
           // Suppose our "users" table has two rows: id and age.
           // Thus, current row will contain two appropriate items with
           // exactly the same order.
           err := res.Scan(&id, &myStr)
   
           // Error handling.
           if err != nil {
               return err
           }
           // do something with data
           fmt.Printf("got id %v, got mystr: %v\n", id, *myStr)
       }
   }
   if res.Err() != nil {
       return res.Err() // handle error
   }

YDB sessions may become staled and appropriate error will be returned. To reduce boilerplate overhead for such cases ydb-go-sdk provides generic retry logic

Credentials

There are different variants to get ydb.Credentials object to get authorized. Spatial cases of credentials for yandex-cloud provides with package ydb-go-yc Package ydb-go-sdk-auth-environ provide different credentials depending on the environment variables. Usage examples can be found here.

Environment variables

Name Type Default Description
YDB_SSL_ROOT_CERTIFICATES_FILE string path to certificates file
YDB_LOG_SEVERITY_LEVEL string quiet severity logging level. Supported: trace, debug, info, warn, error, fatal, quiet
YDB_LOG_NO_COLOR bool false set any non empty value to disable colouring logs
GRPC_GO_LOG_VERBOSITY_LEVEL integer set to 99 to see grpc logs
GRPC_GO_LOG_SEVERITY_LEVEL string set to info to see grpc logs

Ecosystem of debug tools over ydb-go-sdk

Package ydb-go-sdk provide debugging over trace events in package trace. Now supports driver events in trace.Driver struct and table-service events in trace.Table struct. Next packages provide debug tooling:

Package Type Description Link of example usage
ydb-go-sdk-zap logging logging ydb-go-sdk events with zap package trace.Driver trace.Table
ydb-go-sdk-zerolog logging logging ydb-go-sdk events with zerolog package trace.Driver trace.Table
ydb-go-sdk-metrics metrics common metrics of ydb-go-sdk. Package declare interfaces such as Registry, GaugeVec and Gauge and use it for create trace.Driver and trace.Table traces
ydb-go-sdk-prometheus metrics metrics of ydb-go-sdk. Package use ydb-go-sdk-metrics package and prometheus types for create trace.Driver and trace.Table traces trace.Driver trace.Table
ydb-go-sdk-opentracing tracing opentracing plugin for trace internal ydb-go-sdk calls trace.Driver trace.Table

Examples

More examples are listed in examples repository.

Documentation

Index

Constants

View Source 
const (
	MetaRequestType = meta.HeaderRequestType
	MetaTraceID     = meta.HeaderTraceID
)
View Source 
const (
	QUIET = Level(logger.QUIET)
	TRACE = Level(logger.TRACE)
	DEBUG = Level(logger.DEBUG)
	INFO  = Level(logger.INFO)
	WARN  = Level(logger.WARN)
	ERROR = Level(logger.ERROR)
	FATAL = Level(logger.FATAL)
)
View Source 
const Version = meta.Version

Version alias for except cycle import

Variables

This section is empty.

Functions

func IsOperationError 

func IsOperationError(err error) bool

func IsOperationErrorAlreadyExistsError added in v3.5.0 

func IsOperationErrorAlreadyExistsError(err error) bool

func IsOperationErrorCode added in v3.5.0 

func IsOperationErrorCode(err error, codes ...int32) bool

func IsOperationErrorNotFoundError added in v3.5.0 

func IsOperationErrorNotFoundError(err error) bool

func IsOperationErrorOverloaded added in v3.5.0 

func IsOperationErrorOverloaded(err error) bool

func IsOperationErrorSchemeError added in v3.5.0 

func IsOperationErrorSchemeError(err error) bool

func IsOperationErrorUnavailable added in v3.5.0 

func IsOperationErrorUnavailable(err error) bool

func IsRatelimiterAcquireError added in v3.11.0 

func IsRatelimiterAcquireError(err error) bool

func IsTimeoutError 

func IsTimeoutError(err error) bool

func IsTransportError 

func IsTransportError(err error) bool

func IsTransportErrorCancelled added in v3.4.3 

func IsTransportErrorCancelled(err error) bool

func IsTransportErrorCode added in v3.5.0 

func IsTransportErrorCode(err error, codes ...int32) bool

func IsTransportErrorResourceExhausted added in v3.5.0 

func IsTransportErrorResourceExhausted(err error) bool

func IsYdbError added in v3.4.2 

func IsYdbError(err error) bool

func IterateByIssues 

func IterateByIssues(err error, it func(message string, code uint32, severity uint32))

func RegisterParser added in v3.9.0 

func RegisterParser(param string, parser func(value string) ([]config.Option, error)) (err error)

func ToRatelimiterAcquireError added in v3.11.0 

func ToRatelimiterAcquireError(err error) ratelimiter.AcquireError

Types

type Connection 

type Connection interface {
	closer.Closer
	db.ConnectionInfo

	// Table returns table client with options from Connection instance.
	// Options provide options replacement for requested table client
	// such as database and access token
	Table(opts ...CustomOption) table.Client

	// Scheme returns scheme client with options from Connection instance.
	// Options provide options replacement for requested scheme client
	// such as database and access token
	Scheme(opts ...CustomOption) scheme.Client

	// Coordination returns coordination client with options from Connection instance.
	// Options provide options replacement for requested coordination client
	// such as database and access token
	Coordination(opts ...CustomOption) coordination.Client

	// Ratelimiter returns rate limiter client with options from Connection instance.
	// Options provide options replacement for requested rate limiter client
	// such as database and access token
	Ratelimiter(opts ...CustomOption) ratelimiter.Client

	// Discovery returns discovery client with options from Connection instance.
	// Options provide options replacement for requested discovery client
	// such as database and access token
	Discovery(opts ...CustomOption) discovery.Client

	// Scripting returns scripting client with options from Connection instance.
	// Options provide options replacement for requested scripting client
	// such as database and access token
	Scripting(opts ...CustomOption) scripting.Client

	// With returns Connection specified with custom options
	// Options provide options replacement for all clients taked from new Connection
	With(opts ...CustomOption) Connection
}

Connection interface provide access to YDB service clients Interface and list of clients may be changed in the future

func New 

func New(ctx context.Context, opts ...Option) (_ Connection, err error)

New connects to name and return name runtime holder

type CustomOption added in v3.7.0 

type CustomOption func(opts *customOptions)

func WithCustomCredentials added in v3.11.0 

func WithCustomCredentials(creds credentials.Credentials) CustomOption

func WithCustomDatabase added in v3.7.0 

func WithCustomDatabase(database string) CustomOption

func WithCustomToken added in v3.7.0 

func WithCustomToken(accessToken string) CustomOption

func WithCustomUserAgent added in v3.7.0 

func WithCustomUserAgent(userAgent string) CustomOption

type Error added in v3.4.2 

type Error interface {
	error

	Code() int32
	Name() string
}

func OperationErrorDescription added in v3.4.2 

func OperationErrorDescription(err error) Error

func TransportErrorDescription added in v3.4.2 

func TransportErrorDescription(err error) Error

type Level added in v3.5.0 

type Level logger.Level

type LoggerOption added in v3.5.3 

type LoggerOption logger.Option

func WithErrWriter added in v3.5.0 

func WithErrWriter(err io.Writer) LoggerOption

func WithExternalLogger added in v3.5.0 

func WithExternalLogger(external log.Logger) LoggerOption

func WithMinLevel added in v3.3.0 

func WithMinLevel(minLevel Level) LoggerOption

func WithNamespace added in v3.3.0 

func WithNamespace(namespace string) LoggerOption

func WithNoColor added in v3.3.0 

func WithNoColor(b bool) LoggerOption

func WithOutWriter added in v3.5.0 

func WithOutWriter(out io.Writer) LoggerOption

type Option 

type Option func(ctx context.Context, c *connection) error

func MergeOptions added in v3.5.1 

func MergeOptions(options ...Option) Option

func With 

func With(options ...config.Option) Option

func WithAccessTokenCredentials 

func WithAccessTokenCredentials(accessToken string) Option

func WithAnonymousCredentials 

func WithAnonymousCredentials() Option

func WithBalancer added in v3.6.0 

func WithBalancer(balancer balancer.Balancer) Option

func WithCertificate 

func WithCertificate(cert *x509.Certificate) Option

func WithCertificatesFromFile 

func WithCertificatesFromFile(caFile string) Option

func WithCertificatesFromPem 

func WithCertificatesFromPem(bytes []byte) Option

func WithConnectionString 

func WithConnectionString(connectionString string) Option

WithConnectionString accept connection string like 'grpc[s]://{endpoint}/?database={database}'

func WithConnectionTTL added in v3.7.0 

func WithConnectionTTL(ttl time.Duration) Option

func WithCreateCredentialsFunc 

func WithCreateCredentialsFunc(createCredentials func(ctx context.Context) (credentials.Credentials, error)) Option

func WithCredentials 

func WithCredentials(c credentials.Credentials) Option

func WithDatabase added in v3.2.1 

func WithDatabase(database string) Option

func WithDialTimeout 

func WithDialTimeout(timeout time.Duration) Option

func WithDiscoveryInterval 

func WithDiscoveryInterval(discoveryInterval time.Duration) Option

func WithEndpoint added in v3.2.1 

func WithEndpoint(endpoint string) Option

func WithInsecure added in v3.8.6 

func WithInsecure() Option

func WithLogger added in v3.3.0 

func WithLogger(details trace.Details, opts ...LoggerOption) Option

func WithMinTLSVersion added in v3.9.1 

func WithMinTLSVersion(minVersion uint16) Option

func WithRatelimiterOptions added in v3.11.0 

func WithRatelimiterOptions(opts ...ratelimiterConfig.Option) Option

func WithSecure added in v3.7.0 

func WithSecure(secure bool) Option

func WithSessionPoolCreateSessionTimeout 

func WithSessionPoolCreateSessionTimeout(createSessionTimeout time.Duration) Option

func WithSessionPoolDeleteTimeout 

func WithSessionPoolDeleteTimeout(deleteTimeout time.Duration) Option

func WithSessionPoolIdleThreshold 

func WithSessionPoolIdleThreshold(idleThreshold time.Duration) Option

func WithSessionPoolKeepAliveMinSize 

func WithSessionPoolKeepAliveMinSize(keepAliveMinSize int) Option

func WithSessionPoolKeepAliveTimeout 

func WithSessionPoolKeepAliveTimeout(keepAliveTimeout time.Duration) Option

func WithSessionPoolSizeLimit 

func WithSessionPoolSizeLimit(sizeLimit int) Option

func WithTLSSInsecureSkipVerify added in v3.11.0 

func WithTLSSInsecureSkipVerify() Option

func WithTableConfigOption 

func WithTableConfigOption(option tableConfig.Option) Option

func WithTraceCoordination added in v3.10.0 

func WithTraceCoordination(trace trace.Coordination) Option

WithTraceCoordination returns coordination trace option

func WithTraceDiscovery added in v3.10.0 

func WithTraceDiscovery(trace trace.Discovery) Option

WithTraceDiscovery returns discovery trace option

func WithTraceDriver 

func WithTraceDriver(trace trace.Driver) Option

WithTraceDriver returns deadline which has associated Driver with it.

func WithTraceRatelimiter added in v3.10.0 

func WithTraceRatelimiter(trace trace.Ratelimiter) Option

WithTraceRatelimiter returns ratelimiter trace option

func WithTraceScheme added in v3.10.0 

func WithTraceScheme(trace trace.Scheme) Option

WithTraceScheme returns scheme trace option

func WithTraceScripting added in v3.10.0 

func WithTraceScripting(trace trace.Scripting) Option

WithTraceScripting scripting trace option

func WithTraceTable 

func WithTraceTable(trace trace.Table) Option

WithTraceTable returns table trace option

func WithUserAgent added in v3.7.0 

func WithUserAgent(userAgent string) Option

Source Files

View all Source files

Directories

Path Synopsis
config
config
internal
balancer
balancer/list
balancer/multi
balancer/rr
balancer/single
balancer/stub
closer
cluster
cluster/entry
conn
coordination
credentials
db
deadline
decimal
Package decimal provides tools for working with YDB's decimal types.
 Package decimal provides tools for working with YDB's decimal types.
discovery
dsn
endpoint
endpoint/info
errors
feature
lazy
logger
meta
net
operation
proxy
rand
ratelimiter
ratelimiter/errors
ratelimiter/options
repeater
resolver
response
scheme
scripting
table
table/scanner
timeutil
value
wrap
config
config
config
config
options
result
result/indexed
result/named
stats
types
timeutil
timeutil/timetest

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.