sqlccl

Documentation

Overview

Package sqlccl is a generated protocol buffer package.

It is generated from these files:

cockroach/pkg/ccl/sqlccl/backup.proto

It has these top-level messages:

BackupRangeDescriptor
BackupDescriptor

Index

• Variables
• func AllRangeDescriptors(txn *client.Txn) ([]roachpb.RangeDescriptor, error)
• func Ingest(ctx context.Context, txn *client.Txn, path string, checksum uint32, ...) error
• func IntersectHalfOpen(start1, end1, start2, end2 []byte) ([]byte, []byte)
• func MakeKeyRewriterForNewTableID(desc *sqlbase.TableDescriptor, newTableID sqlbase.ID) storageccl.KeyRewriter
• func MakeKeyRewriterForNewTableIDs(tables []*sqlbase.TableDescriptor, newTableIDs map[sqlbase.ID]sqlbase.ID) (storageccl.KeyRewriter, error)
• func MakeRekeyMVCCKeyValFunc(newTableID sqlbase.ID, f func(kv engine.MVCCKeyValue) (bool, error)) func(engine.MVCCKeyValue) (bool, error)
• func Restore(ctx context.Context, db client.DB, base string, table parser.TableName) ([]sqlbase.TableDescriptor, error)
• type BackupDescriptor
  • func Backup(ctx context.Context, db client.DB, base string, endTime hlc.Timestamp) (desc BackupDescriptor, retErr error)
  • func (*BackupDescriptor) Descriptor() ([]byte, []int)
  • func (m *BackupDescriptor) Marshal() (dAtA []byte, err error)
  • func (m *BackupDescriptor) MarshalTo(dAtA []byte) (int, error)
  • func (*BackupDescriptor) ProtoMessage()
  • func (m *BackupDescriptor) Reset()
  • func (m *BackupDescriptor) Size() (n int)
  • func (m *BackupDescriptor) String() string
  • func (m *BackupDescriptor) Unmarshal(dAtA []byte) error
• type BackupRangeDescriptor
  • func (*BackupRangeDescriptor) Descriptor() ([]byte, []int)
  • func (m *BackupRangeDescriptor) Marshal() (dAtA []byte, err error)
  • func (m *BackupRangeDescriptor) MarshalTo(dAtA []byte) (int, error)
  • func (*BackupRangeDescriptor) ProtoMessage()
  • func (m *BackupRangeDescriptor) Reset()
  • func (m *BackupRangeDescriptor) Size() (n int)
  • func (m *BackupRangeDescriptor) String() string
  • func (m *BackupRangeDescriptor) Unmarshal(dAtA []byte) error

Variables

var (
	ErrInvalidLengthBackup = fmt.Errorf("proto: negative length found during unmarshaling")
	ErrIntOverflowBackup   = fmt.Errorf("proto: integer overflow")
)

Functions

func AllRangeDescriptors

func AllRangeDescriptors(txn *client.Txn) ([]roachpb.RangeDescriptor, error)

AllRangeDescriptors fetches all meta2 RangeDescriptor using the given txn.

func Ingest

func Ingest(
	ctx context.Context,
	txn *client.Txn,
	path string,
	checksum uint32,
	startKey, endKey roachpb.Key,
	newTableID sqlbase.ID,
) error

Ingest loads some data in an sstable into an empty range. Only the keys between startKey and endKey are loaded. If newTableID is non-zero, every row's key is rewritten to be for that table.

func IntersectHalfOpen

func IntersectHalfOpen(start1, end1, start2, end2 []byte) ([]byte, []byte)

IntersectHalfOpen returns the common range between two key intervals or (nil, nil) if there is no common range. Exported for testing.

func MakeKeyRewriterForNewTableID

func MakeKeyRewriterForNewTableID(
	desc *sqlbase.TableDescriptor, newTableID sqlbase.ID,
) storageccl.KeyRewriter

MakeKeyRewriterForNewTableID creates a KeyRewriter that rewrites all keys from a table to have a new tableID. For dependency reasons, the implementation of the matching is in storageccl, but the interesting constructor is here.

func MakeKeyRewriterForNewTableIDs

func MakeKeyRewriterForNewTableIDs(
	tables []*sqlbase.TableDescriptor, newTableIDs map[sqlbase.ID]sqlbase.ID,
) (storageccl.KeyRewriter, error)

MakeKeyRewriterForNewTableIDs creates a KeyRewriter that rewrites all keys from a set of tables to have a new tableID. For dependency reasons, the implementation of the matching is in storageccl, but the interesting constructor is here.

func MakeRekeyMVCCKeyValFunc

func MakeRekeyMVCCKeyValFunc(
	newTableID sqlbase.ID, f func(kv engine.MVCCKeyValue) (bool, error),
) func(engine.MVCCKeyValue) (bool, error)

MakeRekeyMVCCKeyValFunc takes an iterator function for MVCCKeyValues and returns a new iterator function where the keys are rewritten inline to the have the given table ID.

func Restore

func Restore(
	ctx context.Context, db client.DB, base string, table parser.TableName,
) ([]sqlbase.TableDescriptor, error)

Restore imports a SQL table (or tables) from a set of non-overlapping sstable files.

Types

type BackupDescriptor

type BackupDescriptor struct {
	EndTime cockroach_util_hlc.Timestamp        `protobuf:"bytes,1,opt,name=end_time,json=endTime" json:"end_time"`
	Ranges  []BackupRangeDescriptor             `protobuf:"bytes,2,rep,name=ranges" json:"ranges"`
	SQL     []cockroach_sql_sqlbase1.Descriptor `protobuf:"bytes,3,rep,name=sql" json:"sql"`
	// TODO(dan): Consider also including total file size and per-range data and
	// file size.
	DataSize int64 `protobuf:"varint,4,opt,name=data_size,json=dataSize,proto3" json:"data_size,omitempty"`
}

BackupDescriptor represents a consistent snapshot of ranges.

Each range snapshot includes a path to data that is a diff of the data in that key range between a start and end timestamp. The end timestamp of all ranges in a backup is the same, but the start may vary (to allow individual tables to be backed up on different schedules).

func Backup

func Backup(
	ctx context.Context, db client.DB, base string, endTime hlc.Timestamp,
) (desc BackupDescriptor, retErr error)

Backup exports a snapshot of every kv entry into ranged sstables.

The output is an sstable per range with files in the following locations: - /<base>/<node_id>/<key_range>/data.sst - <base> is given by the user and is expected to eventually be cloud storage - The <key_range>s are non-overlapping.

TODO(dan): Bikeshed this directory structure and naming.

func (*BackupDescriptor) Descriptor

func (*BackupDescriptor) Descriptor() ([]byte, []int)

func (*BackupDescriptor) Marshal

func (m *BackupDescriptor) Marshal() (dAtA []byte, err error)

func (*BackupDescriptor) MarshalTo

func (m *BackupDescriptor) MarshalTo(dAtA []byte) (int, error)

func (*BackupDescriptor) ProtoMessage

func (*BackupDescriptor) ProtoMessage()

func (*BackupDescriptor) Reset

func (m *BackupDescriptor) Reset()

func (*BackupDescriptor) Size

func (m *BackupDescriptor) Size() (n int)

func (*BackupDescriptor) String

func (m *BackupDescriptor) String() string

func (*BackupDescriptor) Unmarshal

func (m *BackupDescriptor) Unmarshal(dAtA []byte) error

type BackupRangeDescriptor

type BackupRangeDescriptor struct {
	// An empty path means the range is empty.
	Path      string                                           `protobuf:"bytes,1,opt,name=path,proto3" json:"path,omitempty"`
	StartKey  github_com_cockroachdb_cockroach_pkg_roachpb.Key `` /* 143-byte string literal not displayed */
	EndKey    github_com_cockroachdb_cockroach_pkg_roachpb.Key `` /* 137-byte string literal not displayed */
	StartTime cockroach_util_hlc.Timestamp                     `protobuf:"bytes,4,opt,name=start_time,json=startTime" json:"start_time"`
	CRC       uint32                                           `protobuf:"varint,5,opt,name=crc,proto3" json:"crc,omitempty"`
}

BackupRangeDescriptor represents a file that contains the diff for a key range between two timestamps.

func (*BackupRangeDescriptor) Descriptor

func (*BackupRangeDescriptor) Descriptor() ([]byte, []int)

func (*BackupRangeDescriptor) Marshal

func (m *BackupRangeDescriptor) Marshal() (dAtA []byte, err error)

func (*BackupRangeDescriptor) MarshalTo

func (m *BackupRangeDescriptor) MarshalTo(dAtA []byte) (int, error)

func (*BackupRangeDescriptor) ProtoMessage

func (*BackupRangeDescriptor) ProtoMessage()

func (*BackupRangeDescriptor) Reset

func (m *BackupRangeDescriptor) Reset()

func (*BackupRangeDescriptor) Size

func (m *BackupRangeDescriptor) Size() (n int)

func (*BackupRangeDescriptor) String

func (m *BackupRangeDescriptor) String() string

func (*BackupRangeDescriptor) Unmarshal

func (m *BackupRangeDescriptor) Unmarshal(dAtA []byte) error