autoid

package
v1.1.0-beta.0...-c0b86a3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 18, 2024 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// SystemSchemaIDFlag is the system schema/table id flag, uses the highest bit position as system schema ID flag, it's exports for test.
	SystemSchemaIDFlag = 1 << 62
	// InformationSchemaDBID is the information_schema schema id, it's exports for test.
	InformationSchemaDBID int64 = SystemSchemaIDFlag | 1
	// PerformanceSchemaDBID is the performance_schema schema id, it's exports for test.
	PerformanceSchemaDBID int64 = SystemSchemaIDFlag | 10000
	// MetricSchemaDBID is the metrics_schema schema id, it's exported for test.
	MetricSchemaDBID int64 = SystemSchemaIDFlag | 20000
)

Attention: For reading cluster TiDB memory tables, the system schema/table should be same. Once the system schema/table id been allocated, it can't be changed any more. Change the system schema/table id may have the compatibility problem.

View Source 
const (
	// AutoRandomShardBitsDefault is the default number of shard bits.
	AutoRandomShardBitsDefault = 5
	// AutoRandomRangeBitsDefault is the default number of range bits.
	AutoRandomRangeBitsDefault = 64
	// AutoRandomShardBitsMax is the max number of shard bits.
	AutoRandomShardBitsMax = 15
	// AutoRandomRangeBitsMax is the max number of range bits.
	AutoRandomRangeBitsMax = 64
	// AutoRandomRangeBitsMin is the min number of range bits.
	AutoRandomRangeBitsMin = 32
	// AutoRandomIncBitsMin is the min number of auto random incremental bits.
	AutoRandomIncBitsMin = 27
)
View Source 
const (
	// AutoRandomMustFirstColumnInPK is reported when auto_random is not the first column in primary key.
	AutoRandomMustFirstColumnInPK = "column '%s' must be the first column in primary key"
	// AutoRandomNoClusteredPKErrMsg indicates the primary key is not clustered.
	AutoRandomNoClusteredPKErrMsg = "auto_random is only supported on the tables with clustered primary key"
	// AutoRandomIncompatibleWithAutoIncErrMsg is reported when auto_random and auto_increment are specified on the same column.
	AutoRandomIncompatibleWithAutoIncErrMsg = "auto_random is incompatible with auto_increment"
	// AutoRandomIncompatibleWithDefaultValueErrMsg is reported when auto_random and default are specified on the same column.
	AutoRandomIncompatibleWithDefaultValueErrMsg = "auto_random is incompatible with default"
	// AutoRandomOverflowErrMsg is reported when auto_random is greater than max length of a MySQL data type.
	AutoRandomOverflowErrMsg = "max allowed auto_random shard bits is %d, but got %d on column `%s`"
	// AutoRandomModifyColTypeErrMsg is reported when a user is trying to modify the type of a column specified with auto_random.
	AutoRandomModifyColTypeErrMsg = "modifying the auto_random column type is not supported"
	// AutoRandomAlterErrMsg is reported when a user is trying to add/drop/modify the value of auto_random attribute.
	AutoRandomAlterErrMsg = "adding/dropping/modifying auto_random is not supported"
	// AutoRandomDecreaseBitErrMsg is reported when the auto_random shard bits is decreased.
	AutoRandomDecreaseBitErrMsg = "decreasing auto_random shard bits is not supported"
	// AutoRandomNonPositive is reported then a user specifies a non-positive value for auto_random.
	AutoRandomNonPositive = "the value of auto_random should be positive"
	// AutoRandomAvailableAllocTimesNote is reported when a table containing auto_random is created.
	AutoRandomAvailableAllocTimesNote = "Available implicit allocation times: %d"
	// AutoRandomExplicitInsertDisabledErrMsg is reported when auto_random column value is explicitly specified, but the session var 'allow_auto_random_explicit_insert' is false.
	AutoRandomExplicitInsertDisabledErrMsg = "Explicit insertion on auto_random column is disabled. Try to set @@allow_auto_random_explicit_insert = true."
	// AutoRandomOnNonBigIntColumn is reported when define auto random to non bigint column
	AutoRandomOnNonBigIntColumn = "auto_random option must be defined on `bigint` column, but not on `%s` column"
	// AutoRandomRebaseNotApplicable is reported when alter auto_random base on a non auto_random table.
	AutoRandomRebaseNotApplicable = "alter auto_random_base of a non auto_random table"
	// AutoRandomRebaseOverflow is reported when alter auto_random_base to a value that overflows the incremental bits.
	AutoRandomRebaseOverflow = "alter auto_random_base to %d overflows the incremental bits, max allowed base is %d"
	// AutoRandomAlterAddColumn is reported when adding an auto_random column.
	AutoRandomAlterAddColumn = "unsupported add column '%s' constraint AUTO_RANDOM when altering '%s.%s'"
	// AutoRandomAlterChangeFromAutoInc is reported when the column is changing from a non-auto_increment or a non-primary key.
	AutoRandomAlterChangeFromAutoInc = "auto_random can only be converted from auto_increment clustered primary key"
	// AutoRandomAllocatorNotFound is reported when auto_random ID allocator not found during changing from auto_inc to auto_random.
	AutoRandomAllocatorNotFound = "auto_random ID allocator not found in table '%s.%s'"
	// AutoRandomInvalidRangeBits is reported when the auto_random_range_bits is invalid.
	AutoRandomInvalidRangeBits = "auto_random range bits must be between %d and %d, but got %d"
	// AutoRandomIncrementalBitsTooSmall is reported when the auto_random available use space is too small.
	AutoRandomIncrementalBitsTooSmall = "auto_random ID space is too small, please decrease the shard bits or increase the range bits"
	// AutoRandomUnsupportedAlterRangeBits is reported when the auto_random range_bits is changed.
	AutoRandomUnsupportedAlterRangeBits = "alter the range bits of auto_random column is not supported"
)
View Source 
const RowIDBitLength = 64

RowIDBitLength is the bit number of a row id in TiDB.

Variables

View Source 
var (
	ErrAutoincReadFailed    = dbterror.ClassAutoid.NewStd(mysql.ErrAutoincReadFailed)
	ErrWrongAutoKey         = dbterror.ClassAutoid.NewStd(mysql.ErrWrongAutoKey)
	ErrInvalidAllocatorType = dbterror.ClassAutoid.NewStd(mysql.ErrUnknownAllocatorType)
	ErrAutoRandReadFailed   = dbterror.ClassAutoid.NewStd(mysql.ErrAutoRandReadFailed)
)

Error instances.

View Source 
var AllocatorRuntimeStatsCtxKey = allocatorRuntimeStatsCtxKeyType{}

AllocatorRuntimeStatsCtxKey is the context key of allocator runtime stats.

View Source 
var MockForTest func(kv.Storage) autoid.AutoIDAllocClient

MockForTest is exported for testing. The actual implementation is in github.com/pingcap/tidb/pkg/autoid_service because of the package circle depending issue.

Functions

func AutoRandomRangeBitsNormalize 

func AutoRandomRangeBitsNormalize(rangeBits int) (ret uint64, err error)

AutoRandomRangeBitsNormalize normalizes the auto random range bits.

func AutoRandomShardBitsNormalize 

func AutoRandomShardBitsNormalize(shard int, colName string) (ret uint64, err error)

AutoRandomShardBitsNormalize normalizes the auto random shard bits.

func CalcNeededBatchSize 

func CalcNeededBatchSize(base, n, increment, offset int64, isUnsigned bool) int64

CalcNeededBatchSize is used to calculate batch size for autoID allocation. It firstly seeks to the first valid position based on increment and offset, then plus the length remained, which could be (n-1) * increment.

func CalcSequenceBatchSize 

func CalcSequenceBatchSize(base, size, increment, offset, min, max int64) (int64, error)

CalcSequenceBatchSize calculate the next sequence batch size.

func DecodeCmpUintToInt 

func DecodeCmpUintToInt(u uint64) int64

DecodeCmpUintToInt decodes the u that encoded by EncodeIntToCmpUint

func EncodeIntToCmpUint 

func EncodeIntToCmpUint(v int64) uint64

EncodeIntToCmpUint make int v to comparable uint type

func GetStep 

func GetStep() int64

GetStep is only used by tests

func NextStep 

func NextStep(curStep int64, consumeDur time.Duration) int64

NextStep return new auto id step according to previous step and consuming time.

func SeekToFirstAutoIDSigned 

func SeekToFirstAutoIDSigned(base, increment, offset int64) int64

SeekToFirstAutoIDSigned seeks to the next valid signed position.

func SeekToFirstAutoIDUnSigned 

func SeekToFirstAutoIDUnSigned(base, increment, offset uint64) uint64

SeekToFirstAutoIDUnSigned seeks to the next valid unsigned position.

func SeekToFirstSequenceValue 

func SeekToFirstSequenceValue(base, increment, offset, min, max int64) (int64, bool)

SeekToFirstSequenceValue seeks to the next valid value (must be in range of [MIN, max]), the bool indicates whether the first value is got. The seeking formula is describe as below: 

nr  := (base + increment - offset) / increment

first := nr*increment + offset Because formula computation will overflow Int64, so we transfer it to uint64 for distance computation.

func SetStep 

func SetStep(s int64)

SetStep is only used by tests

func TestModifyBaseAndEndInjection 

func TestModifyBaseAndEndInjection(alloc Allocator, base, end int64)

TestModifyBaseAndEndInjection exported for testing modifying the base and end.

Types

type AllocOption 

type AllocOption interface {
	ApplyOn(*allocator)
}

AllocOption is a interface to define allocator custom options coming in future.

type AllocOptionTableInfoVersion 

type AllocOptionTableInfoVersion uint16

AllocOptionTableInfoVersion is used to pass the TableInfo.Version to the allocator.

func (AllocOptionTableInfoVersion) ApplyOn 

func (v AllocOptionTableInfoVersion) ApplyOn(alloc *allocator)

ApplyOn implements the AllocOption interface.

type Allocator 

type Allocator interface {
	// Alloc allocs N consecutive autoID for table with tableID, returning (min, max] of the allocated autoID batch.
	// It gets a batch of autoIDs at a time. So it does not need to access storage for each call.
	// The consecutive feature is used to insert multiple rows in a statement.
	// increment & offset is used to validate the start position (the allocator's base is not always the last allocated id).
	// The returned range is (min, max]:
	// case increment=1 & offset=1: you can derive the ids like min+1, min+2... max.
	// case increment=x & offset=y: you firstly need to seek to firstID by `SeekToFirstAutoIDXXX`, then derive the IDs like firstID, firstID + increment * 2... in the caller.
	Alloc(ctx context.Context, n uint64, increment, offset int64) (int64, int64, error)

	// AllocSeqCache allocs sequence batch value cached in table level（rather than in alloc), the returned range covering
	// the size of sequence cache with it's increment. The returned round indicates the sequence cycle times if it is with
	// cycle option.
	AllocSeqCache() (min int64, max int64, round int64, err error)

	// Rebase rebases the autoID base for table with tableID and the new base value.
	// If allocIDs is true, it will allocate some IDs and save to the cache.
	// If allocIDs is false, it will not allocate IDs.
	Rebase(ctx context.Context, newBase int64, allocIDs bool) error

	// ForceRebase set the next global auto ID to newBase.
	ForceRebase(newBase int64) error

	// RebaseSeq rebases the sequence value in number axis with tableID and the new base value.
	RebaseSeq(newBase int64) (int64, bool, error)

	// Base return the current base of Allocator.
	Base() int64
	// End is only used for test.
	End() int64
	// NextGlobalAutoID returns the next global autoID.
	NextGlobalAutoID() (int64, error)
	GetType() AllocatorType
}

Allocator is an auto increment id generator. Just keep id unique actually.

func NewAllocator 

func NewAllocator(r Requirement, dbID, tbID int64, isUnsigned bool,
	allocType AllocatorType, opts ...AllocOption) Allocator

NewAllocator returns a new auto increment id generator on the store.

func NewAllocatorFromTempTblInfo 

func NewAllocatorFromTempTblInfo(tblInfo *model.TableInfo) Allocator

NewAllocatorFromTempTblInfo creates an in-memory allocator from a temporary table info.

func NewSequenceAllocator 

func NewSequenceAllocator(store kv.Storage, dbID, tbID int64, info *model.SequenceInfo) Allocator

NewSequenceAllocator returns a new sequence value generator on the store.

type AllocatorRuntimeStats 

type AllocatorRuntimeStats struct {
	*txnsnapshot.SnapshotRuntimeStats
	*execdetails.RuntimeStatsWithCommit
	// contains filtered or unexported fields
}

AllocatorRuntimeStats is the execution stats of auto id allocator.

func NewAllocatorRuntimeStats 

func NewAllocatorRuntimeStats() *AllocatorRuntimeStats

NewAllocatorRuntimeStats return a new AllocatorRuntimeStats.

func (*AllocatorRuntimeStats) Clone 

func (e *AllocatorRuntimeStats) Clone() *AllocatorRuntimeStats

Clone implements the RuntimeStats interface.

func (*AllocatorRuntimeStats) Merge 

func (e *AllocatorRuntimeStats) Merge(other *AllocatorRuntimeStats)

Merge implements the RuntimeStats interface.

func (*AllocatorRuntimeStats) String 

func (e *AllocatorRuntimeStats) String() string

String implements the RuntimeStats interface.

type AllocatorType 

type AllocatorType uint8

AllocatorType is the type of allocator for generating auto-id. Different type of allocators use different key-value pairs. 

const (
	// RowIDAllocType indicates the allocator is used to allocate row id.
	RowIDAllocType AllocatorType = iota
	// AutoIncrementType indicates the allocator is used to allocate auto increment value.
	AutoIncrementType
	// AutoRandomType indicates the allocator is used to allocate auto-shard id.
	AutoRandomType
	// SequenceType indicates the allocator is used to allocate sequence value.
	SequenceType
)

func (AllocatorType) String 

func (a AllocatorType) String() string

type Allocators 

type Allocators struct {
	SepAutoInc bool
	Allocs     []Allocator
}

Allocators represents a set of `Allocator`s.

func NewAllocators 

func NewAllocators(sepAutoInc bool, allocators ...Allocator) Allocators

NewAllocators packs multiple `Allocator`s into Allocators.

func NewAllocatorsFromTblInfo 

func NewAllocatorsFromTblInfo(r Requirement, schemaID int64, tblInfo *model.TableInfo) Allocators

NewAllocatorsFromTblInfo creates an array of allocators of different types with the information of model.TableInfo.

func (Allocators) Append 

func (all Allocators) Append(a Allocator) Allocators

Append add an allocator to the allocators.

func (Allocators) Filter 

func (all Allocators) Filter(pred func(Allocator) bool) Allocators

Filter filters all the allocators that match pred.

func (Allocators) Get 

func (all Allocators) Get(allocType AllocatorType) Allocator

Get returns the Allocator according to the AllocatorType.

type ClientDiscover 

type ClientDiscover struct {
	// contains filtered or unexported fields
}

ClientDiscover is used to get the AutoIDAllocClient, it creates the grpc connection with autoid service leader.

func NewClientDiscover 

func NewClientDiscover(etcdCli *clientv3.Client) *ClientDiscover

NewClientDiscover creates a ClientDiscover object.

func (*ClientDiscover) GetClient 

func (d *ClientDiscover) GetClient(ctx context.Context) (autoid.AutoIDAllocClient, uint64, error)

GetClient gets the AutoIDAllocClient.

func (*ClientDiscover) ResetConn 

func (d *ClientDiscover) ResetConn(reason error)

ResetConn reset the AutoIDAllocClient and underlying grpc connection. The next GetClient() call will recreate the client connecting to the correct leader by querying etcd.

type CustomAutoIncCacheOption 

type CustomAutoIncCacheOption int64

CustomAutoIncCacheOption is one kind of AllocOption to customize the allocator step length.

func (CustomAutoIncCacheOption) ApplyOn 

func (step CustomAutoIncCacheOption) ApplyOn(alloc *allocator)

ApplyOn implements the AllocOption interface.

type Requirement 

type Requirement interface {
	Store() kv.Storage
	AutoIDClient() *ClientDiscover
}

Requirement is the parameter required by NewAllocator

type ShardIDFormat 

type ShardIDFormat struct {
	FieldType *types.FieldType
	ShardBits uint64
	// Derived fields.
	IncrementalBits uint64
}

ShardIDFormat is used to calculate the bit length of different segments in auto id. Generally, an auto id is consist of 4 segments: sign bit, reserved bits, shard bits and incremental bits. Take "a BIGINT AUTO_INCREMENT PRIMARY KEY" as an example, assume that the `shard_row_id_bits` = 5, the layout is like 

| [sign_bit] (1 bit) | [reserved bits] (0 bits) | [shard_bits] (5 bits) | [incremental_bits] (64-1-5=58 bits) |

Please always use NewShardIDFormat() to instantiate.

func NewShardIDFormat 

func NewShardIDFormat(fieldType *types.FieldType, shardBits, rangeBits uint64) ShardIDFormat

NewShardIDFormat create an instance of ShardIDFormat. RangeBits means the bit length of the sign bit + shard bits + incremental bits. If RangeBits is 0, it will be calculated according to field type automatically.

func (*ShardIDFormat) Compose 

func (s *ShardIDFormat) Compose(shard int64, id int64) int64

Compose generates an auto ID based on the given shard and an incremental ID.

func (*ShardIDFormat) IncrementalBitsCapacity 

func (s *ShardIDFormat) IncrementalBitsCapacity() uint64

IncrementalBitsCapacity returns the max capacity of incremental section of the current format.

func (*ShardIDFormat) IncrementalMask 

func (s *ShardIDFormat) IncrementalMask() int64

IncrementalMask returns 00..0[11..1], where [11..1] is the incremental part of the current format.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.