gc

Documentation

Overview

Package gc privodes TiDB GC related utilities.

Following graph shows how TiCDC manipulates GC safepoints.

```

          ┌───┐          ┌───┐          ┌──┐                   ┌─────┐
          │CLI│          │API│          │PD│                   │Owner│
          └─┬─┘          └─┬─┘          └┬─┘                   └──┬──┘
            │              │             │                        │
╔═══════════╪═════╤════════╪═════════════╪════════════════════════╪════════════╗
║ "CLI OR API?"   │  CLI   │             │                        │            ║
╟─────────────────┘        │             │                        │            ║
║           │   service GC safepoint     │                        │            ║
║           │   "cdc-creating-<ID>"      │                        │            ║
║           │──────────────────────────>┌┴┐                       │            ║
║           │              │            │ │                       │            ║
║           │    Create changefeed      │ │                       │            ║
║           │──────────────────────────>│ │                       │            ║
╠═══════════╪══════════════╪════════════╪═╪═══════════════════════╪════════════╣
║  API      │              │            │ │                       │            ║
║           │              │        Create changefeed API         │            ║
║           │              │─────────────────────────────────────>│            ║
║           │              │            │ │                       │            ║
║           │              │            │ │   service GC safepoint│            ║
║           │              │            │ │   "cdc-creating-<ID>" │            ║
║           │              │            │ ┌─┐ <────────────────────            ║
║           │              │            │ │ │                     │            ║
║           │              │            │ │ │  Create changefeed  │            ║
║           │              │            │ │ │ <────────────────────            ║
╚═══════════╪══════════════╪════════════╪═╪═╪═════════════════════╪════════════╝
            │              │            │ │ │                     │
            │              │            │ │ │                     │  ╔═════════════════╗
            │              │            │ │ │ GC safepoint "ticdc"│  ║using the minimal║
            │              │            │ │ │ <────────────────────  ║checkpoint among ║
            │              │            │ │ │                     │  ║all changefeeds  ║
            │              │            │ │ │                     │  ╚═════════════════╝
            │              │            │ │ │Remove GC safepoint  │
            │              │            │ │ │"cdc-creating-<ID>"  │
            │              │            └┬└─┘ <────────────────────
            │              │             │                        │
            │              │             │                        │────┐
            │              │             │                        │    │ Start changefeed
            │              │             │                        │<───┘
          ┌─┴─┐          ┌─┴─┐          ┌┴─┐                   ┌──┴──┐
          │CLI│          │API│          │PD│                   │Owner│
          └───┘          └───┘          └──┘                   └─────┘

```

When CLI/API creates a changefeed, it registers a service GC safepoint "ticdc-creating-<ID>" to cover the gap between creating changefeed and handling changefeed.

TiCDC Owner removes the service GC safepoint after new changefeed checkpoint is included in the "ticdc" service GC safepoint.

Index

• Constants
• func EnsureChangefeedStartTsSafety(ctx context.Context, pdCli pd.Client, gcServiceIDPrefix string, ...) error
• func InitMetrics(registry *prometheus.Registry)
• func RemoveServiceGCSafepoint(ctx context.Context, pdCli pd.Client, serviceID string) error
• func SetServiceGCSafepoint(ctx context.Context, pdCli pd.Client, serviceID string, TTL int64, ...) (minServiceGCTs uint64, err error)
• func UndoEnsureChangefeedStartTsSafety(ctx context.Context, pdCli pd.Client, gcServiceIDPrefix string, ...) error
• type Manager
  • func NewManager(gcServiceID string, pdClient pd.Client, pdClock pdutil.Clock) Manager
• type MockPDClient
  • func (m *MockPDClient) Close()
  • func (m *MockPDClient) GetAllStores(ctx context.Context, opts ...pd.GetStoreOption) ([]*metapb.Store, error)
  • func (m *MockPDClient) GetClusterID(ctx context.Context) uint64
  • func (m *MockPDClient) GetTS(ctx context.Context) (int64, int64, error)
  • func (m *MockPDClient) LoadGlobalConfig(ctx context.Context, names []string, configPath string) ([]pd.GlobalConfigItem, int64, error)
  • func (m *MockPDClient) UpdateServiceGCSafePoint(ctx context.Context, serviceID string, ttl int64, safePoint uint64) (uint64, error)

Constants

const (
	// EnsureGCServiceCreating is a tag of GC service id for changefeed creation
	EnsureGCServiceCreating = "-creating-"
	// EnsureGCServiceResuming is a tag of GC service id for changefeed resumption
	EnsureGCServiceResuming = "-resuming-"
	// EnsureGCServiceInitializing is a tag of GC service id for changefeed initialization
	EnsureGCServiceInitializing = "-initializing-"
)

Functions

func EnsureChangefeedStartTsSafety

func EnsureChangefeedStartTsSafety(
	ctx context.Context, pdCli pd.Client,
	gcServiceIDPrefix string,
	changefeedID model.ChangeFeedID,
	TTL int64, startTs uint64,
) error

EnsureChangefeedStartTsSafety checks if the startTs less than the minimum of service GC safepoint and this function will update the service GC to startTs

func InitMetrics

func InitMetrics(registry *prometheus.Registry)

InitMetrics registers all metrics used gc manager

func RemoveServiceGCSafepoint

func RemoveServiceGCSafepoint(ctx context.Context, pdCli pd.Client, serviceID string) error

RemoveServiceGCSafepoint removes a service safepoint from PD.

func SetServiceGCSafepoint

func SetServiceGCSafepoint(
	ctx context.Context, pdCli pd.Client, serviceID string, TTL int64, safePoint uint64,
) (minServiceGCTs uint64, err error)

SetServiceGCSafepoint set a service safepoint to PD.

func UndoEnsureChangefeedStartTsSafety

func UndoEnsureChangefeedStartTsSafety(
	ctx context.Context, pdCli pd.Client,
	gcServiceIDPrefix string,
	changefeedID model.ChangeFeedID,
) error

UndoEnsureChangefeedStartTsSafety cleans the service GC safepoint of a changefeed if something goes wrong after successfully calling EnsureChangefeedStartTsSafety().

Types

type Manager

type Manager interface {
	// TryUpdateGCSafePoint tries to update TiCDC service GC safepoint.
	// Manager may skip update when it thinks it is too frequent.
	// Set `forceUpdate` to force Manager update.
	TryUpdateGCSafePoint(ctx context.Context, checkpointTs model.Ts, forceUpdate bool) error
	CheckStaleCheckpointTs(ctx context.Context, changefeedID model.ChangeFeedID, checkpointTs model.Ts) error
}

Manager is an interface for gc manager

func NewManager

func NewManager(gcServiceID string, pdClient pd.Client, pdClock pdutil.Clock) Manager

NewManager creates a new Manager.

type MockPDClient

type MockPDClient struct {
	pd.Client
	ClusterID        uint64
	GetAllStoresFunc func(ctx context.Context, opts ...pd.GetStoreOption) ([]*metapb.Store, error)

	UpdateServiceGCSafePointFunc func(ctx context.Context, serviceID string, ttl int64, safePoint uint64) (uint64, error)
}

MockPDClient mocks pd.Client to facilitate unit testing.

func (*MockPDClient) Close

func (m *MockPDClient) Close()

Close implements pd.Client.Close() This method is used in some unit test cases.

func (*MockPDClient) GetAllStores

func (m *MockPDClient) GetAllStores(
	ctx context.Context, opts ...pd.GetStoreOption,
) ([]*metapb.Store, error)

GetAllStores gets all stores from PD.

func (*MockPDClient) GetClusterID

func (m *MockPDClient) GetClusterID(ctx context.Context) uint64

GetClusterID gets the cluster ID from PD.

func (*MockPDClient) GetTS

func (m *MockPDClient) GetTS(ctx context.Context) (int64, int64, error)

GetTS implements pd.Client.GetTS.

func (*MockPDClient) LoadGlobalConfig

func (m *MockPDClient) LoadGlobalConfig(
	ctx context.Context,
	names []string, configPath string,
) ([]pd.GlobalConfigItem, int64, error)

LoadGlobalConfig loads global config from PD.

func (*MockPDClient) UpdateServiceGCSafePoint

func (m *MockPDClient) UpdateServiceGCSafePoint(ctx context.Context, serviceID string, ttl int64, safePoint uint64) (uint64, error)

UpdateServiceGCSafePoint implements pd.Client.UpdateServiceGCSafePoint.