cloudstorage

Documentation

Overview

Copyright 2022 PingCAP, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, See the License for the specific language governing permissions and limitations under the License.

Copyright 2022 PingCAP, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, See the License for the specific language governing permissions and limitations under the License.

Copyright 2022 PingCAP, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, See the License for the specific language governing permissions and limitations under the License.

Index

• type DMLSink
  • func NewDMLSink(ctx context.Context, changefeedID model.ChangeFeedID, pdClock pdutil.Clock, ...) (*DMLSink, error)
  • func (s *DMLSink) Close()
  • func (s *DMLSink) Dead() <-chan struct{}
  • func (s *DMLSink) SchemeOption() (string, bool)
  • func (s *DMLSink) WriteEvents(txns ...*dmlsink.CallbackableEvent[*model.SingleTableTxn]) error

Types

type DMLSink

type DMLSink struct {
	// contains filtered or unexported fields
}

DMLSink is the cloud storage sink. It will send the events to cloud storage systems. Messages are encoded in the specific protocol and then sent to the defragmenter. The data flow is as follows: **data** -> encodingWorkers -> defragmenter -> dmlWorkers -> external storage The defragmenter will defragment the out-of-order encoded messages and sends encoded messages to individual dmlWorkers. The dmlWorkers will write the encoded messages to external storage in parallel between different tables.

func NewDMLSink

func NewDMLSink(ctx context.Context,
	changefeedID model.ChangeFeedID,
	pdClock pdutil.Clock,
	sinkURI *url.URL,
	replicaConfig *config.ReplicaConfig,
	errCh chan error,
) (*DMLSink, error)

NewDMLSink creates a cloud storage sink.

func (*DMLSink) Close

func (s *DMLSink) Close()

Close closes the cloud storage sink.

func (*DMLSink) Dead

func (s *DMLSink) Dead() <-chan struct{}

Dead checks whether it's dead or not.

func (*DMLSink) SchemeOption

func (s *DMLSink) SchemeOption() (string, bool)

SchemeOption returns the scheme and the option.

func (*DMLSink) WriteEvents

func (s *DMLSink) WriteEvents(txns ...*dmlsink.CallbackableEvent[*model.SingleTableTxn]) error

WriteEvents write events to cloud storage sink.