synchronizer

README

Auth synchronizer

用途

用于将CMDB的数据同步到权限中心(IAM), 保证所有的CMDB在IAM都有注册，并清理IAM中无效的资源（在CMDB中的对应资源ID已经被删除）

方案设计

同步粒度

一次同步的数据量过大会导致一系列问题：1. CMDDB 和 IAM 系统负载高; 2. 失败重试成本高（需要全量重新开始）。 为了避免这种情况，本模块采用规模适中的粒度作为一次同步任务，比如主机资源的同步按业务进行，一个业务下的主机作为一次同步任务。

设计模式

生产消费者模式

工作机制

• 生产者模块：读取需要同步的scope列表, 组成装同步任务，然后放入channel
• 消费者模块：一次取出一个任务，完成后再取下一个任务

与其它模块交互方式

微服务方式

Documentation

Index

• type AuthSynchronizer
  • func NewSynchronizer(ctx context.Context, authConfig *authcenter.AuthConfig, ...) *AuthSynchronizer
  • func (d *AuthSynchronizer) Run() error
• type Producer
  • func NewProducer(clientSet apimachinery.ClientSetInterface, authManager *extensions.AuthManager, ...) *Producer
  • func (p *Producer) Start()
• type Worker
  • func NewWorker(id int, workerQueue chan meta.WorkRequest, handler meta.SyncHandler) *Worker
  • func (w *Worker) Start()
  • func (w *Worker) Stop()

Types

type AuthSynchronizer

type AuthSynchronizer struct {
	AuthConfig authcenter.AuthConfig

	Workers     *[]Worker
	WorkerQueue chan meta.WorkRequest
	Producer    *Producer
	// contains filtered or unexported fields
}

AuthSynchronizer stores all related resource

func NewSynchronizer

func NewSynchronizer(ctx context.Context, authConfig *authcenter.AuthConfig, clientSet apimachinery.ClientSetInterface) *AuthSynchronizer

NewSynchronizer new a synchronizer object

func (*AuthSynchronizer) Run

func (d *AuthSynchronizer) Run() error

Run do start synchronize

type Producer

type Producer struct {
	ID          int
	WorkerQueue chan meta.WorkRequest
	QuitChan    chan bool
	// contains filtered or unexported fields
}

Producer producer WorkRequest and enqueue it

func NewProducer

func NewProducer(clientSet apimachinery.ClientSetInterface, authManager *extensions.AuthManager, workerQueue chan meta.WorkRequest) *Producer

NewProducer make a producer

func (*Producer) Start

func (p *Producer) Start()

Start do main loop

type Worker

type Worker struct {
	ID          int
	WorkerQueue chan meta.WorkRequest
	QuitChan    chan bool
	SyncHandler meta.SyncHandler
}

Worker represent a worker

func NewWorker

func NewWorker(id int, workerQueue chan meta.WorkRequest, handler meta.SyncHandler) *Worker

NewWorker creates, and returns a new Worker object. Its only argument is a channel that the worker can add itself to whenever it is done its work.

func (*Worker) Start

func (w *Worker) Start()

Start "starts" the worker by starting a goroutine, that is an infinite "for-select" loop.

func (*Worker) Stop

func (w *Worker) Stop()

Stop tells the worker to stop listening for work requests.

Note that the worker will only stop *after* it has finished its work.