config

README

config

Golang process config synchronization using etcd

Documentation

Overview

Package config implements a configuration server using git and zookeeper.

Besides maintaining structured data describing how systems should work together, a good configuration server needs to have the following characteristics:

1. Version control. Should be able to revert to a previous version shall there be any error after a push. 2. Fast, simple, and reliable notification to thousands or tens of thousands servers. 3. Flexible to describe complex systems. 4. Ease of use

For the above reasons, we chose to implement our config system with two components that are already widely used with a rock solid foundation: git, and zookeeper.

It is easy to see that git can handle require 1, 3, and 4, while zookeeper satisfies 2, and 4. The system works as follows:

1. Configurations are stored in a git repo. You can have one file for the whole systems, or you can have hundreds or thousands of files each detailing a part of the whole system. It doesn't matter. The config system deal with repo, and its commits. The sysadmin maintain the repo using whatever git tool he/she choose. Changes to the git repo are not saved to config system. So just use it as a normal repo, branch, edit, etc as you would for a normal repo.

2. For changes to be pushed to our config system, you need to call csi/service/config/pusher/pusher. pusher works on the HEAD of the repo. Whatever you have checked out, when pusher is called, will be pushed onto config system. pusher will update data in a zookeeper ensemble that acts as the persistent, and notification layer. When pusher is called, it is given a root path, which is the root path for the configuration information. This way, it is ok if you just dedicate a subdirectory of your repo for configuration, instead of dedicating a completely new one. All descendent files of the root directory will be saved onto zookeeper, e.g. a/b/c/d will be set in zookeeper as znode a/b/c/d where the content of znode a/b/c/d is the content of file a/b/c/d. Additionally, pusher will collect repo and commit info, encode them in a json structure and store it in the root znode for this configuration system. The json structure contains repo name, branch name, last commit hash, committer email, timestamp, and a files modified in the last commit.

3. The information stored in step 2 on the root znode is important. It allows a client of the configuration system to just listen to one znode, and gets notified of not only a change has happened, but what file(s) got changed, and decided whether or not it should update. This job is done by csi/service/config/client.go. Client.go allows one to register to receive event on any file change.

4. Pusher requires "git" to be in the PATH, as it calls git related command extensively. The following is a list of git commands that pusher uses to find out various repo related info:

# show repo name 00:57 $ basename `git rev-parse --show-toplevel` csi

# show HEAD branch name 00:56 $ git rev-parse --abbrev-ref HEAD master

# show HEAD commit hash 00:54 $ git rev-parse HEAD 6e390e716805599f8f694a5f654649d92ec638f6

# show files affected in a commit 01:02 $ git diff-tree --no-commit-id --name-status -r 6e390e716805599f8f694a5f654649d92ec638f6 common/types/user/login_record.pb.go external/oauth/cache.go make_go.sh pp/frontend/main.go .....

# get timestamp, committer email and subject of a commit 03:03 $ git show --pretty="format:%at:%ce:%s" 6e390e716805599f8f694a5f654649d92ec638f6 | head -n 1 1394199814:alien_lien@htc.com:Add 1) Account Manager and 2) User Management Service

5. Client can listen on the configuration change with a regex string that specifies which files it is interested. It will be notified when any of files that matches the regex has changed. Note: this is achieved by regex match the changed files in the commit log with is published to the root znode.

Index

• Constants
• func Stat(counter stats.Client) func(*clientImpl) error
• type Client
  • func NewClient(conn *etcd.Client, root string, options ...func(*clientImpl) error) (Client, error)
  • func NewFileConfig(root string, uncommitOnly bool) (Client, error)
  • func NewMemConfig(files map[string][]byte) Client
• type CommitInfo
• type ConfigInfo
• type FileConfig
  • func (d FileConfig) AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile
  • func (d FileConfig) ConfigInfo() ConfigInfo
  • func (l *FileConfig) Get(path string) ([]byte, error)
  • func (l *FileConfig) List(path string) (map[string][]byte, error)
  • func (d FileConfig) RemoveListener(ch *chan ModifiedFile)
  • func (l *FileConfig) Stop() error
  • func (d FileConfig) Watch(string, func([]byte) error, chan<- error) error
• type MemConfig
  • func (d MemConfig) AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile
  • func (d MemConfig) ConfigInfo() ConfigInfo
  • func (m *MemConfig) Get(filename string) ([]byte, error)
  • func (m *MemConfig) List(path string) (map[string][]byte, error)
  • func (d MemConfig) RemoveListener(ch *chan ModifiedFile)
  • func (m *MemConfig) Stop() error
  • func (d MemConfig) Watch(string, func([]byte) error, chan<- error) error
• type MockClient
  • func (m *MockClient) AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile
  • func (m *MockClient) ConfigInfo() ConfigInfo
  • func (m *MockClient) Get(path string) ([]byte, error)
  • func (m *MockClient) List(path string) (map[string][]byte, error)
  • func (m *MockClient) RemoveListener(ch *chan ModifiedFile)
  • func (m *MockClient) Stop() error
  • func (m *MockClient) Watch(path string, callback func([]byte) error, errChan chan<- error) error
• type ModifiedFile

Constants

const (
	// PkgName is for creating module counter
	PkgName = "config"
)

Functions

func Stat

func Stat(counter stats.Client) func(*clientImpl) error

Stat sets counter client

Types

type Client

type Client interface {
	// ConfigInfo returns the ConfigInfo protobuf struct that
	// contains infomation w.r.t to repo and last commit
	ConfigInfo() ConfigInfo

	// Get request the content of file at path. If the path doesn't
	// exists then ok returns false
	Get(path string) ([]byte, error)

	// AddListener returns a chan that when any file that matches
	// the pathRegEx changes, the ModifiedFile will be sent over
	// through the channel
	AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile

	// Watch watches change of a file and invokes callback. It also invokes callback for the
	// first time and return error if there's one.
	Watch(path string, callback func([]byte) error, errChan chan<- error) error

	// RemoveListener remove a listener channel
	RemoveListener(ch *chan ModifiedFile)

	// List lists content under certain path
	List(path string) (map[string][]byte, error)

	// Stop is to stop client
	Stop() error
}

Client provides a interface for configuration service

func NewClient

func NewClient(conn *etcd.Client, root string, options ...func(*clientImpl) error) (Client, error)

NewClient return a new Client with option funtions. conn: a etcd client, root: the root path that contains the config tree

func NewFileConfig

func NewFileConfig(root string, uncommitOnly bool) (Client, error)

NewFileConfig return filepath instance for local repo. Optionally you can set uncommitOnly to true which causes List() method only return a subset of uncommitted files.

func NewMemConfig

func NewMemConfig(files map[string][]byte) Client

NewMemConfig creates a memory mapped config

type CommitInfo

type CommitInfo struct {
	TimeStamp      int64  `json:"timestamp,omitempty"`
	CommitterEmail string `json:"committerEmail,omitempty"`
	Subject        string `json:"subject,omitempty"`
}

CommitInfo contains git commit info

type ConfigInfo

type ConfigInfo struct {
	Repo     string         `json:"repo,omitempty"`
	Branch   string         `json:"branch,omitempty"`
	Version  string         `json:"version,omitempty"`
	Commit   CommitInfo     `json:"commit,omitempty"`
	ModFiles []ModifiedFile `json:"modfiles,omitempty"`
	// PathToRepo is the path to git root directory
	PathToRepo string `json:"path_to_repo,omitempty"`
}

ConfigInfo defines a configuration service information

type FileConfig

type FileConfig struct {
	// contains filtered or unexported fields
}

FileConfig is used to test config file in local

func (FileConfig) AddListener

func (d FileConfig) AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile

func (FileConfig) ConfigInfo

func (d FileConfig) ConfigInfo() ConfigInfo

func (*FileConfig) Get

func (l *FileConfig) Get(path string) ([]byte, error)

Get get the file from local directory under root

func (*FileConfig) List

func (l *FileConfig) List(path string) (map[string][]byte, error)

List list all the files of the path

func (FileConfig) RemoveListener

func (d FileConfig) RemoveListener(ch *chan ModifiedFile)

RemoveListener remove a listener channel

func (*FileConfig) Stop

func (l *FileConfig) Stop() error

Stop is to stop config client

func (FileConfig) Watch

func (d FileConfig) Watch(string, func([]byte) error, chan<- error) error

type MemConfig

type MemConfig struct {
	// contains filtered or unexported fields
}

MemConfig is a memroy mapped config client

func (MemConfig) AddListener

func (d MemConfig) AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile

func (MemConfig) ConfigInfo

func (d MemConfig) ConfigInfo() ConfigInfo

func (*MemConfig) Get

func (m *MemConfig) Get(filename string) ([]byte, error)

Get gets file content

func (*MemConfig) List

func (m *MemConfig) List(path string) (map[string][]byte, error)

List lists files under the path TODO: it only returns file without any directories.

func (MemConfig) RemoveListener

func (d MemConfig) RemoveListener(ch *chan ModifiedFile)

RemoveListener remove a listener channel

func (*MemConfig) Stop

func (m *MemConfig) Stop() error

Stop is to stop config client

func (MemConfig) Watch

func (d MemConfig) Watch(string, func([]byte) error, chan<- error) error

type MockClient

type MockClient struct {
	mock.Mock
}

MockClient is the mock struct

func (*MockClient) AddListener

func (m *MockClient) AddListener(pathRegEx *regexp.Regexp) *chan ModifiedFile

AddListener is the mock function.

func (*MockClient) ConfigInfo

func (m *MockClient) ConfigInfo() ConfigInfo

ConfigInfo is the mock function.

func (*MockClient) Get

func (m *MockClient) Get(path string) ([]byte, error)

Get is the mock function.

func (*MockClient) List

func (m *MockClient) List(path string) (map[string][]byte, error)

List is the mock function.

func (*MockClient) RemoveListener

func (m *MockClient) RemoveListener(ch *chan ModifiedFile)

RemoveListener is the mock function.

func (*MockClient) Stop

func (m *MockClient) Stop() error

Stop is to stop client

func (*MockClient) Watch

func (m *MockClient) Watch(path string, callback func([]byte) error, errChan chan<- error) error

Watch watches change of a file and invokes callback. It also invokes callback for the

type ModifiedFile

type ModifiedFile struct {
	Op   string `json:"op,omitempty"`
	Path string `json:"path,omitempty"`
}

ModifiedFile contains a file and its modify operation