objsync

README

Objsync

A distributed synchronization library for Golang built on top of object storage.

Why? See my whitepaper: All You Need Is S3.

Features

• Shared, multi-process, multi-host locks.
• No additional infrastructure required.
• Automatic expiration in the event of a failure.
• Fencing token support, to prevent the use of stale locks.

Limitations

• Not all object storage providers support conditional PUTs which are required to implement locking (e.g. AWS S3).
• Due to object mutation rate limits, it's currently limited to 1 lock per second on most providers (but not Ceph RGW). This makes it more useful for leader election than for fine-grained locking.

Supported Providers

• Ceph RGW (and anyone using it, eg. DigitalOcean Spaces)
• Cloudflare R2
• Google Cloud Storage
• MinIO

Note: This is far from an exhaustive list, and I'm happy to accept PRs.

Usage

p, err := s3.NewProvider(ctx, endpointURL, region, accessKeyID, secretAccessKey)
if err != nil {
	panic(err)
}

mu := objsync.NewMutex(p, bucket, key)

fencingToken, err := mu.Lock(ctx, 5*time.Second)
if err != nil {
	panic(err)
}

// Do something with the mutex, remember to pass along the fencing token.

if err := mu.Unlock(ctx); err != nil {
	panic(err)
}

Contribution Ideas

• Add support for more object storage providers.
• Increase the lock rate limit by using a separate object for each locking attempt (and the necessary cleanup etc).

License

Objsync is licensed under the Mozilla Public License 2.0, see LICENSE.

Documentation

Index

• type Mutex
  • func NewMutex(p provider.Provider, bucket, key string) *Mutex
  • func (mu *Mutex) Lock(ctx context.Context, length time.Duration) (int64, error)
  • func (mu *Mutex) TryLock(ctx context.Context, expiresIn time.Duration) (bool, int64, error)
  • func (mu *Mutex) Unlock(ctx context.Context) error

Types

type Mutex

type Mutex struct {
	// contains filtered or unexported fields
}

Mutex is a distributed mutex.

func NewMutex

func NewMutex(p provider.Provider, bucket, key string) *Mutex

NewMutex creates a new distributed mutex.

func (*Mutex) Lock

func (mu *Mutex) Lock(ctx context.Context, length time.Duration) (int64, error)

Lock acquires the mutex. It blocks until the mutex is available. Length is the maximum duration the lock will be held for.

func (*Mutex) TryLock

func (mu *Mutex) TryLock(ctx context.Context, expiresIn time.Duration) (bool, int64, error)

TryLock attempts to acquire the mutex without blocking.

func (*Mutex) Unlock

func (mu *Mutex) Unlock(ctx context.Context) error

Unlock releases the mutex (if held).