README
¶
Distributed Locks in MongoDB
This package provides a Go client for creating distributed locks in MongoDB.
Setup
Install the package with "go get".
go get "github.com/square/mongo-lock"
In order to use it, you must have an instance of MongoDB running with a collection that can be used to store locks. All of the examples here will assume the collection name is "locks", but you can change it to whatever you want.
Required Indexes
There is one index that is required in order for this package to work:
db.locks.createIndex( { resource: 1 }, { unique: true } )
Recommended Indexes
The following indexes are recommend to help the performance of certain queries:
db.locks.createIndex( { "exclusive.LockId": 1 } )
db.locks.createIndex( { "exclusive.ExpiresAt": 1 } )
db.locks.createIndex( { "shared.locks.LockId": 1 } )
db.locks.createIndex( { "shared.locks.ExpiresAt": 1 } )
The Client.CreateIndexes method can be called to create all of the required and recommended indexes.
Recommended Write Concern
To minimize the risk of losing locks when one or more nodes in your replica set fail, setting the write acknowledgement for the session to "majority" is recommended.
Usage
Here is an example of how to use this package:
package main
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/writeconcern"
"github.com/square/mongo-lock"
)
func main() {
// Create a Mongo session and set the write mode to "majority".
mongoUrl := "youMustProvideThis"
database := "dbName"
collection := "collectionName"
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
m, err := mongo.Connect(ctx, options.Client().
ApplyURI(mongoUrl).
SetWriteConcern(writeconcern.New(writeconcern.WMajority())))
if err != nil {
log.Fatal(err)
}
defer func() {
if err = m.Disconnect(ctx); err != nil {
panic(err)
}
}()
// Configure the client for the database and collection the lock will go into.
col := m.Database(database).Collection(collection)
// Create a MongoDB lock client.
c := lock.NewClient(col)
// Create the required and recommended indexes.
c.CreateIndexes(ctx)
lockId := "abcd1234"
// Create an exclusive lock on resource1.
err = c.XLock(ctx, "resource1", lockId, lock.LockDetails{})
if err != nil {
log.Fatal(err)
}
// Create a shared lock on resource2.
err = c.SLock(ctx, "resource2", lockId, lock.LockDetails{}, -1)
if err != nil {
log.Fatal(err)
}
// Unlock all locks that have our lockId.
_, err = c.Unlock(ctx, lockId)
if err != nil {
log.Fatal(err)
}
}
How It Works
This package can be used to create both shared and exclusive locks. To create a lock, all you need is the name of a resource (the object that gets locked) and a lockId (lock identifier). Multiple locks can be created with the same lockId, which makes it easy to unlock or renew a group of related locks at the same time. Another reason for using lockIds is to ensure the only the client that creates a lock knows the lockId needed to unlock it (i.e., knowing a resource name alone is not enough to unlock it).
Here is a list of rules that the locking behavior follows
- A resource can only have one exclusive lock on it at a time.
- A resource can have multiple shared locks on it at a time [1][2].
- A resource cannot have both an exclusive lock and a shared lock on it at the same time.
- A resource can have no locks on it at all.
[1] It is possible to limit the number of shared locks that can be on a resource at a time (see the docs for Client.SLock for more details). [2] A resource can't have more than one shared lock on it with the same lockId at a time.
Additional Features
- TTLs: You can optionally set a time to live (TTL) when creating a lock. If you do not set one, the lock will not have a TTL. TTLs can be renewed via the Client.Renew method as long as all of the locks associated with a given lockId have a TTL of at least 1 second (or no TTL at all). There is no automatic process to clean up locks that have outlived their TTL, but this package does provide a Purger that can be run in a loop to accomplish this.
Schema
Resources are the only documents stored in MongoDB. Locks on a resource are stored within the resource documents, like so:
{
"resource" : "resource1",
"exclusive" : {
"lockId" : null,
"owner" : null,
"host" : null,
"createdAt" : null,
"renewedAt" : null,
"expiresAt" : null,
"acquired" : false
},
"shared" : {
"count" : 1,
"locks" : [
{
"lockId" : "abcd",
"owner" : "john",
"host" : "host.name",
"createdAt" : ISODate("2018-01-25T01:58:47.243Z"),
"renewedAt" : null,
"expiresAt" : null,
"acquired" : true,
}
]
}
}
Note: shared locks are stored as an array instead of a map (keyed on lockId) so that shared lock fields can be indexed. This helps with the performance of unlocking, renewing, and getting the status of locks.
Development
To work on mongo-lock, clone it to your $GOPATH.
Dependencies
You must use dep to pull in dependencies and populate your local vendor/ directory.
cd $GOPATH/src/github.com/square/mongo-lock
dep ensure
Tests
By default, tests expect a MongoDB instance to be running at "localhost:3000", and they write to a db "test" and randomly generated collection name. These defaults, however, can be overwritten with environment variables.
export TEST_MONGO_URL="your_url"
export TEST_MONGO_DB="your_db"
The randomly generated collection is dropped after each test.
If you have docker, you can easily spin up a MongoDB instance for testing by running docker run --rm -p "3000:27017" mongo.
This will start a MongoDB instance on localhost:3000, and it will remove the image when it's done.
Run the tests from the root directory of this repo like so:
go test `go list ./... | grep -v "/vendor/"` --race
License
Copyright 2018 Square, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Documentation
¶
Overview ¶
Package lock provides distributed locking backed by MongoDB.
Index ¶
- Constants
- Variables
- type Client
- func (c *Client) CreateIndexes(ctx context.Context) error
- func (c *Client) Renew(ctx context.Context, lockId string, ttl uint) ([]LockStatus, error)
- func (c *Client) SLock(ctx context.Context, resourceName, lockId string, ld LockDetails, ...) error
- func (c *Client) Status(ctx context.Context, f Filter) ([]LockStatus, error)
- func (c *Client) Unlock(ctx context.Context, lockId string) ([]LockStatus, error)
- func (c *Client) XLock(ctx context.Context, resourceName, lockId string, ld LockDetails) error
- type Filter
- type LockDetails
- type LockStatus
- type LockStatusesByCreatedAtDesc
- type Purger
Constants ¶
const ( // LOCK_TYPE_EXCLUSIVE is the string representation of an exclusive lock. LOCK_TYPE_EXCLUSIVE = "exclusive" // LOCK_TYPE_SHARED is the string representation of a shared lock. LOCK_TYPE_SHARED = "shared" )
Variables ¶
var ( // ErrAlreadyLocked is returned by a locking operation when a resource // is already locked. ErrAlreadyLocked = errors.New("unable to acquire lock (resource is already locked)") // ErrLockNotFound is returned when a lock cannot be found. ErrLockNotFound = errors.New("unable to find lock") UPSERT = true ReturnDoc = options.After )
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
A Client is a distributed locking client backed by MongoDB. It requires a resource name (the object that gets locked) and a lockId (lock identifier) when creating a lock. Multiple locks can be created with the same lockId, making it easy to unlock or renew a group of related locks at the same time. Another reason for using lockIds is to ensure that only the process that creates a lock knows the lockId needed to unlock it - knowing a resource name alone is not enough to unlock it.
func NewClient ¶
func NewClient(collection *mongo.Collection) *Client
NewClient creates a new Client.
func (*Client) CreateIndexes ¶
CreateIndexes creates the required and recommended indexes for mongo-lock in the client's database. Indexes that already exist are skipped.
func (*Client) Renew ¶
Renew updates the TTL of all locks with the associated lockId. The new TTL for the locks will be the value of the argument provided. Only locks with a TTL greater than or equal to 1 can be renewed, so Renew will return an error if any of the locks with the associated lockId have a TTL of 0. This is to prevent a potential race condition beteween renewing locks and purging expired ones.
A LockStatus struct is returned for each lock that is renewed. The TTL field in each struct represents the new TTL. This information can be used to ensure that all of the locks created with the lockId have been renewed. If not all locks have been renewed, or if an error is returned, the caller should assume that none of the locks are safe to use, and they should unlock the lockId.
func (*Client) SLock ¶
func (c *Client) SLock(ctx context.Context, resourceName, lockId string, ld LockDetails, maxConcurrent int) error
SLock creates a shared lock on a resource and associates it with the provided lockId. Additional details about the lock can be supplied via LockDetails.
The maxConcurrent argument is used to limit the number of shared locks that can exist on a resource concurrently. When SLock is called with maxCurrent = N, the lock request will fail unless there are less than N shared locks on the resource already. If maxConcurrent is negative then there is no limit to the number of shared locks that can exist.
func (*Client) Status ¶
Status returns the status of locks that match the provided filter. Fields with zero values in the Filter struct are ignored.
func (*Client) Unlock ¶
Unlock unlocks all locks with the associated lockId. If there are multiple locks with the given lockId, they will be unlocked in the reverse order in which they were created in (newest to oldest). For every lock that is unlocked, a LockStatus struct (representing the lock before it was unlocked) is returned. The order of these is the order in which they were unlocked.
An error will only be returned if there is an issue unlocking a lock; an error will not be returned if a lock does not exist. If an error is returned, it is safe and recommended to retry this method until until there is no error.
type Filter ¶
type Filter struct {
CreatedBefore time.Time // Only include locks created before this time.
CreatedAfter time.Time // Only include locks created after this time.
TTLlt uint // Only include locks with a TTL less than this value, in seconds.
TTLgte uint // Only include locks with a TTL greater than or equal to this value, in seconds.
Resource string // Only include locks on this resource.
LockId string // Only include locks with this lockId.
Owner string // Only include locks with this owner.
}
Filter contains fields that are used to filter locks when querying their status. Fields with zero values are ignored.
type LockDetails ¶
type LockDetails struct {
// The user that is creating the lock.
Owner string
// The host that the lock is being created from.
Host string
// The time to live (TTL) for the lock, in seconds. Setting this to 0
// means that the lock will not have a TTL.
TTL uint
}
LockDetails contains fields that are used when creating a lock.
type LockStatus ¶
type LockStatus struct {
// The name of the resource that the lock is on.
Resource string
// The id of the lock.
LockId string
// The type of the lock ("exclusive" or "shared")
Type string
// The name of the user who created the lock.
Owner string
// The host that the lock was created from.
Host string
// The time that the lock was created at.
CreatedAt time.Time
// The time that the lock was renewed at, if applicable.
RenewedAt *time.Time
// The TTL for the lock, in seconds. A negative value means that the
// lock does not have a TTL.
TTL int64
// contains filtered or unexported fields
}
LockStatus represents the status of a lock.
type LockStatusesByCreatedAtDesc ¶
type LockStatusesByCreatedAtDesc []LockStatus
LockStatusesByCreatedAtDesc is a slice of LockStatus structs, ordered by CreatedAt descending.
func (LockStatusesByCreatedAtDesc) Len ¶
func (ls LockStatusesByCreatedAtDesc) Len() int
func (LockStatusesByCreatedAtDesc) Less ¶
func (ls LockStatusesByCreatedAtDesc) Less(i, j int) bool
func (LockStatusesByCreatedAtDesc) Swap ¶
func (ls LockStatusesByCreatedAtDesc) Swap(i, j int)
type Purger ¶
type Purger interface {
// Purge deletes expired locks. It is important to note that not just
// locks with a TTL of 0 get deleted - any lock that shares a lockId
// with a lock that has has a TTL of 0 will be removed.
Purge(ctx context.Context) ([]LockStatus, error)
}
A Purger deletes expired locks.
Source Files