snowflake

README

An Lock Free ID Generator for Golang based on Snowflake Algorithm (Twitter announced).

Description

An Lock Free ID Generator for Golang implementation.

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees.

• The first bit is unused sign bit.
• The second part consists of a 41-bit timestamp (milliseconds) whose value is the offset of the current time relative to a certain time.
• The 10 bits machineID(5 bit workid + 5 bit datacenter id), max value is 2^10 -1 = 1023.
• The last part consists of 12 bits, its means the length of the serial number generated per millisecond per working node, a maximum of 2^12 -1 = 4095 IDs can be generated in the same millisecond.
• The binary length of 41 bits is at most 2^41 -1 millisecond = 69 years. So the snowflake algorithm can be used for up to 69 years, In order to maximize the use of the algorithm, you should specify a start time for it.

The ID generated by the snowflake algorithm is not guaranteed to be unique. For example, when two different requests enter the same machine at the same time, and the sequence generated by the node is the same, the generated ID will be duplicated.

So if you want use the snowflake algorithm to generate unique ID, You must ensure: The sequence-number generated in the same millisecond of the same node is unique.

Based on this, we created this package and integrated multiple sequence-number providers into it.

• AtomicResolver (base sync/atomic)

Each provider only needs to ensure that the serial number generated in the same millisecond is different. You can get a unique ID.

Feature

• ✅ Lock Free
• 🎈 Zero configuration, out of the box
• 🚀 Concurrency safety
• 🌵 Support private ip to machineid
• 🐡 Support custom sequence resolver

Installation

$ go get github.com/AniTrack/go-snowflake-modified

Usage

1. simple to use.

package main

import (
    "fmt"

    "github.com/AniTrack/go-snowflake-modified"
)

func main() {
    id := snowflake.ID()
    fmt.Println(id)
    // 1537200202186752
}

1. Specify the MachineID.

package main

import (
    "fmt"

    "github.com/AniTrack/go-snowflake-modified"
)

func main() {
    snowflake.SetMachineID(1)

    // Or set private ip to machineid, testing...
    // snowflake.SetMachineID(snowflake.PrivateIPToMachineID())

    id := snowflake.ID()
    fmt.Println(id)
}

1. Specify start time.

package main

import (
    "fmt"
    "time"

    "github.com/AniTrack/go-snowflake-modified"
)

func main() {
    snowflake.SetStartTime(time.Date(2014, 9, 1, 0, 0, 0, 0, time.UTC))
    id := snowflake.ID()
    fmt.Println(id)
}

1. Parse ID.

package main

import (
    "fmt"
    "time"

    "github.com/AniTrack/go-snowflake-modified"
)

func main() {
    id := snowflake.ID()
    sid := snowflake.ParseID(id)

    fmt.Println(sid.ID)             // 132271570944000000
    fmt.Println(sid.MachineID)      // 0
    fmt.Println(sid.Sequence)       // 0
    fmt.Println(sid.Timestamp)      // 31536000000
    fmt.Println(sid.GenerateTime()) // 2009-11-10 23:00:00 +0000 UTC
}

Best practices

⚠️⚠️ All SetXXX method is thread-unsafe, recommended you call him in the main function.

package main

import (
    "fmt"
    "time"
    "net/http"

    "github.com/AniTrack/go-snowflake-modified"
)

func main() {
    snowflake.SetMachineID(1) // change to your machineID
    snowflake.SetStartTime(time.Date(2014, 9, 1, 0, 0, 0, 0, time.UTC))

    http.HandleFunc("/order", submitOrder)
    http.ListenAndServe(":8090", nil)
}

func submitOrder(w http.ResponseWriter, req *http.Request) {
    orderId := snowflake.ID()
    // save order
}

Advanced

Custom sequence resolver. you can customize the sequence-number resolver by following way:

package main

import (
    "fmt"
    "time"

    "github.com/AniTrack/go-snowflake-modified"
)

func yourSequenceNumber(ms int64) (uint16, error) {

}

// usage

snowflake.SetSequenceResolver(yourSequenceNumber)
snowflake.ID()

License

MIT

Documentation

Overview

Package snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees. The first bit is unused sign bit. The second part consists of a 41-bit timestamp (milliseconds) whose value is the offset of the current time relative to a certain time. The 5 bits of the third and fourth parts represent data center and worker, and max value is 2^5 -1 = 31. The last part consists of 12 bits, its means the length of the serial number generated per millisecond per working node, a maximum of 2^12 -1 = 4095 IDs can be generated in the same millisecond. In a distributed environment, five-bit datacenter and worker mean that can deploy 31 datacenters, and each datacenter can deploy up to 31 nodes. The binary length of 41 bits is at most 2^41 -1 millisecond = 69 years. So the snowflake algorithm can be used for up to 69 years, In order to maximize the use of the algorithm, you should specify a start time for it.

Index

• Constants
• func AtomicResolver(ms int64) (uint16, error)
• func ID() uint64
• func NextID() (uint64, error)
• func PrivateIPToMachineID() uint16
• func SetMachineID(m uint16)
• func SetSequenceResolver(seq SequenceResolver)
• func SetStartTime(s time.Time)
• type SID
  • func ParseID(id uint64) SID
  • func (id *SID) GenerateTime() time.Time
• type SequenceResolver

Constants

const (
	TimestampLength uint8  = 42 // Increased 1 bit
	MachineIDLength uint8  = 10
	SequenceLength  uint8  = 11 // Decreased 1 bit
	MaxSequence     uint16 = 1<<SequenceLength - 1
	MaxTimestamp    uint64 = 1<<TimestampLength - 1
	MaxMachineID    uint16 = 1<<MachineIDLength - 1
)

These constants are the bit lengths of snowflake ID parts.

Functions

func AtomicResolver

func AtomicResolver(ms int64) (uint16, error)

AtomicResolver define as atomic sequence resolver, base on standard sync/atomic.

func ID

func ID() uint64

ID use ID to generate snowflake id, and it will ignore error. if you want error info, you need use NextID method. This function is thread safe.

func NextID

func NextID() (uint64, error)

NextID use NextID to generate snowflake id and return an error. This function is thread safe.

func PrivateIPToMachineID

func PrivateIPToMachineID() uint16

PrivateIPToMachineID convert private ip to machine id. From https://github.com/sony/sonyflake/blob/master/sonyflake.go

func SetMachineID

func SetMachineID(m uint16)

SetMachineID specify the machine ID. It will panic when machined > max limit for 2^10-1. This function is thread-unsafe, recommended you call him in the main function.

func SetSequenceResolver

func SetSequenceResolver(seq SequenceResolver)

SetSequenceResolver set a custom sequence resolver. This function is thread-unsafe, recommended you call him in the main function.

func SetStartTime

func SetStartTime(s time.Time)

SetStartTime set the start time for snowflake algorithm.

It will panic when:

s IsZero
s > current millisecond,
current millisecond - s > 2^41(69 years).

This function is thread-unsafe, recommended you call him in the main function.

Types

type SID

type SID struct {
	Sequence  uint64
	MachineID uint64
	Timestamp uint64
	ID        uint64
}

SID snowflake id

func ParseID

func ParseID(id uint64) SID

ParseID parse snowflake it to SID struct.

func (*SID) GenerateTime

func (id *SID) GenerateTime() time.Time

GenerateTime snowflake generate at, return a UTC time.

type SequenceResolver

type SequenceResolver func(ms int64) (uint16, error)

SequenceResolver the snowflake sequence resolver.

When you want to use the snowflake algorithm to generate unique ID, You must ensure: The sequence-number generated in the same millisecond of the same node is unique. Based on this, we create this interface provide following resolver:

AtomicResolver : base sync/atomic (by default).