README
¶
snowflake
snowflake is a Go package that provides
- A very simple Twitter snowflake generator.
- Methods to parse existing snowflake IDs.
- Methods to convert a snowflake ID into several other data types and back.
- JSON Marshal/Unmarshal functions to easily use snowflake IDs within a JSON API.
- Monotonic Clock calculations protect from clock drift.
- Default nodeID is set to main last part of IPv4 (on en0/eth0 interface), eg. nodeID = 3 when IP is 192.168.1.3
For help with this package or general Go discussion, please join the Discord Gophers chat server.
Status
This package should be considered stable and completed. Any additions in the future will strongly avoid API changes to existing functions.
ID Format
By default, the ID format follows the original Twitter snowflake format.
- The ID as a whole is a 63 bit integer stored in an int64
- 41 bits used to store a timestamp with millisecond precision, using a custom epoch.
- 10 bits used to store a node id - a range from 0 through 1023.
- 12 bits used to store a sequence number - a range from 0 through 4095.
Custom Format
You can alter the number of bits used for the node id and step number (sequence) by setting the snowflake.NodeBits and snowflake.StepBits values. Remember that There is a maximum of 22 bits available that can be shared between these two values. You do not have to use all 22 bits.
Custom Epoch
By default this package uses the Twitter Epoch of 1288834974657 or Nov 04 2010 01:42:54. You can set your own epoch value by setting snowflake.Epoch to a time in milliseconds to use as the epoch.
Custom Notes
When setting custom epoch or bit values you need to set them prior to calling any functions on the snowflake package, including NewNode(). Otherwise the custom values you set will not be applied correctly.
How it Works.
Each time you Next an ID, it works, like this.
- A timestamp with millisecond precision is stored using 41 bits of the ID.
- Then the NodeID is added in subsequent bits.
- Then the Sequence Number is added, starting at 0 and incrementing for each ID generated in the same millisecond. If you Next enough IDs in the same millisecond that the sequence would roll over or overfill then the Next function will pause until the next millisecond.
The default Twitter format shown below.
+--------------------------------------------------------------------------+
| 1 Bit Unused | 41 Bit Timestamp | 10 Bit NodeID | 12 Bit Sequence ID |
+--------------------------------------------------------------------------+
Using the default settings, this allows for 4096 unique IDs to be generated every millisecond, per Node ID.
Getting Started
Installing
This assumes you already have a working Go environment, if not please see this page first.
go get github.com/bingoohuang/snow
Usage
Import the package into your project then construct a new snowflake Node using a unique node number. The default settings permit a node number range from 0 to 1023. If you have set a custom NodeBits value, you will need to calculate what your node number range will be. With the node object call the Next() method to Next and return a unique snowflake ID.
Keep in mind that each node you create must have a unique node number, even across multiple servers. If you do not keep node numbers unique the generator cannot guarantee unique IDs across all nodes.
Example Program
Use default Snowflake Node
// Next return a new generated snowflake ID by the Global snow.DefaultNode
// with the 10-bits node ID which get the last 8-bits from host IP v4.
id := snow.Next()
// Print out the ID in a few different ways.
fmt.Printf("Int64 ID: %d\n", id)
fmt.Printf("String ID: %s\n", id)
Use customized Snowflake Node
// Create a new Node with a Node number of 1
node, err := snow.NewNode(snow.WithNodeID(1))
if err != nil {
fmt.Println(err)
return
}
// Next return a new generated snowflake ID.
id := node.Next()
// Print out the ID in a few different ways.
fmt.Printf("Int64 ID: %d\n", id)
fmt.Printf("String ID: %s\n", id)
fmt.Printf("Base2 ID: %s\n", id.Base2())
fmt.Printf("Base64 ID: %s\n", id.Base64())
fmt.Printf("ID : %d\n", node.Next().Int64())
Performance
With default settings, this snowflake generator should be sufficiently fast enough on most systems to Next 4096 unique ID's per millisecond. This is the maximum that the snowflake ID format supports. That is, around 243-244 nanoseconds per operation.
Since the snowflake generator is single threaded the primary limitation will be the maximum speed of a single processor on your system.
To benchmark the generator on your system run the following command inside the snowflake package directory.
go test -run=^$ -bench=.
Documentation
¶
Overview ¶
Package snow provides a very simple Twitter snowflake generator and parser.
Index ¶
- Variables
- func GetEpoch() int64
- func GetNodeID() int64
- func GetStep() int64
- func GetTime() int64
- func IP2Uint32(ip net.IP) uint32
- type ID
- func Next() ID
- func Next32() ID
- func ParseBase2(id string) (ID, error)
- func ParseBase32(b []byte) (ID, error)
- func ParseBase36(id string) (ID, error)
- func ParseBase58(b []byte) (ID, error)
- func ParseBase64(id string) (ID, error)
- func ParseBytes(id []byte) (ID, error)
- func ParseInt64(id int64) ID
- func ParseIntBytes(id [8]byte) ID
- func ParseString(id string) (ID, error)
- func (f ID) Base2() string
- func (f ID) Base32() string
- func (f ID) Base36() string
- func (f ID) Base58() string
- func (f ID) Base64() string
- func (f ID) Bytes() []byte
- func (f ID) Int32() int32
- func (f ID) Int64() int64
- func (f ID) IntBytes() [8]byte
- func (f ID) MarshalJSON() ([]byte, error)
- func (f ID) NodeID() int64
- func (f ID) Step() int64
- func (f ID) String() string
- func (f ID) Time() int64
- func (f ID) Uint32() uint32
- func (f ID) Uint64() uint64
- func (f *ID) UnmarshalJSON(b []byte) error
- type JSONSyntaxError
- type Node
- type Option
- type OptionFn
Constants ¶
This section is empty.
Variables ¶
var DefaultNode, _ = NewNode()
DefaultNode is the global default snowflake node object. nolint gochecknoglobals
var DefaultNode32, _ = NewNode(
WithNodeBits(2),
WithStepBits(1),
WithTimestampUnit(1*time.Second),
WithEpoch(1577808000000),
)
var ( // ErrInvalidBase32 is returned by ParseBase32 when given an invalid []byte ErrInvalidBase32 = errors.New("invalid base32") )
nolint gochecknoglobals
var ( // ErrInvalidBase58 is returned by ParseBase58 when given an invalid []byte ErrInvalidBase58 = errors.New("invalid base58") )
nolint gochecknoglobals
Functions ¶
func GetEpoch ¶
func GetEpoch() int64
GetEpoch returns an int64 epoch is snowflake epoch in milliseconds.
func GetStep ¶
func GetStep() int64
GetStep returns an int64 of the snowflake step (or sequence) number
Types ¶
type ID ¶
type ID int64
An ID is a custom type used for a snowflake ID. This is used so we can attach methods onto the ID.
func Next ¶
func Next() ID
Next creates and returns a unique snowflake ID To help guarantee uniqueness - Make sure your system is keeping accurate system time - Make sure you never have multiple nodes running with the same node ID
func Next32 ¶
func Next32() ID
Next32 creates and returns a unique snowflake ID for positive int32. only for low frequency usages. unsigned(1) + timestamp(28) + node ID(2) + step(1) can use 2^28/60/60/24/365 ≈ 8.5 年 result example： 459260906 The range of int32 is [-2147483648, 2147483647] and the uint32 is [0, 4294967295].
func ParseBase2 ¶
ParseBase2 converts a Base2 string into a snowflake ID
func ParseBase32 ¶
ParseBase32 parses a base32 []byte into a snowflake ID NOTE: There are many different base32 implementations so becareful when doing any interoperation.
func ParseBase36 ¶
ParseBase36 converts a Base36 string into a snowflake ID
func ParseBase58 ¶
ParseBase58 parses a base58 []byte into a snowflake ID
func ParseBase64 ¶
ParseBase64 converts a base64 string into a snowflake ID
func ParseBytes ¶
ParseBytes converts a byte slice into a snowflake ID
func ParseIntBytes ¶
ParseIntBytes converts an array of bytes encoded as big endian integer as a snowflake ID
func ParseString ¶
ParseString converts a string into a snowflake ID
func (ID) Base32 ¶
Base32 uses the z-base-32 character set but encodes and decodes similar to base58, allowing it to create an even smaller result string. NOTE: There are many different base32 implementations so becareful when doing any interoperation. nolint gomnd
func (ID) IntBytes ¶
IntBytes returns an array of bytes of the snowflake ID, encoded as a big endian integer.
func (ID) MarshalJSON ¶
MarshalJSON returns a json byte array string of the snowflake ID.
func (*ID) UnmarshalJSON ¶
UnmarshalJSON converts a json byte array of a snowflake ID into an ID type.
type JSONSyntaxError ¶
type JSONSyntaxError struct {
// contains filtered or unexported fields
}
A JSONSyntaxError is returned from UnmarshalJSON if an invalid ID is provided.
func (JSONSyntaxError) Error ¶
func (j JSONSyntaxError) Error() string
type Node ¶
type Node struct {
// contains filtered or unexported fields
}
A Node struct holds the basic information needed for a snowflake generator node
func (*Node) GetTime ¶
GetTime returns an int64 unix timestamp in milliseconds of the snowflake ID time.
func (*Node) Next ¶
Next creates and returns a unique snowflake ID To help guarantee uniqueness - Make sure your system is keeping accurate system time - Make sure you never have multiple nodes running with the same node ID
type Option ¶
type Option struct {
// Epoch is set to the twitter snowflake epoch of Nov 04 2010 01:42:54 UTC in milliseconds
// You may customize this to set a different epoch for your application.
Epoch int64 // 1288834974657
// NodeBits holds the number of bits to use for Node
// Remember, you have a total 22 bits to share between Node/Step
NodeBits int8 // 10
// StepBits holds the number of bits to use for Step
// Remember, you have a total 22 bits to share between Node/Step
StepBits int8 // 12
// NodeID for the snowflake.
NodeID int64
// TimestampUnit for the time goes unit, default is 1ms.
TimestampUnit time.Duration
}
Option for the snowflake
type OptionFn ¶
type OptionFn func(*Option)
OptionFn defines the function prototype to apply options.
func WithNodeIDLocalIP ¶
WithNodeIDLocalIP set the customized nodeID with the last 8 bits of local IP v4 and first 2 bits of p.
func WithTimestampUnit ¶
WithTimestampUnit set the customized TimestampUnit n.
Source Files