dbresolver

package
v0.0.1-20220809-0010 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2022 License: MIT Imports: 24 Imported by: 0

README

DBResolver

DBResolver adds multiple databases support to GORM, the following features are supported:

  • Multiple sources, replicas
  • Read/Write Splitting
  • Automatic connection switching based on the working table/struct
  • Manual connection switching
  • Sources/Replicas load balancing
  • Works for RAW SQL

Quick Start

import (
  "gorm.io/gorm"
  "gorm.io/plugin/dbresolver"
  "gorm.io/driver/mysql"
)

DB, err := gorm.Open(mysql.Open("db1_dsn"), &gorm.Config{})

DB.Use(dbresolver.Register(dbresolver.Config{
  // use `db2` as sources, `db3`, `db4` as replicas
  Sources:  []gorm.Dialector{mysql.Open("db2_dsn")},
  Replicas: []gorm.Dialector{mysql.Open("db3_dsn"), mysql.Open("db4_dsn")},
  // sources/replicas load balancing policy
  Policy: dbresolver.RandomPolicy{},
}).Register(dbresolver.Config{
  // use `db1` as sources (DB's default connection), `db5` as replicas for `User`, `Address`
  Replicas: []gorm.Dialector{mysql.Open("db5_dsn")},
}, &User{}, &Address{}).Register(dbresolver.Config{
  // use `db6`, `db7` as sources, `db8` as replicas for `orders`, `Product`
  Sources:  []gorm.Dialector{mysql.Open("db6_dsn"), mysql.Open("db7_dsn")},
  Replicas: []gorm.Dialector{mysql.Open("db8_dsn")},
}, "orders", &Product{}, "secondary"))
Transaction

When using transaction, DBResolver will use the transaction and won't switch to sources/replicas

Automatic connection switching

DBResolver will automatically switch connection based on the working table/struct

For RAW SQL, DBResolver will extract the table name from the SQL to match the resolver, and will use sources unless the SQL begins with SELECT, for example:

// `User` Resolver Examples
DB.Table("users").Rows() // replicas `db5`
DB.Model(&User{}).Find(&AdvancedUser{}) // replicas `db5`
DB.Exec("update users set name = ?", "jinzhu") // sources `db1`
DB.Raw("select name from users").Row().Scan(&name) // replicas `db5`
DB.Create(&user) // sources `db1`
DB.Delete(&User{}, "name = ?", "jinzhu") // sources `db1`
DB.Table("users").Update("name", "jinzhu") // sources `db1`

// Global Resolver Examples
DB.Find(&Pet{}) // replicas `db3`/`db4`
DB.Save(&Pet{}) // sources `db2`

// Orders Resolver Examples
DB.Find(&Order{}) // replicas `db8`
DB.Table("orders").Find(&Report{}) // replicas `db8`
Read/Write Splitting

Read/Write splitting with DBResolver based on the current using GORM callback.

For Query, Row callback, will use replicas unless Write mode specified For Raw callback, statements are considered read-only and will use replicas if the SQL starts with SELECT

Manual connection switching
// Use Write Mode: read user from sources `db1`
DB.Clauses(dbresolver.Write).First(&user)

// Specify Resolver: read user from `secondary`'s replicas: db8
DB.Clauses(dbresolver.Use("secondary")).First(&user)

// Specify Resolver and Write Mode: read user from `secondary`'s sources: db6 or db7
DB.Clauses(dbresolver.Use("secondary"), dbresolver.Write).First(&user)
Load Balancing

GORM supports load balancing sources/replicas based on policy, the policy is an interface implements following interface:

type Policy interface {
	Resolve([]gorm.ConnPool) gorm.ConnPool
}

Currently only the RandomPolicy implemented and it is the default option if no policy specified.

Connection Pool
DB.Use(
  dbresolver.Register(dbresolver.Config{ /* xxx */ }).
  SetConnMaxIdleTime(time.Hour).
  SetConnMaxLifetime(24 * time.Hour).
  SetMaxIdleConns(100).
  SetMaxOpenConns(200)
)

Documentation

Index

Constants

View Source 
const ResolverAllNodesOffline = 1
View Source 
const ResolverReadOnly = 3
View Source 
const ResolverReadyToProcess = 2
View Source 
const ResolverStartingUp = 3
View Source 
const TConsecutive = "consecutive"
View Source 
const TLoadBalancing = "load_balancing"
View Source 
const TRandom = "random"
View Source 
const TRoundRobin = "round_robin"

Letter "T" it's

Variables

This section is empty.

Functions

func Use 

func Use(str string) clause.Interface

Types

type Config 

type Config struct {
	Sources  []gorm.Dialector
	Replicas []gorm.Dialector
	Policy   Policy

	// Logger
	Logger *model.Logger
	// Context, we use it for inside mechanisms like connection monitoring
	Ctx context.Context
	// contains filtered or unexported fields
}

type DBResolver 

type DBResolver struct {
	*gorm.DB

	// Logger
	Logger *model.Logger
	// contains filtered or unexported fields
}

func New 

func New() *DBResolver

func Register 

func Register(
	config Config,
	datas ...interface{},
) (*DBResolver, error)

func (*DBResolver) Call 

func (dr *DBResolver) Call(fc func(connPool gorm.ConnPool) error) error

func (*DBResolver) Initialize 

func (dr *DBResolver) Initialize(db *gorm.DB) error

Initialize this is how gorm initializes our code...

func (*DBResolver) IsTerminating 

func (dr *DBResolver) IsTerminating() bool

func (*DBResolver) LDebug 

func (dr *DBResolver) LDebug() *zerolog.Event

LDebug -> 0

func (*DBResolver) LDebugF 

func (dr *DBResolver) LDebugF(functionName string) *zerolog.Event

LDebugF -> when you need specifically to indicate in what function the logging is happening

func (*DBResolver) LError 

func (dr *DBResolver) LError() *zerolog.Event

LError -> 3

func (*DBResolver) LErrorF 

func (dr *DBResolver) LErrorF(functionName string) *zerolog.Event

LErrorF -> when you need specifically to indicate in what function the logging is happening

func (*DBResolver) LEvent 

func (dr *DBResolver) LEvent(eventType string, eventName string, beforeMsg func(event *zerolog.Event))

func (*DBResolver) LEventCustom 

func (dr *DBResolver) LEventCustom(eventType string, eventName string) *zerolog.Event

func (*DBResolver) LEventF 

func (dr *DBResolver) LEventF(eventType string, eventName string, functionName string) *zerolog.Event

func (*DBResolver) LFatal 

func (dr *DBResolver) LFatal() *zerolog.Event

LFatal -> 4

func (*DBResolver) LInfo 

func (dr *DBResolver) LInfo() *zerolog.Event

LInfo -> 1

func (*DBResolver) LInfoF 

func (dr *DBResolver) LInfoF(functionName string) *zerolog.Event

LInfoF -> when you need specifically to indicate in what function the logging is happening

func (*DBResolver) LPanic 

func (dr *DBResolver) LPanic() *zerolog.Event

LPanic -> 5

func (*DBResolver) LWarn 

func (dr *DBResolver) LWarn() *zerolog.Event

LWarn -> 2

func (*DBResolver) LWarnF 

func (dr *DBResolver) LWarnF(functionName string) *zerolog.Event

LWarnF -> when you need specifically to indicate in what function the logging is happening

func (*DBResolver) Name 

func (dr *DBResolver) Name() string

func (*DBResolver) Register 

func (dr *DBResolver) Register(
	config Config,
	datas ...interface{},
) (*DBResolver, error)

func (*DBResolver) SetConnMaxIdleTime 

func (dr *DBResolver) SetConnMaxIdleTime(d time.Duration) *DBResolver

func (*DBResolver) SetConnMaxLifetime 

func (dr *DBResolver) SetConnMaxLifetime(d time.Duration) *DBResolver

func (*DBResolver) SetMainConfig 

func (dr *DBResolver) SetMainConfig(config driver.Config)

func (*DBResolver) SetMaxIdleConns 

func (dr *DBResolver) SetMaxIdleConns(n int) *DBResolver

func (*DBResolver) SetMaxOpenConns 

func (dr *DBResolver) SetMaxOpenConns(n int) *DBResolver

type Operation 

type Operation string
const (
	Write Operation = "write"
	Read  Operation = "read"
)

func (Operation) Build 

func (op Operation) Build(clause.Builder)

func (Operation) MergeClause 

func (op Operation) MergeClause(*clause.Clause)

func (Operation) Name 

func (op Operation) Name() string

type PConsecutive 

type PConsecutive struct {
	PolicyPingOptions
	PolicyCommon
}

func (*PConsecutive) Resolve 

func (p *PConsecutive) Resolve(resolverOptions *resolveOptions) gorm.ConnPool

type PLoadBalancing 

type PLoadBalancing struct {
	PolicyPingOptions
	PolicyCommon
}

It's hard to know how to spread the traffic evenly, for that we need traffic analysis by communicating directly with the servers... or cluster TODO: We should know in % how busy is a node... TODO: for that we should have instances that are connected with the nodes, and which will read the % very often...

func (*PLoadBalancing) Resolve 

func (p *PLoadBalancing) Resolve(resolverOptions *resolveOptions) gorm.ConnPool

type PRandom 

type PRandom struct {
	PolicyPingOptions
	PolicyCommon
	// contains filtered or unexported fields
}

func (*PRandom) Resolve 

func (p *PRandom) Resolve(resolverOptions *resolveOptions) gorm.ConnPool

type PRoundRobin 

type PRoundRobin struct {
	PolicyPingOptions
	PolicyCommon
	// contains filtered or unexported fields
}

func NewRoundRobinPolicy 

func NewRoundRobinPolicy() (*PRoundRobin, error)

func (*PRoundRobin) GetConnID 

func (p *PRoundRobin) GetConnID() int

GetConnID -> Get's the connection id from the stack based on the policy

func (*PRoundRobin) Resolve 

func (p *PRoundRobin) Resolve(resolverOptions *resolveOptions) gorm.ConnPool

Resolve -> get the connection based on the current policy

type Policy 

type Policy interface {
	Resolve(resolveOptions *resolveOptions) gorm.ConnPool
	// Ping retry...
	SetIsPingRetryEnabled(isEnabled bool)
	SetPingRetryTimes(retryTimes int16)
	SetPingRetryDelaySeconds(delaySeconds uint16)
}

type PolicyCommon 

type PolicyCommon struct {
	// contains filtered or unexported fields
}

func (*PolicyCommon) GetConnPools 

func (p *PolicyCommon) GetConnPools() []detailedConnPool

GetConnPools -> get all connections

func (*PolicyCommon) GetContext 

func (p *PolicyCommon) GetContext() context.Context

func (*PolicyCommon) GetNrOfConns 

func (p *PolicyCommon) GetNrOfConns() int

func (*PolicyCommon) LDebugF 

func (p *PolicyCommon) LDebugF(functionName string) *zerolog.Event

func (*PolicyCommon) LErrorF 

func (p *PolicyCommon) LErrorF(functionName string) *zerolog.Event

func (*PolicyCommon) LInfoF 

func (p *PolicyCommon) LInfoF(functionName string) *zerolog.Event

func (*PolicyCommon) LWarnF 

func (p *PolicyCommon) LWarnF(functionName string) *zerolog.Event

type PolicyPingOptions 

type PolicyPingOptions struct {
	IsPingRetryEnabled    string `default:"yes"`
	PingRetryTimes        int16  `default:"3"`
	PingRetryDelaySeconds uint16 `default:"5"`
}

func (*PolicyPingOptions) GetIsPingRetryEnabled 

func (p *PolicyPingOptions) GetIsPingRetryEnabled() bool

func (*PolicyPingOptions) GetPingRetryDelaySeconds 

func (p *PolicyPingOptions) GetPingRetryDelaySeconds() uint16

GetPingRetryDelaySeconds -> when checking if connection is alive, this is the delay between rechecks

func (*PolicyPingOptions) GetPingRetryTimes 

func (p *PolicyPingOptions) GetPingRetryTimes() int16

GetPingRetryTimes -> how many times it should retry the ping...

func (*PolicyPingOptions) SetIsPingRetryEnabled 

func (p *PolicyPingOptions) SetIsPingRetryEnabled(isEnabled bool)

func (*PolicyPingOptions) SetPingRetryDelaySeconds 

func (p *PolicyPingOptions) SetPingRetryDelaySeconds(delaySeconds uint16)

func (*PolicyPingOptions) SetPingRetryTimes 

func (p *PolicyPingOptions) SetPingRetryTimes(retryTimes int16)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.