datastore

package
v1.0.0-beta.39 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 18, 2024 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package datastore is the database service abstraction layer

Index

Constants

View Source 
const (

	// SortDesc will sort descending
	SortDesc = "desc"

	// SortAsc will sort ascending
	SortAsc = "asc"
)

Defaults for library functionality

View Source 
const (
	Postgres = "postgres"
	JSON     = "JSON"
	JSONB    = "JSONB"
)

index creation constants

Variables

View Source 
var DateFields = []string{dateCreatedAt, dateUpdatedAt, dateModifiedAt}

DateFields are standard known date fields

View Source 
var ErrMigrationError = spverrors.Newf("error occurred during migration")

ErrMigrationError is when something went wrong during migration

View Source 
var ErrNoSourceFound = spverrors.Newf("no source database found in all given configurations")

ErrNoSourceFound is when no source database is found in all given configurations

View Source 
var ErrUnsupportedDriver = spverrors.Newf("sql driver unsupported")

ErrUnsupportedDriver is when the given SQL driver is not determined to be known or supported

View Source 
var ErrUnsupportedEngine = spverrors.Newf("unsupported datastore engine")

ErrUnsupportedEngine is used when the engine given is not a known datastore engine

View Source 
var SQLDatabases = []Engine{
	PostgreSQL,
	SQLite,
}

SQLDatabases is the list of supported SQL databases (via GORM)

Functions

func ApplyCustomWhere 

func ApplyCustomWhere(client ClientInterface, gdb *gorm.DB, conditions map[string]interface{}, model interface{}) (tx *gorm.DB, err error)

ApplyCustomWhere adds conditions to the gorm db instance it returns a tx of type *gorm.DB with a model and conditions applied

func GetColumnName 

func GetColumnName(columnName string, model interface{}, gdb *gorm.DB) (string, bool)

GetColumnName checks if the model has provided columnName as DBName or (struct field)Name Returns (DBName, true) if the column exists otherwise (_, false) Uses global cache store (thread safe) Checking is case-sensitive The gdb param is optional. When is provided, the actual naming strategy is used; otherwise default

func GetModelBoolAttribute 

func GetModelBoolAttribute(model interface{}, attribute string) *bool

GetModelBoolAttribute retrieves a boolean attribute from the model

func GetModelName 

func GetModelName(model interface{}) *string

GetModelName gets the name of the model via reflection

func GetModelStringAttribute 

func GetModelStringAttribute(model interface{}, attribute string) *string

GetModelStringAttribute retrieves a string attribute from the model

func GetModelTableName 

func GetModelTableName(model interface{}) *string

GetModelTableName gets the db table name of the model via reflection

func GetModelType 

func GetModelType(model interface{}) reflect.Type

GetModelType gets the model type of the model interface via reflection

func GetModelUnset 

func GetModelUnset(model interface{}) map[string]bool

GetModelUnset identifies fields in the model that should be unset (i.e., cleared) in the database

func IsModelSlice 

func IsModelSlice(model interface{}) bool

IsModelSlice returns true if the given interface is a slice of models

func IsSQLEngine 

func IsSQLEngine(e Engine) bool

IsSQLEngine check whether the string already is in the slice

func MarshalQueryParams 

func MarshalQueryParams(m QueryParams) graphql.Marshaler

MarshalQueryParams will marshal the custom type

func StringInSlice 

func StringInSlice(a string, list []string) bool

StringInSlice checks whether a string is present in a slice

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the datastore client (configuration)

func (*Client) Close 

func (c *Client) Close() error

Close will terminate (close) the datastore and any open connections

func (*Client) CreateInBatches 

func (c *Client) CreateInBatches(
	_ context.Context,
	models interface{},
	batchSize int,
) error

CreateInBatches create all the models given in batches

func (*Client) DB 

func (c *Client) DB() *gorm.DB

DB returns ready to use gorm instance

func (*Client) Debug 

func (c *Client) Debug(on bool)

Debug will set the debug flag

func (*Client) DebugLog 

func (c *Client) DebugLog(ctx context.Context, text string)

DebugLog will display verbose logs

func (*Client) Engine 

func (c *Client) Engine() Engine

Engine will return the client's engine

func (*Client) Execute 

func (c *Client) Execute(query string) *gorm.DB

Execute a SQL query

func (*Client) GetArrayFields 

func (c *Client) GetArrayFields() []string

GetArrayFields will return the array fields

func (*Client) GetDatabaseName 

func (c *Client) GetDatabaseName() string

GetDatabaseName will return the full database name for the given model name

func (*Client) GetModel 

func (c *Client) GetModel(
	ctx context.Context,
	model interface{},
	conditions map[string]interface{},
	timeout time.Duration,
	forceWriteDB bool,
) error

GetModel will get a model from the datastore

func (*Client) GetModelCount 

func (c *Client) GetModelCount(
	ctx context.Context,
	model interface{},
	conditions map[string]interface{},
	timeout time.Duration,
) (int64, error)

GetModelCount will return a count of the model matching conditions

func (*Client) GetModels 

func (c *Client) GetModels(
	ctx context.Context,
	models interface{},
	conditions map[string]interface{},
	queryParams *QueryParams,
	fieldResults interface{},
	timeout time.Duration,
) error

GetModels will return a slice of models based on the given conditions

func (*Client) GetModelsAggregate 

func (c *Client) GetModelsAggregate(ctx context.Context, models interface{},
	conditions map[string]interface{}, aggregateColumn string, timeout time.Duration,
) (map[string]interface{}, error)

GetModelsAggregate will return an aggregate count of the model matching conditions

func (*Client) GetObjectFields 

func (c *Client) GetObjectFields() []string

GetObjectFields will return the object fields

func (*Client) GetTableName 

func (c *Client) GetTableName(modelName string) string

GetTableName will return the full table name for the given model name

func (*Client) IncrementModel 

func (c *Client) IncrementModel(
	_ context.Context,
	model interface{},
	fieldName string,
	increment int64,
) (newValue int64, err error)

IncrementModel will increment the given field atomically in the database and return the new value

func (*Client) IndexMetadata 

func (c *Client) IndexMetadata(tableName, field string) error

IndexMetadata check and creates the metadata json index

func (*Client) IsDebug 

func (c *Client) IsDebug() bool

IsDebug will return the debug flag (bool)

func (*Client) NewRawTx 

func (c *Client) NewRawTx() (*Transaction, error)

NewRawTx will start a new datastore transaction

func (*Client) NewTx 

func (c *Client) NewTx(_ context.Context, fn func(*Transaction) error) error

NewTx will start a new datastore transaction

func (*Client) Raw 

func (c *Client) Raw(query string) *gorm.DB

Raw a raw SQL query

func (*Client) SaveModel 

func (c *Client) SaveModel(
	_ context.Context,
	model interface{},
	tx *Transaction,
	newRecord, commitTx bool,
) error

SaveModel will take care of creating or updating a model (primary key based) (abstracting the database)

value is a pointer to the model

type ClientInterface 

type ClientInterface interface {
	GetterInterface
	StorageService
	Close() error
	Debug(on bool)
	DebugLog(ctx context.Context, text string)
	Engine() Engine
	IsDebug() bool
}

ClientInterface is the Datastore client interface

func NewClient 

func NewClient(opts ...ClientOps) (ClientInterface, error)

NewClient creates a new client for all Datastore functionality

If no options are given, it will use the defaultClientOptions()

type ClientOps 

type ClientOps func(c *clientOptions)

ClientOps allow functional options to be supplied that overwrite default client options.

func WithCustomFields 

func WithCustomFields(arrayFields []string, objectFields []string) ClientOps

WithCustomFields will add custom fields to the datastore

func WithDebugging 

func WithDebugging() ClientOps

WithDebugging will enable debugging mode

func WithLogger 

func WithLogger(customLogger zLogger.GormLoggerInterface) ClientOps

WithLogger will set the custom logger interface

func WithSQL 

func WithSQL(engine Engine, configs []*SQLConfig) ClientOps

WithSQL will load a datastore using either an SQL database config or existing connection

func WithSQLConnection 

func WithSQLConnection(engine Engine, sqlDB *sql.DB, tablePrefix string) ClientOps

WithSQLConnection will set the datastore to an existing connection for PostgreSQL

func WithSQLite 

func WithSQLite(config *SQLiteConfig) ClientOps

WithSQLite will set the datastore to use SQLite

type CommonConfig 

type CommonConfig struct {
	Debug                 bool          `json:"debug" mapstructure:"debug"`                                       // flag for debugging sql queries in logs
	MaxConnectionIdleTime time.Duration `json:"max_connection_idle_time" mapstructure:"max_connection_idle_time"` // 360
	MaxConnectionTime     time.Duration `json:"max_connection_time" mapstructure:"max_connection_time"`           // 60
	MaxIdleConnections    int           `json:"max_idle_connections" mapstructure:"max_idle_connections"`         // 5
	MaxOpenConnections    int           `json:"max_open_connections" mapstructure:"max_open_connections"`         // 5
	TablePrefix           string        `json:"table_prefix" mapstructure:"table_prefix"`                         // pre_users (pre)
}

CommonConfig is the common configuration fields between engines

type DatabaseLogWrapper 

type DatabaseLogWrapper struct {
	zLogger.GormLoggerInterface
}

DatabaseLogWrapper is a special wrapper for the GORM logger

func (*DatabaseLogWrapper) LogMode 

func (d *DatabaseLogWrapper) LogMode(level gLogger.LogLevel) gLogger.Interface

LogMode will set the log level/mode

type Engine 

type Engine string

Engine is the different engines that are supported (database) 

const (
	Empty      Engine = "empty"
	PostgreSQL Engine = "postgresql"
	SQLite     Engine = "sqlite"
)

Supported engines (databases)

func (Engine) IsEmpty 

func (e Engine) IsEmpty() bool

IsEmpty will return true if the datastore is not set

func (Engine) String 

func (e Engine) String() string

String is the string version of engine

type GetterInterface 

type GetterInterface interface {
	GetArrayFields() []string
	GetDatabaseName() string
	GetObjectFields() []string
	GetTableName(modelName string) string
}

GetterInterface is the getter methods

type QueryParams 

type QueryParams struct {
	Page          int    `json:"page,omitempty"`
	PageSize      int    `json:"page_size,omitempty"`
	OrderByField  string `json:"order_by_field,omitempty"`
	SortDirection string `json:"sort_direction,omitempty"`
}

QueryParams object to use when limiting and sorting database query results

func UnmarshalQueryParams 

func UnmarshalQueryParams(v interface{}) (QueryParams, error)

UnmarshalQueryParams will unmarshal the custom type

type SQLConfig 

type SQLConfig struct {
	CommonConfig       `json:",inline" mapstructure:",squash"` // Common configuration
	Driver             string                                  `json:"driver" mapstructure:"driver"`                      // postgresql
	ExistingConnection *sql.DB                                 `json:"-" mapstructure:"-"`                                // Used for existing database connection
	Host               string                                  `json:"host" mapstructure:"host"`                          // database host IE: localhost
	Name               string                                  `json:"name" mapstructure:"name"`                          // database-name
	Password           string                                  `json:"password" mapstructure:"password" encrypted:"true"` // user-password
	Port               string                                  `json:"port" mapstructure:"port"`                          // 3306
	Replica            bool                                    `json:"replica" mapstructure:"replica"`                    // True if it's a replica (Read-Only)
	TimeZone           string                                  `json:"time_zone" mapstructure:"time_zone"`                // timezone (IE: Asia/Shanghai)
	TxTimeout          time.Duration                           `json:"tx_timeout" mapstructure:"tx_timeout"`              // 5*time.Second
	User               string                                  `json:"user" mapstructure:"user"`                          // database username
	SslMode            string                                  `json:"ssl_mode" mapstructure:"ssl_mode"`                  // ssl mode (for PostgreSQL) [disable|allow|prefer|require|verify-ca|verify-full]
}

SQLConfig is the configuration for each SQL connection (postgresql)

type SQLiteConfig 

type SQLiteConfig struct {
	CommonConfig       `json:",inline" mapstructure:",squash"` // Common configuration
	DatabasePath       string                                  `json:"database_path" mapstructure:"database_path"` // Location of a permanent database file (if NOT set, uses temporary memory)
	ExistingConnection gorm.ConnPool                           `json:"-" mapstructure:"-"`                         // Used for existing database connection
	Shared             bool                                    `json:"shared" mapstructure:"shared"`               // Adds a shared param to the connection string
}

SQLiteConfig is the configuration for each SQLite connection

type StorageService 

type StorageService interface {
	CreateInBatches(ctx context.Context, models interface{}, batchSize int) error
	Execute(query string) *gorm.DB
	GetModel(ctx context.Context, model interface{}, conditions map[string]interface{},
		timeout time.Duration, forceWriteDB bool) error
	GetModels(ctx context.Context, models interface{}, conditions map[string]interface{}, queryParams *QueryParams,
		fieldResults interface{}, timeout time.Duration) error
	GetModelCount(ctx context.Context, model interface{}, conditions map[string]interface{},
		timeout time.Duration) (int64, error)
	GetModelsAggregate(ctx context.Context, models interface{}, conditions map[string]interface{},
		aggregateColumn string, timeout time.Duration) (map[string]interface{}, error)
	IncrementModel(ctx context.Context, model interface{},
		fieldName string, increment int64) (newValue int64, err error)
	IndexMetadata(tableName, field string) error
	NewTx(ctx context.Context, fn func(*Transaction) error) error
	NewRawTx() (*Transaction, error)
	Raw(query string) *gorm.DB
	SaveModel(ctx context.Context, model interface{}, tx *Transaction, newRecord, commitTx bool) error
	DB() *gorm.DB
}

StorageService is the storage related methods

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction is the internal datastore transaction

func (*Transaction) CanCommit 

func (tx *Transaction) CanCommit() bool

CanCommit will return true if it can commit

func (*Transaction) Commit 

func (tx *Transaction) Commit() error

Commit will commit the transaction

func (*Transaction) Rollback 

func (tx *Transaction) Rollback() error

Rollback the transaction

Source Files

View all Source files

Directories

Path Synopsis
Package customtypes encapsulates our custom database field types
 Package customtypes encapsulates our custom database field types
Package sqlite3extended is a workaround for disabled by default math functions in sqlite3 Unfortunately those math functions can be only enabled with build tag, Which means that we would need to force everyone, who is running any command like go build|run|test, to include also -tag "sqlite_math_functions" which looks like pretty big overhead and potential source of many registered issues just because someone overlooked that he needs to set this tag.
 Package sqlite3extended is a workaround for disabled by default math functions in sqlite3 Unfortunately those math functions can be only enabled with build tag, Which means that we would need to force everyone, who is running any command like go build|run|test, to include also -tag "sqlite_math_functions" which looks like pretty big overhead and potential source of many registered issues just because someone overlooked that he needs to set this tag.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.