lys

package module
v0.1.29 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 15, 2024 License: MIT Imports: 28 Imported by: 0

README

lys - LoveYourStack

Packages for rapid development of REST APIs handling database CRUD actions.

Only available for PostgreSQL. Most suitable for "database-first" Go developers.

Example usage

Define store package (wiki)

A store package contains database access functions for a specific table or view, in this case the "category" table in the "core" schema.

Boilerplate is minimized through the optional use of generic database CRUD functions.

package corecategory

// define constants for this database table, which get passed to generic database functions below
const (
	schemaName     string = "core"
	tableName      string = "category"
	viewName       string = "category"
	pkColName      string = "id"
	defaultOrderBy string = "name"
)

// columns required when creating or updating a record
type Input struct {
	Name string `db:"name" json:"name,omitempty" validate:"required"`
}

// columns outputted when selecting a record. Note that Input is embedded
type Model struct {
	Id    int64 `db:"id" json:"id"`
	Input
}

type Store struct {
	Db *pgxpool.Pool
}

// define functions for this table as methods of the Store struct
// use lyspg generic functions if possible, but can also write custom implementations

func (s Store) Delete(ctx context.Context, id int64) error {
	return lyspg.DeleteUnique(ctx, s.Db, schemaName, tableName, pkColName, id)
}

func (s Store) Insert(ctx context.Context, input Input) (newId int64, err error) {
	return lyspg.Insert[Input, int64](ctx, s.Db, schemaName, tableName, pkColName, input)
}

func (s Store) Select(ctx context.Context, params lyspg.SelectParams) (items []Model, unpagedCount lyspg.TotalCount, err error) {
	return lyspg.Select[Model](ctx, s.Db, schemaName, tableName, viewName, defaultOrderBy, gDbTags, params)
}

// etc
Create routes (wiki)

Pass the store package to generic GET, POST, etc handlers to get full REST API CRUD functionality for this table with minimal boilerplate.

package main

func (srvApp *httpServerApplication) getRoutes(apiEnv lys.Env) http.Handler {

	endpoint := "/core-categories"

	// get full CRUD functionality using lys generic handlers, passing the store defined above
	// no framework: free to write custom handlers when needed

	categoryStore := corecategory.Store{Db: srvApp.Db}
	r.HandleFunc(endpoint, lys.Get[corecategory.Model](apiEnv, categoryStore)).Methods("GET")
	r.HandleFunc(endpoint+"/{id}", lys.GetById[corecategory.Model](apiEnv, categoryStore)).Methods("GET")
	r.HandleFunc(endpoint, lys.Post[corecategory.Input, int64](apiEnv, categoryStore)).Methods("POST")
	r.HandleFunc(endpoint+"/{id}", lys.Put[corecategory.Input](apiEnv, categoryStore)).Methods("PUT")
	r.HandleFunc(endpoint+"/{id}", lys.Patch(apiEnv, categoryStore)).Methods("PATCH")
	r.HandleFunc(endpoint+"/{id}", lys.Delete(apiEnv, categoryStore)).Methods("DELETE")
}
Use routes

We can now start the HTTP server app and use the routes above.

curl localhost:8010/core-categories?name=Seafood
curl localhost:8010/core-categories/1
curl --header "Content-Type: application/json" --request POST --data '{"name":"Fruit"}' localhost:8010/core-categories
# etc

See the Northwind sample application for a complete application using these packages.

Features

  • Library only: is not a framework, and does not use code generation, so can be overriden at every step to deal with exceptional cases
  • Support for GET many, GET single, POST, PUT, PATCH and DELETE
  • Support for sorting, paging and filtering GET results via customizable URL params
  • Uses pgx for database access and only uses parameterized SQL queries
  • Support for Excel and CSV output
  • Uses generics and reflection to minimize boilerplate
  • Custom date/time types with zero default values and sensible JSON formats
  • Fast rowcount function, including estimated count for large tables with query conditions
  • Struct validation using validator
  • Distinction between user errors (unlogged, reported to user) and application errors (logged, hidden from user)
  • Provides useful bulk insert (COPY) wrapper
  • Support for getting and filtering enum values
  • Database creation function from embedded SQL files
  • Archive (soft delete) + restore functions
  • and more. See the wiki

Current limitations

  • Only supports PostgreSQL
  • No database obfuscation. Struct "db" tags must be added and must be identical to the "json" tag, unless the latter is "-"
  • Limited support for database date/time arrays

Testing

See CONTRIBUTING.md for setup instructions.

Supported Go and PostgreSQL Versions

Preliminary values:

Go 1.16+ (due to embed.FS)

PostgreSQL 13+ (due to gen_random_uuid)

Documentation

Overview

Package lys is used for rapid development of REST APIs handling database CRUD actions.

Please see the README and wiki for an overview.

Index

Constants

View Source 
const (
	ErrDescBodyMissing        string = "request body missing"
	ErrDescIdMissing          string = "id missing"
	ErrDescIdNotAUuid         string = "id not a uuid"
	ErrDescIdNotAnInteger     string = "id not an integer"
	ErrDescIdNotUnique        string = "id not unique" // the handling func was expecting id to be unique, but it is not
	ErrDescInvalidContentType string = "content type must be application/json"
	ErrDescInvalidId          string = "invalid id" // the Id sent is not present in the relevant table
	ErrDescInvalidJson        string = "invalid json"
	ErrDescRouteNotFound      string = "route not found"
	ErrDescUserInfoMissing    string = "userInfo missing"  // failed to get ReqUserInfo from context
	ErrDescPermissionDenied   string = "permission denied" // authorization failed
)

validation user errors

View Source 
const (
	FormatCsv   string = "csv"
	FormatExcel string = "excel"
	FormatJson  string = "json"
)

output format consts

View Source 
const (
	// status
	ReqSucceeded string = "succeeded"
	ReqFailed    string = "failed"

	// data
	DataArchived string = "archived"
	DataDeleted  string = "deleted"
	DataRestored string = "restored"
	DataUpdated  string = "updated"
)

response constants

View Source 
const ReqUserInfoCtxKey contextKey = "ReqUserInfoKey"

ReqUserInfoCtxKey is the key to be used when binding a ReqUserInfo to a request via context

Variables

View Source 
var (
	ValidFormats = [...]string{FormatCsv, FormatExcel, FormatJson}
)

Functions

func ArchiveById added in v0.1.17 

func ArchiveById(env Env, db *pgxpool.Pool, store iArchiveableById) http.HandlerFunc

Archive handles moving a record from the supplied store into its archived table

func ArchiveByUuid added in v0.1.17 

func ArchiveByUuid(env Env, db *pgxpool.Pool, store iArchiveableByUuid) http.HandlerFunc

Archive handles moving a record from the supplied store into its archived table

func AuthorizeRole 

func AuthorizeRole(allowedRoles []string) func(http.Handler) http.Handler

AuthorizeRole is middleware that checks that the user has one of the supplied allowedRoles is intended for use in subroutes

func DecodeJsonBody 

func DecodeJsonBody[T any](body []byte) (dest T, err error)

DecodeJsonBody decodes the supplied json body into dest and checks for a variety of error conditions adapted from https://www.alexedwards.net/blog/how-to-properly-parse-a-json-request-body

func Delete 

func Delete(env Env, store iDeletable) http.HandlerFunc

Delete handles deletion of a single item using the supplied store

func ExtractFields 

func ExtractFields(r *http.Request, validJsonFields []string, fieldsReqParamName string) (fields []string, err error)

ExtractFields returns a slice of strings parsed from the request's fields param

func ExtractFilters 

func ExtractFilters(urlValues url.Values, validJsonFields []string, getOptions GetOptions) (conds []lyspg.Condition, err error)

ExtractFilters returns a slice of conditions parsed from the request's params to get urlValues from a request: r.Url.Query()

func ExtractFormat added in v0.1.5 

func ExtractFormat(r *http.Request, formatReqParamName string) (format string, err error)

ExtractFormat returns

func ExtractJsonBody 

func ExtractJsonBody(r *http.Request, maxBodySize int64) (body []byte, err error)

ExtractJsonBody reads and validates the body of the supplied request

func ExtractPaging 

func ExtractPaging(r *http.Request, pageReqParamName, perPageReqParamName string, defaultPerPage, maxPerPage int) (page int, perPage int, err error)

ExtractPaging returns paging variables parsed from a request's paging params page defaults to 1, perPage defaults to defaultPerPage

func ExtractSorts 

func ExtractSorts(r *http.Request, validJsonFields []string, sortReqParamName string) (sortCols []string, err error)

ExtractSorts returns an array of SQL sorting statements parsed from the request's sort param

func FileResponse added in v0.1.5 

func FileResponse(filePath, outputFileName string, remove bool, w http.ResponseWriter)

FileResponse opens the supplied file and streams it to w as a file

func Get 

func Get[T any](env Env, store iGetable[T], options ...GetOption) http.HandlerFunc

Get handles retrieval of multiple items from the supplied store

func GetById 

func GetById[T any](env Env, store iGetableById[T]) http.HandlerFunc

GetById handles retrieval of a single item from the supplied store using an integer id

func GetByUuid 

func GetByUuid[T any](env Env, store iGetableByUuid[T]) http.HandlerFunc

GetByUuid handles retrieval of a single item from the supplied store using a text id

func GetEnumValues 

func GetEnumValues(env Env, db *pgxpool.Pool, schema, enum string) http.HandlerFunc

GetEnumValues returns enum values from the supplied schema and enum type name

func GetSimple 

func GetSimple[T any](env Env, selectFunc func(ctx context.Context) (items []T, err error)) http.HandlerFunc

GetSimple handles retrieval of all items returned by selectFunc, which may only take ctx as param

func GetUserNameFromCtx 

func GetUserNameFromCtx(ctx context.Context, defaultVal string) string

GetUserNameFromCtx returns the username if it can be obtained from ctx, otherwise the supplied default value

func GetWithLastSync added in v0.1.9 

func GetWithLastSync[T any](env Env, store iGetableWithLastSync[T]) http.HandlerFunc

GetWithLastSync is a wrapper for Get which adds the lastSyncAt timestamp from the supplied func to the JSON response

func HandleDbError 

func HandleDbError(ctx context.Context, stmt string, err error, errorLog *slog.Logger, w http.ResponseWriter)

HandleDbError returns a generic error message to the API user and includes the failing statement in the error log

func HandleError added in v0.1.25 

func HandleError(ctx context.Context, err error, errorLog *slog.Logger, w http.ResponseWriter)

HandleError is the general method for handling API errors where err could contain wrapped errors of other types

func HandleExtError added in v0.1.25 

func HandleExtError(ctx context.Context, extMessage string, err error, errorLog *slog.Logger, w http.ResponseWriter)

HandleExtError returns the external message to the API user and logs the error

func HandleInternalError 

func HandleInternalError(ctx context.Context, err error, errorLog *slog.Logger, w http.ResponseWriter)

HandleInternalError returns a generic error message to the API user and logs the error

func HandleUserError 

func HandleUserError(statusCode int, userErrMsg string, w http.ResponseWriter)

HandleUserError returns a helpful message to the API user, but does not log the error

func JsonResponse 

func JsonResponse(resp StdResponse, httpStatus int, w http.ResponseWriter)

JsonResponse marshals the supplied StdResponse to json and writes it to w

func Message 

func Message(msg string) http.HandlerFunc

Message returns the supplied msg in the Data field

func MoveRecordsById 

func MoveRecordsById(env Env, db *pgxpool.Pool, moveFunc func(context.Context, pgx.Tx, int64) error, msg string) http.HandlerFunc

MoveRecordsById handles moving record(s) back and forth between the main table and its corresponding archived table

func MoveRecordsByUuid added in v0.1.17 

func MoveRecordsByUuid(env Env, db *pgxpool.Pool, moveFunc func(context.Context, pgx.Tx, uuid.UUID) error, msg string) http.HandlerFunc

MoveRecordsById handles moving record(s) back and forth between the main table and its corresponding archived table

func NotFound 

func NotFound() http.HandlerFunc

NotFound provides a response informing the user that the requested route was not found

func Patch 

func Patch(env Env, store iPatchable) http.HandlerFunc

Patch handles changing some of an item's fields using the supplied store

func PgSleep 

func PgSleep(db lyspg.PoolOrTx, errorLog *slog.Logger, secs int) http.HandlerFunc

PgSleep creates an artifical longrunning query in the db which can be viewed using pg_stat_activity used for testing context cancelation

func Post 

func Post[inputT any, outputT any](env Env, store iPostable[inputT, outputT]) http.HandlerFunc

Post handles creating a new item using the supplied store and returning an output (the new item or its ID) in the response

func ProcessSlice added in v0.1.27 

func ProcessSlice[T any](env Env, processFunc func(context.Context, []T) (int64, error)) http.HandlerFunc

ProcessSlice extracts a slice from the req body and passes it into the supplied processFunc

func Put 

func Put[T any](env Env, store iPutable[T]) http.HandlerFunc

Put handles changing an item using the supplied store

func RestoreById added in v0.1.17 

func RestoreById(env Env, db *pgxpool.Pool, store iArchiveableById) http.HandlerFunc

Restore handles moving a record from the store's archived table back to the main table

func RestoreByUuid added in v0.1.17 

func RestoreByUuid(env Env, db *pgxpool.Pool, store iArchiveableByUuid) http.HandlerFunc

Restore handles moving a record from the store's archived table back to the main table

Types

type Env 

type Env struct {
	ErrorLog    *slog.Logger
	Validate    *validator.Validate
	GetOptions  GetOptions
	PostOptions PostOptions
}

Env (environment) contains objects and options needed by API calls

type GetMetadata added in v0.1.8 

type GetMetadata struct {
	Count                 int   `json:"count"`
	TotalCount            int64 `json:"total_count"`
	TotalCountIsEstimated bool  `json:"total_count_is_estimated"`
}

type GetOption added in v0.1.9 

type GetOption struct {
	GetLastSyncAt func(ctx context.Context) (lastSyncAt lystype.Datetime, err error) // for external data: func to get the last synced timestamp
}

type GetOptions 

type GetOptions struct {
	FormatParamName        string // param name to determine the output format of a GET request, e.g. "xformat"
	FieldsParamName        string // param name to limit the fields returned by a GET request, e.g. "xfields"
	PageParamName          string // param name to define the page offset returned by a paged GET request, e.g. "xpage"
	PerPageParamName       string // param name to define the number of records returned by a paged GET request, e.g. "xper_page"
	SortParamName          string // param name to identify the sort param used by a GET request, e.g. "xsort"
	MultipleValueSeparator string // the string used by a GET request to separate values in a filter where each value should be returned, e.g. "|", usage: "name=Bill|Sam"
	DefaultPerPage         int    // default number of results returned by a paged GET request, e.g. 20
	MaxPerPage             int    // max number of results returned per paged GET request, regardless of what the caller enters in the "PerPageParamName" param, e.g. 500
	MaxFileRecs            int    // max number of records contained in a file output
	CsvDelimiter           rune   // delimiter between values in CSV file output
}

GetOptions contains the options used when processing GET requests, such as paging param names and default values Since the json field names are used as filters, param names should be chosen which will never appear as json field names

func FillGetOptions 

func FillGetOptions(input GetOptions) (ret GetOptions)

FillGetOptions returns input GetOptions if they are passed, and sets any unset fields to a sensible default value

type GetReqModifiers 

type GetReqModifiers struct {
	Format     string
	Fields     []string
	Conditions []lyspg.Condition
	Page       int
	PerPage    int
	Sorts      []string
}

GetReqModifiers contains data from a GET request's Url params which is used to modify a database SELECT statement

func ExtractGetRequestModifiers 

func ExtractGetRequestModifiers(r *http.Request, validJsonFields []string, getOptions GetOptions) (getReqModifiers GetReqModifiers, err error)

ExtractGetRequestModifiers reads the Url params of the supplied GET request and converts them into a GetReqModifiers

type PostOptions 

type PostOptions struct {
	MaxBodySize int64 // bytes
}

PostOptions contains the options used when processing POST or PUT requests

func FillPostOptions 

func FillPostOptions(input PostOptions) (ret PostOptions)

FillPostOptions returns input PostOptions if they are passed, and sets any unset fields to a sensible default value

type ReqUserInfo 

type ReqUserInfo struct {
	Roles    []string `json:"roles"`
	UserId   int64    `json:"user_id"`
	UserName string   `json:"user_name"`
}

ReqUserInfo contains data about the API user which is added to request context after authentication

func GetUserFromCtx added in v0.1.13 

func GetUserFromCtx(ctx context.Context) ReqUserInfo

GetUserFromCtx returns the user from ctx. ReqUserInfoCtxKey and ReqUserInfo must have been assigned in middleware

type RouteAdderFunc 

type RouteAdderFunc func(r *mux.Router) *mux.Router

RouteAdderFunc is a function returning a subrouter

type StdResponse 

type StdResponse struct {
	Status          string            `json:"status"`
	Data            any               `json:"data,omitempty"`
	GetMetadata     *GetMetadata      `json:"metadata,omitempty"`     // only used for GET many
	LastSyncAt      *lystype.Datetime `json:"last_sync_at,omitempty"` // if the data was synced from external source: the last sync timestamp
	ErrType         string            `json:"err_type,omitempty"`
	ErrDescription  string            `json:"err_description,omitempty"`
	ExternalMessage string            `json:"external_message,omitempty"` // user-readable messages passed on from 3rd party API calls
}

StdResponse is the return type of all API routes

type SubRoute 

type SubRoute struct {
	Url        string
	RouteAdder RouteAdderFunc
}

SubRoute contains a Url path and the function returning the subrouter to process that path

Source Files

View all Source files

Directories

Path Synopsis
internal
cmd
cmd/lyscli
lyscmd
sql/ddl
stores/core/corearchivetest
stores/core/coreparamtest
stores/core/coretypetest
stores/core/corevolumetest
Package lysclient contains types and functions to help test a REST API which was created using lys functions.
 Package lysclient contains types and functions to help test a REST API which was created using lys functions.
Package lyserr contains structs related to error handling used in lys and lyspg.
 Package lyserr contains structs related to error handling used in lys and lyspg.
Package lysgen contains experimental functions to generate code from Postgres database tables.
 Package lysgen contains experimental functions to generate code from Postgres database tables.
Package lysmeta contains functions that analyze structs.
 Package lysmeta contains functions that analyze structs.
Package lyspg contains structs and functions providing generic CRUD operations on a Postgres database.
 Package lyspg contains structs and functions providing generic CRUD operations on a Postgres database.
Package lyspgdb contains functions for creating and monitoring Postgres databases.
 Package lyspgdb contains functions for creating and monitoring Postgres databases.
Package lysstring contains string functions.
 Package lysstring contains string functions.
Package lystype contains date/time types used in lys and lyspg.
 Package lystype contains date/time types used in lys and lyspg.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.