qutedb

README

QuTeDB — The QUBIC Test Database

Requirements

• Go version 1.16 or newer

The program depends on the following libraries:

• Fitsio, to read and write FITS files
• go-chart, to produce charts
• GORM, an ORM
• go-sqlite3, a database driver for SQLite
• go.uuid, for generating random UUIDs
• Logrus, for structured logging
• simple-scrypt, for encrypting passwords
• Viper, for reading configuration files

Note that go-sqlite3 is a cgo package, which means that you must have gcc in your path before downloading and compiling this library. This is usually the case on UNIX systems; if you are running Windows, you have to install gcc first (e.g., using the scoop package manager).

Installation

To download all the dependencies, build, and install the package in your $GOPATH/bin directory, run this command:

go build ./...
go install ./...

This will add the two executables qutedb and createqdbcfg in the bin directory of the $GOPATH folder.

You must now create a configuration file. The go install command you just issued installed a small script called createqdbcfg. Run it, and it will print a default configuration to stdout. Save it into a file named config.json; this file can be kept either in the current directory or in $HOME/.qutedb/. Run createqdbcfg --help to get an help of the commands.

Configuration

All the configuration is specified through a file, config.json, which is read from any of the following locations (in this order):

1. The directory from where qutedb was run;
2. The directory .qutedb within the home folder (i.e., $HOME on UNIX systems, %USERPROFILE% on Windows);
3. The directory /etc/qutedb (only on UNIX systems)

Use the program createqdbcfg to create a skeleton for this file; use createqdbcfg --help to get an help of a few parameters you can set from the command line.

Here is an example of configuration file:

{
    "log_format": "json",
    "port_number": 80
}

The following table contains all the configuration parameters that can be used in config.json. A few of them have sensible defaults, if no value is provided.

Parameter	Default	Meaning
cookie_hash_key	None	Hash key used to encode session cookies. It must be encoded using base64 encoding, and the unencoded string should be 32 or 64 characters long
cookie_block_key	None	Block key used to encode session cookies. It must be encoded using base64 encoding, and the unencoded string should be 32 or 64 characters long
log_format	"text"	Format of log messages. Possible values are "text" and "json"
log_output	"-"	File where to write log messages. If equal to "-", write to stderr; if "--", write to stdout
log_level	It depends	Logging level. Possible values are "error", "warning", "info", and "debug", in increasing order of verbosity. The default is "info", unless development mode is turned on
port_number	8080	Socket port number used for publishing the API and the site
read_timeout	15	Timeout for HTTP read operations, in seconds
static_path	static	Path to the directory containing static files (e.g., images) to serve
server_name	127.0.0.1	Name of the server (e.g., www.example.com)
repository_path	.	Path to the folder that contains the QUBIC test data
write_timeout	60	Timeout for HTTP write operations, in seconds

The following environment variables are recognized and take precedence over the corresponding keys in config.json:

Environment variable	Key in config.json
QUTEDB_PORT_NUMBER	port_number
QUTEDB_SERVER_NAME	server_name
QUTEDB_READ_TIMEOUT	read_timeout
QUTEDB_WRITE_TIMEOUT	write_timeout

Authentication

The code uses the "scrypt" encryption algorithm to hash users' passwords. See the article Do not use sha256crypt/sha512crypt for the reasons behind this choice.

The first time the program is started, it will create a new superuser with name admin@localhost and password changeme. Please change the password immediately!

License

This code is released under the MIT license. See the file LICENSE for more details.

Documentation

Index

• Constants
• func CheckUserPassword(db *gorm.DB, email string, password string) (uint, bool, error)
• func DeleteSession(db *gorm.DB, UUID string) error
• func DeleteUser(db *gorm.DB, user *User) error
• func HkDirName(folder string) string
• func InitDb(db *gorm.DB, config *Configuration) error
• func RawDirName(folder string) string
• func RefreshDbContents(db *gorm.DB, repositoryPath string) error
• func SumDirName(folder string) string
• func TimeToCanonicalStr(t time.Time) string
• func UpdateUserPassword(db *gorm.DB, user *User, newPassword string) error
• type Acquisition
  • func QueryAcquisition(db *gorm.DB, acqtime string) (*Acquisition, error)
• type App
  • func NewApp() *App
  • func (app *App) CreateDefaultUser() error
  • func (app *App) Run()
• type Configuration
• type Error
  • func (e Error) Error() string
• type HomeData
• type RawDataFile
• type Session
  • func CreateSession(db *gorm.DB, user *User) (*Session, error)
  • func QuerySessionByUUID(db *gorm.DB, UUID string) (*Session, error)
• type SumDataFile
• type User
  • func CreateUser(db *gorm.DB, email string, password string, superuser bool) (*User, error)
  • func QueryAllUsers(db *gorm.DB) ([]User, error)
  • func QueryUserByEmail(db *gorm.DB, email string) (*User, error)
  • func QueryUserByID(db *gorm.DB, userID uint) (*User, error)

Constants

const QuteDBVersion = "0.5.3"

QuteDBVersion is a string containing the version number of QuteDB

Functions

func CheckUserPassword

func CheckUserPassword(db *gorm.DB, email string, password string) (uint, bool, error)

CheckUserPassword checks that an user with the specified email and password is in the database. If found, return the tuple (userID, true, nil). If the user does not exist, or if the passwords don't match, return (0, false, nil). In the event of an error, the last element in the returned tuple identifies the error.

func DeleteSession

func DeleteSession(db *gorm.DB, UUID string) error

DeleteSession finds a session with a matching UUID in the database and deletes it. If no session is found, returns silently without signaling any error.

func DeleteUser

func DeleteUser(db *gorm.DB, user *User) error

DeleteUser removes an user from the database

func HkDirName

func HkDirName(folder string) string

HkDirName returns the name of the test directory containing the housekeeping files

func InitDb

func InitDb(db *gorm.DB, config *Configuration) error

InitDb creates all the tables in the database. It takes care of not raising errors if the tables are already present. You should call this function only once during the lifetime of the program, as it resets all open sessions.

func RawDirName

func RawDirName(folder string) string

RawDirName returns the name of the test directory containing raw files

func RefreshDbContents

func RefreshDbContents(db *gorm.DB, repositoryPath string) error

RefreshDbContents scans the repository for any file that is missing from the database, and create an entry for each of them.

func SumDirName

func SumDirName(folder string) string

SumDirName returns the name of the test directory containing raw files

func TimeToCanonicalStr

func TimeToCanonicalStr(t time.Time) string

TimeToCanonicalStr converts a standard date/time into a string which is formatted according to the template "YYYYMMSShhmmss". This is the format used in Acquisition.AcquisitionTime

func UpdateUserPassword

func UpdateUserPassword(db *gorm.DB, user *User, newPassword string) error

UpdateUserPassword changes the password associated with a user

Types

type Acquisition

type Acquisition struct {
	ID        uint      `gorm:"primary_key" json:"id"`
	CreatedAt time.Time `json:"created_at"`

	Name          string `json:"name"`
	Directoryname string `json:"directory_name" gorm:"unique_index"`
	// We encode the acquisition time as a string in order to have
	// full control on the formatting, which is always "YYYY-MM-DDThh:mm:ss"
	AcquisitionTime  string        `json:"acquisition_time"`
	RawFiles         []RawDataFile `json:"-"`
	SumFiles         []SumDataFile `json:"-"`
	AsicHkFileName   string        `json:"-"`
	InternHkFileName string        `json:"-"`
	ExternHkFileName string        `json:"-"`
	MmrHkFileName    string        `json:"-"`
	MgcHkFileName    string        `json:"-"`
	CalConfFileName  string        `json:"-"`
	CalDataFileName  string        `json:"-"`
}

An Acquisition represents a set of files within a folder in the repository

func QueryAcquisition

func QueryAcquisition(db *gorm.DB, acqtime string) (*Acquisition, error)

QueryAcquisition returns an Aquisition object with all its fields properly filled

type App

type App struct {
	// contains filtered or unexported fields
}

An App is a structure which encapsulate the whole state of the application

func NewApp

func NewApp() *App

NewApp creates a new application and performs a number of initializations.

func (*App) CreateDefaultUser

func (app *App) CreateDefaultUser() error

CreateDefaultUser creates a superuser with a standard password, if no user exists.

func (*App) Run

func (app *App) Run()

Run opens the database and starts the main loop.

type Configuration

type Configuration struct {
	// Full path of the file containing the configuration
	ConfigurationFileName string `json:"-"`

	DatabaseFile string `json:"database_file"`

	LogFormat string `json:"log_format"`
	LogLevel  string `json:"log_level"`
	LogOutput string `json:"log_output"`

	PortNumber int    `json:"port_number"`
	ServerName string `json:"server_name"`

	ReadTimeout  int64 `json:"read_timeout"`
	WriteTimeout int64 `json:"write_timeout"`

	StaticPath string `json:"static_path"`

	RepositoryPath string `json:"repository_path"`

	CookieHashKey  []byte `json:"cookie_hash_key"`
	CookieBlockKey []byte `json:"cookie_block_key"`
}

A Configuration type contains all the settings loaded from environment variables and configuration files. It is initialized using the Viper library.

type Error

type Error struct {
	// contains filtered or unexported fields
}

Error contains information about an HTTP error

func (Error) Error

func (e Error) Error() string

type HomeData

type HomeData struct {
	User            User
	AcquisitionList []Acquisition
}

HomeData contains the data passed to the "index.html" template

type RawDataFile

type RawDataFile struct {
	ID            int    `json:"id" gorm:"primary_key"`
	FileName      string `json:"file_name"`
	AsicNumber    int    `json:"asic_number"`
	AcquisitionID int    `json:"-"`
}

A RawDataFile represents the file containing raw data acquired with one ASIC

type Session

type Session struct {
	gorm.Model
	UUID   string `gorm:"size:36;unique_index"`
	UserID uint
}

A Session records who is currently allowed to access the site. This only happens if a user has successfully logged in.

func CreateSession

func CreateSession(db *gorm.DB, user *User) (*Session, error)

CreateSession inserts a new "Session" object in the database. The object is uniquely identified by its UUID.

func QuerySessionByUUID

func QuerySessionByUUID(db *gorm.DB, UUID string) (*Session, error)

QuerySessionByUUID searches for an active session and returns a Session object if found.

type SumDataFile

type SumDataFile struct {
	ID            int    `json:"id" gorm:"primary_key"`
	FileName      string `json:"file_name"`
	AsicNumber    int    `json:"asic_number"`
	AcquisitionID int    `json:"-"`
}

A SumDataFile represents the file containing science data acquired with one ASIC

type User

type User struct {
	gorm.Model
	Email          string `gorm:"unique_index"`
	HashedPassword []byte
	Superuser      bool
}

An User is somebody which can have (read-only) access to the data

func CreateUser

func CreateUser(
	db *gorm.DB,
	email string,
	password string,
	superuser bool,
) (*User, error)

CreateUser creates a new "User" object and initializes it with the hash of the password and the other parameters as well. The new object is saved in the database.

func QueryAllUsers

func QueryAllUsers(db *gorm.DB) ([]User, error)

QueryAllUsers returns a list of all the users defined in the database

func QueryUserByEmail

func QueryUserByEmail(db *gorm.DB, email string) (*User, error)

QueryUserByEmail searches in the database for an user with the matching email and returns a poitner to a User structure. If the user is not found, the pointer is nil. The "error" variable is set to something else than nil only if a real error is occurred.

func QueryUserByID

func QueryUserByID(db *gorm.DB, userID uint) (*User, error)

QueryUserByID searches in the database for an user with the matching user ID and returns a pointer to a User structure. If the user is not found, the pointer is nil. The "error" variable is set to something else than nil only if a real error is occurred.