migration

README

Migrations

Directory db/rdbms/migration contains all migrations which are supported by ConTest. These consist both in .sql and pure go migrations as explained in the documentation for the migration tool. Please refer to the documentation of the tool to understand how the migrations are applied. This document explains each supported migration.

Every migration is associated to a version number, which eventually will represent the db schema version.

0001_add_extended_descriptor.sql

The add_extended_descriptor migrations adds extended_descriptor column. This was part of #118, which fixed an issue in handling job descriptor and test steps descriptors. extended_descriptor field is introduced in the jobs table to track the initial job descriptor submitted by the user, and the resolved test descriptors (which are obtained by the test fetcher at runtime, just after submitting the job) in a single data structure. Before, they would be part of the Request object, populated at different times of the lifecycle of the job. This also resulted in an additional bug when rebuilding status of a job: we would fetch test descriptors from backend, instead of using the object serialized at event submission time.

0002_migrate_descriptor_to_extended_descriptor.go

The migrate_descriptor_to_extended_descrptor migration backfills existing entries in the jobs table and populates them with an extended_descriptor. After #118, ConTest has a dependency over the extended_descriptor being populated and it won't be able to re-build status for any job which hasn't had the extended_descriptor backfilled.

0003_add_jobs_state_column.sql

The add_jobs_state_column migration adds the state column to the jobs table to keep track of the job state without making queries against the framework_events table. Migration script backfills the column for existing jobs and server maintains the column going forward.

0004_add_job_tags_table.sql

The add_job_tags_table migration creates the job_tags table that maintains the relationship between job id and job tags to facilitate efficient lookups of jobs by tag(s). Migration script backfills the column for existing jobs and server maintains the column going forward.

0005_event_payload_mediumtext.sql

TBD

0006_add_indices.sql

The add_indices migration creates the indices required to cover SELECT requests issued by JobRunner.

Documentation

Index

• Variables
• func NewDescriptorMigration(ctx xcontext.Context) migrate.Migrate
• type DescriptorMigration
  • func (m *DescriptorMigration) Down(tx *sql.Tx) error
  • func (m *DescriptorMigration) DownNoTx(db *sql.DB) error
  • func (m *DescriptorMigration) Up(tx *sql.Tx) error
  • func (m *DescriptorMigration) UpNoTx(db *sql.DB) error
• type Request

Variables

var TargetManagers = []target.TargetManagerLoader{
	csvtargetmanager.Load,
	targetlist.Load,
}

Functions

func NewDescriptorMigration

func NewDescriptorMigration(ctx xcontext.Context) migrate.Migrate

NewDescriptorMigration is the factory for DescriptorMigration

Types

type DescriptorMigration

type DescriptorMigration struct {
	Context xcontext.Context
}

Schema v0002 introduces the concept of extended_descriptor, which is defined as follows:

type ExtendedDescriptor struct {
		JobDescriptor
		TestStepsDescriptors []test.TestStepsDescriptors
}

We remove TestDescriptors from Request objects, and we store that information side-by-side with JobDescriptor into an ExtendedDescriptor. We then store this ExtendedDescriptor in the jobs table so that all the test information can be re-fetched by reading extended_descriptor field in jobs table, without any dependency after submission time on the test fetcher.

func (*DescriptorMigration) Down

func (m *DescriptorMigration) Down(tx *sql.Tx) error

Down implements the down migration of DescriptorMigration

func (*DescriptorMigration) DownNoTx

func (m *DescriptorMigration) DownNoTx(db *sql.DB) error

DownNoTx implements the down migration of DescriptorMigration in a non-transactional manner

func (*DescriptorMigration) Up

func (m *DescriptorMigration) Up(tx *sql.Tx) error

Up implements the forward migration

func (*DescriptorMigration) UpNoTx

func (m *DescriptorMigration) UpNoTx(db *sql.DB) error

UpNoTx implements the forward migration in a non-transactional manner

type Request

type Request struct {
	JobID         types.JobID
	JobName       string
	Requestor     string
	ServerID      string
	RequestTime   time.Time
	JobDescriptor string
	// TestDescriptors are the fetched test steps as per the test fetcher
	// defined in the JobDescriptor above.
	TestDescriptors string
}

Request represent the v1 job request layout