db

package
v2.5.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2018 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	BatchStatusPending   = "pending"    // Not yet built or in the process of being built
	BatchStatusQCReady   = "qc_ready"   // Ready for ingest onto staging
	BatchStatusOnStaging = "on_staging" // On the staging server awaiting QC
	BatchStatusFailedQC  = "failed_qc"  // On staging, but QC failed it; it needs to be pulled and fixed
	BatchStatusPassedQC  = "passed_qc"  // On staging, passed QC; it needs to be pulled from staging and pushed live
	BatchStatusLive      = "live"       // Batch has gone live; batch and its issues need to be archived
	BatchStatusLiveDone  = "live_done"  // Batch has gone live; batch and its issues have been archived and are no longer on the filesystem
)

These are all possible batch status values

View Source 
const (
	JobObjectTypeBatch = "batch"
	JobObjectTypeIssue = "issue"
)

Object types for consistently inserting into the database

Variables

View Source 
var DB *magicsql.DB

DB is meant as a global accessor to a long-living database connection

View Source 
var Debug bool

Debug should be set to true if operations should be logged to stderr

Functions

func Connect 

func Connect(connect string) error

Connect takes a connection string, opens the database, and stores both the source sql.DB and the wrapped magicsql.DB

func CreateAuditLog 

func CreateAuditLog(ip, user, action, message string) error

CreateAuditLog writes the given data to audit_logs

func DeleteMOC 

func DeleteMOC(id int) error

DeleteMOC removes the MOC with the given id

func RandomBatchName 

func RandomBatchName(seq int) string

RandomBatchName generates a unique name for the given sequence. The names are guaranteed not to duplicate any of the component pieces until one component's values have all been used. The full names are guaranteed not to duplicate until the longest list has been used up, at which point all lists will be reshuffled.

func ValidMOC 

func ValidMOC(code string) bool

ValidMOC returns true if the given code is in the database

Types

type AuditLog 

type AuditLog struct {
	ID      int       `sql:",primary"`
	When    time.Time "sql:\"`when`\""
	IP      string
	User    string
	Action  string
	Message string
}

AuditLog represents the audit_logs table

type Batch 

type Batch struct {
	ID          int `sql:",primary"`
	MARCOrgCode string
	Name        string
	CreatedAt   time.Time
	Status      string
	Location    string
	// contains filtered or unexported fields
}

Batch contains metadata for generating a batch XML. Issues can be associated with a single batch, and a batch will typically have many issues assigned to it.

func CreateBatch 

func CreateBatch(moc string, issues []*Issue) (*Batch, error)

CreateBatch creates a batch in the database, using its ID to generate a unique batch name, and associating the given list of issues. This is inefficient, but it gets the job done.

func FindBatch 

func FindBatch(id int) (*Batch, error)

FindBatch looks for a batch by its id

func (*Batch) AwardYear 

func (b *Batch) AwardYear() int

AwardYear uses the batch creation date to produce the "award year" - this is the most similar value we can produce

func (*Batch) FullName 

func (b *Batch) FullName() string

FullName returns the name of a batch as it is needed for chronam / ONI.

Note that currently we assume all generated batches will be _ver01, because we would usually generate a completely new batch if one were in such a state as to need to be pulled from production.

func (*Batch) Issues 

func (b *Batch) Issues() ([]*Issue, error)

Issues pulls all issues from the database which have this batch's ID

func (*Batch) Save 

func (b *Batch) Save() error

Save creates or updates the Batch in the batches table

func (*Batch) SaveOp 

func (b *Batch) SaveOp(op *magicsql.Operation) error

SaveOp saves the batch to the batches table with a custom operation for easier transactions

type Issue 

type Issue struct {
	ID int `sql:",primary"`

	// Metadata
	MARCOrgCode   string
	LCCN          string
	Date          string
	DateAsLabeled string
	Volume        string

	// Titles are tied to issues by LCCN, and all issues must have a title in the
	// database, so we load these when we load the issue data
	Title *Title `sql:"-"`

	// This field is a bit confusing, but it is the NDNP field for the issue
	// "number", which is actually a string since it can contain things like
	// "ISSUE XIX"
	Issue         string
	Edition       int
	EditionLabel  string
	PageLabelsCSV string
	PageLabels    []string `sql:"-"`

	BatchID                int                 // Which batch (if any) is this issue a part of?
	Error                  string              // If set, a metadata curator reported a problem
	Location               string              // Where is this issue on disk?
	MasterBackupLocation   string              // Where is the master backup located?  (born-digital only)
	HumanName              string              // What is the issue's "human" name (for consistent folder naming)?
	IsFromScanner          bool                // Is the issue scanned in-house?  (Born-digital == false)
	HasDerivatives         bool                // Does the issue have derivatives done?
	WorkflowStepString     string              `sql:"workflow_step"` // If set, tells us what "human workflow" step we're on
	WorkflowStep           schema.WorkflowStep `sql:"-"`
	WorkflowOwnerID        int                 // Whose "desk" is this currently on?
	WorkflowOwnerExpiresAt time.Time           // When does the workflow owner lose ownership?
	MetadataEntryUserID    int                 // Who entered metadata?
	ReviewedByUserID       int                 // Who reviewed metadata?
	MetadataApprovedAt     time.Time           // When was metadata approved / how long has this been waiting to batch?
	RejectionNotes         string              // If rejected (during metadata review), this tells us why
	RejectedByUserID       int                 // Who did the rejection?
	Ignored                bool                // Is the issue bad / in prod / otherwise skipped from workflow scans?
}

Issue contains metadata about an issue for the various workflow tools' use

func FindAvailableIssuesByWorkflowStep 

func FindAvailableIssuesByWorkflowStep(ws schema.WorkflowStep) ([]*Issue, error)

FindAvailableIssuesByWorkflowStep looks for all "available" issues with the requested workflow step and returns them. We define "available" as:

- No owner (or owner expired) - Have not been reported as having errors

func FindInProcessIssues 

func FindInProcessIssues() ([]*Issue, error)

FindInProcessIssues returns all issues which have been entered in the workflow system, but haven't yet gone through all the way to the point of being batched and approved for production

func FindIssue 

func FindIssue(id int) (*Issue, error)

FindIssue looks for an issue by its id

func FindIssueByKey 

func FindIssueByKey(key string) (*Issue, error)

FindIssueByKey returns the first issue with the given key

func FindIssueByLocation 

func FindIssueByLocation(location string) (*Issue, error)

FindIssueByLocation returns the first issue with the given location

func FindIssuesByBatchID 

func FindIssuesByBatchID(batchID int) ([]*Issue, error)

FindIssuesByBatchID returns all issues associated with the given batch id

func FindIssuesByKey 

func FindIssuesByKey(key string) ([]*Issue, error)

FindIssuesByKey looks for all issues in the database that have the given issue key. Having more than one is an error, but we allow users to save metadata in "draft" form, so we have to be able to test for dupes later in the process

func FindIssuesInPageReview 

func FindIssuesInPageReview() ([]*Issue, error)

FindIssuesInPageReview looks for all issues currently awaiting page review and returns them

func FindIssuesOnDesk 

func FindIssuesOnDesk(userID int) ([]*Issue, error)

FindIssuesOnDesk returns all issues "owned" by a given user id

func FindIssuesReadyForBatching 

func FindIssuesReadyForBatching() ([]*Issue, error)

FindIssuesReadyForBatching looks for all issues which are in the WSReadyForBatching workflow step and have no batch ID

func FindIssuesWithErrors 

func FindIssuesWithErrors() ([]*Issue, error)

FindIssuesWithErrors returns all issues with an error reported by metadata entry personnel

func NewIssue 

func NewIssue(moc, lccn, dt string, ed int) *Issue

NewIssue creates an issue ready for saving to the issues table

func (*Issue) ApproveMetadata 

func (i *Issue) ApproveMetadata(reviewerID int)

ApproveMetadata moves the issue to the final workflow step (e.g., no more manual steps) and sets the reviewer id to that which was passed in

func (*Issue) Claim 

func (i *Issue) Claim(byUserID int)

Claim sets the workflow owner to the given user id, and sets the expiration time to a week from now

func (*Issue) DateEdition 

func (i *Issue) DateEdition() string

DateEdition returns the date+edition string used by our general schema

func (*Issue) Key 

func (i *Issue) Key() string

Key returns the standardized issue key for this DB issue

func (*Issue) RejectMetadata 

func (i *Issue) RejectMetadata(reviewerID int, notes string)

RejectMetadata sends the issue back to the metadata entry user and saves the reviewer's notes

func (*Issue) Save 

func (i *Issue) Save() error

Save creates or updates the Issue in the issues table

func (*Issue) SaveOp 

func (i *Issue) SaveOp(op *magicsql.Operation) error

SaveOp creates or updates the Issue in the issues table with a custom operation

func (*Issue) SchemaIssue 

func (i *Issue) SchemaIssue() (*schema.Issue, error)

SchemaIssue returns an extremely over-simplified representation of this issue as a schema.Issue instance for ensuring consistent representation of things like issue keys

func (*Issue) Unclaim 

func (i *Issue) Unclaim()

Unclaim removes the workflow owner and resets the workflow expiration time

type Job 

type Job struct {
	ID          int       `sql:",primary"`
	CreatedAt   time.Time `sql:",readonly"`
	StartedAt   time.Time `sql:",noinsert"`
	CompletedAt time.Time `sql:",noinsert"`
	Type        string    `sql:"job_type"`
	ObjectID    int
	ObjectType  string
	Location    string
	Status      string

	// The job won't be run until sometime after RunAt; usually it's very close,
	// but the daemon doesn't pound the database every 5 milliseconds, so it can
	// take a little bit
	RunAt time.Time

	// Extra information any job might need - e.g., the issue's next workflow
	// step if the job is successful
	ExtraData string

	// QueueJobID tells us which job (if any) should be queued up after this one
	// completes successfully
	QueueJobID int
	// contains filtered or unexported fields
}

A Job is anything the app needs to process and track in the background

func FindJob 

func FindJob(id int) (*Job, error)

FindJob gets a job by its id

func FindJobsByStatus 

func FindJobsByStatus(st string) ([]*Job, error)

FindJobsByStatus returns all jobs that have the given status

func FindJobsByStatusAndType 

func FindJobsByStatusAndType(st string, t string) ([]*Job, error)

FindJobsByStatusAndType returns all jobs of the given status and type

func FindJobsForIssueID 

func FindJobsForIssueID(id int) ([]*Job, error)

FindJobsForIssueID returns all jobs tied to the given issue

func FindRecentJobsByType 

func FindRecentJobsByType(t string, d time.Duration) ([]*Job, error)

FindRecentJobsByType grabs all jobs of the given type which were created within the given duration or are still pending, for use in pulling lists of issues which are in the process of doing something

func (*Job) Logs 

func (j *Job) Logs() []*JobLog

Logs lazy-loads all logs for this job from the database

func (*Job) Save 

func (j *Job) Save() error

Save creates or updates the Job in the jobs table

func (*Job) SaveOp 

func (j *Job) SaveOp(op *magicsql.Operation) error

SaveOp creates or updates the job in the jobs table using a custom operation

func (*Job) WriteLog 

func (j *Job) WriteLog(level string, message string) error

WriteLog stores a log message on this job

type JobLog 

type JobLog struct {
	ID        int `sql:",primary"`
	JobID     int
	CreatedAt time.Time `sql:",readonly"`
	LogLevel  string
	Message   string
}

JobLog is a single log entry attached to a job

type MOC 

type MOC struct {
	ID   int `sql:",primary"`
	Code string
}

MOC contains MARC org codes

func AllMOCs 

func AllMOCs() ([]*MOC, error)

AllMOCs returns the full list of MOCs in the database, sorted by their org code

func CreateMOC 

func CreateMOC(code string) (*MOC, error)

CreateMOC adds a new MOC to the database with the given code

func FindMOCByCode 

func FindMOCByCode(code string) (*MOC, error)

FindMOCByCode searches the database for the given MOC and returns it if it's found, or nil if not

type Title 

type Title struct {
	ID           int `sql:",primary"`
	Name         string
	LCCN         string
	Embargoed    bool
	Rights       string
	ValidLCCN    bool
	SFTPDir      string
	SFTPUser     string
	SFTPPass     string
	MARCTitle    string
	MARCLocation string
}

Title holds records from the titles table

func FindTitle 

func FindTitle(where string, args ...interface{}) (*Title, error)

FindTitle searches the database for a single title

func FindTitleByID 

func FindTitleByID(id int) (*Title, error)

FindTitleByID wraps FindTitle to simplify basic finding

func (*Title) Save 

func (t *Title) Save() error

Save stores the title data in the database

func (*Title) SchemaTitle 

func (t *Title) SchemaTitle() *schema.Title

SchemaTitle converts a database Title to a schema.Title instance

type TitleList 

type TitleList []*Title

TitleList holds a full list of database titles for quick scan operations on all titles, such as is needed to do mass lookups of issues' LCCNs

func Titles 

func Titles() (TitleList, error)

Titles returns all titles in the database for bulk operations

func (TitleList) Find 

func (tl TitleList) Find(identifier string) *Title

Find looks for the title by either directory name or LCCN to give a simpler way to find titles in a general case

func (TitleList) FindByDirectory 

func (tl TitleList) FindByDirectory(dir string) *Title

FindByDirectory looks up a title by the given directory string, matching it against the sftp_dir field in the database

func (TitleList) FindByLCCN 

func (tl TitleList) FindByLCCN(lccn string) *Title

FindByLCCN returns the title matching the given LCCN or nil

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.