processor

Documentation

Overview

Package processing provides functionality for running a submission and its snapshots through the Impendulo tool suite.

Index

• Constants
• Variables
• func Compiler(p *FileProcessor) (tool.Compiler, error)
• func JPF(p *FileProcessor) (tool.T, error)
• func MonitorStatus() error
• func PMD(p *FileProcessor) (tool.T, error)
• func RunTools(file *project.File, target *tool.Target, p Processor) error
• func Serve(maxProcs uint) error
• func Shutdown() error
• func ShutdownMonitor() error
• func TestTools(p *TestProcessor, tf *project.File) ([]tool.T, error)
• func Tools(p *FileProcessor) ([]tool.T, error)
• type FileProcessor
  • func NewFileProcessor(sid bson.ObjectId) (*FileProcessor, error)
  • func (fp *FileProcessor) Compile(fid bson.ObjectId, t *tool.Target) error
  • func (fp *FileProcessor) Process(fid bson.ObjectId) error
  • func (fp *FileProcessor) ProcessArchive(a *project.File) error
  • func (fp *FileProcessor) ProcessSource(f *project.File) error
  • func (fp *FileProcessor) ProcessTest(test *project.File) error
  • func (fp *FileProcessor) ResultName(t tool.T) string
  • func (fp *FileProcessor) Start(fc chan bson.ObjectId, dc chan util.E)
  • func (fp *FileProcessor) StoreFile(n string, d []byte) (*project.File, error)
  • func (fp *FileProcessor) Tools() []tool.T
• type Helper
  • func NewHelper(sid bson.ObjectId) *Helper
  • func (h *Helper) Handle(onDone chan util.E, fq *list.List)
  • func (h *Helper) SetDone()
• type Monitor
  • func NewMonitor() (*Monitor, error)
  • func (m *Monitor) Monitor()
  • func (m *Monitor) Shutdown() error
• type Processor
• type Server
  • func NewServer(maxProcs uint) (*Server, error)
  • func (s *Server) Serve()
  • func (s *Server) Shutdown() error
• type TestProcessor
  • func NewTestProcessor(tf *project.File, fp *FileProcessor) (*TestProcessor, error)
  • func (tp *TestProcessor) Compile(fid bson.ObjectId, t *tool.Target) error
  • func (tp *TestProcessor) Process(fid bson.ObjectId) error
  • func (tp *TestProcessor) ResultName(t tool.T) string
  • func (tp *TestProcessor) Tools() []tool.T

Constants

const (
	LOG_PROCESSOR = "processing/processor.go"
)

const (
	LOG_SERVER = "processing/server.go"
)

const (
	LOG_TOOLS = "processing/tools.go"
)

Variables

var (
	MAX_PROCS = max(runtime.NumCPU()-1, 1)
)

Functions

func Compiler

func Compiler(p *FileProcessor) (tool.Compiler, error)

Compiler retrieves a compiler for a Processor's language.

func JPF

func JPF(p *FileProcessor) (tool.T, error)

JPF creates a new instance of the JPF tool.

func MonitorStatus

func MonitorStatus() error

MonitorStatus begins keeping track of Impendulo's current processing status.

func PMD

func PMD(p *FileProcessor) (tool.T, error)

PMD creates a new instance of the PMD tool.

func RunTools

func RunTools(file *project.File, target *tool.Target, p Processor) error

RunTools runs all available tools on a file. It skips a tool if there is already a result for it present. This makes it possible to rerun old tools or add new tools and run them on old files without having to rerun all the tools.

func Serve

func Serve(maxProcs uint) error

Serve launches the default Server. It listens on the configured AMQP URI and spawns at most maxProcs goroutines in order to process submissions.

func Shutdown

func Shutdown() error

Shutdown signals to the default Server that it can shutdown and waits for it to complete all processing. It then shuts down all active producers as well as the status monitor.

func ShutdownMonitor

func ShutdownMonitor() error

ShutdownMonitor

func TestTools

func TestTools(p *TestProcessor, tf *project.File) ([]tool.T, error)

func Tools

func Tools(p *FileProcessor) ([]tool.T, error)

Tools retrieves the Impendulo tool suite for a Processor's language. Each tool is already constructed.

Types

type FileProcessor

type FileProcessor struct {
	// contains filtered or unexported fields
}

func NewFileProcessor

func NewFileProcessor(sid bson.ObjectId) (*FileProcessor, error)

NewFileProcessor creates a Processor and sets up the environment and tools for it.

func (*FileProcessor) Compile

func (fp *FileProcessor) Compile(fid bson.ObjectId, t *tool.Target) error

Compile compiles a file, stores the result and returns any errors which may have occured.

func (*FileProcessor) Process

func (fp *FileProcessor) Process(fid bson.ObjectId) error

func (*FileProcessor) ProcessArchive

func (fp *FileProcessor) ProcessArchive(a *project.File) error

Extract extracts files from an archive, stores and processes them.

func (*FileProcessor) ProcessSource

func (fp *FileProcessor) ProcessSource(f *project.File) error

ProcessFile extracts archives and runs tools on source files.

func (*FileProcessor) ProcessTest

func (fp *FileProcessor) ProcessTest(test *project.File) error

func (*FileProcessor) ResultName

func (fp *FileProcessor) ResultName(t tool.T) string

func (*FileProcessor) Start

func (fp *FileProcessor) Start(fc chan bson.ObjectId, dc chan util.E)

Process listens for a submission's incoming files and processes them.

func (*FileProcessor) StoreFile

func (fp *FileProcessor) StoreFile(n string, d []byte) (*project.File, error)

StoreFile creates a new project.File given an encoded file name and file data. The new project.File is then saved in the database.

func (*FileProcessor) Tools

func (fp *FileProcessor) Tools() []tool.T

type Helper

type Helper struct {
	// contains filtered or unexported fields
}

Helper is used to help handle a submission's files.

func NewHelper

func NewHelper(sid bson.ObjectId) *Helper

NewHelper creates a new Helper for the specified Submission.

func (*Helper) Handle

func (h *Helper) Handle(onDone chan util.E, fq *list.List)

Handle helps manage the files a submission receives. It spawns a new Processor which runs in a seperate goroutine and receives files to process from this Helper. fq is the queue of files the submission has received prior to the start of processing.

func (*Helper) SetDone

func (h *Helper) SetDone()

SetDone indicates that this submission will receive no more files.

type Monitor

type Monitor struct {
	// contains filtered or unexported fields
}

Monitor is used to keep track of and change Impendulo's processing status.

func NewMonitor

func NewMonitor() (*Monitor, error)

NewMonitor

func (*Monitor) Monitor

func (m *Monitor) Monitor()

Monitor begins a new monitoring session for this Monitor. It handles status updates and requests.

func (*Monitor) Shutdown

func (m *Monitor) Shutdown() error

Shutdown stops this Monitor

type Processor

type Processor interface {
	ResultName(tool.T) string
	Compile(bson.ObjectId, *tool.Target) error
	Tools() []tool.T
	Process(bson.ObjectId) error
}

Processor is used to process individual submissions.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is our processing server which receives and processes submissions and files.

func NewServer

func NewServer(maxProcs uint) (*Server, error)

NewServer constructs a new Server instance which will listen on the coinfigured AMQP URI.

func (*Server) Serve

func (s *Server) Serve()

Serve spawns new processing routines for each submission started. Added files are received here and then sent to the relevant submission goroutine.

func (*Server) Shutdown

func (s *Server) Shutdown() error

Shutdown stops Serve from running once all submissions have been processed.

type TestProcessor

type TestProcessor struct {
	// contains filtered or unexported fields
}

func NewTestProcessor

func NewTestProcessor(tf *project.File, fp *FileProcessor) (*TestProcessor, error)

func (*TestProcessor) Compile

func (tp *TestProcessor) Compile(fid bson.ObjectId, t *tool.Target) error

func (*TestProcessor) Process

func (tp *TestProcessor) Process(fid bson.ObjectId) error

func (*TestProcessor) ResultName

func (tp *TestProcessor) ResultName(t tool.T) string

func (*TestProcessor) Tools

func (tp *TestProcessor) Tools() []tool.T