Multiple Evaluator Workflow - Functional design

This is the first iteration of a functional design for multiple evaluation workflow

See also evaluation workflow related requirements

Scenarios

Looking at the user stories we identified three basic scenario's for evaluation:

Single evaluation - Student receives one evaluation per submission by one evaluator (current situation).
Multiple independent evaluations - Student receives N evaluations per submission, created by N different evaluators that work independently.
Team evaluation - Student receives one evaluation per submission created by N different evaluators that work together. Usually one of the evaluators on the team is the lead evaluator that communicates the evaluation to the student. The lead evaluator uses the evaluations of the other evaluators as an advice.


Scenario 1 workflow	Scenario 2 workflow	Scenario 3 workflow

Terminology

We use the term evaluatable item as an abstract term for things that can have an evaluation workflow. An evaluatable item can be a matrix cell or portfolio presentation. Since the assignments team is also working on evaluation workflow an evaluatable item could perhaps also be an assignment.

UI Mockups

These mockups give an impression of the screens a site administrator uses for defining the workflow.


Screen 1	Screen 2a	Screen 2b

Screen 3a	Screen 3b

These mockups give an impression of the screens an evaluator uses creating an evaluation.


Screen 5	Screen 6a	Screen 6b

Use cases

Use case name	Evaluate evaluatable item
Version	1.0
Goal	The evaluator wants to provide formative or summative feedback to the student and assess the formal progression of the student through the curriculum.
Actors	Evaluator
Preconditions	The evaluator is logged in to the system. A student has submitted an item for evaluation. An evaluation workflow has been created for the item under evaluation. The evaluatable item has been routed to the evaluator or group of evaluators the evaluator is a member of.
Triggers	The evaluator has received a notification that an item needs evaluation.
Basic course of events	The evaluator decides to evaluate an evaluable item. The system shows the evaluable items that are routed to the evaluator or to the group the evaluator is member of. The evaluator selects an evaluatable item from the overview. The system shows the contents of the evaluatable item with the submitted result of the student. The evaluator reads the contents of the evaluatable item and other available evaluations he has access to (if any). The evaluator fills out the evaluation form, provides a rating and then saves the form when complete. The system stores the evaluation with the evaluatable item. The evaluator completes the evaluation workflow step. The system initiates the next step in the workflow. If the evaluator is the final evaluator then the system marks the evaluatable item as EVALUATED and sends an optional notification to the student If the evaluator is not the final evaluator then the system routes the evaluatable item to the next evaluator or group of evaluators. The system sends an optional notification to the next evaluator or to the evaluators in the group that have not yet contributed an evalutation to this evaluatable item. The status of the evaluatable item remains UNDER EVALUATION. The system removes the evaluatable item from the overview of items he can/should evaluate.
Alternative paths	Step 4. If the evaluator decides to complete the evaluation on a later time he saves the evaluation without completing the workflow step. The system shows the evaluatable item. Goto step 1.
Postconditions	The evaluatable item has been evaluated The evaluatable item can no longer be evaluated by the evaluator The student has received an evaluation OR the evaluatable item is routed to the next evaluator if the evalation criteria are not yet met.
Business rules
Notes	See also screens 5, 6a & 6b
Author and date	Mark Breuker, Hugo Jacobs April 4th, 2008

Use case name	Design evaluation workflow
Version	1.0
Goal	The administrator wants to allow an matrix cell or portfolio to be evaluated by an evaluator.
Actors	Administrator
Preconditions	The administrator is logged in to the system An evaluatable item has been created No students have previously submitted the evaluatable item
Triggers	A portfolio or matrix cell has been created that needs to be evaluated.
Basic course of events	1. The administrator selects the evaluatable item for which s/he wants to define evaluation (workflow) settings. The system renders a link to the evaluation settings for the evaluatable item. 2. The administrator follows the link to the evaluation settings. 3. The administrator selects a workflow type: paralllel or sequential. 4. For each step in the evaluation the administrator defines an evaluation step. 5. If the administrator selected a sequential workflow s/he arranges the steps in the appropriate sequence. 6. The administrator sets the global evaluation options. 7. The administrator saves the settings. The system shows the evaluatable item.
Alternative paths	Step 4: The administrator removes a previously defined evaluation step. Goto step 5 Step 4. The administrator edits a previously defined evaluation step. Goto step 5
Postconditions	An evaluation workflow has been defined for the evaluatable item.
Business rules
Notes	See also screen mockups 1, 2 & 3
Author and date	Mark Breuker April 7th 2008

Use case name	Obtain evaluation for evaluatable item
Version	1.0
Goal	The student wants to obtain a formative or summative rating of an evaluatable item.
Actors	Student
Preconditions	The student is logged in to the system. The student is permitted to submit the evaluatable item The evaluatable item has not yet been submitted (status = READY)
Triggers	The student has completed work on the evaluatable item.
Basic course of events	1. The student submits the evaluatable item; 2. The system determines the first evaluator(s) in the evaluation workflow and changes the status of the evaluatable item to PENDING; 3. The system sends a notification to the evaluator(s) if the evaluator indicated he wishes to be notified of new items in his list of items for evaluation; 4. The students waits until the evaluatable item changes its status to EVALUATED. Optionally the system sends a notification to the student when the status changes; 5. The student opens the evaluatable item. The system shows a list of all available evaluation(s); 6. The student opens each evaluation to read the contents. Once the student has read the evaluation the system marks the evaluation as read.
Alternative paths
Postconditions	The student has obtained one or more evaluations for his submitted evaluatable item. The status of the evaluatable item is EVALUATED
Business rules
Notes	If the evaluation in unsatisfactory to the student and the student is allowed to resubmit the item then the basic course of events is repeated. In step 4 the status changes to EVALUATED when the evaluation workflow is completed.
Author and date	Mark Breuker April 7th 2008