Child pages
  • Multiple Evaluator Workflow - Functional design
Skip to end of metadata
Go to start of metadata

This is the first iteration of a functional design for multiple evaluation workflow

(warning) See also evaluation workflow related requirements

Scenarios

Looking at the user stories we identified three basic scenario's for evaluation:

  1. Single evaluation - Student receives one evaluation per submission by one evaluator (current situation).
  2. Multiple independent evaluations - Student receives N evaluations per submission, created by N different evaluators that work independently.
  3. Team evaluation - Student receives one evaluation per submission created by N different evaluators that work together. Usually one of the evaluators on the team is the lead evaluator that communicates the evaluation to the student. The lead evaluator uses the evaluations of the other evaluators as an advice.

Scenario 1 workflow

Scenario 2 workflow

Scenario 3 workflow

Terminology

We use the term evaluatable item as an abstract term for things that can have an evaluation workflow. An evaluatable item can be a matrix cell or portfolio presentation. Since the assignments team is also working on evaluation workflow an evaluatable item could perhaps also be an assignment.

UI Mockups

These mockups give an impression of the screens a site administrator uses for defining the workflow.

Screen 1

Screen 2a

Screen 2b

 

Screen 3a

Screen 3b

 

These mockups give an impression of the screens an evaluator uses creating an evaluation.

Screen 5

Screen 6a

Screen 6b

Use cases



Use case name

Evaluate evaluatable item

Version

1.0

Goal

The evaluator wants to provide formative or summative feedback to the student and assess the formal progression of the student through the curriculum.

Actors

Evaluator

Preconditions

  • The evaluator is logged in to the system.
  • A student has submitted an item for evaluation.
  • An evaluation workflow has been created for the item under evaluation.
  • The evaluatable item has been routed to the evaluator or group of evaluators the evaluator is a member of.

Triggers

The evaluator has received a notification that an item needs evaluation.

Basic course of events

  1. The evaluator decides to evaluate an evaluable item. The system shows the evaluable items that are routed to the evaluator or to the group the evaluator is member of.
  2. The evaluator selects an evaluatable item from the overview. The system shows the contents of the evaluatable item with the submitted result of the student.
  3. The evaluator reads the contents of the evaluatable item and other available evaluations he has access to (if any).
  4. The evaluator fills out the evaluation form, provides a rating and then saves the form when complete. The system stores the evaluation with the evaluatable item.
  5. The evaluator completes the evaluation workflow step. The system initiates the next step in the workflow.
    1. If the evaluator is the final evaluator then the system marks the evaluatable item as EVALUATED and sends an optional notification to the student 
    2. If the evaluator is not the final evaluator then the system routes the evaluatable item to the next evaluator or group of evaluators. The system sends an optional notification to the next evaluator or to the evaluators in the group that have not yet contributed an evalutation to this evaluatable item. The status of the evaluatable item remains UNDER EVALUATION.
  6. The system removes the evaluatable item from the overview of items he can/should evaluate.

Alternative paths

Step 4. If the evaluator decides to complete the evaluation on a later time he saves the evaluation without completing the workflow step. The system shows the evaluatable item.
Goto step 1.

Postconditions

  • The evaluatable item has been evaluated
  • The evaluatable item can no longer be evaluated by the evaluator
  • The student has received an evaluation OR the evaluatable item is routed to the next evaluator if the evalation criteria are not yet met.

Business rules

 

Notes

See also screens 5, 6a & 6b

Author and date

Mark Breuker, Hugo Jacobs
April 4th, 2008


Use case name

Design evaluation workflow

Version

1.0

Goal

The administrator wants to allow an matrix cell or portfolio to be evaluated by an evaluator.

Actors

Administrator

Preconditions

  • The administrator is logged in to the system
  • An evaluatable item has been created
  • No students have previously submitted the evaluatable item

Triggers

A portfolio or matrix cell has been created that needs to be evaluated.

Basic course of events

1.      The administrator selects the evaluatable item for which s/he wants to define evaluation (workflow) settings. The system renders a link to the evaluation settings for the evaluatable item.
2.      The administrator follows the link to the evaluation settings.
3.      The administrator selects a workflow type: paralllel or sequential.
4.      For each step in the evaluation the administrator defines an evaluation step.
5.      If the administrator selected a sequential workflow s/he arranges the steps in the appropriate sequence.
6.      The administrator sets the global evaluation options.
7.      The administrator saves the settings. The system shows the evaluatable item.

Alternative paths

Step 4: The administrator removes a previously defined evaluation step. Goto step 5
Step 4. The administrator edits a previously defined evaluation step.
Goto step 5

Postconditions

An evaluation workflow has been defined for the evaluatable item.

Business rules

 

Notes

See also screen mockups 1, 2 & 3

Author and date

Mark Breuker
April 7th 2008


Use case name

Obtain evaluation for evaluatable item

Version

1.0

Goal

The student wants to obtain a formative or summative rating of an evaluatable item.

Actors

Student

Preconditions

  • The student is logged in to the system.
  • The student is permitted to submit the evaluatable item
  • The evaluatable item has not yet been submitted (status = READY)

Triggers

The student has completed work on the evaluatable item.

Basic course of events

1.      The student submits the evaluatable item;
2.      The system determines the first evaluator(s) in the evaluation workflow and changes the status of the evaluatable item to PENDING;
3.      The system sends a notification to the evaluator(s) if the evaluator indicated he wishes to be notified of new items in his list of items for evaluation;
4.      The students waits until the evaluatable item changes its status to EVALUATED. Optionally the system sends a notification to the student when the status changes;
5.      The student opens the evaluatable item. The system shows a list of all available evaluation(s);
6.      The student opens each evaluation to read the contents. Once the student has read the evaluation the system marks the evaluation as read.

Alternative paths

 

Postconditions

  • The student has obtained one or more evaluations for his submitted evaluatable item.
  • The status of the evaluatable item is EVALUATED

Business rules

 

Notes

If the evaluation in unsatisfactory to the student and the student is allowed to resubmit the item then the basic course of events is repeated.
In step 4 the status changes to EVALUATED when the evaluation workflow is completed.

Author and date

Mark Breuker
April 7th 2008

  • No labels