Title:	Evaluator Selection Redesign
Jira Number:	SAK-13577
Related Jira Numbers:	SAK-13285
Component(s):	OSP Matrix/Wizard/Evaluations
Author:	Beth Kirschner (UM)
Date:	May 19, 2008
Demo Status and Date(s):	Status: Complete (checked into trunk June 12, 2008) Dates: Reviewed June 2, 2008 Available on osp-nightly June 4, 2008

Part 1: Functional Description

Evaluator Selection Redesign (1)

Summary (1)

Update the logic defining the selection of evaluators in matrix and wizard cells:

Only qualified users can be selected as evaluators (e.g. must have either osp.matrix.evaluate or osp.wizard.evaluate permission)
If no evaluators are selected, by default all users with evaluate permission will be auto-selected

Update the Matrix/Wizard properties terminology:

Restate the 'Reviewer Group Access' options as 'Reviewer/Evaluator Group Access'

Update the logic of the Evaluations tool:

Evaluatable items will be filtered by group – users will only see those items by users restricted by group (if matrix/wizard group filtering is enabled).

Rationale (1)

The current implementation of Evaluator Selection is flawed – consider the current functionality:

Any user can be selected as an evaluator, even if they have no evaluate permission (or if they are not a member of an appropriate group). Users can be filtered by group (via 'browse for individuals') without consideration of group restrictions.
Any role can be selected as an evaluator, even if they have no evaluate permission (or if they are not a member of an appropriate group).
If a user is selected as an evaluator, but does not have the evaluate permissions, they may not use the Evaluations tool.
If a user is selected as an evaluator, but cannot evaluate in the matrix/wizard tool due to group restrictions, they can evaluate from the Evaluations tool.
If a user is not selected as an evaluator, but has evaluate permission, they can evaluate in the matrix/wizard tool, but not in the Evaluations tool.

Origin (1)

This issue arose out of the need to support group filtering in the Evaluations tool (i.e. if group filtering is enabled in a matrix, the same filtering must be enabled in the Evaluations tool for consistency).

User Stories (1)

The User Stories should paint a picture of what it is like for a user to make use of the enhancement. The actors should be based on real users with definable tasks and goals. Include as many stories as necessary to demonstrate how the enhancement would be used by different types of users.

Actors and Stakeholders

Actor: Faculty/Evaluator
Actor: Peer Advisor/Reviewer
Actor: Student
Stakeholder: Educational institute (high quality, consistent evaluations)

Michigan Story

One matrix serves hundreds of students, peer advisors and faculty, with reviewing/evaluating group access set to "Normal: may only review work of fellow group members". Any (faculty) user with evaluate permission is allowed to evaluate student users in their group. Evaluations may be performed via the Evaluations or the Matrix tool.

Indiana Story

One matrix serves students over the course of several years, with different users allowed to review/evaluate different cells. Evaluations may be performed via the Evaluations or the Matrix tool.

Functional Details (may be added after community demo) (1)

Implementation notes:

Wizard and WizardPageDefinition classes will be modified such that siteId is an Id (not String). This is consistent with the Scaffolding class and allows faster loading of available evaluatable items in Evaluations tool. There does not appear to be a performance degradation elsewhere (the siteId String was often converted to an Id in much of the code). Warning: this is a broad/shallow change affecting 29 files.
AudienceTool class may sort lists of groups, users, roles if performance isn't shown to suffer.

Interaction and Implications (1)

Multiple Evaluators: These designs do not contradict/negate plans for Multiple evaluators workflow, though the Multiple Evaluator Workflow should be reviewed and revised in light of this Evaluator Selection Redesign.

Diagrams and Mockups (3)

The secondary browse for individuals page, which allowed filtering users by group and/or role, has been removed. It's functionality has been merged into Select Evaluators and Share Portfolio as appropriate.

old browse for individuals

The redesigned Select Evaluators page removes the secondary browse for individuals page (rationale: filter by role would be confusing when on the same page as selected by role; filter by group would be confusing given how groups are used in matrix/wizards). Instead, users/roles are first filtered by permission before being displayed in user list.

old evaluator select	new evaluator select

Because the Share Portfolio page shares the same underlying code, it has also been redesigned. The browse for individuals page has been merged into the primary selection page (allowing filter by groups, but not filter by roles, since this would be confusing when on the same page as selected by role).

old portfolio share	new portfolio share

Community Acceptance (4)

Initial Discussion: May 19, 2008
Proposal Review: June 2, 2008

Part 2 of the Proposal for Enhancement Template: The Specification
The specification should be filled out once the feature is clearly defined.

Specification Template (5)

Behavior

Select Evaluators (Default)
Prerequisite Conditions or Step:	User creates matrix cell or wizard page and does not select evaluators
Behavior:	By default, all user roles with osp.matrix.evaluate or osp.wizard.evaluate permission will be auto-selected as evaluators
Post-step Conditions or Next Step:	Evaluations

Select Evaluators
Prerequisite Conditions or Step:	User selects evaluators for matrix cell or wizard page
Behavior:	Only users and roles with osp.matrix.evaluate or osp.wizard.evaluate permission are displayed
Post-step Conditions or Next Step:	Evaluations

Evaluations (Normal Group Access)
Prerequisite Conditions or Step:	Student submits matrix cell or wizard page for evaluation; Group Access is "Normal: Reviewer/Evaluator may only view work of fellow group members."
Behavior:	Selected evaluators for cell/page will only see this evaluatable item if they are in the Student's group
Post-step Conditions or Next Step:	Evaluation proceeds as previously implemented

Evaluations (Unrestricted Group Access)
Prerequisite Conditions or Step:	Student submits matrix cell or wizard page for evaluation; Group Access is "Unrestricted: Reviewer/Evaluator may view all members of the site, regardless of group membership."
Behavior:	All selected evaluators for cell/page will see this evaluatable item
Post-step Conditions or Next Step:	Evaluation proceeds as previously implemented

Interaction

Matrix Tool
- Audience (Evaluator Selection) Helper Tool
Wizards Tool
- Audience (Evaluator Selection) Helper Tool
Portfolio
- Audience (Share Portfolio) Helper Tool
Evaluations Tool

Quality Metrics

None of the affected tools (especially the Evaluations tool) should suffer with degraded performance from the addition of new logic controlling group filtering. If possible, performance should be improved.

Assumptions

n/a

Outstanding Issues

n/a

Browser not supported