---

Part 1: Functional Description

---

Evaluator Selection Redesign (1)

Summary (1)

Update the logic defining the selection of evaluators in matrix and wizard cells:

• Only qualified users can be selected as evaluators (e.g. must have either osp.matrix.evaluate or osp.wizard.evaluate permission)
• If no evaluators are selected, by default all users with evaluate permission will be auto-selected

Update the Matrix/Wizard properties terminology:

• Restate the 'Reviewer Group Access' options as 'Reviewer/Evaluator Group Access'

Update the logic of the Evaluations tool:

• Evaluatable items will be filtered by group – users will only see those items by users restricted by group (if matrix/wizard group filtering is enabled).

Rationale (1)

The current implementation of Evaluator Selection is flawed – consider the current functionality:

1. Any user can be selected as an evaluator, even if they have no evaluate permission (or if they are not a member of an appropriate group). Users can be filtered by group (via 'browse for individuals') without consideration of group restrictions.
2. Any role can be selected as an evaluator, even if they have no evaluate permission (or if they are not a member of an appropriate group).
3. If a user is selected as an evaluator, but does not have the evaluate permissions, they may not use the Evaluations tool.
4. If a user is selected as an evaluator, but cannot evaluate in the matrix/wizard tool due to group restrictions, they can evaluate from the Evaluations tool.
5. If a user is not selected as an evaluator, but has evaluate permission, they can evaluate in the matrix/wizard tool, but not in the Evaluations tool.

Origin (1)

This issue arose out of the need to support group filtering in the Evaluations tool (i.e. if group filtering is enabled in a matrix, the same filtering must be enabled in the Evaluations tool for consistency).

User Stories (1)

The User Stories should paint a picture of what it is like for a user to make use of the enhancement. The actors should be based on real users with definable tasks and goals. Include as many stories as necessary to demonstrate how the enhancement would be used by different types of users.

Actors and Stakeholders

• Actor: Faculty/Evaluator
• Actor: Peer Advisor/Reviewer
• Actor: Student
• Stakeholder: Educational institute (high quality, consistent evaluations)

Michigan Story

One matrix serves hundreds of students, peer advisors and faculty, with reviewing/evaluating group access set to "Normal: may only review work of fellow group members". Any (faculty) user with evaluate permission is allowed to evaluate student users in their group. Evaluations may be performed via the Evaluations or the Matrix tool.

Indiana Story

One matrix serves students over the course of several years, with different users allowed to review/evaluate different cells. Evaluations may be performed via the Evaluations or the Matrix tool.

Functional Details (may be added after community demo) (1)

Implementation notes:

• Wizard and WizardPageDefinition classes will be modified such that siteId is an Id (not String). This is consistent with the Scaffolding class and allows faster loading of available evaluatable items in Evaluations tool. There does not appear to be a performance degradation elsewhere (the siteId String was often converted to an Id in much of the code). Warning: this is a broad/shallow change affecting 29 files.
• AudienceTool class may sort lists of groups, users, roles if performance isn't shown to suffer.

Interaction and Implications (1)

• Multiple Evaluators: These designs do not contradict/negate plans for Multiple evaluators workflow, though the Multiple Evaluator Workflow should be reviewed and revised in light of this Evaluator Selection Redesign.

Diagrams and Mockups (3)

The secondary browse for individuals page, which allowed filtering users by group and/or role, has been removed. It's functionality has been merged into Select Evaluators and Share Portfolio as appropriate.

The redesigned Select Evaluators page removes the secondary browse for individuals page (rationale: filter by role would be confusing when on the same page as selected by role; filter by group would be confusing given how groups are used in matrix/wizards). Instead, users/roles are first filtered by permission before being displayed in user list.

Because the Share Portfolio page shares the same underlying code, it has also been redesigned. The browse for individuals page has been merged into the primary selection page (allowing filter by groups, but not filter by roles, since this would be confusing when on the same page as selected by role).

Community Acceptance (4)

Initial Discussion: May 19, 2008
Proposal Review: June 2, 2008

---

Part 2 of the Proposal for Enhancement Template: The Specification
The specification should be filled out once the feature is clearly defined.

---

Specification Template (5)

Behavior

Interaction

• Matrix Tool
  • Audience (Evaluator Selection) Helper Tool
• Wizards Tool
  • Audience (Evaluator Selection) Helper Tool
• Portfolio
  • Audience (Share Portfolio) Helper Tool
• Evaluations Tool

Quality Metrics

None of the affected tools (especially the Evaluations tool) should suffer with degraded performance from the addition of new logic controlling group filtering. If possible, performance should be improved.

Assumptions

n/a

Outstanding Issues

n/a