...
- Evaluation forms are also used to rate goals linked to a matrix cell or wizard page via the goal management tool.
- Evaluation forms, specified evaluators, and the goal management process may also be used with portfolios via the association of evaluation forms with portfolio templates.
- When evaluators add comments to an evaluation form instance, they may designate them as public or private comments.
- When each evaluator submits an instance of an evaluation form for a matrix cell, wizard page, or portfolio, the calculations for each criterion or goal in that form instance, as well as the mean score for the form instance, are combined with calculations from all other evaluation form instances for the owner of that cell, page, or portfolio to provide a mean score for each criterion or goal and a total mean score across all evaluators.
- The set of evaluators associated with a matrix cell, wizard page, or portfolio template can include one or more peer advisers whose ratings are advisory only and do not enter into the calculation of mean scores.
- The calculation of the mean score for each criterion or goal identifies an acceptable range of inter-rater reliability for each rating and flags ratings that require a third raterevaluator.
- A flagged evaluation form instance requires the submission of an additional evaluation form instance by a third evaluator before the cell, page, or portfolio can be complete. Ratings from the additional evaluator replace the scores from the evaluator farthest from the calculated mean score for the cell, page, or portfolio.
- When all evaluators have completed their ratings, the cell, page, or portfolio becomes complete and its contents and score cannot be changed.
- The owner of a completed matrix cell, wizard page, or portfolio can view combined ratings and public comments of all evaluators.
...