User Guide

Owned by Former user (Deleted)
Last updated: Nov 15, 2007 by Former user (Deleted)Version comment
2 min read

Information

The Evaluation System is used to conduct formative and summative (end of term) evaluations and review the results. An evaluation consists of a series of scaled and/or free text questions, which are delivered to students over a fixed time period (typically a few days).  This guide will explain how to create questions, templates, and evaluations, deliver the evaluation and view the results. Please see the administrator's guide for permissions setting and creating scales.

Overview of Evaluation Process

The process of using the Evaluation Tool involves the following general steps:

  1. Create your items (this is optional as you can also create items while creating a template)
  2. Create a template
  3. Add items to your template
  4. Create an evaluation
    1. Set the parameters for the evaluation
  5. Run the evaluation
  6. View the results

Each of these steps are explained in the following sections.

Evaluations Dashboard

The Evaluations dashboard provides a summary of group information and the current status of your evaluations.  You'll see two boxes on the screen which are described below:

  • Site/Group Listing: displays the groups that you can be evaluated in and the groups you can evaluate
  • Evaluations I am creating or administering: displays the title, status and date of your evaluations 
    • Clicking on the Title of the evaluation will bring up a preview of the evaluation. 
    • The Status column will indicate whether an evaluation is active or if results are viewable.  You can click on the View Report link to gain direct access to the report for that particular evaluation. 
    • The Date column will indicate the due date of the evaluation or the date of when results became viewable.

Creating items

  1. Click My Items.  You'll see both the items you've created and the expert items that come with the tool or have been added by others.
  2. Select the type of item you'd like to add (Scaled/Survey, Short Answer/Essay, Text Header) and click the Add item button.
  3. For Scaled/Survey items
    1. Enter the Item Text (this is the question or statement that the evaluator will respond to)
    2. Select the Scale
    3. Select the Sharing level (private or public)
    4. Indicate if this is an Expert item
      1. If this is an expert item, enter the Expert Description (describes the appropriate usage of this item for template authors)
    5. Set Display Settings Hints (these optional settings allow the item author to provide hints as to how this item should be displayed)
      1. Select the Scale Display Setting
      2. Indicate if N/A will be an option
      3. Select Item Category
    6. Click the Save item button
  4. For Short Answer/Essay items
    1. Enter the Item Text (this is the question or statement that the evaluator will respond to)
    2. Select the Sharing level (private or public)
    3. Indicate if this is an Expert item
      1. If this is an expert item, enter the Expert Description (describes the appropriate usage of this item for template authors)
    4. Set Display Settings Hints (these optional settings allow the item author to provide hints as to how this item should be displayed)
      1. Select the Response size
      2. Indicate if N/A will be an option
      3. Select Item Category
    5. Click the Save item button
  5. For Text Header items
    1. Enter the Item Text (this is the question or statement that the evaluator will respond to)
    2. Select the Sharing level (private or public)
    3. Indicate if this is an Expert item
      1. If this is an expert item, enter the Expert Description (describes the appropriate usage of this item for template authors)
    4. Set Display Settings Hints (these optional settings allow the item author to provide hints as to how this item should be displayed)
      1. Indicate if N/A will be an option
      2. Select Item Category
    5. Click the Save item button

Creating templates

  1. Click the Add Template link at the top
  2. Fill in the Title and Description of the template and select the Sharing level (private or public)
  3. Click the Save button.  You will now have the ability to add questions to the template.

Add items to your template

  1. Click on My Templates
  2. Click on the title of the template you want to modify
  3. For new items
    1. Select the type of item you'd like to add (Scaled/Survey, Short Answer/Essay, Text Header, Existing, Expert) and click the Add button.
    2. For Scaled/Survey, Short Answer/Essay and Text Header items see the Creating Items section above.
  4. For Existing items
    1. Select the items you wish to include in the template
    2. Click the Insert Items button
  5. For Expert items
    1. Click on the Category name you want to select from
    2. If available, choose the Objective you want to select from
    3. Select the items you wish to include in the template
    4. Click the Insert Items button
  6. Reordering items
    1. Click and drag:  Click on an item and drag it to the new location
    2. Drop-down box:  From the drop-down box, choose the new order for the item
  7. Creating blocks of items
    1. Select the items that you would like to have included in the block.  Note that all items need to have the same scale.  Once the first item is selected, all other items with the same scale will have an asterisk appear next to the checkbox for that item.
    2. Click the Create Block button
    3. Fill in the Item Header Text
    4. Indicate if N/A will be an option
    5. Indicate if Use Ideal Coloring will be used
    6. Select Item Category
    7. Click the Save item button

Creating evaluations

  1. Click the Add Evaluation link at the top
  2. Fill in the Title and Instructions and choose the Template to use for this evaluation.
  3. Click the Continue to Settings button.  There are several settings to define for delivering the evaluation:  Evaluation Dates, Evaluation results viewable by, Administrative Settings, Evaluation Extras, Evaluation Reminders.
  4. For Evaluation Dates
    1. Set the Start Date (Users may begin submitting evaluation responses on this date.)
    2. Set the Due Date (Evaluation ends on this date, no more responses after this date.)
    3. Set Stop Date (Users may still submit responses up until this date - defines a grace period).
    4. Set the View Date (Results are not viewable until this date.)
  5. For Evaluation results viewable by
    1. Indicate if you want results to be Private
  6. For Administrative settings
    1. Select the type of Authentication control setting to use for this evaluation
    2. Set the Instructor opt in setting
  7. For Evaluation Extras
    1. Enter a Category name.  This is used to group related evaluations and for the category view.
  8. For Evaluation Reminders
    1. A default notification email is sent to out to all students assigned to an evaluation the morning of the day the evaluation goes live.  Click the View/Edit the notification email? link to edit this notification email.
    2. Select the Email reminder to non-respondent students interval.
    3. A default reminder email is sent out to all students who have not yet completed the evaluation at the interval specified above.  Click the View/Edit the reminder email? link to edit this reminder email.
    4. Fill in the Email From/Reply-to Address: if different from the system setting.
  9. Click the Continue to Assign to Courses button
  10. Select the courses that will use this evaluation
  11. Click the Save Assigned Groups button.  You will see each group with their associated enrollments listed.
  12. Click the Done button
  1. explain basic usage (create template, start eval, take evals, view results), details about setting up the permissions correctly in Sakai would be helpful, explanation of the summary (dahsboard) view
  2. how to create a template with details about various things like inserting expert items, reusing existing items, block items, etc.
  3. how to start evaluations with details about the start dates and the various options
  4. explanation of the email reminders and how they work
  5. ways to access an evaluation to take it (email links, summary page, category view, direct links)
  6. Viewing reports options with details on how to download csvs and view text responses