Skip to end of metadata
  • Created by Former user, last modified by Former user on Feb 21, 2012
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 30 Next »

Evaluation Test Plans

Prior to creating evaluations for the first time, administrators need to define system settings and permissions for their institution.

Please see the Administrator Guide for information on System Settings, Creating Scales and Setting up Permissions.

General Information

The Evaluation System is used to conduct formative and summative (End of Term) evaluations and review the results. An evaluation consists of a series of scaled and/or free text questions, which are delivered to students over a fixed time period (typically a few days). This guide will explain how to create items (questions), templates, and evaluations, deliver the evaluation and view the results. Permissions have been assigned for the "maintain" and "access" roles.  Instructors with the maintain role can create a template, assign an evaluation to a template, and be evaluated in an evaluation.  Students with the access role can complete and submit an evaluation.  Test IDs should be created for an Instructor and Students to assist the tester with this effort.

 The test plan is setup to record results from multiple testers.  It's advisable that you enter your Name, Date, Tag, and Browser information in the heading.  There is also a column to enter JIRA ticket numbers for any issues you create, the Operating System (eg, Windows 7, Mac OS 10.6) and Browser (eg, Internet Explorer 9.0, Firefox 10.0.2) you used to test with and any other pertinent information you have into the Notes section.

Overview of Evaluation Process

The process of using the Evaluation Tool involves the following general steps:
1. Create your items (this is optional as you can also create items while creating a template)
2. Create a template
3. Add items to your template
4. Create an evaluation
a. Set the parameters for the evaluation
5. Run the evaluation
6. View the results

Each of these steps are explained in the following sections.

Preparation

If you are new to QA testing for Sakai, there are some things that you do before you begin:

Creating test accounts and sites on the Sakai QA servers

  • The QA5 server has been used in the past by UM for testing Release candidates.  It can be found here:
    Quick link: http://qa5-us.sakaiproject.org/portal
  • Login to the Qa5 server: 
    • UserID = admin
    • Password = (traditional Sakai login)  contact Cathy Crouch if you need additional information (crouch@umich.edu)
  • On the Sakai QA servers, each user can create his or her own test accounts and sites.
  • Test accounts have been setup for the following IDs:  Instructor01, Student01, and Student02.  See the Information page of the test plan for details.
  • Navigation for Teaching Evaluations for the 1.4.x Release:  Click on "More Sites" at the top of the page > "TQ-test" site > Evaluation System tool
  • Set up 2 Sakai course and/or project sites, i.e. EvalTest01 and EvalTest02
  • Add an instructor to each site
  • Add at least 1 student to each site.  Multiple students should be added to at least 1 site to fulfill the requirements for testing as a Teaching Assistant and to simplify the process of testing many student test conditions.

Evaluation Dashboard

The Evaluations Dashboard provides a summary of group information and the current status of your evaluations. Links at the top of the page when selected will display the following:

  • My Evaluations
  • My Templates
  • My Items
  • My Scales
  • My Email Templates
    • + Add Template
    • + Add Evaluation

You'll also see two boxes on the screen which are described below:

  • Site/Group Listing: displays the groups that the instructor can be evaluated in and the groups they can evaluate.
  • Evaluations I am creating or administering: displays the Title, Status and Date of the instructor's evaluations
    • Clicking on the Title of the evaluation will bring up a preview of the evaluation.
    • The Status column will indicate whether an evaluation is active or if the results are viewable. You can click on the View Report link to gain direct access to the report for that particular evaluation.
    • The Date column will indicate the due date of the evaluation or the date of the when results become viewable.

Test Plan

All of the functional testing features are listed in the Google workbook named TQ 1.4 Release - Test Plan*.  Each worksheet identifies a feature to test and contains one more sets of tasks as described below. Click on a feature's link to view test conditions for each step.

  • Instructions- Instructions for completing the test plan and preliminary steps that are necessary to complete before testing features in the test plan.
  • Create Items - Test scripts that focus on creating, modifying and removing items.  These are the questions or statements which the evaluator responds to.
  • Create Templates - Templates store items in a layout and are used to create evaluations.  This page contains test scripts for creating and editing templates.
  • Create Evaluations - Test scripts that focus on authoring assessments. The conditions will focus on creating new and updating previously creating evaluations.

N. Teaching Assistant - User is a student (evaluator) and an instructor (evaluatee)  

The following default setting must be turned on in EvalSys > Administrate, "Enable Teaching Assistant Item category"    

  1. Log into Sakai as an instructor who is teaching courses and currently enrolled for at least one course. Both the course that the instructor is teaching and the course that the instructor is taking has evaluations created.
  2. Click on the Evaluation link in the left menu.
  3. Instructor clicks on a link under the "Groups I can be evaluated in."  Can only preview the evaluation.
  4. Instructor clicks on a link under the "Groups I can evaluate" and completes the evaluation.
  5. Click Submit

P. General Student Conditions. The following tests require the tester to become a student that matches each test condition.

  1. Log in as a student who is taking courses but no evaluations have been assigned. The Evaluation tool is not listed in the menu.
  2. Create an evaluation for a class that the student is taking.  Student receives an email notification for the evaluation.  (NA for 1.3.0 testing)
  3. Log in as a student who is enrolled in multiple courses to which an evaluation has been assigned.
  4. Click on Evaluation tool in the left menu.  Should not have options to add an evaluation or create templates.  Student sees evaluations under "Groups I can evaluate."
  5. It is between the start date and end date. The evaluation has an active link.
  6. Click on link.  The correct evaluation appears for the class.
  7. Complete evaluation and click Submit. The evaluation is submitted and the student is taken to the main evaluation dashboard.
  8. Student clicks on the link for another active evaluation. – The correct evaluation appears for the class.
  9. Answer a few of the questions and Submit (SAVE is not an option).
  10. Open the above evaluation. – The completed answers appear in the evaluation.
  11. Answer the rest of the questions and click the submit button.  The evaluation is submitted and the student is taken to the main evaluation dashboard.  Can submit multiple times through the End Date.
  12. Student clicks on the link for another active evaluation. – The correct evaluation appears for the class.
  13. Student has an evaluation that has not been taken and it is past the end date. – Link for the evaluation is inactive. Status = Closed.
  14. Student has an evaluation but it is prior to the start date. – Students should not see evaluations until after the Start Date.
  15. Student has an evaluation which has not been taken two days past the start date. – Student receives an email reminder containing a link to the evaluation. (NA for 1.3.0 testing)
  16. Student clicks on the link in the email. – Student is taken to a login page. (NA for 1.3.0 testing)
  17. Student logs in. – Student is taken to the evaluation. (NA for 1.3.0 testing)
  18. Student is taking a class with only one instructor.  Student sees the correct instructor name listed on the evaluation and questions appear one time.
  19. Student is taking a class with only multiple instructors.  Student sees both instructor names listed on the evaluation and questions appear once for each instructor.
  20. Once a student submits an evaluation, email reminders should stop.  – Verify that a student does not receive an email reminder during the evaluation period. (NA for 1.3.0 testing)
  21. Logout as a student and login as an instructor in one of the above classes where an evaluation has been submitted.
  22. Click on the Evaluation link in the left menu. – Verify that the count of submitted evaluations changed.

Q. AFTER End Date

  1. Delete existing evaluation – Not allowed. Evaluations cannot be deleted at this time.

Re-order question block using menus Re-ordering blocks with drop-down menus is not implemented in 1.3.x

  1. Go back and edit template.
  2. Try to reorder the items inside a bock of items by using the drop-down menu box for the numbered items.
  3. Exit edit mode and re-enter template and determine if order was saved.
  4. Preview template to be sure the order of items is displayed correctly.
  5. Go back to template edit view (from preview mode) and make sure order is still retained.

For ideas on how to format your test plans, take a look of some of the ones linked here: http://confluence.sakaiproject.org/display/QA/2.7.0+Changes. Also, the OSP test plans are very thorough: http://confluence.sakaiproject.org/display/OSP/System+Test+Scripts

Testing Results

  • No labels