Test Plans

Evaluation Test Plans

Prior to creating evaluations for the first time, administrators need to define system settings and permissions for their institution.

Please see the Administrator Guide for information on System Settings, Creating Scales and Setting up Permissions.

A. Create template and add/remove items

1. Create template.
2. Add several items from "My Items" bank of questions.
3. Remove one item.
4. Exit edit mode and re-enter template and confirm the item was removed.

B. Test drag and drop re-ordering

1. Try to reorder the items by using the drag-and-drop UI.
2. Click "Save Order."
3. Exit edit mode and re-enter template and determine if order was saved.

C. Test drop-down menu re-ordering

1. Go back and edit template.
2. Try to reorder using the drop-down menu box for the numbered items.
3. Exit edit mode and re-enter template and determine if order was saved.

D. Add new questions and test drag and drop re-ordering

1. Try to add new items (versus selecting existing items in "My Items" bank of questions)
2. Try to reorder them by using the drag-and-drop UI.
3. Click "Save Order."
4. Exit edit mode and re-enter template and determine if order was saved.

E. Add new questions and test re-ordering within a group using UI drag-and-drop

1. Go back and edit template.
2. Try to reorder using the drop-down menu box for the new items.
3. Exit edit mode and re-enter template and determine if order was saved.
4. Create question group and test ordering
5. Click the checkbox next to several items using the same evaluation scale and then click the "Create Question Group" button.
6. Preview template to be sure blocked items are displayed correctly.
7. Try to reorder the items inside a bock of items by using the drag-and-drop UI.
8. (There is no need to click the "Save Order" button, because the changes are automatically saved; the "Save Order" button is disabled.)
9. Exit edit mode and re-enter template and determine if order was saved.

F. Test Instructor items on evaluation

1. Edit a survey question and click the "Evaluatee/Instructor" radio button in the "Category" area.
2. Save the question.
3. Preview the survey and confirm that the question for the "Instructor" is showing up for each of the preview instructors.

G. Test Creating a Group with a blank item header text

1. Select several items with the same rating scale.
2. Click "Create Group".
3. Leave Item Header Text blank and see if system lets you enter it with a blank.
4. Should be presented with the screen again where the Item Header Text box is still blank (expecting to be filled in).
5. Test "Cancel" button.
6. Type in header text.
7. Click Save item.

H. Test hierarchy template features

1. Go to Administrate.
2. Check the "Use Hierarchy menus, options, and data in the system" checkbox and then click "Save Settings."
3. Click on "Control Hierarchy".
4. Click on the "Add" link next to the "Root" hierarchy level.
5. Name the node "Department 1" and click "Save Item."
6. Repeat those steps to create another node immediately under "Root" called "Department 2".
7. Go back to edit template view. 
8. Add node assignments to two different items.  Edit the item and choose "Department 1" from the "Hierarchy Assignment" menu for one item.  Choose "Department 2" for a different item.
9. Preview template to be sure the order of items is displayed correctly and in the correct nodes.

I. Set up courses, instructors, and students

1. Set up two Sakai course sites, eval_test_1 and eval_test_2.  (additional course sites may be added now to fulfill requirements for sections N - P below)
2. Add an instructor level account (ex. instructor1) to eval_test_1 as Instructor; add two instructors (ex. instructor1 and instructor2) to eval_test_2.
3. Add at least one student account (ex. student1) to each course. (Multiple students should be added to the courses to fulfill requirements for sections N - P below)

J. Set up evaluation

1. Log into Sakai (or super-user in) as "instructor1".
2. Go to the Evaluation System tool and click on My Evaluations.
3. Click "Add Evaluation."
4. Give the evaluation a title and choose the template you have previously created.
5. Select a template from the list.
6. Configure the evaluation to launch immediately.
7. Click "Continue to Assign to Courses."
8. Do not select anything from the Hierarchy Level (already defined on the template).  Select the Course Title under "Assign to Evaluation Groups" (you can also Assign the evaluation to an adhoc group)
9. Click "Save Assigned Groups."
10. Click "Done."
11. Setup additional evaluations to fulfill requirements for sections N - P below.

K. Check evaluation

1. Log into Sakai (or super-user in) as "student1".
2. Click into the evaluations for each course.
3. Confirm that questions are in correct order.
4. In eval_test_1 evaluation, confirm that only "instructor1" item block appears.
5. In eval_test_2 evaluation, confirm that both instructors' item blocks appear.
6. In eval_test_1 evaluation, confirm that "Department 1" items appear and "Department 2" items do not appear.
7. In eval_test_2 evaluation, confirm that "Department 2" items appear and "Department 1" items do not appear.
8. Confirm that reminder e-mails are sent on schedule.
9. Confirm that reminder messages are not sent after a student completes an evaluation.

L. Change evauation (BEFORE Start Date)

1. Change an existing order START_DATE or END_DATE
2. Change an existing order - Template change.
3. Instructor changes existing order. Add/Remove an Instructor question.
4. Add/Remove an Instructor
5. Remove an existing evaluation.

M. AFTER Start Date and BEFORE End date

1. "Evaluations are Available"  and email notifications are generated to students and instructors.
2. Change existing order End Date should not be an available option.
3. Add or remove students from the roster for a course
4. Remove an eval should not be an available option

N. Teaching Assistant - User is a student (evaluator) and an instructor (evaluatee)  

The following default setting must be turned on in EvalSys > Administrate, "Enable Teaching Assistant Item category"    

1. Log into Sakai as an instructor who is teaching courses and currently enrolled for at least one course. Both the course that the instructor is teaching and the course that the instructor is taking has evaluations created.
2. Click on the Evaluation link in the left menu.
3. Instructor clicks on a link under the "Groups I can be evaluated in."  Can only preview the evaluation.
4. Instructor clicks on a link under the "Groups I can evaluate" and completes the evaluation.
5. Click Submit

O. User is an instructor (evaluatee) only. The following tests require the tester to become an instructor that matches each test condition.

1. Login as an instructor who is teaching one course but not enrolled for any courses
2. Click on the Evaluation link in the left menu.
3. Evaluation for the course displays under "Groups I can be evaluated in."
4. Open evaluation.  Instructor can view questions for all instructors teaching the course, but is unable to edit questions or submit.
5. Login as an instructor who is teaching multiple courses.
6. Click on the Evaluation link in the left menu.
7. Evaluation links display for all courses.
8. Open evaluations.  Instructor can view questions for all instructors teaching the course, but is unable to edit questions or submit.

P. General Student Conditions. The following tests require the tester to become a student that matches each test condition.

1. Log in as a student who is taking courses but no evaluations have been assigned. The Evaluation tool is not listed in the menu.
2. Create an evaluation for a class that the student is taking.  Student receives an email notification for the evaluation.  (NA for 1.3.0 testing)
3. Log in as a student who is enrolled in multiple courses to which an evaluation has been assigned.
4. Click on Evaluation tool in the left menu.  Should not have options to add an evaluation or create templates.  Student sees evaluations under "Groups I can evaluate."
5. It is between the start date and end date. The evaluation has an active link.
6. Click on link.  The correct evaluation appears for the class.
7. Complete evaluation and click Submit. The evaluation is submitted and the student is taken to the main evaluation dashboard.
8. Student clicks on the link for another active evaluation. – The correct evaluation appears for the class.
9. Answer a few of the questions and Submit (SAVE is not an option).
10. Open the above evaluation. – The completed answers appear in the evaluation.
11. Answer the rest of the questions and click the submit button.  The evaluation is submitted and the student is taken to the main evaluation dashboard.  Can submit multiple times through the End Date.
12. Student clicks on the link for another active evaluation. – The correct evaluation appears for the class.
13. Student has an evaluation that has not been taken and it is past the end date. – Link for the evaluation is inactive. Status = Closed.
14. Student has an evaluation but it is prior to the start date. – Students should not see evaluations until after the Start Date.
15. Student has an evaluation which has not been taken two days past the start date. – Student receives an email reminder containing a link to the evaluation. (NA for 1.3.0 testing)
16. Student clicks on the link in the email. – Student is taken to a login page. (NA for 1.3.0 testing)
17. Student logs in. – Student is taken to the evaluation. (NA for 1.3.0 testing)
18. Student is taking a class with only one instructor.  Student sees the correct instructor name listed on the evaluation and questions appear one time.
19. Student is taking a class with only multiple instructors.  Student sees both instructor names listed on the evaluation and questions appear once for each instructor.
20. Once a student submits an evaluation, email reminders should stop.  – Verify that a student does not receive an email reminder during the evaluation period. (NA for 1.3.0 testing)
21. Logout as a student and login as an instructor in one of the above classes where an evaluation has been submitted.
22. Click on the Evaluation link in the left menu. – Verify that the count of submitted evaluations changed.

Q. AFTER End Date

1. Delete existing evaluation – Not allowed. Evaluations cannot be deleted at this time.

Re-order question block using menus Re-ordering blocks with drop-down menus is not implemented in 1.3.x

1. Go back and edit template.
2. Try to reorder the items inside a bock of items by using the drop-down menu box for the numbered items.
3. Exit edit mode and re-enter template and determine if order was saved.
4. Preview template to be sure the order of items is displayed correctly.
5. Go back to template edit view (from preview mode) and make sure order is still retained.

For ideas on how to format your test plans, take a look of some of the ones linked here: http://confluence.sakaiproject.org/display/QA/2.7.0+Changes. Also, the OSP test plans are very thorough: http://confluence.sakaiproject.org/display/OSP/System+Test+Scripts

Testing Results

• Evaluation Test Results 1.3.0-RC3