samigo: Setting feedback date earlier than due/late submission date lets students see feedback DURING test.
Description
Attachments
- 01 Dec 2016, 08:56 AM
- 01 Dec 2016, 08:54 AM
Activity
Tiffany Stull October 4, 2021 at 7:28 PM
Some of the work here to make the feedback date check obey exceptions depends on the changes in SAK-34323.
Shawn Foster February 17, 2021 at 10:30 AM
Discussed this on today's Sakai UX call today. @Tiffany Stull said that UVa's fix has helped avoid this issue locally. I've assigned it to @Mitchell Dunn, as Tiffany says he's working on the local port for their upgrade.
Tiffany Stull April 29, 2020 at 8:53 AMEdited
Here's a usability article that I think sums up well why it's better to simply prevent the instructor from selecting an inappropriately early feedback date: Preventing User Errors: Avoiding Unconscious Slips
I didn't think about it before, but I like the idea of presenting a reasonable default. If we include logic in this fix to automatically populate the feedback date with the latest submission date allowed, I think that would be the most effective solution:
On selecting Feedback at a specific date, populate date field with latest submission date ("retract date" in the code) allowed.
If instructor edits the date to be earlier, block save with the appropriate error message at the top of the screen.
Tiffany Stull April 29, 2020 at 8:20 AMEdited
I don't think it's the more intuitive thing to try to add logic that would identify, on a per-student basis, whether a take exists, in order to determine whether the instructor's unintentional feedback settings should be ignored to display differently for that specific student. It would make me angry if something silently overrode the settings I selected. I think it makes the most sense to just stop the instructor from making the mistake in the first place.
Implementing the feedback date check was the fix specifically suggested by two instructors at UVA who fell prey to the issue by accidentally publishing tests copied from their previous sites (where they forgot that the feedback date was still set to a previous semester's date).
Additionally, it's my understanding that the point of the feedback at a specific date is to curtail cheating by preventing students from seeing the feedback before others take the test, and sharing that feedback with the students who didn't take it yet. To make the option work as intended and be fair to all students, no students should have the ability to see the feedback before that date.
Finally, the new logic you propose will unquestionably break so many things and be INCREDIBLY difficult to test with all the other mess of settings that already exist! Simpler is better, and date checks already exist in settings, so this isn't too hard a fix to implement.
Laura Gekeler April 29, 2020 at 8:16 AM
@Tiffany Stull I am really tired of not being able to do the right thing, the more intuitive thing, because the code would be too difficult to change, aren't you?
Issue: Samigo's Feedback Delivery option Feedback will be displayed to the student at specific dates does not check to ensure the show feedback date you enter is later than the assessment's final submission deadline (either due date or late submission deadline).
As a result, if you enter a date in Feedback Delivery that is EARLIER than the submission deadline, and students take the test AFTER the show feedback date, Samigo kindly treats the setting as if the instructor had selected Immediate Feedback and lets the student see any selected feedback WHILE they're in the process of taking the test. The issue is most likely to present itself when an instructor has duplicated/imported an older assessment - they can easily forget to update the Show feedback date even though they must update due and/or late submission dates for students to access the test.
Samigo should verify that the Feedback Delivery date is later than the final submission deadline and if not, warn and prevent publishing on selecting Save, Save Settings and Publish or Publish (from any screen/drop-down menu that has a publishing option available) like it does for a due date earlier than a release date.
In discussion on Teaching & Learning call, folks suggested that some language should display to the instructor about using Immediate Feedback to make feedback available to students while they take the test. The linked https://sakaiproject.atlassian.net/browse/SAK-34674#icft=SAK-34674 includes language to address this concern.
Suggested fix:
If the instructor tries to set the Show Feedback Date to be earlier than the Final Submission Deadline (a.k.a., retract date in the code) for the assessment or any exceptions, block save with an appropriate error message at the top of the screen which will allow the instructor to fix the error, e.g.:
The Show Feedback Date cannot be earlier than the Final Submission Deadline of [date here].
If the instructor attempts to enable a Show Feedback Date while a Final Submission Deadline is empty/null, thus having no deadline, block save with the following error message:
A Final Submission Deadline must be set to enable feedback on specific dates. Please enter a date for the Final Submission Deadline.
To reproduce:
Create an assessment with several questions, where you enter Correct/Incorrect Answer Feedback in the boxes for some of the questions.
Select Settings.
Under Availability and Submissions, enter a due date in the future.
Click on Grading and Feedback. Select Feedback will be displayed to the student at a specific date and enter some date in the past. Select Release Questions and the following and check ALL available feedback type boxes.
Log in as a student and start taking the test.
Click on the handy Feedback button to see the answers and Question-Level Feedback to the question you're viewing.
Select some correct answers and some incorrect answers, then click on the Table of Contents; you'll see your question and part scores, because those have also been made available in the feedback.