...
Gap analysis between Samigo and Respondus question types and file formats
I conducted a comparison of the IMX XML files generated by Samigo v2.3 and Respondus v3.5 with the following results.
Testing Results
Import Results for 10/3/06
Initial assessment import testing of a Respondus v3.5 xml file with 8 questions, one of each type available. This testing is prior to any code changes or additions. The testing environment was my local Sakai v2.3.
Original Type | Translated Type | Question Text | Points | Answers | Feedback Types | Feedback Text |
|---|---|---|---|---|---|---|
Multiple Choice/Multiple choice single choice (mcsc) | Incorrectly typed | Text imported correctly | Value correct | Defaulted to Model Short Answer - only the last answer displayed | general feedback --> copy to both correct and incorrect question feedback fields? | Feedback was not saved |
True-False (TF) | Correctly typed | Text imported correctly | Value correct | True and false answers displayed; correct answer not selected | general or answer-level feedback options --> copy general to both question-level feedback fields? | Feedback was not saved |
Essay | Correctly typed | Text imported correctly | Value incorrect - original of 5.0 not saved | n/a to translation | general feedback | Feedback was not saved |
Matching | Correctly typed | Text imported correctly | Value correct | Pairs did not translate | general feedback --> copy to both correct and incorrect question feedback fields? | Feedback was not saved |
Fill-in-Blank | Correctly typed | Text imported twice on question list; not at all in Modify Question | Value correct | no - question text not visible in Modify Question | general feedback --> copy to both correct and incorrect question feedback fields? | Feedback was not saved |
Multiple Response (Grading Method: Right Less Wrong) | Correctly typed | Text displayed correctly in list, but not in Modify where it generated UI problems | Value correct | Answers displayed correctly in list, but correct answers not indicated; UI problems in Modify | general feedback --> copy to both correct and incorrect question feedback fields? | Feedback was not saved |
Multiple Response (Grading Method: All Points or None) | Incorrectly typed | Text imported correctly | Value correct | Defaulted to individual Model Short Answer records - only the last answer displayed | general feedback --> copy to both correct and incorrect question feedback fields? | Feedback was not saved |
Algorithmic* | Instructions and link to Flash movie displayed; link fails due to failure to import SWF | Value correct | Defaulted to 5 individual Model Short Answer records | n/a | n/a | n/a |
Of note here is that the feedback models for Samigo are more numerous and flexible than for Respondus. Samigo offers both question-level and answer-level incorrect and correct feedbacks, depending on the question type. Respondus offers only general feedback except for true-false question type.
The question arises as to which field is most appropriate to migrate a general feedback response into - the correct answer feedback, incorrect answer feedback, both or the answer-level feedback fields. The answer may vary with the instructor or school - and may additionally vary based on the desired assessment feedback setting.
*Also of note here is that Samigo does not curently support algorithmic question types. Respondus exports this question type as a .swf or Flash movie and the question text is actually the JavaScript launch code and the xml answer output is similar to a multiple-choice single-correct answer. It would be interesting to determine if the SWF could be imported as an attachment and whether the user's response could be sent back for scoring.
Finally, there are differences in grading method that should be noted in the gap analysis so that instructors understand the potential grading changes in their translations.