Respondus Migration

(Note on Dec 16, 2008: This applies to all Samigo v2.x versions. )

Gap analysis between Samigo and Respondus question types and file formats

I conducted a comparison of the IMX XML files generated by Samigo v2.3 and Respondus v3.5 with the following results.

Testing Results

Import Results for 10/3/06

Initial assessment import testing of a Respondus v3.5 xml file with 8 questions, one of each type available. This testing is prior to any code changes or additions. The testing environment was my local Sakai v2.3.

Original Type	Translated Type	Question Text	Points	Answers	Feedback Types	Feedback Text
Multiple Choice/Multiple choice single choice (mcsc)	Incorrectly typed Defaults to Essay type	Text imported correctly	Value correct	Defaulted to Model Short Answer - only the last answer displayed	general feedback --> copy to both correct and incorrect question feedback fields?	Feedback was not saved
True-False (TF)	Correctly typed	Text imported correctly	Value correct	True and false answers displayed; correct answer not selected	general or answer-level feedback options --> copy general to both question-level feedback fields?	Feedback was not saved
Essay	Correctly typed	Text imported correctly	Value incorrect - original of 5.0 not saved	n/a to translation	general feedback	Feedback was not saved
Matching	Correctly typed	Text imported correctly	Value correct	Pairs did not translate	general feedback --> copy to both correct and incorrect question feedback fields?	Feedback was not saved
Fill-in-Blank	Correctly typed	Text imported twice on question list; not at all in Modify Question	Value correct	no - question text not visible in Modify Question	general feedback --> copy to both correct and incorrect question feedback fields?	Feedback was not saved
Multiple Response (Grading Method: Right Less Wrong)	Correctly typed	Text displayed correctly in list, but not in Modify where it generated UI problems	Value correct	Answers displayed correctly in list, but correct answers not indicated; UI problems in Modify	general feedback --> copy to both correct and incorrect question feedback fields?	Feedback was not saved
Multiple Response (Grading Method: All Points or None)	Incorrectly typed Defaults to Essay type	Text imported correctly	Value correct	Defaulted to individual Model Short Answer records - only the last answer displayed	general feedback --> copy to both correct and incorrect question feedback fields?	Feedback was not saved
Algorithmic*	Instructions and link to Flash movie displayed; link fails due to failure to import SWF	Value correct	Defaulted to 5 individual Model Short Answer records	n/a	n/a	n/a

Of note here is that the feedback models for Samigo are more numerous and flexible than for Respondus. Samigo offers both question-level and answer-level incorrect and correct feedbacks, depending on the question type. Respondus offers only general feedback except for true-false question type.

The question arises as to which field is most appropriate to migrate a general feedback response into - the correct answer feedback, incorrect answer feedback, both or the answer-level feedback fields. The answer may vary with the instructor or school - and may additionally vary based on the desired assessment feedback setting.

*Also of note here is that Samigo does not curently support algorithmic question types. Respondus exports this question type as a .swf or Flash movie and the question text is actually the JavaScript launch code and the xml answer output is similar to a multiple-choice single-correct answer. It would be interesting to determine if the SWF could be imported as an attachment and whether the user's response could be sent back for scoring.
Finally, there are differences in grading method that should be noted in the gap analysis so that instructors understand the potential grading changes in their translations.