Early Experiences with WebCT-to-Sakai Exam Question Migration (CalcQuest)

Owned by Former user (Deleted)
Last updated: Mar 23, 2009Version comment
4 min read
Update: the code for conversion has now been included in a separate page, which can be found here.

Presentation of 12 November 2008 (Virginia Tech)

Institutional Context:

After productive use for several years at the University of Wyoming, WebCT CE 4.1 is to be phased
out by May 2009. We hope to adopt Sakai as one of our main options for online teaching and other
collaborative activities. Our instance is currently hosted by rSmart, to be run in-house eventually.

Our Physics faculty are heavy users of WebCT Calculated questions. Investigation shows that the
"Calculated" type is evidently a question set generated under a script given some parameters. Our
course website support policy has consistently emphasized maintaining materials locally (independent
of the course platform). However, we would like to provide help to our science instructors who have
grown to rely on WebCT quizzes. We require no batch facility-- faculty will convert their own materials,
with our assistance.

The "Calculated" question type in WebCT is evidently a question set generated from these instructor inputs:

  • a formula involving demarcated variables,
  • ranges of values for variables,
  • the number of decimal places to be used in the student answer,
  • the number of variations desired.
    The question variants are generated at the time of question creation and one is selected from that
    question set when a student takes the exam, in the same way that a WebCT question can be determined
    by a pre-determined question set and random selection; the variants are not generated dynamically when
    the student takes the exam.
Two main tasks emerge:
  1. Conversion of current questions, already in WebCT quizzes, to Sakai format.
  2. Creation of new calculated questions through an interface that simulates the WebCT "Add New Question"
    procedure.

I. Conversion of Current Questions

Many scenarios present themselves, distinguished primarily by the type of output from the WebCT course
and the type of input to the Sakai worksite.

Output options (from WebCT CE 4.1.5.8):
  1. Copying individual question parts from the WebCT screens (the completely manual option, appropriate
    for limited circumstances).
  2. Export of the entire WebCT course.
    1. Backup (in course Control Panel) and download of proprietary zip file.
    2. Execution of the Content Migration Utility from the vendor, Blackboard, (formerly WebCT). Accomplished
      by execution of cp_api.pl on the server, with course IDs listed in the file courses.txt in the DATA folder.
      Produces the same question, quiz, and quiz-properties files as does the "Export Content" command in the
      course Control Panel, plus more files for the other course components. (Although the Blackboard website
      claims that this separate feature in WebCT 4.0 was incorporated into the course shell in WebCT 4.1, presumably
      as the Control Panel option "Export Content" mentioned below, we investigated each. The "Export Content"
      option applies only to Content Modules and the Quizzes tool elements, not the whole course.) Output files
      remain on the WebCT server in an "ims_export" area separate from the course materials.
      1. Output to folder (default), with two subfolders containing user files and course data.
      2. Output to Zip file, containing both user files and course data, with "compress" option.
        (This form serves as input to the UNE Migration Utility, as described below.)
    3. UNE Migration Utility. We will use this for the larger task, as it provides browsing and other facilities,
      but our current concern is only quizzes, and within that, only calculated questions. The UNE utility uses
      the results of the Content Migration process with the compress option, same as Export Content, to populate
      various Sakai tools. Calculated questions are skipped, however; they do not appear in the Quizzes.zip output.
  3. Export of WebCT Quizzes and Surveys tool components
    1. Tagged text. This text format is unique to WebCT (as far as we know), capturing each question parameter
      in a text field identified with a tag, such as ":TYPE"", ":TITLE:", ":QUESTION:", ":FEEDBACK:", ":ANSWER1:",
      and so forth. Available only in the Question Database, not for Quiz and Survey objects.
    2. "Export Content," in course Control Panel, of option "Quizzes, Surveys, and Question Database" (where the
      other option is export of Content Modules). This operation produces, in a Zip file in the course file space
      ("Manage Files"), the same result as the Content Migration Utility run with the "compress" option.
      Calculated questions are so designated with an HTML comment and tagged from the namespace www.webct.com/IMS.
      They include tags <qmd_itemtype>Numerical</qmd_itemtype> and <mat_extension> nodes with tags
      webct:x_webct_v01_dynamicmattext and webct:x_webct_v01_dynamicdata.
    3. QTI through Content Migration Utility. This is part of the result of the operation of option 2.b above.
      Exports calculated questions in the same way.
    4. Third Party Tool: QTI through Respondus, with calculated questions as Flash ".swf" format dynamically
      formulating multiple choice answers. Produces a logically equivalent but procedurally odd interface.
Migration Input Options (to Sakai 2.5.2):

Sakai has no "calculated" type as such,, but import as a question pool
will work, with short-answer or numeric answers. (Then use "Edit" on
the Part to specify random draw of i questions from a pool of n
variants.)

  1. Typing, or pasting, individual question parts into the Sakai edit windows (the completely manual
    option, appropriate for limited circumstances)
  2. For entire assessment-- Import option:
    1. To short-answer type (IMS presentation (FIB)
    2. To numeric type (IMS notation FIN)
  3. Quick-create with plain text file of proper format, optionally created from WebCT tagged text
    download through script cqfacility.py. To short-answer questions (IMS presentation FIB);
    no numeric type available here.

II. Creation of New "Calculated" Questions

WyoSakai import options for future simulation of calculated questions:

  • Script cqfacility.py for command line interface with two output options--
    1. Plain text output for Quick Create.
    2. XML QTI-compliant text output for Import into Question Pool.

Both of these options use the Sakai "Fill-in-the-blank" question type, which requires string
matching, just as the WebCT calculated question type did. WebCT elicits the number of decimal
places in the answer to ensure accuracy of assessing the student answer against the intended
answer. Wouldn't it be better to allow a tolerance in the answer with real numeric (floating-point)
representation? The answer is clearly "yes," but here are the implications:

  • Questions must be imported into Sakai as "Numeric" from an IMS QTI-compliant file.
  • No Quick-Create is possible in Sakai because there is no syntax provided for the Numeric question type.

Conclusions

Aaron Zeckoski, in a DG: User message of 2 October 2008, volunteers to turn the code into a web service.

Still to be determined:

  • What is the maximum size of the Questions Database the UNE conversion will handle?
  • What does Blackboard intend WebCT calculated question users to do?
  • Can Sakai just incorporate the facilities of LON-CAPA (http://www.lon-capa.org/whatis.html), as
    suggested by Chuck Severance in a DG: User message of 2 October 2008, since the computed question
    type and other advanced assessment facilities for the sciences are already there?