Can I modify a published test?

You can't modify a test after it's published. Period.

You can retract it (equivalent of taking it away from students!) and republishing a new version with the fixes, under a new name.

Do not publish a test until you have reviewed it thoroughly and it is truly "publication" material.

How can I preview an assessment as it will be seen by others?

  1. In the menubar of the relevant course or project site, click Tests & Quizzes. From the Assessments page, under "Core Assessments", locate the desired test and click its title.
  2. Click Preview Assessment. (Look for this in a sub-menu in a gray bar just above the "Add Question Drop-down menu there are three choices: Add Part | Settings | *Preview Assessment*
  3. Then click Begin Assessment. Click Done to return to the Questions page.

Can you make the fill in answer take multiple versions of a correct answer?

The system will just match what you put in the { }. For synonyms or multiple answers, you can insert a "|" (pipe) between the answers. For example, put {yellow|red|purple} to accept any of those for the answer. It's a good idea to check the fill in the blank answers the first few times to make sure that students get credit if they use a variation on the answer I put in the { }. It is easy enough to do and over-ride the answers marked wrong if you choose to.

The pipe key ( | ) is found on the keyboard above the Return/Enter key - use the shift key, too.

I'm unsure what the "mark for review" tag is for. Is that for students to mark questions they want to go back to or for students to let the instructor know when a rationale is there for them to review?

That's it. You got it. Imagine if you have a long or difficult test and students are stumped on some questions. Instead of straining and losing time, they can check "mark for review." When they are done, the questions they marked with be tagged in the table of contents and they can quickly return and finish them.

Note: this feature only works if settings allows for random access to questions (under the Assessment Organization category). If the settings are set to linear access, then students can't mark for review and return to skipped questions. Before you create your test you should check the template settings and make Assessment Organization editable if you might want to change this depending on the test.

What is Metadata for?

Taking the time to complete the metadata field will pay off in the long run for searches. If you have a large question pool, having good metadata in your tests will allow you to zero in to the right question that covers a specific topic/term. The goal is to have search functionality in every tool.

What do you do if a student is taking a test and they are having problems with their system or computer? How do you allow them to retake?

If you have set the test up for a single entry, the deadline for submitting has passed, or if they submit accidentally you can't allow them to retake the same test. The only option that I know of is to set up a new test for the student.

Can the final page URL be edited? What URL page? We have not talked about a URL page here. Please clarify.

Yes, it can be edited. Go to template settings to the Submission Message category and check the Can be Edited for Submission message or URL (or both).

The URL page is one you want to take your students to once they submit the test. If you have a final page that you have designed and published somewhere on the web, you can add that URL there. It may be a place to see solutions to the exam or model answers or a CONGRATULATIONS web page/image.

If you don't have a URL (web page), then simply adding a message for the final page, is sufficient. Something to greet students when they are done, like: Thank you for your effort. The grades will be available in a week.

It helps to alleviate worries that student have about whether or not their instructor received the test -- also then they don't send me emails asking if the test was received.

I can't figure out how to view my quiz as a student.

To preview a test, you must do it prior to publishing it, under Core Assessments. Here is what to do:

  1. Click on the name/title of one of your tests under Core Assessments.
  2. The test will open up. Close to the top, there are three links; they are centered above the first question.
    | add part | settings | preview assessment |
  3. Click on preview assessment.

When I previewed, I could see all of the questions, and practiced wrong answers, correct answers, save and review response, and unanswered questions. However, once done, the links for unanswered and review did not work. So I did not get to review questions marked save and review, or go back to unanswered questions.

Nothing is saved when you preview a quiz. It's just for preview purposes to go through the questions and see what students see. You are not really taking a test as a student - just previewing it.

When creating a quiz, I used a fill-in-the-blank type question, but when I previewed the test, I was unable to type anything in the block. I simply could not type in the space during the test.

This is a function of your viewing the text in "preview" mode. The test isn't meant to be taken, just previewed, so the field is grayed out.

When creating test questions, it wouldn't allow me to assign a pool name. Why not?

You can't assign a question to a pool UNLESS you first create pool to send questions to. Go to the question pools area first, create some pools and sub-pools, and then when you create tests, you can send questions to appropriate pools.

I have a pool and a subpool. I would like to copy questions from the pool to the subpool, but when I click on copy under the question, the only option that shows up is the other pool. I am able to copy the other direction - from subpool to pool. Am I doing something wrong or is copying only one direction?

You are correct. It only goes one way, from subpool up to pool. So if you put a question into the wrong subpool, you can't move it into the correct subpool.

Where can you set the time for an exam? Like say you want to have a half hour or a one hour test and you want to limit the time for the students.

It's located under test settings => timed assessment. This is another area where you will want to check your template to see if these settings can be edited. Do this BEFORE you create your test.

The timed feature works really well. If you select "auto-submit" when time expires, the software says, "time is up," submits the answers to the database, and returns students to the T & Quizzes home page. Also, students have a green bar showing them how much time they have left and with 5 minutes to go, it turns red.

I keep getting a JavaScript error in the test editor (see below). I only mentioned it because the developers might want to know that the errors were occurring.

Line 1116
Char 4
Error: 'HTMLArea.l18N.msg' is null or not an object

Yes, it is related to browser settings. I have JavaScript enabled and because I write JavaScript, in IE I have enabled the setting that shows the JavaScript error. FireFox puts JavaScript errors in the JavaScript console which can be shown from the tools menu.

If I allow late submissions, can a student who has submitted a test see a test a second time even though further submissions are disallowed, or is he prevented from even looking at the test?

If students have taken the test, even if it is past due, they CAN see it again and (possibly) their scores. They may not be able to see the correct/incorrect answers and their score based on your feedback settings, but they can access the test. There is no difference in this behavior for tests that allow late submissions or not. The only thing you control is access to scores/feedback. Students can always access active exams and "submitted" exams (even if they are inactive/past due).

Students who have not taken a test cannot see it at all after it's inactive (past due). Retracted exams (you remove them from the active list manually, ideally after the due date) are not visible to students who have not taken them, but they are visible to those who have taken them. The latter is a bug, I believe. Need to check with Stanford U on the behavior prior to posting an issue on it. I thought that retracted exams should never be visible to any students.

I noticed the editor for the quiz creation is different than the editor for the module creation, which also appears perhaps different from the editor in other areas. For math class, not having the equation editor in there makes it rough to create a meaningful math quiz/exam. In addition, it really prevents a student from responding using proper mathematical notation.

Yes, right now, only Melete has the commerical Sferyx editor with the equation editor plug-in that we purchased. The other tools use an open source javascript light editor.

Our goal is to allow users (students and faculty) to choose one of the two editors from MyWorkspace >> Preferences. For some disciplines, the Melete editor with MathML and foreign language support is an overkill. For others, like yourself, they can't teach in the system and author assignments, tests, or content without it.

I don't know the timeline for this functionality. It is very likely that it would be available in the fall of 2006.

The equation editor is missing some features that are essential for my math class. Assuming the equation editor is improved upon, are there plans to standardize the editor such that we will always see the same editor, regardless of which tool we are in (hopefully with the equation editor included)? This would allow for proper mathematical notation anywhere where an editor pops up.

As for the improvement of the equation editor, I would have to check with the company where we purchased it from to find out how often they expect to put out new releases. As I said in the other post, it's a new product (released in Nov. of '05) that will only get better.

The equation editor is an add-on plug-in that we purchased for the Sferyx Editor in Melete. We are not in charge of its future enhancements nor in a position to extend it (not our code base, and outside of our area of expertise), but we will be getting the upgrades from the vendor, as they come out. Keep in mind that the equation editor for Melete was just released in November by the company (beta version), so it is a young product that will only get better.

A published quiz has vanished and is no longer visible to me under inactive quizzes.

It could be that you set the test to be visible to "anonymous" users, which is nobody in your class.

What process can I use to upload questions from a computerized test bank into the system's question bank (pool)?

This would be done using the Import function, as long as the test was in the proper, compatible format. (You see Import as soon as you click on the test tool, just under "Create a new assessment" and "Title.") The publisher would need to support (provide test banks in) the international standard QTI IMS specification that the Tests & Quizzes import function uses.

If I check the "Late Submissions NOT Accepted" button, and the due date passes, can I later uncheck it to allow a late student about whom I forgot to take the test?

No, you cannot change a test from "Late Submissions NOT Accepted" to "Late Submissions will be accepted but will be tagged during grading" after it's published (active or inactive).

But you CAN activate an inactivated exam:

  1. Click on settings of inactive exam.
  2. Change the delivery date (past due) to future.
  3. REMOVE the retract date from field, if filled.
  4. Save settings

It will now be moved up to the active list for students to see and take.

If I allow more than one correct choice for a MC problem, how does the system assess point? All or nothing?

The newest release DOES take off points for incorrect selections in multiple-answer m/c questions. The software determines the possible max value per question and takes off points for wrong answers down to zero. It's nice to students and won't go negative.

For example: You have a m/c question that is worth 5 points and it has give choices (A, B, C, D, and E), with A and B being correct. If a student chooses all five, he will get zero points (-3 + 2).

Will there be a way to copy, import or whatever the quizzes from an existing CMS site to the system's site?

You can now request to have your classic ETUDES tests from x course site/term converted to an XML file that you can import into your new system site. Because there are hundreds of you that need this done, we'll have to process requests for those who go live in the spring quarter and summer terms first. There is a link on the faculty support page that you need to fill out and submit to start the process.

Is there a good way to include several essays on a test for students to choose from AND still have an accurate point total for the test? I have found that if I post three 15 point essay questions and ask students to pick two to write on, the Test tool still assumes that there are 45 points possible for the essay section.

Hmm, here is one thought. Offer one question worth 30 points and then outline 3 different essay questions within that one question. Instruct students to pick two of the three questions to respond to. It will show up as a test worth 30 points and they answer two of the questions.

I have a question pool, but I can't see what's there. I can see that there are 4 questions, but I can't access the questions themselves.

The interface needs a bit work in the question pool area. You might have to open your browser window completely, and scroll down and horizontally, to access the links. You want to click on the pool/subpool name to access that pool. Then scroll down to see the questions. Click on a question to open it up for editing.

Am I able to see how long the students took on a test? Also, I didn't want my students to get booted out for exceeding the time limit - my intention was to penalize them for doing so.

No, the system does not have student tracking at this time. It is a requirement that teachers want, so I trust it will be available some time in the future. Students get booted out if they exceed the time limit. The software respects the time limit you set. This is a forced requirement with timed exams - tests can be set to "auto-submit" when the specified time expires.

OK, this is just an odd one - that I am sharing for information purposes. A student, John Doe, contacted me as he thought his quiz score was oddly low. I went in and while his score was recorded as a 4, he had 8 correct - and this was indicated by checks and so on. When I updated his grade, he had the proper score of 8. Perhaps an error that came about as a consequence of the using multiple parts?

That is odd. Be sure that if you are using pools, the point value of the questions within a single pool are the SAME. Otherwise, if you use 'randomization' from pools that have questions with different point values, the random feature might give students questions that are worth varied values, getting a totally different score in the exam than other classmates!

It could it be that you have multi-part multiple choice questions in your pools, and that in addition to checking the right answers, your student checked incorrect answers and got points off? There is a penalty for checking incorrect selections in m/c multi-answer questions.

I created a survey in one course's tests & quizzes. I need to export it into my other courses. But I got a long, hard to read error message.

When you create test questions, please do not paste Word code into your exams! Your survey/exam was full of non-standard code that Microsoft Word is equipped to understand, but hardly any other editor.

The editor is a standard simple javascript editor. It is not smart. It accepts what you paste. The export function needs clean data to convert it to xml. There was no other way to clean this up but by going to each of your questions one-by-one! You should be able to export your assessment now.

If your questions are in Word, copy/paste the questions into Notepad or Simple Text first, then copy and paste from there into your exams. Or Save As the Word document into text format (.txt, NOT .doc) before using that text in Tests & Quizzes. That will strip out the bad Word code.

A student wrote: "I was taking the test and put to review and thought it would let me go answer the questions that I knew and go back to the ones I had to search for and now I have an F." There is no review button, is there?

There is a "mark for review" feature. If students mark a question for review, they can then click on the 'table of contents' link located at the top of the window at any time and go through each question to review them and change answers. This will be available to students IF you have made the choice to allow for a table of contents to show. Additionally, there is "save and exit" and return to the test and go through it one more time, and THEN, click on "submit for grading."

Now, if your student clicked on 'submit for grading,' his/her answers were sent to you and it's too late. 'Submit for grading' means submit for grading. This mistake is very easy to do without paying attention, because the Submit for Grading button is on the last question in the same location that Save and Continue is located in previous questions. We had posted an announcement to warn students to not submit for grading until they are done. You might want to send that reminder (again) to them.

I'm writing a quiz question and click on the insert image button. I get a place to put a URL into, but no option to browse my computer (which is where the image I want to use is). Surely there is a way to insert a jpg into a quiz?

Unfortunately, the editor currently used in the Tests & Quizzes tool does not support image upload. It only supports inserting an image URL. This means that you need to have your images somewhere on the web and point to them. In other words, you will need to add the full image URL ending with the image file name. http://......gif or http://....jpg

The rest of the tools will be upgraded to a new editor that supports image uploads soon. I'm not sure when the test tool will implement the new editor. They are discussing it.

While results are not available until the exam is past due, is there nothing to prevent the printing out of exams? Students have FULL ACCESS to their ungraded exams after they have submitted it for grading?

No, there is no way to stop students from printing exams. Yes, students have full access after submitting.

I just realized that after students take a quiz, they can click on it and see the whole test at one time. I chose to have each question on a separate web page so that they could not print out the whole test at one time and share the information with other students. I went in as a student into my class and took a quiz. Then I went back before feedback was released, and I could click on the quiz and see my answers. After feedback with the correct answers was released, I could go back and see the whole quiz on one page with all the answers. At the moment, I am in the process of adding questions to pools for each chapter, but right now I only have 10 questions for each module and so randomizing the quizzes will not work yet. If I was using this for a mid-term or final exam, I would be extremely unhappy with this feature.

For high-stakes exam, you should set your settings to "no feedback." The problem with this is that it is not the best for learning, as it doesn't allow students to learn from their mistakes and their successes. But, no matter what, if you allow feedback to show after a certain date, even screen-by-screen, there is no stopping students from printing it in their home computers and sharing it with others. Yes, a combo of drawing from a large bank and randomization is the best alternative to proctored exams.

Until you have a larger test bank, set high-stakes exams to 'no feedback.'

Maybe overall feedback is a place to provide feedback for student performance on the exam. Is this the case? If so, I cannot locate where one would insert overall test performance feedback.

First, let me clarify/confirm the following:

There is a big problem with the last item above.... (a bug). The grader's comments are not visible to students, even if you check the box under "feedback." This should be fixed in the end of March release.

I seem to have published my first quiz without entering the date for feedback to be released. Is there anything I can do about that, now that the students have taken it and are clamouring to know what the right answers were?

Once a test has been published, it can't be modified.

Wouldn't it make sense for the program to reject the option of release on X date unless there is a date filled in?

Yes - I put the request to Stanford about a month ago and they have incorporated a fix for it (a warning to faculty). It will be in our production code in the next major upgrade (early April).

I have also asked that they consider giving faculty flexibility in changing their mind about the feedback settings after publication and deactivation of an assessment. I just asked for this recently, so I am sure it won't be in the next release.

I'm setting up a template, mostly because I want to have the same message about "save and exit" vs. "submit" appear at the beginning of each quiz. I can't figure out how to put that in my template!

No, you are not blind. The "Assessment Information" textbox is not available in the template. However, you can put the instructions for students in the "Template Information" >> "Description/Intro" box of your template, and before you publish a test, under settings, you can copy/paste your default instructions from the "Template Information" box to the "Assessment Information" box just below it. It's a quick workaround. At least you don't have to search for the information.

P.S. I've passed this on as a feature request to Stanford U.

3/16/06