U-TIC Project (Updating the TurnItIn Integration Code)

All features of current integration are present: UI options and configuration file options

The new integration code works with both Sakai 10 and Sakai 11

Full technical documentation

TurnItIn-enabled assignments created with previous integrations must be easily accessible, ie, the reports must be able to be retrieved

Errors are handled correctly

All tests pass  (Unit and mocked integration)

Vericite plagiarism service can still be used with Sakai

There shall be no known security issues

The code is accepted into the Sakai 10 and 11 codebase with the appropriate licence

Informative error reporting is available through the web UI in new Admin tool

New TurnItIn Administration tool which works correctly with Sakai 10 and 11 and can be added to Administration Workspace

n/a

Informative error reporting is available through the web UI in Assignments tool

User can track where their submission has got to

Modified Assignments tool works with Sakai 10 and 11

Ability to verify each call to TII API via a mocked-up endpoint

Endpoint should not depend upon an SQL database

Every test runs as part of Maven build process

Automatic set up of a selection pre-migration assignments for the purposes of testing all the acceptance criteria of the migration code

Automatic deployment of database updates / migration scripts

Automatic verification that new integration code can retrieve old originality reports

Can handle 1 submission per second

Can dispatch 20 submissions per minute (assuming TurnItIn can accept this many)

Hibernate is NOT mandated

MySQL version 5.5 and above

Oracle version 12.1 and above (I think!)

Successful build with Maven

Build includes successful running of mocked integration test

Code is appropriately document

No incompatible libraries are used

Any changes to the implementation of Content Review must be agreed with Vericite developers / Longsight

No new external service dependencies without unanimous agreement from Sakai developer community

Use existing Sakai frameworks for logging, configuration etc

Queuing should be driven through the database and not be held purely in-memory

Pull request is successfully merged

User Logging infrastructure

Use Sakai events service

Etc!

Assignment is created "as normal" with a checkbox option to "Enable TurnItIn"

Success message displayed if an assignment is correctly be created in TII via LTI

Human understandable on-screen message explaining the problem if an assignment cannot be created in TII via LTI.

If TII is down then present message telling the user to try enabling TII at a later date. Do not display this message if TII is up and running and it is some other aspect that is causing the problem.

Just as currently happens, a TII enabled assignment should have a special icon with

alt="Originality of attachments will be verified by Turnitin." title="Originality of attachments will be verified by Turnitin."

If Vericite is used then (obviously) "TurnItIn" should not be mentioned / "VeriCite" should be used instead.

If not already done, replace the silk icon with a RED 'fa-certificate' character

Assignments written before the deployment of the new code must still function correctly

Originality Reports generated before the deployment of the new code must still function correctly up to the point where the old API is removed

Demonstrate that assignment titles can use non-ASCII characters

Demonstrate that assignment titles can contain ampersands

Demonstrate that assignment titles can use non-ASCII characters

Demonstrate that assignment titles can contain ampersands

Done

Two sites with the same name must both be able to contain assignments (which also have exactly the same name as each other) that provide access to TII reports

NB - if this is a problem then it is acceptable to modify site titles prior to sending them to TII (eg, append a parenthesised site ID to site titles?) Special attention must be paid to what happens to legacy Originality Reports and Assignments - users must still be able to retrieve report that were created using the old integration code.

Allow two assignments in the same site to have exactly the same name.

Allow two assignments in different sites to have exactly the same name.

Ensure that using the 'Duplicate' option to copy an Assignment does not lead to problems of being unable to see / generate originality reports, ie, both copies of the Assignment should function correctly.

NB: it may be necessary to append the Assignment ID to assignment titles to force uniqueness.

If somebody gives a site a single character name then TII-enabled Assignments on that site must function correctly.

NB site title must be at least 2 characters and assignment must have 3 characters

Option presented in UI - default is that they can re-submit

Student is able to submit one or more revised copies of documents and a TII report should always be returned if option is enabled.

Student cannot resubmit if option isnt enabled

Check it can be changed and the changed is reflected in TII.

Problems with multiple files

Student is able to attach multiple documents to a TII enabled assignment

Check it can be changed and the changed is reflected in TII.

A separate TII originality report is generated for each submission

Display choice of Submitting to

1. Standard Paper Repository or 
2. No Repository or 
3. Institutional Repository

Allow the displaying of "Institutional Repository" to be suppressed. (Oxford doesn't have one.)

Ensure that both options work correctly

Check it can be changed and the changed is reflected in TII.

If an establishment doesnt have an Institutional repository then allow the display of the option to be masked from the UI. I guess the default is to display the option but we could canvas opinion on the lists and having no institutal repo may be the norm?

1. If an institutional repository exists and the property is set accordingly, then display the option in the UI and check that it is passed through correctly by using it and verifying via the corresponding Assignment in TII
2. If the institution doesnt have one and the option is NOT displayed in the Assignments tool UI then check that the assignment is created correctly

Display: Immediately or On Due Date

Ensure that both options work correctly

Check it can be changed and the changed is reflected in TII.

Submit a response in the in-page Textbox ('in-line submission') and show it gets an originality report

Check it can be changed and the changed is reflected in TII.

Ensure that both options function correctly, ie, if students aren't allowed to view the reports then demonstrate that they are unable to do so and vice versa.

Check it can be changed and the changed is reflected in TII.

1. Exclude bibliographic materials from Similarity Index for all papers in this assignment
2. Exclude quoted materials from Similarity Index for all papers in this assignment
3. Exclude small matches

Ensure that zero or more of the following options can be selected and function correctly.

Check originality against:

1. Turnitin paper repository
2. Institutional Repository
   
3. Current and archived internet
4. Periodicals, journals, and publications

Allow the displaying of "Institutional Repository" to be suppressed. (Oxford doesn't have one.)

Check it can be changed and the changed is reflected in TII.

(This option is not currently seen in Assignments.)

Allow option to allow submissions of any type or reject non-supported options (checkbox).

Default setting should be reject non-supported file types.

Check it can be changed and the changed is reflected in TII.

(This option is not currently seen in Assignments.)

Present option that allows date to be set for when TurnItIn feedback is available to students.

Check that chosen date is obeyed.

Check it can be changed and the changed is reflected in TII.

(This option is not currently seen in Assignments.)

Question: how would this interact with Sakai's peer mark option?

(This option is not currently seen in Assignments.)

Check rubric is attached and is visible in both assignments and in TurnItIn

(This option is not currently seen in Assignments.)

Check grammar checking options are passed through to TurnItIn

Check grammar report is generated

Check setting can be changed

Use the same colours for Originality Report icons as current integration

Use new font Awesome Icon (for the morpheus look). Use "fa-flag" as icon.

Use the same logic to decide which colours to show

Mouse over text should show report 'score' (just as currently happens)

Click on icon to visit full Originality Report (just as currently happens)

Should use the same text as current "View originality report: score = xx%"

If resubmissions are allowed then show the most recent score

If one or more members of a site does not have an email address then this should NOT cause the Roster sync to fail. TurnItIn requires all site participants to have an email.

Highlight such users in the Assignment so that staff can see what has happened. (Highlighting method to be agreed).

Site participants with missing names should not cause the whole assignment to fail.

The assignment should be set up and submissions processed even if people dont have the correct names (TII requires first and second name).

Highlight such users in the Assignments tool so that staff can see what has happened. (Highlighting method to be agreed).

TurnItIn-enabled Assignments can be assigned to "Internal Subgroups" (Sections)

Only the members of the subgroup can see and take the assignment

Add an option entitled "Allow TurnItIn GradeMark to return marks to Gradebook" (US locale) under the TurnItIn Options section of an assignment set-up. (This is the equivalent setting to "Allow external Tool to return outcomes" that one sees in the standard LTI Tool interface.)

If assignment is not set up to grade using numeric 'points' then user must be notified. They must use points-based scoring for GradeMark to be available.

For the UK (EN/GB locale) the option should be "Allow TurnItIn GradeMark to return marks to Markbook".

If TurnItIn's Grademark facility is used to mark student work, then the marks should be pulled back to the Sakai Gradebook tool (Sakai 10) and Gradebook NG (Sakai 11). (NB there is an option to get TII to push grades to Sakai but we will not use this method - we will always 'pull' grades back.)

The process must be able to cope with TII being unavailable when an attempt is made to get grades, ie, we must record the status and try again if necessary.

TII GradeMark returns marks to Sakai Assignments tool.

NB This needs some discussion

Optionally allow marks to be sent to Assignments tool.

Add an option entitled "Allow TurnItIn GradeMark to return marks to Assignment" under the TurnItIn Options section of an assignment set-up. (Question: is this allows as well as returning marks to Gradebook or instead of?)

If this is enabled then the marks must appear in Assignment tool.

If Gradebook is linked to the assignment from the Assignment Tool UI, then the marks should appear there as well.

If marks are also set up to be passed back to Gradebook then "something sensible" should happen.

NB: this is a bit of an unknown issue. Do we have 2 sets of marks in assignments? One set entered in sakai and one set posted by Grademark. How does all this work if marks are also to be sent back to Grade book? We need to clarify as to how this would work as one can also (presumably) manually add marks from within Assignments tool.

The mark will be added next to the report. Also, for single file assignments or only inline type, it will:

-Override assignments grade if it's empty.

-Add warning if there grades in both systems.

Classes expire after 7 years so warn if an assignment is created with a due date after class will expire.

If age of class cannot be determined, then use the site creation date as basis of calculations

(This apparently should not be an issue in the new integration as a unique and non-changing User ID is used to identify an individual)

Find an assignment with submissions, for a given staff member, change their email address then try to access originality reports (after a short while)

Apparently this will not be an issue as a user ID is used to identify the student.

It works but the email is not updated on TII

If an assignment has been routed via TurnItIn then it should not be able to be unattached.

The "enable TurnItIn" button should be greyed out once set and the assignment created.

If TurnItIn is not present when the assignment is set up then it should be possible to retrospectively attach it.

The user should be warned if TurnItIn cannot be enabled. They should be told to try again later and should also be told why there was a problem, ie, highlight that it's the fault of TurnItIn and not Sakai.

If there are already submissions when TII routing is enabled then these essays should be dispatched for review straight away.

Essays routed via TurnItIn must be available in the Sakai site.

Ideally upload into Sakai and then dispatch to TII.

Place a link to "TurnItIn Assignment Inbox" on 'View submissions' page underneath the submissions table but above the "> Assignments details" "DIV"

Clicking on the link should open a new browser tab positioned at the inbox for the current assignment

Unsupported type is accepted by a red "Standard Sakai warning" is displayed on screen

Read acceptable file types from a configuration property OR (if possible) dynamically retrieve from TII and cache for 24 hours

?Q: I think there's a TII config setting that says what file types are accepted - how does this work with that setting?

We have a patch at Oxford: https://jira.oucs.ox.ac.uk/jira/browse/WL-3155 - may need modifying

Mixed with any type property code

If configuration says so, the student should get a receipt acknowledging submission.

If configuration says not, then the student should not receive notification.

See: https://jira.oucs.ox.ac.uk/jira/browse/WL-2488

They can access it on the TII LTI site

There should be a (new?) TII service configuration parameter giving maximum file size (unless this can be retrieved dynamically). I believe the current limit set by TII is 20MB.

If a student attempts to submit a file larger than is permitted, the file should be accepted by Assignments but the student should be informed that it will not be checked

Related: https://jira.oucs.ox.ac.uk/jira/browse/WL-3787 - please add this to Assignments tool

Added

If a student doesn't have a name (or only has 1 name) then they should see in the Assignments UI that their TII report will not be generated.

NB If we are able to accept submissions from people with one name then only warn if no names are set

Students with non ASCII characters, eg, unicode characters in their name should be able to submit essays and receive originality reports

The full unicode / UTF-8 character set must be accepted

This facility should be configurable

Default is that they should not be allowed to opt out

If the tool is configured to allow opt out, the student should be able to select an option and not have their document checked.

Tutor should see they opted out rather than just not seeing a link to the report

(This apparently should not be an issue in the new integration as a unique and non-changing User ID is used to identify an individual)

Set up assignment for a site, for a given student, change their email address then try to submit (after a short while)

Essays routed via TurnItIn must be available in the Sakai site.

Ideally upload into Sakai and then dispatch to TII.

Config option is proven to work - we have this at Oxford

Current property is used

Config option is proven to work.

Current property is used

Configuration file option to prevent email messages being sent from specific TII endpoint (eg, a sandbox server), this 'overrides' all other email settings, ie, one only has to flip one setting when in "sandbox" mode

Verify that emails arent sent.

TurnItIn integration works and uses the TII IMS LTI tool on each site

The TII LTI tool does not appear in External tools for any user (except Sakai admins in Administration Workspace)

The TII LTI tool does not appear in Plugins part of Edt Tools page

A site maintainer cannot accidentally disable or remove the LTI tool from the site.

There should be a property to disable the “allow resubmissions” option in the UI. The default should be to show the option but if institutions are worried about resubmissions then they could suppress the display of the option.

Automatically build a “simulated live environment” in the TurnItIn Sandbox that contains assignments created using old integration (over several sites)

Auto-submit a variety of documents by various students to these assignments

Then automatically create assignments using new integration and check that they are created.

Automatically submit essays to these assignments and check they are accepted

Then check that reports can be obtained from both types of assignment

Originality Reports for Assignments submitted using the old integration ('Legacy API)' must be accessible. (If people are using the older content-review code ('Open API') must upgrade via the newer integration.)

Option by option analysis of existing assignments in a system using the old integration compared to these assignments in a system set up to use new integration.

Basically, make sure that when a staff member or student accesses an old assignment, the options that were used when it was set up are still obeyed, eg, if an old assignment was set up and prevented students from being able to access their originality report then this setting must still be in place. (I seem to be having trouble explaining myself here!)

A property should dictate that the LTI-based "new integration" is used for assignment creation, submission and viewing reports.

This property should also be able to be set on a site-by-site basis (via the Sites tool) so specific sites can be set up to use the new integration. This will allow us to pilot new integration with a handful of sites before the "big switch-over"

Another property should control whether links to reports generated / accessible via the old API are shown. When the old API is switched off, the property will be set so that the old links are suppressed.This property should affect all sites, there is no need for it to be set on a site by site basis. (Alternatively, this could be done automatically - Sakai checks and if the old API isnt there, the links are suppressed.)

Tool is available to be added via Sites tool

An administrator can see all events in Sakai

(We may need to talk about this.)

If the new TII content-review code is being used, create a site with ID '!tii-admin' and add the TII dashboard to it

Provide migration script to create this site and add the dashboard for existing installations that wish to use the new TII code

Tool is available in Edit Tools menu

User with correct permissions can see events from the current site

Interface is not tied to a particular back-end solution

Interface is acceptable to Longsight

Default pagination is 250 items - it is a form of "raw database dump"

All events are shown in reverse chronological order. Columns should be sortable and contain

• Select item (checkbox)
• Site Title
• Site ID
• Assignment name - see note
• Assignment ID
• Submitter 'Display ID'
• Submitter name
• Created
• Current Status - clickable
• Link to Originality Report - see note

Clicking on Current Status gives a modal pop-up showing transaction log of timestamped status transitions, ie,a history of a particular submission.

NB If it is expensive to retrieve the Assignment name or to link to Originality Report then do not display this / these column(s), we think that retrieving assignment names and reports may not be scalable.

Filter by 'Display ID', or

Filter by Site ID

NB: No date search, no faceted search

Each entry in table has checkbox (including filtered results)

'Select All' in table header

'Actions' for selected items

• Cancel (ie, just give up, no retries)
• Cancel and Resubmit (or possibly Resubmit if it isn't necessary to cancel first)

NB some experimentation or advice from iParadigms may be needed to make sure that matters wont be made worse if a stuck submission is resubmitted

Pause all TurnItIn activity for "N hours" starting at DD/MM/YYYY HH:MM - in case we know of TII outages

Accept submission but just add to the queue

Additional "tab" to allow tracking roster syncs

Columns:

• Select item (checkbox)
• Site Title
• Site ID
• Created
• Updated
• Status including error mesaages

'Select All' in table header

'Actions' for selected items

• Resync

NB some experimentation or advice from iParadigms may be needed to make sure that matters wont be made worse if faulty roster sync is retried

Sakai Administrators should be able to see, edit and disable tool placements from within Admin Workspace

Could

Queue submission in a similar fashion to the current implementation

If submission is not possible must wait and retry

log everything to events table for use in 'TII dashboard' when this is implemented

Choose between one "Big Roster Sync" job or smaller-grained individual syncs with sensible inter-dependencies, ie, only allow (say) 4 concurrent syncs at any one time. (Set max concurrent threads via a property.)

NB: assess how much work each option would be then make a decision in conjunction with Oxford

Classes must be up to date in TurnItIn

Student added to site is able to submit to an assignment pretty much straight away

Only need to do a roster sync if a new person is added to a site which contains the assignment tool with at least one TurnItIn-enabled assignment

Queue Roster syncs so as not to flood TurnItIn after overnight site membership updates at the start of a new term

NB: do we still need to do a roster sync or is there a better way now?

Add new configuration property to specify length of time reports are kept (assuming this is possible). Choose a sensible default - 1 year.

Check on TII website that the "delete date" has been set (assuming this is possible).

NB I'm a little unclear about this.

Logs all web service calls and responses if configured to do so

Use terse logging if not configured to be verbose.

Enabled by property.

When student visits assignment and clicks on title, they are placed in LTI TII tool in the main content iFrame and can submit from there.

When staff try to view report they are put into the LTI TII tool in the main content iFrame.

Make it work like Canvas in these:

1. http://vimeo.com/114083671
   
2. http://vimeo.com/114084561
3. http://vimeo.com/114085336

Student are only allowed to track their own submissions (by default).

Staff with appropriate permissions (assignment.submit) can track all site member's submissions.

The state that a report is in should be reflected in the UI. A 'wait time' threshold should be used to flag abnormally long delays: this should be set in a property. Possible states:

• 'Awaiting submission'
• 'Delayed'
• 'Awaiting report - submitted on DD/MM/YYYY at HH:MM:SS'
• 'Awaiting report delay threshold exceeded - submitted on DD/MM/YYYY at HH:MM:SS'
• 'Rejected. Reason: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' (assuming we can retrieve a reason for rejection)
• 'Temporary Failure (but will retry). Reason: XXXXXXXXXXXXXXXXXXXXXXXXXXXX' (assuming we can retrieve a reason for failure)
• 'Permanent Failure. Reason: XXXXXXXXXXXXXXXXXXXXXXXXXXXX' (assuming we can retrieve a reason for failure)
  
• 'Report generated at DD/MM/YYYY at HH:MM:SS'
• 'Report expired' (assuming we can find out when a report will disappear from the TurnItIn server)
  
  

Non TurnItIn-enabled Assignments must work correctly as they wont have a "Report Status" column.

Assignments that use Vericite must work correctly.

A staff member should know whether the report is queuing for dispatch, has been sent for checking with date, has been processed at a specific time(and is therefore available), has expired with a date (ie, been deleted from TII services), has been abandoned (due to an error) with date plus explanation of the error or some other state.