Sakai 2.9 Functional Accessibility Evaluation Protocol (Accessibility Walkthroughs)

Walkthrough scripts were created to assist in evaluating the functional accessibility of a given tool. The goal of a using a walkthrough script is not to test the user, but rather to guide the evaluator through the tool, exercising as much of the tool as possible to look for accessibility issues that make it difficult (or impossible) to operate the tool or perceive the presented information. It is not meant to be a usability test of the tool or a comprehensive QA test. The walkthrough scripts should be written generically so they can be used with any selected combination of adaptive technology (screen reading software, screen magnification software, voice dictation software, keyboard only users, etc.) and/or disability.

Workflow / Process

This workflow was developed to support the Sakai 2.9 Accessibility Review. As part of that review, the process for using this protocol is as follows:

• Sign up:
  • Add your name to the participants section of the Sakai 2.9 Accessibility Review (2011) page.
  • Indicate the tool you are evaluating and the adaptive technology + disability combination in the table under the "Walkthrough Functional Accessibility
    Evaluation Signup" column.
• Get a reporting template:
  • Create a copy of the Sakai 2.9 Walkthrough Scripts Results Template (it is a child page of the Sakai 2.9 Accessibility Review Results (2011) page) for recording the results.
    • Log into Confluence
    • Browse to the Sakai 2.9 Walkthrough Scripts Results Template page.
    • From the "Tools" menu at the top of the page, select the "Copy" option.
    • Rename the page title (in the textbox at the top), use the format: "Sakai 2.9 [TOOL NAME] Tool Walkthrough Script Results".
    • Fill in the tool name, your name, the date, etc. in the header at the top of the reporting template.
    • Unless you are ready to enter issues, click "Save" to store the template for your work.
• Get server access and prepare the tool:
  • In most cases, to perform the testing a username and password for a user account that has been given access to a course or project site is needed. Frequently, the tool may need to be configured in advance with some content to work with.
  • If an accessibility walkthrough script is available for the tool, use the walkthrough script to help you more fully exercise the tool (if needed). Walkthrough scripts can be found on the Sakai Accessibility Walkthrough Scripts page and are linked to from the tools list table on the Sakai 2.9 Accessibility Review page. The walkthrough script will have details on any pre-walkthrough configuration that needs to be performed.
  • If you don't know the usernames and passwords to use, contact the Accessibility Review Lead (Brian Richwine: brichwin@indiana.edu).
  • Be sure to use the appropriate Sakai 2.9 QA Server. (Listed at the bottom of the Sakai 2.9 Accessibility Review pate).
• Perform the evaluation:
  • The best evaluation setup is to have a native user of the selected adaptive technology (if any) and a second person who will read the walkthrough script instructions to the first while recording the results.
  • Many of the walkthrough scripts have a setup section that details any required content that must be setup before hand.
  • Although the walkthrough scripts aren't meant to test the participant(s), it is best of the person carrying out the steps hasn't read the walkthrough script before hand as the script may include questions they will be asked to test whether they can access information that is presented by the tool.
  • When an issue is found, record the issue on the results page created above.
    • Describe the issue.
    • Record the steps required to both reproduce and to detect the issue (if not relatively obvious).
    • Record any recommendations that you have to resolve the issue.
    • Record the severity of the issue (see list of severity ratings and their definitions on the reporting template/page).
• Once complete:
  • Indicate that the evaluation is complete by entering the word "Yes" next to the "Evaluation Complete Yes/No?" entry in the recording page template.
  • Place a link to the results page in the Tools Listing table on the Sakai 2.9 Accessibility Review Page.
• After the completed evaluation results are posted, the Accessibility lead will review the results, search for existing JIRAs on the issues, and create any new JIRAs as required (JIRAs for each issue will be listed on the results page next to the issue).

Note: If you can't or don't feel comfortable using the JIRA system for recording the results, a word file version of the template is available for downloading. Place the results in that template and mail them to the Accessibility Review Lead (Brian Richwine: brichwin@indiana.edu). Please be sure to use the subject line "Sakai 2.9 Accessibility Review Results" so the email will get noticed!