What is it?
Admittedly, design is a tricky thing to get a handle on. Most of us know good design when we see it, but how does that help when it comes to improving bad design?
The UX Scorecard aims to do just that. It's a measuring stick that we'll use to gage the "goodness" or "badness" of Sakai's design. We'll then use that knowledge to make measurable and meaningful changes.
How does it work?
This image is just a sample of what the UX Scorecard will look like.
Before embarking on any *new design project, UX experts will review the previous design using the scorecard as both a guide and documentation template. The result will produce a clean and consolidated report.
Each design project (for example: re-designing the schedule tool) will have a confluence page containing a scorecard. The scorecard will go back as far as three design versions and display a percentage difference column to help with horizontal analysis.
*The UX Scorecard is a component of the UX Improvement Initiative and therefore will only be used after an initial re-design. This way, all subsequent design efforts will have a baseline starting with the first re-design.
Why not use the score card to evaluate the current state of Sakai prior to an initial re-design? Learn the answer to this and other questions by reading the UX FAQ.
How accurate is it?
The scorecard is made up of various parts (see below). Some of these parts have some statistically valid information, other parts tend to be more subjective in nature, and yet some are based on industry best practices. In any case, evaluation will be handled by a team of UX experts – so the results should be fairly reliable.
Call for Help: Did you pay attention in statistics class (or know someone who did)? Are you fascinated by the study of quality, 6-Sigma, performance metrics, TQM? Perhaps you've heard of Juran, Deming, Kaplan, Norton, or Kanri?
Then we could use your help. Any guidance you can offer to help us keep our statistics accurate and our process efficient will go a long way in helping make Sakai's design the best it can be!
Parts of the UX Scorecard (An explanation of each item)
Note: This section will be flushed out at a later date once the scorecard is ready to be put in practice. But as a brief overview, the major areas include:
A. JIRA Tickets (Bugs)
JIRA will be used to answer two things: 1. Have the number of UX complaints increased or decreased from one design to the next. 2. How efficient was the UX team in addressing the complaints from one design to the next.
B. Usability Heuristics
In a nutshell, heuristics can be thought of as industry best practices. If you've never heard of them or simply want to learn more about them, why not Google them?
The Fluid team has also put together lots of good information as well as a solid process that uses heuristics, cognitive walk-throughs, and related tools to study design.
For an introduction to usability heuristics, I recommend starting here: Fluid's heuristics overview
Also, while you're at it, take a look at Fluid's UX toolkit
C. User Satisfaction Polls
Taking sample surveys will help us get a better handle on what our end-users think. The plan is to target administrators, faculty, and yes... even students!
D. Visual Design Review
It's one thing for a page to be busy, but its another thing for it to be noisy! Good visual design will turn down the volume and help users concentrate on the task at hand.
Visual design is more than skin deep. What we'll be looking for is symmetry, good use of contrast, scale, proportion, and various other attributes that make each screen easier to visually comprehend.
E. Compliance with Trends & Standards
In this section, the scorecard will try to tackle good front-end coding principles, compliance with accessibility standards, adherence to style guides, as well as questions related to relevance (for example: Is Sakai's "user experience" what most user's expect given todays technology and design standards?).
For more information on accessibility analysis, take a look at what ~melledge put together: Sakai Test Protocol--Firefox Accessibility Extension and WAVE 3.5 Combined. This, and other Fluid inspired protocols, will be heavily relied on to make sure this scorecard give both you (the stakeholder) and us (the designers) the most reliable information on the status of Sakai's design quality. Because let's face it, it's hard to fix something if you don't know what's broken.