Page Comparison

1.0

User Experience

1.1

Consistency / Best practices

10

UI Checklist is needed.

1.2

Usability

10

Very subjective, how to evaluate?

MJN:Perhaps a rating system?

1.3

Accessibility

10

UI Checklist is needed.. Screen reader trial - 0/1. Accessibility should be a requirement, given University regulations.

Will the existing protocols and templates suffice? http://confluence.sakaiproject.org/confluence/x/3ok
MMM: I see no problem using the process we have in place. 

1.4

Internationalization

20

Number of languages supported (max of 20).

1.5

Ease of use

0

Same as usability and same problems.

MJN: Merge into usability.

1.6

User testing

10

Results scaled vs. 10 points.

2.0

Technical

2.1

Browser support

6

IE6 - 1, IE7 - 1, Firefox2 - 1, Firefox3 - 1, Safari - 1, Opera - 1, Others - 1

MJN: What is our policy?

2.2

Code review

10

Results graded on 10 points.

MJN: This is formal code review.

2.3

Unit testing

10

Percent coverage scaled to 10 points.

2.4

Functional regression testing

10

Percent coverage scaled to 10 points.

MJN: Requires a test plan.

2.5

Integration testing

10

Test harness - 5. Results - 5

MJN: What is tested here?

2.6

Performance testing

10

Test harness - 5. Results - 5

2.7

Internationalization

10

Checklist. Strings externalized into bundle - 0/1

2.8

Licensing

10

Percent files labeled scaled to 10 points.

2.9

Outstanding JIRA bugs

10

None - 10, Some - 5, Many - 0.

MJN: This is a bit vague.

2.10

Packaging (code structure)

10

Checklist is needed.

2.11

Static code review

10

Code review harness - 5. Results - 5.

MJN: This is an automated review.

2.12

Validation/spec conformance

10

Checklist is needed.

MJN: Wasn't this supposed to replace spec validation?

2.13

DB support

10

Hypersonic - 3, MySQL - 3, Oracle - 3, Others - 1

MJN: This could be better expressed.

2.14

DB best practices

10

Checklist is needed.

2.15

Security review

10

Checklist is needed.

2.16

Technical

0

Not sure what this means. Checklist is needed.

MJN: Drop this in favor of code reviews.

2.17

Event tracking

2

All - 2, Some - 1, None - 0.

3.0

Descriptive

3.1

Bundled Help

10

Coverage scaled to 10 points.

3.2

Test plan

10

Coverage scaled to 10 points.

3.3

Javadocs

10

Coverage - 5 points, Quality - 5 points.

3.4

Design Documentation

3

Good - 3, Fair - 2, Some - 1, None - 0.

3.5

Wiki/website

0

What would be on these pages?

3.6

Deployment doc

3

Good - 3, Fair - 2, Some - 1, None - 0.

3.7

End user external docs.

3

Good - 3, Fair - 2, Some - 1, None - 0.

3.8

Issue Tracking (Jira)

3

Good - 3, Fair - 2, Some - 1, None - 0.

3.9

Events documented

3

Good - 3, Fair - 2, Some - 1, None - 0.

3.10

Licensing Documented

0

How is this different than above?

3.11

Configuration (sakai.properties)

3

Good - 3, Fair - 2, Some - 1, None - 0.

4.0

Community Support

4.1

Team size

20

Number of regular participants (max of 20)

MJN: Big projects will get a boost because of this.

4.2

Team diversity (institution, dev/ux/qa)

4

Dev - 1, UX - 1, QA - 1, Others - 1

4.3

Responsiveness (Average time to respond to JIRAS)

10

Response time scaled to 10 points.

MJN: Possible, but a LOT of math.

4.4

Production experience - length, scale, diversity

3

Proposed - 0, New - 1, Mature - 2, Ancient - 3

4.5

Communications and openness.

10

Response scaled to 10 points

MJN: How to measure?