Sousa Scorecard

The following scorecard is for sousa_2-5-x-1 released on July 16, 2008. All evaluations were done by Mark Norton, the project leader of the Sousa Project.

The following evaluation methods are proposed for the scorecard items selected in the Scorecard BOF at the Ninth Sakai Conference in Paris, France. Please feel free to edit in place, add comments in other colors, or append comments below.

Note that point scales for individual items vary considerably. This is not a problem if we develop a weighting schemes when scores are rolled up into the four evaluation dimensions.

See also Scorecard Item Definitions

No.

Scorecard Item

Max Points

Score

Comments

1.0

User Experience

60

16

26%

1.1

Consistency / Best practices

10

5

Some conventions are used. Sakai CSS.

1.2

Usability

10

5

Improvements are needed.

1.3

Accessibility

10

2

No formal assessment. Basic support included.

1.4

Internationalization

20

2

Only one langauge (English)

1.5

Ease of use

0

 

 

1.6

User testing

10

2

Very limited user testing, informal feedback.

2.0

Technical

148

23

15%

2.1

Browser support

6

3

FF2, FF3, IE6

2.2

Code review

10

0

No formal code review yet.

2.3

Unit testing

10

0

No test, no results.

2.4

Functional regression testing

10

0

No test, no results.

2.5

Integration testing

10

0

No tests, no results.

2.6

Performance testing

10

0

No test, no results.

2.7

Internationalization

10

5

Strings externalized. UTF-8 support.

2.8

Licensing

10

5

Currently on a personal license, not Sakai.

2.9

Outstanding JIRA bugs

10

0

Not part of Jira yet.

2.10

Packaging (code structure)

10

8

Conforms to naming conventions. Maven 1/2 support.

2.11

Static code review

10

0

No tests, no results.

2.12

Validation/spec conformance

10

0

No test, no results.

2.13

DB support

10

n/a

Based on CHS, no databases.

2.14

DB best practices

10

n/a

Bsed on CHS, no databases.

2.15

Security review

10

2

Check for basic permissions.

2.16

Technical

0

 

 

2.17

Event tracking

2

0

No support for event tracking.

3.0

Descriptive

48

29

60%

3.1

Bundled Help

10

0

No bundled help.

3.2

Test plan

10

6

Test plan exists, needs work.

3.3

Javadocs

10

9

Very good coverage.

3.4

Design Documentation

3

9

Well documented.

3.5

Wiki/website

0

 

 

3.6

Deployment doc

3

2

Exists, needs work.

3.7

End user external docs.

3

3

Good documentation.

3.8

Issue Tracking (Jira)

3

0

Not included in Jira at this time.

3.9

Events documented

3

0

No support for events.

3.10

Licensing Documented

0

 

 

3.11

Configuration

3

1

No configuration properties at this time.

4.0

Community Support

47

16

34%

4.1

Team size

20

1

 

4.2

Team diversity

4

1

 

4.3

Responsiveness

10

5

I do what I can.

4.4

Production experience

3

1

Still new after one year of development.

4.5

Communications and openness.

10

8

YMMV.