Below is a summary of the QA 2.0.1 Post-Release Review conference call on 8/22/2005. Feel free to add additional items or modify any ITEM/ACTION shown below.
Attendees: Michael Beasley, Nadine Blackwood, Carol Dippel, Clay Fenlason, David Haines, Peter Knoop, Margaret Petit, Seth Theriault, Anthony Whyte.
# | Review Item | Action |
|---|---|---|
1 | Improve the process to manage release documentation resulting from the QA Deployment (DP) test effort. | Continue to use the QA space in Confluence for draft release documentation purposes. There is public access to view and comment in this space. Upon launch, completed documents will be posted to the release web site and the main Confluence space entitled Release Information. Clay and Dave will collaborate to improve the management of the release web site (ex. http://cvs.sakaiproject.org/release/2.0.1/). |
2 | The DP effort needs to begin earlier in the DEV cycle so the QA team can begin test and document creation earlier. | The QA DP team should become involved in the Release Engineering process to enable an earlier work effort. |
3 | Improve the project planning process to allow creation of standard test documentation such as schedules, test plans and test conditions. We need to move from an ad-hoc to a more structured QA effort. | Carol is working with Mike Elledge & Chuck Severance to ensure QA time in-between functional & maintenance releases to allow for QA planning and analysis phases where test documentation is created. Michael Beasley, Nadine Blackwood, Margaret Petit & Carol Dippel are currently in-process of test documentation creation based on 2.0.1 OOTB functionality. |
4 | There is a concern re: how 2.1.0 will be deployed both as a clean install and upgrade process. At present, we don't have the capability to test the upgrade process due to machine and human resource constraints. | The Sakai Project funds adequate QA resources. |
5 | Improve Jira problem Descriptions to help DEV & QA teams more accurately and efficiently fix and regression test Jira items. | Send communication to the Sakai Community re: the importance of making clear, written problem Descriptions in Jira. |
6 | Improve Jira resolution Comments to assist the QA team. | Send communication to sakai-dev requesting Jira Comments for regression test guidance. With SVN, there is now a new Subversion Commits tab in Jira that provides improved information to guide the test effort. |
7 | Early in the 2.0.1 release cycle, the QA team experienced significant delays resulting from the project's conversion from CVS to Subversion (SVN). | Send communication to Sakai Community re: SVN Process & Policies. Dave will prepare a document for review. |
8 | The Jira Fix Version/s column is used before and during a release cycle to manage Jira items included in the release. Better management of this meta data is needed to help ensure inclusion of items earlier in the release cycle. | Send communication to Sakai Community re: the importance of managing the Fix Version/s column in Jira. |
9 | New &/or significant changes late in the release cycle increases the probability of release delays due to regressions. | The majority of significant changes planned for a given release should be included early in the release cycle (ex. ~m1 or ~m2). |
10 | Samigo is the most complex Sakai application but is currently understaffed in both DEV & QA. Additional complex Sakai applications will continue to be created and the Samigo experience may be a foreshadow of future DEV/QA issues. | Risk & complexity analysis should be conducted for each Sakai application to ensure adequate staffing for both DEV & QA. Complex functionality should be implemented early in the release cycle. |
11 | How can we continue to recruit QA participants? | Suggestions are requested from the Sakai Community. |
12 | For 1.5.0, 1.5.1, 2.0.0 & 2.0.1 releases, the QA team did not have time between releases for adequate planning and analysis for the next release. Test documentation has not been created resulting in an ad hoc test effort. | Improve project planning to take into consideration QA schedule overlap between functional (ex. 2.0.0) & maintenance (ex. 2.0.1) releases. This will allow adequate time for QA planning and analysis for each release. Advance QA receipt of requirements and specifications at least one month before the test cycle begins will allow time for creation of test documentation. See Item #3. |
13 | Should Sakai OOTB (aka Enterprise Edition) include changes to all components in a given release OR segregate components into their own release cycles? If segregated, how should the release process be redefined? | Suggestions are requested from the Sakai Community. |
Others?