M&M and Provisional-Contrib-Core discussion notes

Notes from afternoon discussion, part I of Maintenance and Minor Release discussion

Branch management tracking
Patching is for a bug, not a feature
info required on testing already achieved and where it is running

Doesn't sound like branch processes are broken
Michael proposes the following (picking on Seth and David): get them to write up the process and get it posted. Then we will start following it. E.g. the Jira type is bug etc.
may be value in broadening the branch mgmt process to get second opinions on whether it passes.   Doing it in some areas such as involving Jim Eng in Resources issues. Document this as part of the process.   Recommendation of a project leader is a good stamp of approval.
    2.5.1 and # of components that were touched and the number of tests needed.
    Can these fixes be clustered in narrower focus to make testing easier.
    Could be tough to track the modules and the clustering
    Megan says can work around needed fixes by version release vs maintenance branch
   If 2.5.2 and 2.5.3 come out only a month apart this makes tracking easier

Time boxing helps with local resource allocation. Need is to have a goal for maintenance release.
Counter difficulty is it is hard to take a lot of smaller releases if you have large mods and your own copy of the whole system (e.g. if it takes a month to apply the minor releases)
Fine for 2.5.x to only accept bug fixes.   Then maintenance release tests only certain areas and pulls other code from the previous releases.
Michael's goal is to take down a certain release and put up something better for people to download.
Seth likes the idea of targeted tested areas with time frames. 2.5.x continues. But maintenance release is 2.5.1 - announced for xx months ahead. Fixes in that would be focused on certain functional areas (with security bugs an exception). Announce the coming next release as make a call for QA assistance and choose the functional area for the release based on the QA resource that is volunteered.
    2.5.x evolves as branch manager sees fit.
    2.5.2, 2.5.3 evolve as scoped by what QA resources are available in a relatively short time frame as volunteered.
    And Security trumps all.
    It ought to be okay to take down prior maintenance releases (for public download)
    For a major security release and a month or more to go to next planned release then make a tag for that and include only the security patch

Post this to Management/coordination page as notes (as per Peter)

For new functionality: also try to introduce something that comes out more regularly and probably QA driven.
UCD wasn't running official 2.4.x GB - something sooner. What is problematic though is that anybody can introduce anything.
Note: there is no enforcement mechanism now to talk about what you are doing and to expose it to the community. There are so many moving parts and people working on different things that it is really problematic. Gets more and more difficult to get the thing running.
Seth's proposal is an equivalent of patching PMC for functional changes: very small group of people with technical person, UIperson,
e.g. Indiana's chat tool. Promised in a conference call that there would be no UI changes but that turned out not to be true.
Related issue is things are developed as full package: functionality, UI and API
The way Apache does it is with some amount of technical review. Some people just need some guidance.
But a big question is who has the expertise to be on such a committee. Who is it and do they have the time.

topic: if want to do minor release on top of 2.5---what is an example.:
one example is an optional feature that you can swap out.
Another question is why do we have to release all the tools at the same time. Could have 5 GB releases when other tools have different time scales.   Feature Release branches....
Eclipse model is coordinated subprojects and each project is responsible for its own release. But this feeds into K1 discussion and different individual numbering schemes.
Installer model would be really nice.
Changes that don't require changes to underlying services.
What do we want to see improved in the shorter term
   One example: camtools by October
   Another: email templating service (new services is easier)
Lance distinguishes between experimental releases and enterprise releases (Fedora vs RedHat)

Maintenance Release:
Propose to community that some technical governance group QA+Branch Mgmt will review
No decision yet on functional releases (minor releases).
Talking about minor risk is in part documenting what we are already doing.
New topics: technical governance of functional
decoupling general tool releases from functional additions

Notes from afternoon discussion, part II of Maintenance and Minor Release discussion and Provisional/Contrib/Core

The group began discussion about minor releases (2.6 2.7, ect) by considering whether it is a good idea to introduce new features and how to do this in a more rapid succession.

Concerns that motivated this idea are

Features are all introduced at the same time. When we decide to release, there's a large scope for QA which takes a lot of resources
Long wait times to introduce user requested features.
Taking on the burden of more local maintenance

Jason framed some thoughts with question about 2 major stakeholders-

Q: Do existing deployers care about new features? Jason isn't sure
Q: Do adopters care about new features? Jason believes yes.

Essentially, we are trying to discover how we get back to the benefits of the 6 month release cycle without a 6 month cycle?

Seth pointed out that we might be biting off more than we can chew. Not only will we have 2.5 maintenance releases, but we'll also have folks in the 2.6 vein that will also start clammoring for maintenance releases. This lead into discussion of the ability of projects to manage there own releases. Some projects (ie Samig) are successfully doing this; however most are not. Someone poised the question "is QA the responsibility of the project teams?" There was speculation about whether teams would follow though on this. Megan pointed out that minor releases were an opportunity to enforce the requirement for unit/integration tests.

Problem with current QA process. Testing is getting harder and harder with more and more coming into the release. Further, we are not effectively communicating out what changes are occurring. For example, Davis was surprised at the work in the 2. gradebook

Core - Provisional - Contrib

The core-provisional-contrib model does not accurately convey the meaning of quality in Sakai.

Tool Score Cards: quality review of components.
There are tools outside of core that meet quality standards. We do have a basis for the Contrib to Provisional status and a draft for Provisional to core.
This isn't to determine what goes into the release this is about documenting the status of the

Who's going to execute it and how do we go about doing it?

This can be done a couple ways
1) Project teams - self reporting
2) 3rd Party - pick someone that you trust
3) Have someway of empirically proving it

To begin with, we'll begin with project teams doing a self evaulation. From a security perspective - Anthony wants to ensure the XSS doesn't occur. Needs to be multi-faceted

Let's assume we have criteria and methods to get community rating. Now what? What goes into the release?

Right now you can get something into trunk and abandon it. Quality assessment should be up to date and meet certain standards. If not there there are decision points
1) In the release or out of the release?
2) licensing - does the code in question fit into our licensing model

Talk about different types of packaging of the product.

Sakai Core: tools very well tested
Sakai Evaluation: tools that come from trunk or contrib and that have clean licensing.

Thought experiment
Oliver suggested we take a look at Section Info. UCB did the development on the tool and is largely focused on other work at this time. What would happen with that tool? Would it be dropped?

Stephen Marquard - discussed in great deal that we have to take a hard line on tools that are not up to snuff. First analysis of the current state needs to be performed and the community determines a baseline.

Who defines Sakai as a project? Jess noted that we were discussing Sakai in terms of two attribute

accountability
value judgment

Discussion of inter dependencies between modules. It's very hard to decouple some of theses tools. Perhaps we should be working toward modularity and flexibility so that there are more opportunities. For example, we can't build Sakai without sections, assignments and gradebook!

As to determining what is in the release

People around the block doesn't need release, they need the a fore mentioned rating.
People that need to evaluate Sakai are the ones that really need the release.

Q: Who handles choosing what is in the 'Evaluation' release? Michael proposed the Foundation does by performing market analysis/ competitive analysis. The group was in agreement about the to the idea of distribution of Sakai evaluation to the Foundation that was not a criteria base decision.

Q: Can we drop stuff from the evaluation release? Yes - is like a showcase.

Thumbs up to scraping the Core-Provisional Contrib!

Proposal

Establishing a WG to define a set of criteria (ie score card). This WG will work from the existing set of provisional criteria.
2-3 months before the release of 2.6 folks need to fill out the scores card.