Product Council Review: Community Review

State of community product management functions

What do you think of the product development lifecycle?

Philosophically the reviewers support the product development lifecycle. It has the right structure and is logical. The need to bring order to what are currently haphazard development processes in the community is acknowledged. The deeper into substance, though, the more concerns expressed. These are summarized below.

One general criticism of the model is that it focuses too much on getting products into the release, rather strong and integrated way of handling plug-ins to Sakai (which would be a small core) and having an "app store" like experience for Sakai admins to add things into their Sakai installation (similar to JIRA for example). Similarly, the model does not address code that is in independent release, e.g., Profile 1.3 and 1.4.

Several reviewers described a disconnect with the maintenance team and process. There is a lack of clarity about authority – the maintenance team and release management work groups do not have advance purview of projects so are unable to ascertain if a product is maintainable. Inclusion in the release is described as synonymous with a project being in maintenance mode, but this is not how things work in practice and there should be an extra step between developed and in maintenance (which ties to being in a release).
Generally, the reviewers were skeptical of focus on the Foundation driving the development model, as opposed to a community driven model, and questioned if developers would be motivated to follow this model.

Specific comments:

Criteria for inclusion in the release should also include QA and release management for at least a couple versions of Sakai.
The product lifecycle does not address onboarding new project. It's quite difficult to gain all the necessary knowledge of core Sakai API's, component manager, and portal lifecycle, not to mention the wide array of front and backend technology. Existing technical documentation is either outdated non-existent or thin, and facilitating good recommendations and guidelines for the R&D state would help avoid later rework and more smoothly move projects through the lifecycle.
It is good to have included End-of-Life, but the lifecycle doesn't take into consideration previous attempts to classify Sakai lifecycle (for example, contrast the above lifecycle with the Contrib-Provisional-Supported-Deprecated criteria listed at Criteria for Supported Status and Promote-2.5-OSP).
The deprecation policy needs to be finalized

What are the critical functions that the community needs in the context of managing the product?

Guidance on product coherence. Teams may have an unclear understanding of the exact drivers that lead to issues they face. When these issues potentially deal with the coherence of Sakai as a product it would be useful to have a body that could be consulted for advice. An example of this is database and browser support.
A UI style guide would be helpful when new tools/plugins are being developed or changes are made to existing parts of the system. Better than this for example: http://confluence.sakaiproject.org/display/UI/Style+Guide
The critical functions include ensuring that the code is usable, reliable, extensible, maintainable, scalable, secure, well documented, easy to install and not least, relevant
A clear set of features that the product must always provide to meet user needs
Maintainability
Expand/improve functionality
Quality assurance
Technology selection
User interface consistency, accessibility
Scalability
Internationalization
Release management
Branch management
Security issue resolution
Increasing adoption
Documentation and Communication. While the SakaiProject.org site has been cleaned up and now serves as a nice introduction, confluence remains a morass of misleading, contradictory and out-dated information (for example, see above documents on "product development lifecycle". Policies should be clearly, unambiguously stated, with up-to-date information.
Coordinating Release Management Work Group, Kernel Team, Maintenance Team, Maintenance Branch managers, Community Infrastructure managers (JIRA, svn, QA servers), Quality Assurance, Internationalization and up-to-date Localization, product parts creators and maintainers.

For which of those critical functions, if any, do you think a product council is needed?

The Sakai Product Council should serve to make sure critical functions do not "fall through the cracks" with no one taking responsibility. It should delegate often, but follow up to make sure critical functions are addressed.
Coordination amongst all the necessary product management components and teams
The PC should encourage/enforce the minimum set of features that must be present in the release. This will need to be evaluated based on other systems available, possibly also a survey within the community, so that we are confident the product is meeting the needs of users, and or can focus on areas where we might need more development.

How does the product council relate to other groups working on these critical functions?

Currently interaction is not happening. The external reviewers expressed several ideas about what should happen.
There is consensus amongst the reviewers that the role of the Product Council in incubating project, and the role of the production teams in managing the production code base are necessarily separate activities. Three models of future coordination are presented:

Give the Product Council oversight authority over the working groups, responsible for delegation of activities, coordination, and success
Make the Product Council and advisory body for the other groups
Have no role with respect to QA, release management, the maintenance team, the kernel team, the i18n or accessibility teams and the Nakamura/Sakai 3 teams. This would either be a severely limited role, or an abolishment of the Product Council.

Regarding options 1 or 2, the reviewers note that the strength in the Product Council rests in its wide perspective due to purposefully diverse membership: it is a representative microcosm of the community. Assuring equal consideration is given to functional, technical, and operational viewpoints would assure good decisions and full consideration of the impact of decisions.

Which of those functions are missing from current activity or processes?

Documentation as a specific community deliverable.
Development of a more refined distribution mechanism for Sakai (e.g., app store).
Pro-active approach to Sakai security
Staffing and training: ensuring that projects are staffed appropriately and are supported by more than a single institution
Decisions or guidance beyond 2.7, including early discussion of what we want to accomplish with the next Sakai release (the 2.8 discussion should be occurring now). More broadly, direction about where the product is headed.
Willingness to review not simply new projects but existing projects in the core build regarding their adherence to the development process
Stewarding the documentation and creating a product roadmap well ahead of release schedule
Marketing and adoption support
The current free-for-all in development frameworks is a mess. The PC should analyze the technologies being used currently, and recommend what people use to develop with. This could even be a factor when considering what tools/services make the cut for releases.

The mission and charter of the Product Council
When you heard about the product council, what did you hope the product council might achieve?
A focused place for coordinating and contributing to the enhancement of the Sakai software, where vision could unfold, and where critical elements of the development process could be determined. People hoped for tangible results, such as the development of a Product Scorecard. Specific actions include:

Help articulate the product vision of Sakai
Provide a clear path for when releases should occur, what each release will contain, as well as expected improvements that need to be developed for each release.
Encourage the professionalization of the Sakai development process; provide a coherent set of production goals that community teams could work to actualize
Provide timely guidance on issues relating to the "product."
Provide scrutiny to any major refactoring effort or major dependency upgrade and ensure that the benefit out ways the negative impact to existing implementations and those managing those implementations.

How close to your hopes did the product council charter come (have you read it)?

There is a gap between the what the community reviewers would have liked the charter to be and the actual charter. Several areas that lack were cited:

The charter linked here outlines the composition of the council but says nothing of its role or function. That seems to be outlined in several places in the product space. (e.g. http://confluence.sakaiproject.org/display/MGT/Product+Council+FAQ)
These pages mention that the council makes decisions "relation to Sakai's overall roadmap as a coherent product." However its unclear if a roadmap exists for Sakai 2
The charter addresses qualifications, selection process, and governance, but does not address what the Product Council will actually do.
Release management decisions were handled differently in the past, so it is not clear that this was a need to be filled.
Community engagement is lacking.

What would you change about the charter?

The external reviewers call for clarity, definitiveness, and inspiration:

The role and scope of the product council needs to be clearly described in the charter and consistently communicated. There seems to be a discrepancy between what the council sees as its mandate and what elements of the community expect or understand that mandate to be.
It needs to inspire me to do more for Sakai.
A clear definition of authority, responsibility and actions.
A clear statement about what it is out to achieve, where the product is going, and how it will help project teams get there.
The council should provide oversight that insures all the community view points are represented equally when making important decisions, like what goes into a release, communicating the scope of releases, overseeing decisions about major dependency upgrades, and overseeing the value of major refactors, before they happen. In addition to the knowledge listed in the charter, security, internationalization and accessibility should be represented on the Product Council.
They should have some role in end of life decisions, and help orchestrate the move from one tool to another (like the mailtool).
They should make and enforce decisions about platform support: things like databases, application servers, operating systems, and browsers.
The charter should require a chairperson to lead the product council.

What impression, if any, do you think the product council has on people looking at Sakai from outside the community?

The appearance of a product council is positive, and the membership reflects well on the community. There is concern that the substance of the Product Council actions does not reflect well to the external community.

Membership

Have we got the membership right? What constitutes the right mix of people on the Council?

Individuals on the Council are respected, and generally the aggregate skills and experience of the group is viewed positively. There are suggestions for changes and improvement.
Suggestions for the group composition:

Bridges to other teams/working groups
- Sakai production operations mgmt
- rep from release mgmt
- Rep from maintenance team
- Development representatives (probably need one from each project)
- Functional expert (in Sakai and online learning)
- Usability expert (handles i18n and accessibility concerns in addition to the other stuff).
Operations staff who represent the production point of view
Technical people along with the managers
Teachers
Development oriented people to provide the input and recommendations on how the product is actually being implemented and the technologies being used.
If the PC serves as the gatekeeper of the development process, then it's short of technical and operational expertise.
If the PC is interested in helping transform user wants and desires into working code then I'd like to see more users (e.g., faculty, students) involved.
Members must be contributing code, working with an organization that is running, piloting, or at least considering adopting Sakai,

How should members be selected?

There is no consensus on this. Opinions included that the members be appointed by appoint by vote, ED selection, or working group selection. There are no suggestions about who would be part of an electorate body.

Should Board members be on the Council?

Reviewers either responded no, no concern one way or the other, or that overlap with the board would be helpful. No votes were more numerous, but there was not consensus.

What expectations should there be on council members?

The consensus amongst the community reviewers is that more coordination with the community's team and working groups is needed. Awareness, interaction, and involvement in these teams are needed. Trust is needed.

Personal expectations of Product council members include that every member be a leader, every member be a doer. Reading email and participating in meetings is note enough. There needs to be concrete actions: projects that are tracked and completed.

What the Product Council has done

What do you think the product has focused its attention and energy on?

Self-definition and self process
2.7 new capabilities review
General conceptual Sakai 3 discussions.

What do you see as the successes of the product council so far? What are the disappointments?

The responses of the community reviewers were overwhelmingly negative. The only positive responses related to the new 2.7.0 capabilities, which was consistently praised as useful, both for its impact on the 2.7.0 product, and for its helpfulness to the one developer who responded.

Disappointments included the lack of work beyond the 2.7.0 capabilities, as well as the slowness to completion of this piece of work. Specific examples include:

Communications (both quality and timing) to the release teams was problematic.
No improvement to upgrades
No coherent involvement with discussions around tool deprecations and 2.7
No review of existing tools in line with current trends and user expectations

Process disappointments were also cited:

All too often the PC has not been firing on all pistons, resulting in a council considered slow and generally unproductive
Group mentality is unimpressive

What has been the net pay-off of the product council thus far?

Community reviewers have mixed feelings about the net payoff of the Product Council. The 2.7.0 capability reviews are the only positive outcome, but some question whether the slowness of this work resulted in a delayed product release. As one reviewer said, "I think the community is better off with a product council. The jury's still out (for me) as to whether or not the product is better off. "

However, some reviewers believe the concept of a product council is the right one. Reviewer commented,

"With some more guidance and direction coming from the PC, we should be able to instill confidence in the project teams so they know they are on the right track and they are working towards a common goal."

"Given the evolution of 2.x and 3.x a case can be made for separate structures overseeing the evolution of each code base. I'm thinking lightweight technical/operational structures here along with the PC evolving in the direction of a visioning body, a group of people tasked not with assessing the technical merits of whether or not Profile2 is suitable for 2.7.0 but with helping frame (and answer) the larger questions of where we want to take this Community and the software it produces, a Community presence or orchestrator of fine ideas that if handled right becomes more important to the life to the Community than the Sakai Foundation or Sakai Board."

Is the Sakai 2 product better off?

Unclear.

Sakai 3?

For Sakai 3 benefits, if any, have yet to be realized

Management / Project Coordination

Product Council Review - Community Responses