Portfolio-related vignettes

Assess

- Organizers may set up work flows using various portfolio tools to assess participant performance in relation to standards or outcomes.

- Organizers should have an easy to use interface to design and create assessment work flows that associate goals and standards with student work and instructor evaluation via a minimum number of tools.

- From Boston Sakai: Functionality for assessment should provide the ability to look at the big picture to determine where students are in their learning and where the program is in facilitating student learning. Programs and institutions should be able to use assessment functionality to express their curricula in ways that students can understand.

Document

- Organizers can create matrices, wizards, forms, and portfolio templates that allow users to document their learning by associating items they have created with reflections and learning outcomes.

- Users should be able to document their learning regardless of the existence of specific data structures by locating evidence and associating it with particular learning outcomes. Users should also be able to associate metadata with any item they have created in Sakai that documents how and why the item was created and its relationship to their learning.

- From Boston Sakai:

• Any user can document work, which would include space for reflection/meta-reflection.
• Documentation could be guided by templates that are flexible/modifiable for student, faculty, and admins.
• Documentation should be able to cross layers of work/roles/etc.
• Documentation should support users (students/faculty/admin) to engage in a discussion/dialogue about the work.
• Documentation should have a mechanism for grading/ratings.
• Documentation should be usable for accreditation.

Own

- Users control access to items in their My Workspace Resources folder, determine who is invited to view their portfolios, and usually decide when to submit a matrix cell, wizard page, or wizard for evaluation.

- Users have the right to privacy and the right to own their work globally and by artifact.

-

Empower

- The portfolio layouts and styles tools may be used to provide users with a mechanism to organize and present information about themselves as they wish.

- Users can move beyond the preordained structure of a portfolio implementation to organize their own work and use it for their own purposes.

-

Re-purpose

- Users may be invited by their institution to take previous work and apply it to new contexts.

- At any time, users may take previous work and assign it new meaning in new contexts.

-

Express

- Rich text editors in forms allow creative expression. The portfolio layouts and styles tools may be used to provide users with a mechanism to creatively display and share information about themselves.

- Users have multiple options requiring different levels of technological expertise to creatively display their work.

-

Migrate

- Institutional migration of portfolio data from one version of Sakai/OSP to the next version is generally smooth. Migration of portfolio data to another system is very difficult.

- Portfolio implementations can be efficiently and easily moved from one platform to another.

-

Preserve

- Users may access any item they have stored in Resources and/or any other item they have created as long as the site, tool, data structure, and/or instance with which they are associated has not been deleted or hidden.

- Any user should be able to see all of the items they have created at any time regardless of whether the sites, tools, data structures, or instances associated with them have been deleted or hidden. These items or copies of these items should be located in the user's own data store. Other users with permission should also be able to access these items.

- From Boston Sakai: Users should be able to export their data along with their data structures. Exports should have the capability to stand alone. Exports should allow historical snapshots to see change over time, a process that would be distinct from current ability of the multi-column matrix where students deposit artifacts in columns based over time. Instead, the student would modify portfolio over time and institutions (along with portfolio owners) could use 'play-back' to see evolution of student's evidence. Google Wave is a conceptual model of this process.

Customize

- Organizers may customize OSP data structures for predetermined workflows guiding participants through processes designed for personal representation, teaching and learning, and/or assessment and accreditation.

- Customization should take place via user interfaces instead of XML coding. Customized workflows should involve far fewer tools and more effective guidance.

- From Boston Sakai: Customization needs to be linked with flow. Customization of flow brings components back into the process.

Usability

Current user interface is not intuitive and complex

Ideal user interface would be easy for administrators, faculty and students to use

-

Present

- Participants may create portfolio from available portfolio templates or from a combination of available portfolio layouts and styles.

- Participants should have an easy to use and multi-faceted interface to use in designing and creating an aesthetically pleasing as well as meaningful display of a selection of any items they have available to them in Sakai.

- From Boston Sakai: Portfolios should employ templates that allow multiple views. There should be more student control over how portfolios are presented with the potential for a different view for each audience.

Collaborate

- Users can grant collaborative permission to shared users, but there is no ability to have collaborative and read-only shared users at the same time.

- Users should be able to create group, course, department, and institutional portfolios without having to share individual accounts.

- From Boston Sakai: There should be a way for users to create group portfolios. It is also important to preserve current feedback functionality allowing students to comment on each others' work.

Guide

- Organizers may use the FCK editor in Matrices and Wizards to provide instructions, rationale, and/or examples for participants completing each Matrix cell, approaching a Wizard as a whole, or completing each Wizard page. Forms and Form fields allow organizers to add simple text instructions. Form fields may also be given more elaborate prompts using a custom renderer. Portfolio Templates allow organizers to provide simple text instructions for each item participants select and/or for participants add text to Portfolios via Outline Options.

- Guidance should be more closely associated with workflow and should provide an overt way to allow participants to easily understand what is expected of them.

- From Boston Sakai: Template-based page creation in 3.0 may be a benefit or a challenge. Should template mechanisms form the basis for a guided experience or do templates open up a need for greater guiding? What advantages or complications do new structures like converging lines or "memory palaces" offer? What metaphors exist for guiding? Should guidance look like a matrix or should it be more like the forums tool or other tools in Sakai?

Collect

- Participants collect items for association with Forms, Matrix cells, Wizard pages, and Portfolio Templates in Resources. Matrices, Wizards, and Portfolio Templates allow participants to select a portion of their items from Resources to submit for evaluation in a Matrix or Wizard or share with other via a Portfolio.

- Any item created by the user in Sakai should be available in the user's collection for reflection, tagging, association, sharing, evaluation, and reporting.

- From Boston Sakai:

• Users should be able to collect material across Sakai including assignments, blog posts, videos, resources, etc.
• Users should also be able to collect material outside of Sakai including tests managed outside of Sakai, URLs, other databases, etc.

Associate

- Items from Resources may be associated with Forms, Matrix cells, Wizard pages, and Portfolio Templates. Participants may complete Form instances to associate with Matrix cells, Wizards, Wizard pages, and Portfolio Templates. Organizers may automatically associate submitted Assignments with Matrix cells.

- Users should be able to associate any item they have created in Sakai with a portfolio workflow.

- From Boston Sakai:

• Admin should be able to connect/associate items. An example is: Assignment 4 meets Competency 2.
• Faculty should be able to connect/associate items. An example is: Assignment 4 is related to blog post on Japanese Art history and website about woodblocks.
• Users should be able to connect/associate items (blogs, assignments, etc.) in Sakai for sharing with others or personal learning. |

Reflect

- Organizers may supply prompts in Forms for Reflection to be included in Matrix cells, Wizards, or Wizard pages. Reflection forms may also be included in Portfolio Templates as options for participant selection or via automatic inclusion from matrices or wizards. Organizers may also add reflection prompts for participants to fill in via Outline Options in Portfolio Templates.

- Any item in Sakai should allow a reflection to be associated with it.

- From Boston Sakai: Reflection can be freeform or structured, self-evaluation (possibly using a rubric). Ideas to enhance reflection in Sakai 3.0 include:

• Allowing students to reflect anywhere
• Providing options to record ideas or notes (that refer to a specific piece of work); academic thought, insight, --a scratchpad or notepad
• Offering Twitter-like functionality (ability to record things from anywhere from variety of device types)
• Offering the ability to assemble and share/present piece of the scratchpad
• Ensuring that users have a private workspace
• Allowing reflection in other contexts
• Offering the ability to see one's own growth
• Providing a time line of reflections or growth on a specific topic.
• Offering options for non-linear display

Tag

- Participants may tag resources item with metadata in Resources, but this data (with the exception of copyright prompts) is only available to the participant. The Goal Management tools allow tagging of goals from the same site or from associated sites, but this tool is still in Contrib. Matrix cells may also be tagged with goals from the Goal Management tool.

- Any item a user creates in Sakai should be tagable.

- From Boston Sakai:

• Users should be able to add tags.
• Tags should be applicable at the text, document, and item levels.
• Institutions should be able to push tags to users.
• Faculty should be able to push/define tags for users.
• Auto tagging of meta-data like time/data.
  
  - From T. Summers, VT: We would like to see better ability to move work between portfolios, matrices, and personal resource areas.  Multiple portfolios should be easy to combine into single ones; items from matrices should be able to be collectively downloaded and collected together, etc.

Evaluate

- Users with permission may evaluate student submissions for Matrix cells, Wizards, or Wizard pages by filling out and saving Evaluation Forms. Users with permission may also provide feedback at any time to ongoing work in Matrix cells, Wizards, or Wizard pages. Users with permission may grade submitted Assignments and/or rate goals linked to submitted Assignments. Participants may allow comments in Portfolios shared with others.

- Evaluation should allow much more flexible workflows and roles.

- From Boston Sakai: Functionality for evaluation should include the option to create rubrics and apply them to almost anything. Users with permission should also be able to share rubrics, use them for peer evaluation, and allow rubrics to feed the Gradebook or a report. Rubrics should be linked to learning outcomes and/or facilitate additional ways to present expectations for learning. Functionality for evaluation should be flexible enough to support multiple models of evaluation, single or multiple, double blind, and/or peer evaluation.

Share

- Participants may share Portfolios with any other user in the site, with users in other sites in which the user is a member, with any email address, or as a public URL. Participants may also submit Matrix cells, Wizards, Wizard pages, and Assignments for evaluation by other users with permission to evaluate.

- Users should be able to assemble, display, and share any item they have created in Sakai.

- From T.Summers, VT: It would be nice to have one interface for creating presentation portfolios. - From Boston Sakai: Student users should be able to share with different audiences including each other, instructors, institutions/departments, specific individuals (prospective employers or relatives), and the public at large. Each audience will likely require a different view of the same portfolio.

Report

- Users with permission may use Report definitions to run predetermined queries on data in pre-populated data warehouse tables. The data is displayed according to the specifications of the report definition, which may include more than one view of the data.

- Users with permission should be able to write queries on any item in Sakai and determine how the resulting data is displayed.

- From R. Hill, UWyo: We are interested in this function, to be restored or enhanced, allowing access to the portfolio artifacts and meta-data such as counts. More details and use cases on request.
- From T. Summers, VT: We would like to see more ability to report out other statistical information about matrices without the need for a separate reporting tool to be customized per matrix.  For example, retrieving statistics such as a report of who has Cell A2 empty/full, number of items added to Cell C3, or an overall percentage of who have cell F22 in the "Completed" status would be very useful.

Workflow

- The OSP tools are used in combination to provide a work flow for participants. The lefthand menu bar and/or the home page for the site may be customized with prompts to assist participants with the intended workflow.

- Work flow should be much more closely coordinated with guidance and should include options for organizers to make the intended work flow within a site much more understandable for participants.

- From T.Summers, VT: We would like to see improved interaction between the OSP tools and the XSD-based forms that drive them.  For instance, we need to be able to edit and replace forms after matrices are in place (published); to have a better tool for creation of these forms; and to have a system that isn't going to slow down the more forms we create.
- From Boston Sakai: Do we need a flow engine? What is the starting point for a work flow? Should there even be a starting point? What about transfer students? Users need to be able to jump in at any point. What metaphors could be used to represent work flow?

Export

- Users may download any portfolio they create to their desktop for use outside of the portfolio system.

- Users can take their portfolios with them to use on their own desktops and in other portfolio systems.

- From Yitna Firdyiwek, U of Virginia: Users should have the ability to viewed the content of their portfolios in a stand-alone mode or reconstituted in another environment. It should be possible to de-couple a portfolio from the tools and environment in which it was created.