Present & Collaborate & Share

Summary

Users will be able to create portfolio presentations from institutional templates with configurable rigidity (from blank pages, through sample content to be replaced, to enforced restrictions on the number of pages). Portfolio presentations may be created by a single (student) user or by multiple users collaborating on a single portfolio presentation. Portfolio presentations will follow the WYSIWYG (what you see is what you get) design implementation.

User Stories

• Expressive Portfolio: Students are given a portfolio "template" with pages on separate subjects (e.g. Welcome, Goals, Philosophy, Examples of Work and Resume). Student replaces sample content with content specific to their experience, adding and removing pages as appropriate. Faculty or peer advisers will review the portfolio and provide feedback indicating whether or not enough or appropriate content is included in the portfolio.
• Collaborative Interview Portfolio: A group of students will be conducting interviews, each filling out a form corresponding to the interview, and collaboratively creating one portfolio from the multiple student interviews.
• Collaborative Grant Evaluation: A group of researchers will be creating a portfolio for a grant evaluation and have different users contribute different pieces.
• Collaborative Shared Project Experiences: A student service learning group or leadership group wants to build shared portfolios about project experiences, without requiring a designated person be finally responsible for owning the portfolio.
• Collaborative Institutional Portfolio: Institutions and programs seeking to create and share institutional portfolios demonstrating the accomplishments of an institution or program want to build shared portfolios to highlight the assessment of student learning and other institutional or programmatic outcomes.

Functional Details

Portfolio presentations will be created as a type of Sakai 3 site, with extended properties and services to implement the actions described below.

Portfolio prseentation creation will follow a "suggestive model" of creation, where sample content and pages are provided, but specified page content are not enforced by portfolio software. This provides the flexibility fo create guided or non-standard portfolios subject to review by faculty or peer advisers. Longer term functionality could be added that would restricted the number of portfolio pages to the initial sample set.

Actions:

• Create: create multiple page-level presentations with explicit navigation links (need pages, and intra-page navigation or sub-pages: perhaps intra-page navigation and allow formatted)
  • Sakai 3 provides multiple page navigation for portfolios/sites; including hierarchical navigation. Full featured site-templates are being developed (/wiki/spaces/KERNDOC/pages/22665136063)
  • Sakai 3 does not (yet) support ability to create a page in Site A based on a page from Site B (perhaps based on page templates?)
• Rigidity: Adding/removing pages may be optionally restricted
  • Sakai 3 does not yet provide this functionality
• Theme: choose a style theme that controls how the presentation is displayed, allow user personalization to themes (ex:  change banner image, colors of headers, etc.)
  • Sakai 3 currently supports themes for sites
• Collaborate: enable collaborative editing of presentation by multiple users
  • Sakai 3 currently supports "collaboration" in the form of adding members to sites with varying access rights
• Duplicate: create a copy of existing portfolio presentation site
  • Sakai 3 does not yet support this
• Publish: Presentation and presentation contents will be copied to a persistent location independent of course-management content which may be removed. Presentations may be added as workflow artifacts and they may be unpublished at any time. Presentation URLs must be short and readable.
  • Sakai 3 should be extended to allow duplication of (portfolio) sites to facilitate publishing presentations. Publishing a presentation would involve duplicating and setting access rights as public. Additional publishing options might be required to adjust rendering, re-naming the presentation.
• Export: download presentation site as a HTML or PDF document
  • Sakai 3 does not yet support this
• Print: Printable representation of portfolio presentation site (e.g. PDF)
  • Sakai 3 does not yet support this
• Share: share with a restricted or unrestricted (public) set of users
  • Sakai 3 supports sharing with individual users (e.g. site members), but needs to support sharing by group or optionally support for guest access, email notification
• Comment/Feedback: optionally enable or request comments (which must not interfere with style & visualization of portfolio presentation)
  • Sakai 3 does not yet support this
• Delete: Remove portfolio presentation site
  • Sakai 3 currently supports
• Tag: tag with semantic or free-text tags (e.g. indicate portfolio meets a specified competency or indicate it's been "approved/vetted" for display in a gallery of institutional portfolios)
  • Sakai 3 needs to support services and UX for this, but underlying Sling/Kernel framework does support tagging
• Web-services: query for URLs of published, public portfolios (e.g. for displaying in a public gallery of portfolios)
  • Sakai 3 does not yet supoort
• Browse/Sort/Filter: Browse and sort portfolio presentation sites
  • Sakai 3 supports but additional sorting/filtering functionality may be needed
• Search: Search portfolio content leveraging Sakai 3 search functionality
  • Sakai 3 currently supports

Attributes

• Artifacts: any single piece of content (e.g. form, assignment, ...)
• Owner(s): users who may edit portfolio
• Comment: optionally enable user comments (subject to same style as portfolio, or on a separate page); portfolio owner can choose to approve comments for public display (e.g. blog-like threads) or leave them private
• Pages: support paging within expressive/free-form or templated portfolio
• Template: optional constraints on what content can be selected and how the portfolio will be rendered

Goals

• All artifacts (images, etc) included in a portfolio must be easily viewed by any shared user
• Email notification of shared portfolio presentations
• Sharing a portfolio should be distinct from collaborating on a portfolio (different user sets)
• Collaborative portfolios should either support equal ownership, or support transferring primary ownership to another

Assumptions

• Share-by-role functionality is being explicitly dropped (share-by-group meets real user needs and share-by-role is confusing to users)
• Base Sakai 3 share functionality will be leveraged and extended as needed to support portfolio sharing
  • Guest user access to shared content may be implemented as a lightweight "token" authorization scheme such as is used by Google Calendar & Google Docs

Diagrams & Mockups

Creating a portfolio presentation site:

Viewing a portfolio presentation site:

Implementation Details

• Will site templates support permission structure where a portfolio user creates a portfolio site from a template, but is not allowed to edit the site (only enter data into the structured form data widgets)?
• When a portfolio is published, widgets should be rendered differently (hiding structured form data input buttons)
• Perhaps Portfolios could become part of the student's profile (published portfolio part of their sakai social network)
• Could collaborative portfolios benefit from something like EtherPad (http://code.google.com/p/etherpad/)?
• Should comments be threaded? Should they be inline? Should they be specific to a particular portfolio page or generally applied to entire portfolio?
• Perhaps portfolios with fully required content would be created automatically as the user creates the required content?
• Need to be able to add a page to Site A based on a page from Site B (perhaps based on cross-site page-templates?)
• Portfolio Name/Title - user's may want a different 'name/title/heading' for portfolio than the site title (e.g. same portfolio title, different portfolio for grad school vs job application).

Workflow & Guide

Summary

The Sakai 3.0 Workflow tool combines the functionality of the Sakai 2.x Matrix, Wizard and Evaluation tools, in addition to new functionality to meet current needs. Guidance, in the form of static content (e.g. instructions, rational, rubrics and examples) is available at the top-level of a workflow as well as on individual pages. Workflow progressions can be optional, involve choices, or enforce a required process.  A sortable, filtered "dashboard" view is available to enable evaluators and reviewers easy access to student's content, as well as a traditional "matrix" view that allows students to better visualize their progress. Workflows will be independent of any course or site boundaries.

User Stories

• Teacher Education Program (IU): Students are asked to upload different types of evidence into separate pages of a workflow (with no progression). The evaluators don't evaluate the evidence in the individual pages, but instead evaluate the entire workflow collection using a rubric developed by the instructor.

• Undergraduate Research Opportunity Program (UM): Students collect structured data in a workflow (with no progression) organized into Lectures, with each page associated with an enriching activity. No implicit page progression is needed or defined. Students view the workflow in a Matrix/Horizontal visualization. Some of the structured data collected will be re-used in a portfolio presentation.

• Adverse Event Reflective Exercise (UM): Workflow (with open progression) for medical students, with each category representing an curriculum area and each page within the curriculum area representing a unit of time (year). Administrators need to select a subset (cohort) of students and batch update the status of their workflow pages (opening up the next year of pages for those students). Students fill out structured data within each page (collect), optionally solicit or receive feedback, create a structured reflection and submit the page for evaluation.

• Expressive Portfolio (UM): Workflow (with no progression) with a single category with several pages create a guided portfolio creation experience. Pages collect structured data for re-use in a portfolio presentation (e.g. welcome page, goals page, philosophy statement, examples of work, resume page, ...). In-place feedback of artifacts is important, as well as re-use of artifacts from other workflow collections. Ideally, this would look like a content hosting system with sample/prototype data content in each page.

• Dental Hygiene (UM): Open workflow with each category representing a competency domain and each page with the competency domain representing a unit of time (mini-mester). Feedback per artifact is important, as well as having different progressions for different categories. The last category leverages page status and submission for evaluation. The ability to define different evaluators for different pages is important.

Functional Details

Terminology

• Artifact: single piece of content (e.g. form, assignment, uploaded-file) or a published portfolio presentation
• Page: collection of content (e.g. matrix cell or wizard page) that will be stored or referenced in a common directory/folder
• Category: collection of pages (e.g. matrix row or wizard category) that will be stored in a common directory/folder
• Workflow Collection: collection of categories (e.g. wizarrd of matrix) that will be stored in a common directory/folder
• Progression: the rules defining how a Page Status changes from LOCKED to READY

Workflow Definition

Workflow categories can be defined with one of four progression options:

1. None: page status and evaluator attributes will not be used at all
2. Open: page status will not automatically change based on a defined defined progression, and it's initial status may be READY or LOCKED
3. Sequential: each individual page will be unlocked (set to READY) when the previous page in the category is set to PENDING
4. Custom: each individual page can be defined as a trigger for another page, so a non-linear progression can be defined that unlocks a subsequent page when a given page's status becomes PENDING. This allows for choices, such as a user can set either Page-A or Page-B to pending, in order to unlock Page-X.

Defining a workflow involves the following actions:

• Define a category (or sub-category) and category progression (none, open, sequential, custom)
• Define page within a category
• Optionally duplicate page
• Optionally duplicate category
• Optionally delete, modify, or move page
• Optionally delete, modify, or move category
• Optionally "bulk" modify the status of multiple pages in one operation
• Import/Export/Duplicate workflow definition
• Authorize workflow usage for specified users or user groups

Each Page of a workflow has the following attributes:

• Name
• Owner (e.g. student): permission to read/write artifacts and submit page for evaluation (page status to PENDING)
• Guidance artifacts: such as instructions, rubrics, examples, attachments
• Selected artifacts: such as specific forms, structured reflections, assignments, uploaded files, external URLs, or any artifact
• Bound artifacts: required or automatically associated artifacts (e.g. specified assignment)
• Reviewer(s): List of users from whom feedback/review is requested
  • Review (feedback) artifact (may be implemented as part of more global review anywhere functionality)
• Evaluator: single user with permission to read artifacts, write evaluation/feedback, re-assign to another evaluator
  • Evaluation artifact (may be implemented as part of more global review anywhere functionality)
• Status: READY, PENDING, COMPLETE, LOCKED
• Tag(s): Pre-defined semantic tags associated with a page (e.g. defining a learning objective or outcome) that facilitates finding artifacts with the same tag
• Change History: changes to status, add/delete/modify artifacts, changes to structured reflection/feedback/evaluation, evaluator changes, last-modified-date, etc.

Each Category has the following attributes:

• Name
• Progression (e.g.none, open, sequential, custom)

Each Workflow Collection has the following attributes:

• Name
• Owner (e.g. student): permission to add reflection, request reviewers and submit workflow for evaluation (page status to PENDING)
• Guidance artifacts: such as instructions, rubrics, examples, attachments
• Selected artifacts: such as specific forms, structured reflections, assignments, uploaded files, external URLs, or any artifact
• Reviewer(s): List of users from whom feedback/review is requested
  • Review (feedback) artifact (may be implemented as part of more global review anywhere functionality)
• Evaluator: single user with permission to read artifacts, write evaluation/feedback, re-assign to another evaluator
  • Evaluation artifact (may be implemented as part of more global review anywhere functionality)
• Status: READY, PENDING, COMPLETE, LOCKED
• Change History: changes to status, changes to pages, changes to structured reflection/feedback/evaluation, evaluator changes, last-modified-date, etc.

Workflow Visualization

Once a workflow has been defined, student and evaluator users are given a choice of visualization options that can be dynamically selected and changed. Student users can view their own workflows along with feedback requests from others within the dashboard visualizations. Evaluator and administrative users can view workflows from multiple students/users within the dashboard visualizations.

• Matrix/Horizontal: Available only for workflows defined with a consistent number of categories and page (e.g. 3x4 or 1x5) and only available to students (not evaluators of  workflow collections). Categories are represented as horizontal rows. This view is best when the number of pages can be easily displayed across the top without requiring horizontal browser scrolling.
• Matrix/Vertical: Available only for workflows defined with a consistent number of categories and page (e.g. 3x4 or 1x5) and only available to students (not evaluators of workflow collections). Categories are represented as vertical columns. This view is best when the number of categories can be easily displayed across the top without requiring horizontal browser scrolling.
• Dashboard: Hierarchical listing of categories and pages that allows sorting and filtering by page attributes

The Dashboard Visualizations should leverage ideas originally defined in http://confluence.sakaiproject.org/confluence/display/OSP/UMichigan+-+OSP+Dashboard.

Workflow Usage

... tbd ... 

Diagrams & Mockups

Matrix/Horizontal Student Screenshot

Dashboard Student Screenshot

 

Dashboard Instructor Screenshot

 

Implementation Details

• Workflow views of specific pages must provide contextual clues as to where the user is within the workflow (e.g. when looking at a page, we must display where they are in the workflow, either via "breadcrumbs", or perhaps a navigation frame with the entire workflow hierarchy available).
• Should Sakai write it's own workflow engine or leverage an existing project such as Drools?
• Workflow Rules (within a page and/or category) should be modular and extensible.
• Workflow presentation should be skin-able with institution-specific stylesheets
• Need some way to selectively show/hide workflow (by date or by choice); this will show/hide workflow but workflow collection content remains available
• Should we automate moving students from one page to the next when submitted? For example, Upon submitting page one, page two is not only open, but it is (optionally) displayed?
• How can we include attributes such as due-dates and grades? Are the due dates for content within the page (e.g. assignment), or for the page itself? If the content has an associated due date, there's the possibility of more than one due date, and also the need for a sakai-wide due-date attribute. If the page itself is considered a type of assignment, with due dates and grading associated with it, we would have to integrate with a broader sakai assignment services.

Reflect

Summary

Portfolios and workflows will leverage two different types of reflections in Sakai 3.0:

• Structured Reflection - form based structured content within the context of a workflow
• Unstructured Reflection - free-form text which may be attached to any resource or artifact outside the context of portfolio tools (e.g. reflect anywhere)

User Stories

• Medical Student needs to reflect on adverse events by filling out a structured form within a workflow page
• Business Student uploads a document describing his summer internship responsibilities, and wishes to add some unstructured commentary/reflection for later use

Functional Details

• The user interface for creating both structured and unstructured reflections must be intuitive and simple. The ease of use will have a strong impact on the effectiveness of students creating, finding, and reusing reflections in their portfolios and other learning areas.

Diagrams & Mockups

-

Implementation Details

• Unstructured reflections may leverage a universal "relates-to" property implemented for all Sakai content.
• Long term implementation may include personal student collections of content which allow a (one-to-many) unstructured reflection (perhaps in the form of a portfolio or personal workflow?)
• Reflections perhaps could show up as a blog view? Alternately show a reflect icon next to all content with an attached reflection
• How to implement structured rich-text reflections -- especially when integrated with structured form widgets (rich text widget within a rich-text page doesn't make much sense). Note same problem as noted for structured rich-text feedback and evaluations.

Collect & Associate & Tag

Summary

Content may be selected by users, instructors or administrators to be collected or associated within a workflow or portfolio presentation:

• Portfolio presentations allow selective collection where students select content to be included in their presentation based on suggestions in the presentation template.
• Workflows allow selective collection where students select content to be included on a workflow page (e.g. to demonstrate a certain competency)
• Workflows allow binding collection where instructors or administrators bind a required piece of content to a workflow page (e.g. assignment submission that automatically is displayed on a competency page)
• Tagging allows associating content with pages in a workflow (e.g. competencies) or associating content with other content

User Stories

• Jane Student uploads a document describing her summer internship and adds the global tag "Example of Work". Later that year, she is using a Workflow that has a page requesting examples of work, she easily finds her summer internship document by searching all documents tagged as "Example of Work".

• An administrator would like to connect/associate Assignment #4 as meeting Competency #2. He creates a Competency Workflow that includes a page for Competency #2 and creates a corresponding "Competency #2" tag. He also tags the assignment with the "Competency #2" tag. When a student submits Assignment #4, it will appear in the Competency #2 page of the student's Competency Workflow.

• An instructor would like to connect/associate Assignment #4 with his blog post on Japanese Art history and also with a website about woodblocks. He creates a global "Japanese Art History" tag and associates it with the assignment, the blog post, and the URL of the website.

Functional Details

Definitions

• Artifact: a single piece of content (e.g. form, assignment, uploaded-file, external url)
• Semantic Tag: tag defined by authorized users for identifying content
• Collection: student/user explicitly creates artifacts within a defined workflow or attaches artifacts to a page within a workflow
• Association: student/user creates an artifact outside of a workflow, but uses a defined semantic tag to associate the artifact with a workflow page

Goals

• Program staff (faculty or coordinator) can create outcome/goal sets to capture learning objectives.
• Learners can review the outcomes relevant to their course and portfolio work artifacts
• Learners can associate artifacts (content, artifacts, files, assignments) with outcomes to demonstrate proficiency.
• Program staff can create expectations (Guide) specific to given materials (structured data/forms).
• Learners can freely associate artifacts with goals or outcomes outside of the structured expectations.
• Learners can collect or associate artifacts created in Sakai 2 or Sakai 3 for their workflows and portfolio presentations.
• Artifacts (content) must include different media types (e.g. pdf, video, audio, structured form content, ...)
• Artifacts (content) must be easily re-used between portfolios

Tagging

Semantic Tags facilitate associating content with each other, or with competencies, objectives or outcomes. All tagging usage described here assumes that semantic tags are defined by authorized users for use by all users.

Authorized users create Semantic tags within the context of a workflow (e.g within the 'Public Health' workflow, a 'Leadership' tag may be created and associated with one or more pages). Semantic tags may be applied to any content. Tags may be promoted to global tags by authorized users. Tags are created with a context (optional), name, and description (tags without a context are considered global). Content may be discovered by searching by tag.

Diagrams & Mockups

-

Implementation Details

• Leveraging tagging to create an "intentional repository"

Evaluate & Feedback

Summary

Portfolio evaluation and feedback may be solicited, iterative, open to specified group(s), or incorporated into a workflow, and could be either structured or unstructured.

User Stories

Presentation Feedback & Evaluation

• Jane Student has completed one page of her portfolio presentation, and would like feedback from her instructor as to whether or not it is on track. She selects feedback is requested.
• Joe Instructor receives an email notificationand review's Jane's work. She seems to need a  lot of guidance, so he provides inline comments explaining where she needs more work.
• Jane Student receives an email notification, reads the inline feedback, and proceeds to complete her work. She marks the work as locked and completed.
• Joe Instructor receives an email notification and reviews Jane's work. It is much improved, and adds an Unstructured Global Evaluation before marking his evaluation is complete.
• Fred Advisor receives an email notification and reviews Janes work and Joe's evaluation comments. He fills out a Structured Evaluation form and then marks his evaluation as complete.
• Jane Student now receives notification that her work has been evaluated. She reads both evaluations.

Functional Details

Both Feedback and Evaluations can be defined as one of the following types:

• Structured Feedback/Evaluation
• Unstructured Global Feedback/Evaluation
• Unstructured Inline Feedback/Evaluation

Structured Feedback/Evaluation leverages a Structured Data Widget (e.g. form) as it's basic building block (also used in presentation and workflow pages).

Unstructured Global Feedback/Evaluation leverages a framework as the "Reflect anywhere" , using a "relates-to" attribute attachable to all Sakai objects.

Unstructured Inline Feedback/Evaluation may leverage a framework such as is used by Etherpad.

All feedback and evaluations will employ an optional notification mechanism (e.g. email) to alert students of new feedback/evaluations and to alert evaluations of content that is ready for evaluation.

Evaluations may optionally be "blind", where the evaluator does not know the student's identity.

Evaluations may optionally be "hidden", where the student cannot read the evaluation until released to the student.

Multiple Evaluations may optionally be configured, where one evaluation is handed off to another evaluator (and so on), until the last evaluator completes the evaluation process.

Configuring Evaluation and Feedback decision trees must balance flexibility against undo complexity. One approach would be to define a Evaluation and Feedback "Template" with presets for the following options:

• Feedback Type(s): [Structured | Unstructured-Global | Unstructured-Inline ]
• Evaluation Type(s): [Structured | Unstructured-Global | Unstructured-Inline ]
• Student Notification of Feedback: [ opt-in | opt-out ] [ email | SMS | ... ]
• Evaluator Notification of Content: [ opt-in | opt-out ] [ email | SMS | ...]
• Student option to solicit feedback: [ from none | from all | from specified groups ]
• Student option to receive feedback: [ from none | from all | from specified groups ]
• Blind Evaluation: [ true | false ]
• Hidden Evaluation: [ true | false ]
• Evaluations: [ from none | from specified user | from specified groups | multiple (ordered list) ]

Diagrams & Mockups

This diagram describes the workflow for evaluations and feedback:

Implementation Details

• How to implement structured rich-text feedback and evaluations -- especially when integrated with structured form widgets (rich text widget within a rich-text page doesn't make much sense). Note same problem as noted for structured rich-text feedback and reflections.