Post Sakai Conference OSP Meeting - Sakai Newport Beach 12-7-07
Conveners: Melissa Peet and Janice Smith
Scribe: Janice Smith
The meeting began with a two-hour session on gathering newcomer needs and determining how to divide up the rest of the day into small working groups. The documentation conversation took place in the whole group.
Before lunch there was one breakout of two groups: Enhancements to Goal Management and the OSP Functional Team meeting on proposed procedure for documenting and approving new OSP enhancements.
After lunch there were two more sets of break-outs with two groups each. The first breakout covered ways to facilitate entry into the OSP community and a discussion of the review of the OSP Matrix user interface. The second breakout covered an improved user interface for OSP tools and enhancements for better feedback and evaluation processes.
Notes from each discussion are as follows.
OSP Newcomer Needs
- More information on how to use OSP tools
Creating and using forms for the matrix and portfolio templates
Designing and using the matrix
Using matrix data in a portfolio template
Documentation on tying the OSP authoring process together
Quick tips on how to do things in OSP
- Need OSP training day at future Sakai Conference or during extra day at LaGuardia Conference - sooner rather than later
- Improve Sakai Confluence by:
Improving methods for finding OSP content
Clarifing where OSP information is located
Rewriting the OSP Whitepaper
Providing information on how to contribute documents to OSP Help
- Improve the OSP Library by:
Providing a newcomer page with links to important resources, websites, and FAQs
Including an OSP community roster with contact information and areas of expertise for all participants in the OSP community. (This list would also be useful to call up anytime major decisions need to be made in the community.)
Providing OSP authoring tutorials
- Provide information in the OSP Library on
How institutions are using OSP
How pedagogy can determine ways to use OSP
How companies can support OSP
How institutions can get started with OSP
How to catch up on OSP activities to date
How to understand where OSP documentation is in relation to the current release
Recap of participant experiences at the Sakai Conferences
OSP 2.4 use cases for supporting assessment and accreditation
OSP 2.4 use cases for supporting life long learning
Additional OSP use cases
- Clarify the tension between portfolios for assessment and portfolios for personal expression
- Better defining structure for providing information on OSP
- Clarify entry points into the OSP community - where and how to contribute
- Find more ways to publicize the Monday OSP call and the phone number to use for the call.
- Write an introductory OSP article for Educause Quarterly
- Provide a mentoring program
- Develop leadership in the OSP community in key areas
- Once the process for creating and approving new OSP requirements is approved, publicize it and follow it. Train newcomers in using it.
- Help newcomers understand the history of OSP, how weaker areas continue to be strengthened, and how they can participate in the process of improvement.
- Network with other Sakai groups
Teaching and Learning (Pedagogy)
Sakai Functional Leadership
- Lower barriers to ease of using OSP by providing tools like a portfolio template builder or for now come up with less painful ways to create portfolio templates, portfolio layouts, and styles.
- Provide better usability for matrices. A possible way to do this is to integrate the roster tool with the matrix.
- Provide more options for editing a published matrix that is already in use.
- Improve OSP tool usability for better workflow.
- Create a visioning group to integrate portfolio processes for assessment and personal expression
Summaries of Small Group Work
Proposed OSP Refinements Gathering and Approval Process
The OSP Functional Team met to refine the following two documents to be voted on for approval during the OSP community call at 11 AM EST on Monday, December 17 (812-856-7060, 348#, 72524#).
Proposed Procedure for Documenting and Approving OSP Requirements
Proposed OSP Enhancements Template
- John Bolte of IU participated in the meeting representing Documentation for IU and for the Sakai community.
- The Documentation office manages information about the use of Sakai, keeps the information current, identifies what is needed, and collects the information for incorporation into the Sakai Knowledge Base and Help
- The Documentation office does not produce manuals. They work with smaller segments of crucial information.
- Information for the Sakai Knowledge Base (https://www.indiana.edu/~sakaikb/) should be emailed to KB@iu.edu.
- Also email the Sakai Knowledge base to indicate what information is needed.
- The OSP Library could also collect documentation in the form of manuals, knowledge base entries, or tutorials. It could also collect information that can eventually be added to Help for OSP tools.
- There is a potential to use Web Services and Resources for similar purposes.
- There is a need to remedy communication gaps between the OSP community and this office. IU manages Help and the OSP community wants to provide and continue to manage OSP Help.
- Is there a need for an OSP group on documentation?
- There is a need to determine when and how OSP documentation gets sent to the Knowledge Management Team at the Documentation office. Currently OSP documentation available to the community is significantly behind the current release.
Proposed Enhancements to Goal Management
- See IU 2.6 Goal Management Requirements at http://bugs.sakaiproject.org/confluence/display/OSP/Indiana+University+-+Future+Directions+and+Priorities
- How will the goal management tools be moved out of Contrib into Provisional and Core?
- There is a need to continue support of goal management beyond the funding contributed by Syracuse University. Will the IU improvements to the tools do this or is more support needed?
- There is a need for more inter-institutional exchange on ways of using goal management.
- Integration with OSP so that all tools interact cohesively
- Extractor for reporting to institutional portfolio
- Defined groupings of goals for assignment to instructors charged with having their students meet those goals
- Auto recording of student work in My Workspace similar to the portfolio interaction folder for the matrix
- Rubrics (holistic or collection of criteria to be met by one assessment)
- Multiple evaluators for goal management (similar to the matrix)
- Options for providing feedback as well as evaluation
- Portability across institutions (according to IMS standards)
- Clarify how goal management differs from using a matrix or wizard in more standard ways
Improving Information for Newcomer Entry into OSP
- Better and greater amount of information to newcomers
- Improve quality and quantity of information from authoritative sources on osportfolio.org
- Provide pedagogical rationale behind ePortfolios
- Provide sample data structures
- Glossary of tools
- Information on what can be done with OSP
- Reflection and evaluation guidelines
- How to plan for a portfolio implementation
- How to get involved in the community, including links
- Make information already on Sakai Confluence more easily available and add to it
- Tutorials for using OSP
- Why would people visit the site?
To learn about portfolios
To provide examples
To provide information on the project
To suggest ways that Sakai/OSP can relate to the larger community
Improved User Interface for the Matrix
- See Sean Keesler's review of the matrix UI at http://bugs.sakaiproject.org/confluence/display/OSP/OSP.2_5.UIReview.Manage+Matrices
- Rationale: Better grouping of processes, rewording instructions, etc.
- Group identified a few ways to modify Sean's recommendations, but some of this work has already been done. Should it be changed again?
- Very involved process - difficult to pull people in to vet the process
Feedback and Evaluation
- A feedback dashboard should be constructed to provide additional functionality and ease of use.
- Forms to collect and document evidence for a matrix cell or wizard page should allow faculty to view the collection/evidence form at the same time that they fill in a feedback or evaluation form.
- Artifacts need to be visible on the screen when providing feedback and evaluation.
- Comments on the feedback and evaluation forms should be keyed to specific fields or groups of fields on the collection/evidence form along with a summary field at the close of the form to tie the feedback or evaluation all together.
- There should be multiple evaluation roles to cover options for different types of evaluators including peers.
- There should be a way to specify a particular evaluator for the work of a particular site participant in a matrix cell or wizard page.
- Evaluations and Feedback should be group aware.
- The provider of feedback or evaluation should be able to revise it.
- The owner of a matrix cell or wizard page should be notified by email when they receive feedback or evaluation.
- Owners of matrix cells or wizard pages should be able share them like portfolios in order to ask for feedback or evaluation from specific evaluators. This is especially needed in order to facilitate peer review processes.
- Anyone with permission should be able to track recent changes to participant and evaluator activity in a matrix or a wizard.
- Viewers of shared portfolios (via templates or freeform) should be able to highlight areas of the portfolio to mark it up with comments.
- Future improvements may include a method for evaluating inter-rater reliability and for doing full text searching as can be done with In Vivo or HyperResearch.
- Reports need to continue to be improved in order to get evaluation results and evaluated artifacts out of the matrix and wizard
Ideas for an Improved OSP User Interface
During the UI breakout session, we couldn't decide on a "general UI" topic, but we were able to identify that the freeform portfolio tools have been really neglected for the past couple years. The UI exposed to users is somewhat functional but awkward. We thought that the freeform portfolio would be a great place to implement DHTML interfaces in concert with the Fluid group. We discussed a couple of other rich authoring environments.
Consistent with previous discussions, there is a need to present a simple, unified interface to the user that brings together the CSS style tool, the XHTML page Layout tool, the ability to author html/rich text and incorporate assets (such as structured data and images) in a simple, intuitive UI. We discussed that there may be possibilities to work with Fluid to accomplish this with UI features such as the ability to drag and drop assets onto pages and dynamic creation of Page Layouts. A freeform portfolio layout editor might be thought of as a precursor to a template builder.
The discussion expanded beyond UI issues and into new functionality that we believed would be expected of the freeform portfolio tools. Providing a portfolio owner with the ability to invite others to collaboratively authoring a portfolio is functionality that would be useful.
Darren Cambridge noted that many of the items we talked about were in the 2.1 requirements document and that we should go back and read it again before going forward.