Child pages
  • Promote-2.5-Tests & Quizzes
Skip to end of metadata
Go to start of metadata

See the Comment section at the bottom of this page for discussion on this tool's promotion.

Core Criteria Matrix

The table below is derived from the Criteria for Supported Status document and is intended as a quick-reference guide on how this tool measures up to the requirements for a Core tool.

Key:
(tick) - Criteria met.
(warning) - Work needs to be done to meet criteria.
(error) - Will not meet criteria by code freeze.
N/A - Not Applicable.

Community Support Criteria

Tests & Quizzes (a.k.a. Samigo)

There must be an identified group of committers within the Sakai community who take responsibility for the tool. This group can be comprised of the original author(s) or others from the community willing to take responsibility for the application.

(tick)
(Stanford & others TBD)

The application source code must reside in Sakai's source control system. Organization and management of the source control system is defined in a different Sakai Community Practice (to be determined).

(tick)

At least two Sakai production sites must be utilizing the tool successfully in their production environments; please list the sites. Those sites should be able to report that the tool runs properly and is useful to their users. These sites must also report that the application does not cause any problems with other parts of the Sakai release such as poor performance, memory leaks, etc.

(tick)
(Univ. of Cape Town, Claremont Univ. Consortium, Arizona State Univ./IDEAL, Stanford, all...)

The developer(s) of the tool must be committed to maintaining the application across Sakai version changes including any necessary data conversion for their elements of Sakai if storage structure changes.

(tick)

The developer(s) of the tool must be willing to answer questions about their application on Sakai dev list.

(tick)

The developer(s) of the tool must commit to helping to develop test plans and specifications for the application. Until those test plans are in place, the author(s) will commit to join the Sakai release/QA process and perform the necessary QA on their application. To become an official tool, the author(s) of the tool must contribute test plans and specifications (to satisfy the requirements of the QA group).

(tick)
(current set)

Bugs and feature requests for the tool must be tracked in Sakai's bug tracking system.

(tick)

Technical Elements Criteria

Tests & Quizzes (a.k.a. Samigo)

If the tool persists data to a database, it must support all official Sakai databases (currently HSQL, MySql, and Oracle).

(tick)

The tool must work properly with the Sakai AutoDDL approach. The application must properly create tables when tables do not exist and the tables must be named so as not to conflict with other Sakai tables.

(tick)

All database access to tables that the tool does not own will take place via published Sakai APIs provided by the application that manages those tables. The tool should use APIs for access to its own data, but it must use APIs for access to data from others.

(tick)

The tool must participate in the system-wide configuration (sakai.properties) and not require any local configuration to be hand-edited.

(tick)

The tool must properly operate in the Sakai Authorization and tool placement structure. It must either use existing appropriate security functions or introduce new security functions for the application.

(tick)

The tool will not require patches to other Sakai tools or to the Sakai framework. A application may require changes to other areas of Sakai, but those changes should be negotiated ahead of time and should be part of the full distribution.

(tick)

The tool must fully work in a clustered Sakai application server environment.

(tick)

The tool cannot force new jars into shared/lib or common/lib. Sakai's goal is to keep these jar footprints as small as possible. Any new jar requirement in those areas requires significant discussion.

(tick)

There are a number of system-wide elements in Sakai including Spring, Hibernate, and others--the application must work with the versions of these elements that are part of the Sakai release.

(tick)

Licensing must be clean before the application is moved into Sakai's source control system and put into a Sakai distribution. The license must not violate the policies set by the Sakai Foundation. This also means that a tool cannot require other application or jar with a proprietary or ECL-incompatible license to be part of the distribution.

(tick)

Interaction and Visual Design Criteria

Tests & Quizzes (a.k.a. Samigo)

The tool UI must look like the rest of the Sakai application and properly inherit skins from Sakai (i.e. when the sites color changes the tool colors change as well). The tool should not look "out of place" amongst other Sakai tools.

(tick)

The tool should follow general interaction guidelines outlined in the Sakai Style Guide (SG) so users have consistent experiences and expectations about how to complete actions such as "paging in a list", "navigating between pages", "taking action of items in a list", etc across tools. The SG is a guide and should be used to help inform interaction decisions but is not meant to have all the answers.

(tick)

UI components available in the Sakai library should be used where possible (e.g. wysiwyg editor, calendar, paging widget).

(tick)

The application must support all of the browsers currently supported by Sakai including FireFox/Mozilla and Internet Explorer.

(tick)

The tool should provide basic help which integrates seamlessly with the Sakai Help system. The help should have a look and feel and writing style similar to the other elements of help.

(tick)

Desirable Elements Criteria

Tests & Quizzes (a.k.a. Samigo)

The tool is properly internationalized with all of its user-displayed strings derived from properties files.

(tick)

The tool should be fully accessible and pass a Sakai accessibility review.

(tick)
(needs a new review, last review was 2.0)

If the tool has any persistence or business rule code it should be factored into a Sakai Component with a published API which has complete javaDoc.

(warning)
(javadoc can be improved)

This component should be designed so as to allow other applications and/or system processes to be able to work with the business objects handled by the application.

(tick)

Creation of web services API for the tool. The authors should provide a web-service layer which sits on top of their API.

(error)
(did create proof-of-concept for alti-lab demo in 2005)

The tool should participate in the Sakai cross-site import and export if it persists business objects.

(tick)

The tool should be inter-operable with other Sakai tools where appropriate.

(tick)
(currently gradebook, and site import/export)

The tool should generate event codes that are triggered minimally on new, revise, and delete actions on the basic objects created by the tool.

(warning)
(filed as SAK-11137)

Hibernate is not required but if the tool is doing significant ORM, it should be done using Hibernate - the application should work with the current version of Hibernate used in the Sakai release.

(tick)

In order for Sakai to seen to be a seamless application, effort needs to be made to minimize overlap in functionality with other Sakai "bundled" tools. At times, as tools are evolving, or complete replacement tools are being produced, overlap is acceptable. Where overlap will happen between tools under consideration, developers should first look to fixing/improving the existing tools rather than simply writing a new tool with a few features. Each tool which adds overlap may result in the long term need to "clean up" the overlap to maintain a good user experience.

(tick)

Notification and Process Criteria

Tests & Quizzes (a.k.a. Samigo)

30-day notification prior to code freeze of the following

 

What it is

(tick) homepage

How to install/configure it

(tick) 2.4 release notes

Known issues

(tick)
see release notes

Whom to contact

(tick)
see homepage

  • No labels

8 Comments

  1. After several discussions with people involved in T&Q and other assessment projects related to Sakai, it is my believe that Tests and Quizzes (SAMigo) should be returned to fully supported (core) status in Sakai.

    Beyond the outstanding work that the T&Q team did (esp. the folks at Stanford), I'm reminded that the "SAM" in SAMigo stands for Sakai Assessment Manager. Personally, I think it's essential that Sakai have a fully integrated assessment manager. SAMigo, derrived from it's predecessor, Navigo, IS that assessment engine. There are no other candidates at this time for a Sakai assessment manager. T&Q meets or exceeds the requirements for a core tool.

    Naturally, I also think that more work is needed to improve T&Q. I've been in coversation with some of the people involved to create a body of Sakai assessment requirements. Initial work will be explanded to produce a set of requirements that can be used to drive future enhancements, features, and requirements.

    Meanwhile, it's important for the Sakai community to acknowledge that T&Q works and is reliable for most use cases.

    +1 to make T&Q a Fully Supported Tool.

  2. On behalf of some of our 90 clients running Sakai, I'll advocate as well for Samigo to move to core status. I'lve polled some of our clients (Brevard, Kentucky Christian, Ohio Valley in the Appalachian College Association consortium and Walsh and others) and get a consistent response that Samigo is working in production, useful and meeting needs. Some of our client colleges have undertaken training programs for other colleges in their regions; there is much interest. Nearly all of our clients are using T&Q routinely, so I don't see how we could categorize this other than core.

    Improvements needed? Let's hope that's ALWAYS the case because it reveals that we have a dynamic and demanding community.

    Core status for Tests and Quizzes: yes, absolutely.

  3. There is a lot to like about T&Q and it would be difficult to argue that T&Q should not be considered a core tool... So I won't really try to do that. We (UC Davis) will likely stealth it after having run it in pilot for year. That is primarily due to performance issues, but in part, to make sure we manage the usage a little... it needs some 'managing' w.r.t faculty, IMO.

    My only real concern is with this requirement for promotion: 'These sites must also report that the application does not cause any problems with other parts of the Sakai release such as poor performance, memory leaks, etc.' I think T&Q needs more attention to this.

    Strictly speaking Sakai would be pretty thin if this were adhered to with any rigour (smile) But T&Q has some serious battles waging between hibernate and JSF that could be resolved to make this less of an issue. In the end, I don't like the way the code looks very much, but that may have little to do with it. If SiteAction were at issue, it would fail on those grounds.

    So, since it's been pretty much core anyway...and, while it's not perfect, it has effectively met all the requirements ... and is the best showing in it's space... and more attention is exactly what making it core would give it... and, even though we will probably stealth it immediately...

    +1

  4. We've been running Samigo for years now for both ASU and IDEAL, back around when Samigo was demoted to provisional status the vast majority of our support issues where from Samigo, it was a VERY painful time, however those days are now thankfully long since past, with every release since then the tool has become increasingly stable and performance has improved considerably while a number of features that our faculty and faculty support staff see as absolute must haves have been added. There is of course still plenty of work to be done to Samigo on several fronts, but the same can be said for much of core Sakai.

    In the context of a course setting we don't see a system like Sakai being very practical with out a feature rich fully integrated assessment manager and we've never considered running with out one, not to say that we won't consider other options if they fulfill all our needs better some day in the future but as it stands now Samigo should be the core Sakai assessment manager.

    +1

  5. Samigo has been an integral part of the rSmart Sakai CLE since our initial release a year and a half ago. A majority of schools we support make use of it in their production instances. While it has not been devoid of issues in the past, we believe that major progress has been made to improve both it's functionality, and it's ability to run in production without major implications for performance, memory leaks, etc, within Sakai. Over the years, we've also been EXTREMELY impressed with the responsiveness of the Samigo team in addressing these in both a thoughtful and expeditious manner.
    We would, therefore, like to add our voice to those calling for promotion of Samigo to core.

  6. UCT supports the promotion of T&Q to core. T&Q no longer presents significant support issues for us, and is a stable part of our production system.

  7. I support putting it in core, largely because we can't really have a CMS without tests and quizzes. New sites are going to find it confusing not to have this in the default configuration.

    A lot of work has gone into Samigo in the last year, and it shows. However I need to register a concern. If you asked faculty that are preparing complex tests, they would say that Sakai as a whole is not ready for production use, because of problems with Samigo. We see two types of issues:

    • assessment and tool UI's do not have the necessary facilities to manage tests that use large pools of questions. This is a very common situation for intro courses. We've added code, and I believe it will be in 2.5, but his is based on experience with a single team of faculty. As we get more users on in late August, we're hoping that they'll find everything they need.
    • we're seeing bugs at a higher rate than in other parts of Sakai. The same faculty member has run into two bugs so far that turned out to be minor, but would be show-stoppers if we weren't able to look at the code and figure out what is triggering it.
  8. Here in Claremont McKenna College, we vote for promoting the Tests and Quizzes into core status. We were WebCT campus. We believe that any system used as the LMS should have a robust and fully Q-n-Aed online test and quiz tool as one of the core components.

    We have been using Samigo since our Sakai went to production in Fall 06. We rolled out a trial to allow our freshmen to take the Math placement exams in Sakai this Summer. The process has been encouraging. We plan to do it again in future. To us, Samigo is one of the most critical tools in Sakai.

    Though we do encounter some bugs and user experience issues while using Samigo, we also found that the Samigo team has been extremely helpful.

    BTW, I have polled the other members of Claremont Colleges consortium at one of our Sakai Admin Team meeting. The other Claremont Colleges members voted the same way unanimously.