Every year IU's IT organization, UITS, does a user survey. They use some kind of sampling method, so the survey isn't sent out to everyone. The results are all available on the web. I don't personally have any involvement in the survey except to read the results with great interest every year.
What I've done here is to cull the Sakai-related text comments from both the IU-Bloomington survey and the IUPUI survey. Note that at IU, Sakai is branded Oncourse CL. When people refer to the "old Oncourse" or "original oncourse", they are talking about our former homegrown CMS that preceded our adoption of Sakai. Currently, both campuses are running both systems in parallel, while phasing out the original Oncourse.
As this survey is done every year, it is possible to compare last year's satisfaction ratings with the two prior years'. Satisfaction ratings are on a five-point scale, with 1="not at all satisfied" and 5="very satisfied". 2005 would be pre-Sakai, 2006 initial Sakai+legacy, 2007 more Sakai+legacy.
Satisfaction rating at IU-Bloomington with Oncourse (overall w/Instructional & Computing Services):
2005: 3.90 (4.04)
2006: 3.39 (4.02)
2007: 3.71 (4.16)
and at IUPUI:
2005: 4.15 (4.12)
2006: 3.62 (4.01)
2007: 3.55 (4.08)
In sending out news of the survey results, Garland Elmore cautions, "No single comment should be viewed as authoritative in isolation, but any opinion expressed by those we serve can point us to opportunities to improve."
I would love to see similar survey results from other Sakai institutions.
Best,
Mark
--
Mark Notess
Development Manager / Usability Specialist
Digital Library Program / UITS
Indiana University
Bloomington, Indiana 47405
812.856.0494 (w)
mnotess@indiana.edu
2007 Survey Comments that Mention Oncourse
IUB Faculty Comments:
The Oncourse CL system needs to be made cleaner and less cumbersome for students to use, especially those using it at a distance. I can't tell you how many problems I've had with using it as a distance ed tool. My IU students are resigned to it, but don't like it. My non-IU students keep asking me why we don't use another, more user friendly system and they, quite honestly, loathe it. Instead of moving toward greater transparency, where the system becomes more intuitive in its design, Oncourse seems to become more clumsy, with far too many screens for students to click through to get to where they need to be. I find the system fatiguing to work in--something I never thought I'd say, after over a decade of web-based instruction, about the technology.
Oncourse CL is in need of signnificant improvement. The interface is confusing and the usability is poor. I am an expert in usability, and have observed numerous problems and complaints about CL. The worst part of CL is the Discussion tool (part of Message Center ). It is awful, and impairs our distance students who MUST have a good discussion tool. The interface to the discussion tool is literally baffling to DOCTORAL and MASTERS students in order to do simple things such as find a post, edit it (impossible), or make an attachment to a post. The Oncourse CL discussion tool was so bad that students preferred to use Discus Pro in my class last summer. The discussion tool actually has a serious bug. Sometimes the bottom part of the display fails to show, and so you don't know what to do. If you click refresh, then it will "suddenly" appear. But why do you need to click refresh to make it work at all? But in general the discussion tool is simply awful. Simply awful. Really bad. Not OK. Hello? Fix it!!! The CL interface is very difficult and next to impossible to use in 800x600 resolution on laptops with small screens. You really need a 17-inch or higher display in 1024x768 to read and navigate CL. The real problem is the windows-within-windows concept that forces awkward layouts and excessive scrolling within small windows -- Onestart is similarly impaired. Those programmers with their 21-inch monitors probably don't notice. But the thousands of users who have smaller display devices that must be run in lower resolutions in order to simply READ the screen are screwed. Make Oncourse work on my cell phone, and then you'll get it right--it needs to scale up and down for different size displays. For a humorous, embarrassing and sad example of the problems created by this interface design, see: [IRD] On a positive note, the ability to form projects in CL is excellent. This has been very helpful for creating research groups.
I might be useful for UITS to poll faculty on issues such as voice mail or OnCourse to determine desired features instead of going to a new system and telling the faculty that this is what they will now use. The first version of new oncourse was not useful, yet UITS told us all that we had to go with it (and then opt out). ... Bottom line, get input before telling the campus to change, and then solicit input and consider it after making a major change.
As indicated above, the one real problem that I have encountered is the new oncourse system. Last I tried, there were still serious problems for example with the gradebook and it's also extremely user-unfriendly. I even took a class and still have trouble managing some functions. The old system was just perfectly fine. Any chance we can go back to some of the old features? (Sorry to be so defaitist.)
The continuous changes in Oncourse are very annoying. I prefer to have a system which is not perfect, but I can handle.
I have called into 855-6789 (during the day) with an OnCourse question and have received a different response each time I call or have a student call (with the same question). There needs to be consistent help given each time.
There should be surveys specific to oncourse. I believe there is a large group that dislike oncourse cl. Additionally I know of major flaws and lack of functionality but rarely are end-users asked, and when asked, no one seems to be listening. I know of people that will be developing alternatives to Oncourse CL if the "original" oncourse is shut down. Other than Oncourse and PeopleSoft - UITS has been great. The people I talk with are the best.
Oncourse CL is an improvement over original Oncourse, but still has odd glitches and frustrating limitations. When I tried to report three of them, I received a response that addressed one of the three inadequately, and didn't even mention the other two.
Oncourse CL is good but can still be improved.
entering grades in onestart is a total joke. I would enter a few grades, click save, and it would do some updating and erase some of my grades. I tried to capture a movie of this happening, but the movie didn't save for some reason. If it had, it would have been a classic. This interface is BROKEN. At least oncourse cl works (and if oncourse is your fallback, you are really temping fate). Otherwise I couldn't be more pleased with support here at IU.
1. Oncourse CL is the bane of my teaching life because it promises so much and then routinely fails to deliver. It has improved since it was first introduced, but the improvements have come too slowly, and failure is still routine. A favorite example: I used Oncourse-CL to send a time-sensitive message to my class about an assignment. How clever that I can post an announcement, and email my students at the same time! Unfortunately, the email was not received for over 18 hours. Students who did not by chance look at the Oncourse website missed the info.
2. Oncourse CL is far too limited as a research environment. It is very much a step down from previous centralized storage facilities because I cannot directly access data stored in CL. I have to download it first. Extra step, extra time.
Need a class or tutorial in how to best use Onestart/Oncourse to enhance our teaching and would like to find a tutorial or course for those of us not as computer savvy to help us create a faculty webpage.
IUPUI Faculty Comments:
I have continued to use Oncourse Original because the functionalities were not present in CL. Original should not be shut down until CL has at least all the functionality of Original. And I do understand that Original is based on an older system that had to be replaced.
But I will say over the last year I have valued UITS services more than CNC, perhaps because I use Oncourse CL and the knowledge base so much.
Need more training and input into Oncourse and continual changes
It would be most helpful if Oncourse could be more user-friendly and that faculty could have input into development of Oncourse.This was difficult to answer because in the past we were informed that Oncourse was NOT part of the UITS, but now it seems that it is. I am very satisfied with all parts of the technology services BUT the new Oncourse. ... However, the new Oncourse is dreadful. Clearly, the designers never thought to have a group of students or faculty members try things before it was dumped on us. After reading about the new components on oncourse,(primarily the ability to add comments as well as a grade-evidence that those who designed this do not teachimagine giving a student a grade but no feedback on what he/she did wrong)-I tried it again this semester. Unfortunately, I just learned that when students send completed work from oncourse and I receive notice of this on my email account (a good thing), this does not mean that the work is available on the oncourse site. Since, the only redeaming value of Oncourse is that it keeps a record of what students and faculty have sent and received (without cluttering your personal mailbox),why on earth would the designers of oncourse have a way that students can send things from oncourse, and you have to go on to oncourse to reply, yet there is no record of what was sent? I am not able to figure out if this occurs when sent as a private message or email messages since both appear on my account when I have tried this. Unfortunately, it is now to late to switch. However, I have learned my lesson. Last semester I did everything via email and it worked like a charm. From now on, I will only use oncourse for turning in final grades.
OnCourse CL is still clunky and SDIS though a pwerful tool does not (easily) meet its potential. As a faculty member I often feel like the UITS folks do not care for my input (for example it took a major revolt to get the attention of the CL designers to fix things like e-mail, grade books etc. and there are still problems with the system) Informational technology should be designed to enhance learning not impeed it.
Oncourse CL is a sink of money with very little beneficial return. How much more is going to be put into software that has yet to reach the levels of the software it was designed to replace after years and so much money being spent. The SIS system is horrible and another example that someone is making choices with the university's technology that has no idea what the everyday user is dealing with.
IUB Staff Comments:
OnCourse CL is a joke. It's two steps backwards after *years* of development.
Fix OnCourse so that it makes sense and is usable.
IUPUI Staff Comments:
OnCourse-hard to evaluate as it a constant work in progress and needs additional usability features and improved stability;
Other than the overall Oncourse CL movement, I think UITS is doing a great job dealing with numerous and complicated services. The course management system transfer to a new environment decision setback UITS in terms of faculty views and trust. However I think that in time, that will be repaired and hopefully it will be business as usual again.
Oncourse CL has a LONG way to go to be the same as the Original Oncourse. It is so slow to bring up information when a link is clicked. I do like the feature though, that allows one to export a file to Excel with ease.
Improve discussion forum functionality in Oncourse CL. The current tool is in the dark ages when compared to commercial and open source discussion boards. Add search capabilities to every tool in CL, but especially message center, resources, announcements, assignments, and calendar.
IUB Graduate Student Comments:
oncourse is okay at best.
I can only copy folders of files (not individual files) into Oncourse Resources via WebDAV from my Mac without getting an error. What's up with that?! I really don't like ONCOURSE resources--the old locker system was much easier and FASTER to access and use.
oncourse self services are less than intuitive. the bursar page would be nice if it could show line-item costs. (this, i understand, is not your problem). suffice to say, billing and the oncourse experience could be more clear.for instructors, oncourse pages should NOT expire automatically. send an email asking which pages instructors would like to maintain. you could limit overall space, but let instructors choose how that space is allocated in terms of "archived" class pages.
Last I checked, the new Oncourse did not allow gradebooks to specify assignments by percentage of total grade; it only allowed you to do it as a function of total points. This is a complaint that many of my colleagues and I share, and it is the reason we stay with the original Oncourse.
Oncourse is complicated and confusing. No one seems to use it in the same way so it is difficult to know where to look for or post information. Some classes allow you to post and others don't. Some professors post files through Resources and others use email attachments. Most frustrating, emails sent through Oncourse do not let you reply to the group. It would seem one has to log in and find the correct section through which to post a reply... only sometimes you can't post as a student. I find it limits online participation more than it enables it. (Although I do like having a "resources" section where files can be posted and retrieved. I just wish it were easier for users to communicate.)
Professors need to be educated in how to use Oncourse CL. It has been a major source of frustration. The calendar feature is great... if everyone would use it.
- The Oncourse system is *horrible*. Extremely user-unfriendly and unclear. The old oncourse (which some of my instructors still use) is significantly clearer and thus better.
Oncourse is really inferior even for purposes of basic group work; students I know have resorted instead to Yahoo! Groups and Facebook. Oncourse discussion areas are also very difficult to navigate and use efficiently.
What interaction I have had with Oncourse CL has been terrible, but I don't have to use it very often or the rating would likely be lower.
IUPUI Graduate Student Comments:
Oncourse is a very frustrating service. It is slow, the menus are not user friendly, and the technology does not seem to work very often. I am currently taking my first online class solely through Oncourse and it is the worst experience that I have ever had at IUPUI. The course is organized well, but Oncourse regularly does not post my papers, comments, and other work. I get more error messages than is practical from any computer application. I have used Oncourse for years and the new "CL" is simply the worst computing experience. The entire structure needs to be scrapped.
Great job with one exception. Please include the names of the classes on the new Oncourse pages! How am I supposed to remember 10 6 digit phrases?
The new OnCourse is horrible! The old system was much easier to use and allowed students to see other comments posted rather than having to open all the different sites and hunt for them. It frequently freezes and take forever to load and it frequently loses messages that I post. It acts like it's posting but the message never appears on the site. Go back to the old system!
I would also like one version of Oncourse. Enough is enough.
The original Oncourse is easier to use than Oncourse CL. My usage of your services is generally restricted to registering for classes and tracking my bursar account with OneStart, and accessing class resources and taking online quizzes with Oncourse. The e-mail provisions with Oncourse are a pain to someone used to Outlook or a similar product.
The New Oncourse is terrible in terms of uploading personal files and management of group spaces. New oncourse is not very user friendly to non-IT folks!
Oncourse CL still needs some easier features. Class e-mail would be easier for instructors and students if there was a 'forwarding' feature available or if you could select only a few recipients like the Old Oncourse allowed.
I would recommend changing curriculum management software (i.e Oncourse). I find it to be not very user friendly or intuitive. I also work with ANGEL in the medical school and while it is not my favorite it is more intuitive and user friendly.
You've already heard all the complaints about Oncourse CL.
The new Oncourse is the weakest link in the IT system. In many respects it is less user friendly than the old system.
IUB Undergraduate Student Comments:
The old Oncourse had a better interface, loaded faster, was easier to navigate.-I still think oncourse needs some work
Oncourse is great, just wish it was easier to use. For examples I want to see my current classes on top.
The original version of oncourse and the new version need to be combined or one needs to be the only one.
I was not the biggest fan of the change in Oncourse, I feel the original was much more user friendly. Now it's not as much of a hassle because all instructors use it, however I still liked the past version. However UITS seems to supply plenty of help.
The new Oncourse is really annoying and hard to use. I makes my life more difficult.
Oncourse is terrible. There is no back button which is very bad. Also, there is no point of having my workspace and the class listing. Why cant they jsut be one thing!
There have been several occasions where I have needed to access Oncourse for homework purposes and have not been able to do so. It has affected my performance in school. If the tech services are not capable of dealing with thousands of students using their services at the same time, then I don't think that UITS should be so pivotal to the coursework at IUB.
-Doing away with OneStart, Oncourse, and possibly even webmail, and then investing in a user-friendly system would make a lot of students and faculty very happy.
IUPUI Undergraduate Student Comments:
The new oncourse is still difficult enough that it hinders my use. I believe one issue is that online classes, at least with my experience, generally use the old format. Can we focus on one instead of bouncing back and forth?
It's frustrating when OnCourse frequently has issues, making it so that I cannot access it.
The old oncourse is 100 times better then the new one. I waste so much of my time waiting for the new one. I don't see anything that the new one does better then the old one and I feel it is a waste of money and time.
Oncourse CL should not take the place of the original Oncourse. As a student, I have taken many online classes, and currently still do, and Oncourse CL is not user-friendly. The original Oncourse is very basic and simple to use, even the "tab" look is helpful. It makes each item very clear. I would not recommend Oncourse CL to professors to use right away, especially when they have not had the proper training, which was the case this past summer in one of my online classes. Thanks!
Everyone that I know HATES the new oncousre CL. It's hard to use and navigate through. We would like for you to not use, but keep the old oncourse. Thank you for KEEPING the OLD oncourse.
The new Oncourse is not as easy to navigate as the original Oncourse. There are features that are better but for the most part it is difficult to get through. Before making changes like those to the major websites, I would say ask the students before hand.
Please make Oncourse CL easier to navigate.
Oncourse CL is a lot more difficult to navigate and use than the old oncourse. Many teachers I have had refuse to use Oncourse CL.
On the top of Oncourse CL there is a box that says "My Workspace" and lists four classes. It would be convenient if these classes were classes that a student is currently enrolled in, or the option that the student can choose which classes to put in their workspace.
The new oncourse should have a course list like the old oncourse does, instead of the top nav bar and the drop down menu. The course pages layout is better though. Other than the above, I've had nothing but good experiences with UITS.
I don't like the navigation of the new oncourse.
I really would like more workshop about the new oncourse cl because I am still trying to figure out the email setting with the new oncourse.
I take all online courses and use the original oncourse. I have seen the new oncourse used and it seems to have made improvements and I hope that my teachers use the new one next semester. I think oncourse is a great system that helps busy online students keep in contact with professors and what is going on in a particular course.
Please combine the new and old oncourse systems into one system so that I don't have to check both, it would be less complicated that way.
The new design of Oncourse, in my opinion, is unnecessary. The original version was quicker and easier to use. There have been times when Oncourse does not work properly for one reason or another, mainly when trying to log on, which is very infuriating. But the people who are available to help correct the problems are very good at their job.
I do NOT like the new Oncourse CL compared to the original oncourse.
Though the additional features of the new Oncourse are beneficial, the original Oncourse was much more user friendly and more easily navigated. Most of my professors over the past year have preferred it as well. So, I have classes in both old and new Oncourse. It gets quite confusing sometimes. Frankly, I wish the new features could be integrated into the original Oncourse.
Make the Central Log in system truly central so I don't have to keep logging in when I switch services (i.e. switching from One Start to Oncourse)
The old oncourse seems to be much better. More straightforward.