Child pages
  • 21st Century Test Engine Wish List
Skip to end of metadata
Go to start of metadata

2015 Apereo Conference
Baltimore, MD
June 2, 2015
2:30 - 3:15 pm EDT


What do we need in a higher ed test engine?


Issues:
------

* 1. Proctoring

Proctoring services - e.g., ProctorU, Remote Proctor Now, etc.; drawback = expensive

Limit to high stakes exams?

2. Assessment Types (too many? too few?)

3. Training

4. Partial submission

5. Extend time (technical issues)

6. Instructor/administrator submit exam on behalf of student

7. Student contending exam was taken/completed

8. Video questions (limited time for student to answer)

9. Realtime adaptive testing - as you take test, it learns about you (e.g., language placement test; 10 placement questions)

Maplesoft product - "stem-based" questions, e.g., Physics diagram and related calculations

10. Test engine which is scalable (can handle large number of examtakers at same time, particularly, when random draw exam
from question pool(s))

Manual workaround is stagger examtakers (schedule in groups)

*11. Anti-cheating

Respondus lockdown browser

ID verification before taking exam

Timed exam (limit time so impractical to look up answers)

Add security feature to authenicate periodically during live exam?


12. Support for mobile devices for exam taking

13, Ease of use

14. Intuitiveness

15. Partial credit for exam questions (e.g, scientific/math question type which takes a question from a stylus)

16. Ability to handle complicated scientific/math formulas

17. Comprehensive testing tool (does it exist?)

18. "Test in progress" - does not display user's names; need JIRA to incorporate this feature

19. Ensure "Open", "Close" and "Accept Until" dates are in sync


Current status of Tests & Quizzes:
---------------------------------

1. Defined list of assessment types (e.g., multiple choice, fill in the blank, T/F, audio, etc.); auto grade if so
defined or otherwise (e.g., essay), instructor must manually grade

2. Templates problematic if wrong template chosen (e.g., anonymous survey selected in error)

3. Question pools owned by individual - instructor can author or import (QTI)


Real case:
---------

Notre Dame

Lecture Hall at Notre Dame with 350 students; administered small quiz, no grade; notebooks brought by students (must be fully
charged, not enought power sources).

Samigo could not generate enough randomly generated exams (capacity was exceeded) even with limited number of questions on
exam (i.e., 15-20 questions).

Conclusion: Need test engine which is scalable


Real Case:
---------

Western University

25-40 quizzes per semester, random draw from question pool, administered by an instructor every week having response time
issues; turned out issue was in Samigo; every question from question pools was being pulled down for every question on exam

Conclusion: Fix issue in database including indexing (alternate workaround - stagger exams)


Proposals:
---------

1. Samigo Lite - accommodates 80%+ of common testing scenarios; either activate Samigo Lite or Samigo Advanced

Perhaps remove randomized questions

Ref: Examsoft (used particularly for high stakes exams), for example, for legal bar exams

2. Samigo Advanced - comprehensive test engine

 

 

 

 

 

  • No labels