Maths CAA Series: August 2003
Page Guide: Home > Articles > maths-caa-series >
|
|
|
1. Introduction - What is this? |
|
2. Technical details |
|
3. The current position |
|
4. the Situation in Mathematics |
|
5. What this means to you |
|
6. More information |
Abstract
This short article provides a summary of the current position of the new international specification for computer based questions and tests, for those of you who are using, or planning to use, computer based assessments in your courses.
1. Introduction - What is this?
Computers are increasingly being used to help assess learning, knowledge and understanding. IMS Question and Test Interoperability (QTI) is an international specification for a standard way of sharing such test and assessment data. It is one of a number of such specifications being produced by the IMS Global Learning Consortium to support the sharing of computer based educational material such as assessments, learning objects and learner information.
This new specification is now being implemented within a number of assessment systems and Virtual Learning Environments. Some systems store the data in their own formats but support the export and import of question data in IMS QTI format. Other systems operate directly on IMS QTI format data. Having alternative systems conforming to this standard format means that questions can be shared between institutions that do not use the same testing systems. It also means that in theory banks of questions can be created that will be useable by many departments.
2. Technical details
The QTI specification uses XML (Extensible Markup Language) to record the information about assessments. XML is a powerful and flexible markup language that uses 'tags' rather like HTML. The IMS QTI specification was designed to be pedagogy and subject neutral. It supports five different types of user response ( item selection, text input, numeric input, xy-position selection and group selection) that can be combined with several different input techniques ( radio button, check box, text entry box, mouse xy position dragging or clicking, slider bar and others). It is able to display formatted text, pictures, sound files, video clips and even interactive applications or applets. How any particular question appears on the screen and what the user has to do to answer it may vary between different systems, but the question itself, the knowledge or understanding required to answer it, the marks awarded and the feedback provided should all remain the same.
3. The current position
The specification is relatively new. Version 1.2 was made public in 2002, and a minor upgrade to Version 1.2.1 was made early in 2003, that corrected some errors and ambiguities. The specification is a complex entity comprising nine separate documents. Various commercial assessment systems (e.g. Question mark, Granada, MedWeb, Canvas Learning) have already implemented some degree of IMS QTI compatibility for their assessments. A number of new academic systems are also being developed to comply with the specification. These include the TOIA project system which will have editing and course management facilities, the SToMP project testing system which was used with students for the first time during this last academic year, and a Scottish Enterprise system called Oghma which is currently being developed.
4. the Situation in Mathematics
One of the disadvantages of any such specification towards a standard system within mathematics is the lack of support for facilities like the entry of algebraic expressions, the handling of both accuracy and precision of numbers, the use of alternative number bases, the provision of randomised values, graphical input and multi-stage questions. All of these have, in one form or another, been included in science specific systems in the past, but they have not been included in this specification yet. Due in part to the fact that one of the members of the Physical Sciences LTSN Centre is taking a close interest in the specification, the handling of numbers is now being addressed. A proposal has been lodged with the IMS Consortium for an extension to the specification that would support the four features mentioned above to do with the handling of numbers (accuracy, precision, randomisation and number bases). Two QTI compatible systems have already been modified to support these features and one of them (the SToMP system) is available for free trial now, so you can see how such questions appear.
Unfortunately, support for the entry and manipulation of algebraic expressions within the QTI specification is not yet under consideration. However, it is planned to hold a meeting under the auspices of MSOR and SCROLLA (Scottish Centre for Research into On-Line Learning and Assessment) towards the end of August at Heriot-Watt University in Edinburgh. At this meeting some proposed extensions will be discussed with a larger group of mathematicians. In addition it is hoped to address the issue of how to include algebraic expressions as answers within any emerging specification.
5. What this means to you
If you are starting or planning to start using computer based tests, then you need to be aware of the advantages of using an emerging standard-compliant system. It is clearly a good idea to choose a system that will allow you to move your assessments to another system at a later time with the minimum of effort or to be able to import assessments authored elsewhere.
A consideration to bear in mind, however, is that at this early stage in the life of the specification there will be a range of legacy differences between various implementations. It will also remain possible with some 'compliant' systems to create non-standard question formats if implementation specific extensions are used. The degree of conformity of any one system is a parameter that is difficult to assess at any time. Tools to assist with this are now beginning to be discussed, but it will be some time before objective measures of conformance will be available. In view of this it is a good idea to keep in touch with those interested in the development of the specification, and the best way within UK HE is probably via the CETIS Assessment Special Interest Group website listed below.
It is clearly important that a specification such as this must have subject specific input from interested academics. The needs of different disciplines are not always well known and the lack of specific features can make adoption difficult. If you can spare the time, have a look at the examples on the CETIS assessment web site (in the CETIS rendering tool) and let us know where your needs are not being met.
6. More information
The full QTI specification, along with other IMS specifications is published on the IMS Global Learning Consortium website
CETIS is the JISC funded Centre for Educational Technology Interoperability Standards. It has a website containing useful information on all the IMS specifications
http://www.cetis.ac.uk
The CETIS Assessment Special Interest Group has a website at
http://www.cetis.ac.uk/assessmentSome websites of commercial and academic testing systems and support.
Questionmark
http://www.questionmark.comCanvas Learning Author and Player
http://www.canvaslearning.comTestwise (Granada Learning)
http://trp.demo.granada-learning.com/preview.htmlSCAAN (Scottish Computer Assisted Assessment Network)
http://www.scaan.ac.uk/ims.htmlScottish Centre for Research into On-Line Learning and Assessment
http://www.scrolla.ac.ukQTI Ready (XDL Soft)
http://www.xdlsoft.com/products.htmlAssesst Designer
http://www.xdlsoft.com/ad/ad.htmlJcourse (Carnegie Mellon University)
https://course.phil.cmu.edu:8443/jcourse/index.htmlLearn eXact (Giunti Labs)
http://www.learnexact.com/Act Elearning
http://www.actelearning.comRespondus
http://www.respondus.comIntrallect
http://www.intrallect.comSToMP
http://www.stomp.ac.ukWebMCQ
http://www.webmcq.comMiranda
http://cvu.strath.ac.uk/new_information/miranda.htmlMAP: Monitoring Assessment and Provision
http://alto.aber.ac.uk/caa/maphome.aspRiva e.test
http://www.riva.com/etest/etest.asp