Introduction
The Scottish Centre for Research into On-Line Learning and
Assessment is
funded for 3 years through a research and development grant from the Scottish Higher
Education Funding Council. Its aim is to develop an infrastructure for interdisciplinary
research throughout Higher Education institutions in Scotland. There are three main
strands to the Centre’s activities:
1. ICT Policy led by the University of Edinburgh
2. Networked Learning led by the University of Glasgow
3. On-Line Assessment led by Heriot-Watt University
At Heriot-Watt, the research activities will be aided by the
assessment system called CUE. CUE resulted from collaboration between:
1. CALM (Computer-Aided Learning in Mathematics at
Heriot-Watt)
2. UCLES (University of Cambridge Local Examination Syndicate)
3. EQL Livingston
This system is currently delivering the on-line formative and
practice summative assessments within the SCHOLAR programme. The programme covers the
Scottish Qualifications Authority (SQA) Advanced Higher and Higher in Biology, Chemistry,
Computing, Physics and Maths. Although these are Scottish qualifications, the levels
translate roughly to the two years covering AS and A level in England and Wales.
With the focus in SCROLLA on interdisciplinary research, the
intention is to make CUE freely available for research purposes to any interested
institution within Scotland (and perhaps the rest of Great Britain) in the very near
future. In this way, stronger collaboration in projects to advance on-line assessment will
be possible without the added difficulties of differences between assessment systems.
Naturally institutions will want to consider the features and capabilities of CUE before
proceeding. A dedicated server at Heriot-Watt University will be maintained and available
to colleagues who use CUE and who wish to test and share their research work with others.
Aspects and Features of CUE
There is a demonstration of the features of CUE on the web at
http://www.calm.hw.ac.uk
References to the demo are given in the form CUEq1 for question 1 in
the demo.
Systems Information:
CUE is an on-line assessment system but can be run through a local
Intranet using a Windows 2000 server. It will also run as a stand-alone assessment system
on a local server. This is extremely useful for presentations and demonstrations when
installed on a laptop.
On the programming side, CUE is written in Delphi (Borland). The
system, however, includes question and test editors providing interfaces that are easy to
use by the subject authors, lecturers and academics.
General Features:
CUE tests can be set to run in a variety of ways.
Although commonly described as help, practice and exam modes, there are multiple options
for feedback and navigation. (figure 1)
Figure 1
Whilst the reveal button is useful for the student, CUE also
provides the options for multiple keyparts and steps to these keyparts. The keyparts allow
a complicated question to be broken down into several tasks and help to guide the student
through the techniques required. All keyparts need to be answered to gain full marks.
(CUEq2)
Steps are used to offer similar but lower level help. The student
decides whether to take the steps or not. If the steps are chosen the student must answer
each question but is given simpler smaller tasks to do that lead to the answer required in
the key part. This gives weaker students additional help yet still allows them to progress
and gain marks. (CUEq2)
Any question type can be used for each key part and step. The test
size is flexible, allowing the author to decide on the number of questions within each
test.
There is a comprehensive results database where data from tests is
gathered and stored. This is done incrementally to safeguard against network failure
during a test. Information from the database can be made available to the student and the
registered tutor.
Marking within CUE gives ticks and crosses, feedback in a separate
window for most question types and the option of partial credit. Every CUE question has
the same basic structure in the question editor (figure 2) although this example includes
two key parts and one step in each.
Figure 2
Question Types:
These are best described in detail from the authoring viewpoint as
there is a growing trend for types to differ according to the maker or developer.
Judged Mathematical Expression: CUEq1
and CUEq2
This is one of the most powerful types available and perhaps could
be better described as Numerical, Algebraic and Mathematical Expression. A detailed
description is given as this question type is rarely available. This type will
automatically mark mathematical equivalent expressions in addition to numerical and
algebraic answers. Random parameters can be used adding a greater flexibility to the
questions.
In figure 3 there are 2 key parts, one with a step.
Figure 3
The question shows the use of random parameters (within curly
brackets). For clarity the question is shown as figure 4.
Figure 4
The parameters ‘a’ and ‘b’ show normal tags with
parameter ‘c’ displaying a math tag with an id attribute. This id attribute
allows the choice of options at figure 5:
Figure 5
There are tabs above the prompt in the example question. (figure 6).
Figure 6
They offer comprehensive accuracy settings for decimal places,
absolute/relative differences and significant figures. (figure 7)
Figure 7
Length and String restrictions provide other ways of limiting
answers with ‘allowed‘ and ‘not allowed‘ characters and
maximum/minimum character length in the answer box. (figure 8)
Figure 8
CUE has options for global and conditional restrictions to be
imposed on randoms. In this question type the user inputs the answers in string equivalent
form. On revealing any answer both this form and the rendered version of the maths is
shown to help the students if they are finding this difficult.
Multiple Choice: CUEq3,4,5
This system offers various rendering and display styles (button
image, radio group, choice content, drop down list) and layout (number of columns). It is
possible to randomise the order of the choices, use random parameters in maths expressions
and use animation or images in the choices. Feedback can be given for each choice. Some
styles offer submit buttons before the selection is logged. From a choice of options the
user has to select one correct answer.
Multiple Response: CUEq6
This type allows for more than one correct answer and there are
comprehensive marking possibilities allowed that will give partial credit for a lesser
number of correct answers than required. The multiple choice template option forms the
basis of this type but the style (check box) allows for the alternative answers. The user
selects more than one correct answer from a choice in multiple response types.
Word Match: CUEq7and CUEq8
With this question type it is possible to have several correct
alternative answers (chosen by the author), case sensitivity and phrase match. Word match
will work with any keyboard character and can accept the accented letters in languages
such as French. Correct marking of root recognition is also possible. Figure 9 shows the
entry in the editor for alternatives as the answer to the question 'Name one primary
colour'.
Figure 9
The user types in the answer in normal text format.
Hotspot: CUE q9,q10,q15
In CUE this type of question allows the author to dictate the shape
of the ‘spot‘ rather than force a rectangle. Multiple hotspots correct or
otherwise, are allowed on an image and it is possible to use multiple images as hotspots.
The user chooses an answer or answers by clicking on the correct image or part of it.
Essay: CUEq11
The student is able to type in an essay answer but it is not yet
possible to automatically mark it. The results service will however record that the
question has been accessed. The student can save, copy and print it. In formative mode the
author can prepare a model answer that is available to the student under the reveal
button. The size of text box available on screen is set by the author.
Hint/Information: CUEq12
This facility allows a hint or information to be given. Marks can be
deducted if required when the hint is within a step. There is no user input for this type.
Multimedia: CUEq13,14,16
In general, all question types can accept images and animation using
packages such as Flash. Sound files can also be included.
Developments
Although CUE has many strengths there are still areas for
development. These include the introduction of new question types, additional features
such as hot links to on-line resources and improvement in the general usability and
stability of the system.
Contacts
There is a demonstration test on the web showing these features in
CUE:
http://www.calm.hw.ac.uk
- follow the links to CUE demo.
This demo will also be available shortly through the SCROLLA
websites at: http://www.scrolla.hw.ac.uk and http://www.scrolla.ac.uk
For further information contact Jane Paterson, Research Fellow in
|
|
©Copyright,
Higher Education Academy - Maths, Stats & OR Network
Maintained by R.L.Surowiec@bham.ac.uk
Last revised: Friday, 07-Oct-2005 18:00:00 BST
|
