Maths CAA Series: July 2002

Computer-Aided Assessment in Statistics at Heriot-Watt University

Index to article

	Abstract
	1. Setting the scene
	2. A Typical Test over the Web
	3. SCHOLAR Statistics
	4. The CUE assessment system
	References

Abstract

A version of this article was first published in the May issue of MSOR Connections and describes the practice of delivering tests over the web for a range of Service Statistics courses at Heriot-Watt University in Edinburgh. Mention is also made of the on-line learning materials for Statistics being created as part of the SCHOLAR Project [1]. SCHOLAR is delivering on-line resources to over 90% of Scottish secondary schools next year.

1. Setting the scene

For many years the three major service courses in Statistics at Heriot-Watt University have been assessed using traditional paper-based multiple-choice examinations. So, when web Tests were created for these courses in 1996 a conscious decision was made to present the questions in the style of the final examination at the end of the course. The three courses, Statistics for Business I and II and Statistics for Science & Engineering, are now all assessed using 85% from the end of course paper-based examination and 15% from two Web Tests. The Web Tests were prepared by third year students on paid summer projects under the supervision of staff.

2. A Typical Test over the Web

From the earlier CALM Project for Computer Aided Learning in Mathematics [2] it had long been recognised that randomisation was an important driver in CAA for Science and Engineering undergraduates. The same lesson emerged from the experiences with the Business students taking Service Statistics. Below is a typical test with three questions. Note that randomisation is employed in two ways: to pick different questions from a bank of similar questions and to vary the data presented to the screen. Thus, each time the students take a practice test different scenarios appear and the numbers in each scenario change. This gives the students plenty of practice to hone their basic skills.

The structure of one of the Web Tests is illustrated below with some sample screen displays:

Q1 (3 multiple-choice parts)

This question involves the sampling distribution of the mean; random choice of one out of six questions, each involving a different application or scenario and each with a randomised mean and standard deviation from suitable ranges.

Q2 (2 multiple-choice parts)

This second question involves confidence intervals for a mean; random choice of one out of four questions, two for large n with randomised sample mean and standard deviation, and two for small n with twenty different versions with Minitab output.

Q3 (2 multiple-choice parts)

The third question involves confidence intervals for a proportion; random choice of one out of four questions, with x and n randomised in each one.

There are approximately 150 students in each of the three groups.
They are encouraged to practice these tests as often as they wish and then they take a formal test supervised in one of the computing laboratories during a tutorial hour. It is important to be able to validate the students' performance in this way. For more information on Heriot-Watt web tests see reference [3].

3. SCHOLAR Statistics

Scholar [1] has been developed by Heriot-Watt University in collaboration with Local Education Authorities and Further Education Colleges, and currently involves Mathematics, Physics, Chemistry, Biology and Computing for Advanced Higher, Higher, 'A'-level, HNC Units and first year courses at Heriot-Watt University. Academic year 2000/1 was designated a pilot year with some 44 secondary schools and 5 FE Colleges participating. This year it has been used in about 75% of Scottish secondary schools (approximately 300 schools in all). Over the next academic year 29 out of the 32 Education Authorities in Scotland have opted into SCHOLAR providing an estimated 40,000 students on-line. The resources delivered include written materials, on-line simulations, CUE assessments [4], discussion areas and frequently asked questions. The on-line resources contain the ingredients of a VLE (Virtual Learning Environment). Materials for the Higher Statistics unit (equivalent to 'AS'-level in England, Wales and Northern Ireland) are now being prepared by schoolteachers on secondment to Heriot-Watt University. The unit on Statistics covers the topic Exploratory Data Analysis.

4. The CUE assessment system

At the heart of the SCHOLAR Programme is the CUE assessment system. CUE developed from a fruitful collaboration between CALM, UCLES (University of Cambridge Local Examination Syndicate) and the commercial firm EQL of Livingston in West Lothian. CUE [4,5] provides a wide variety of question types and response types which are designed to be suitable for all subjects including Mathematics and Statistics. The CUE system includes question and test editors which are easy to use by subject authors and academics. The intention is that CUE will become freely available to any interested institution within Scotland (and perhaps beyond) wishing to implement CAA research. CUE has been adopted as part of the work of the recently formed Scottish Centre for Research into On-Line Learning and Assessment [6]. It is also worth noting that EQL have now developed a commercial product called i-assess [7] which has been employed very successfully in high stakes examinations for the Accountancy profession.

References

[1] http://scholar.hw.ac.uk/
[2] http://www.calm.hw.ac.uk/
[3] http://test.hw.ac.uk/
[4] http://www.calm.hw.ac.uk/cue.html
[5] J S Paterson, The CUE System, MSOR Connections Vol 2 number 2, 22 - 25 (2002). And also at http://ltsn.mathstore.ac.uk/articles/maths-caa-series/apr2002/index.shtml
[6] http://www.scrolla.hw.ac.uk/
[7] http://www.i-assess.co.uk/