Maths CAA Series: October 2002
Page Guide: Home > Articles > maths-caa-series > |
Introduction |
|
1. Background and Project Aims |
|
2. Problems |
|
3. Project Changes |
|
4. Implementation |
|
5. Assessment |
|
6. Module and Project Outcomes |
|
7. Conclusions |
|
8. The Next Phase(s) |
Introduction
This short article describes a project, the aim of which was to use a Virtual Learning Environment (VLE) in conjunction with CAA in a Level 0 mathematics (mainly calculus) and basic statistics module for Computer Science and Physics students. This was a second semester module following on from a first semester module. The first semester module used continuous assessment via regular assignments based on selected questions on exercise sheets to be handed in, marked and returned. A similar assessment profile was planned for the second semester, except that CAA would be used for the continuous assessment.
There were 70 students in this class with varied pre-university mathematics backgrounds (as usual). The School of Mathematics and Statistics, at the University of Newcastle, promoted the project and I applied and obtained funding from LTSU (Learning and Teaching Support Unit) Newcastle for the project to run from October 1 2001 until June 14 2002.
As well as describing this project and its outcomes, I have detailed some of the difficulties, misunderstandings and frustrations encountered. Also I have included brief details of two further projects in this area which are now under way offering great promise for the future.
Perhaps the main lesson is that if VLEs are introduced into an Institution, then there must be a solid and well thought out infra-structure of support, both technical and pedagogic within the Institution. Otherwise, their use and development, especially in MSOR, is doubtful. Your help in this is sought at the end of this article so please read on.
1. Background and Project Aims
The University of Newcastle had recently obtained Blackboard and LTSU Newcastle was charged at this time with its implementation and use. However, the use of Blackboard within the School of Mathematics and Statistics was very limited. There had been complaints about crucial administrative failures in student registration. Also the "selling" of Blackboard to the School was by general consensus poor with very little relevant information and examples forthcoming. These factors helped create a negative attitude towards the VLE in the School. Another important factor was the lack of a useable assessment tool within Blackboard. Multiple Choice Questions and variants thereof are offered by most VLEs but are not considered adequate to test mathematics or statistics at HE.
The project then was timely in terms of:
a) Testing and demonstrating the use of a VLE together with CAA in mathematics and statistics.
b) Promoting the use of VLEs in mathematics and statistics within the School.The original aim of the project was to pilot the integration of TransMATH, a well used CAA tool for Level 0 mathematics and statistics, into Blackboard. This was to enable the use of CAA in Blackboard and to use the communication and other facilities within Blackboard to:
1. Provide an enriched learning environment for students studying a Foundation Level Mathematics module.
2. Enable the use of efficient continuous assessment.
3. Provide the means by which students can practice their skills using the assessment tool at any time and from anywhere using Blackboard.
4. Use the Blackboard gradebook tool and communication facilities to enable efficient and timely feedback and advice to students as to their progress.
This integration was to be achieved between October 2001 and December 2001 with:
- Module preparation from November 2001 to February 2002.
- Training of Teaching Assistants in January and February 2002. (Two Postgraduate Teaching Assistants were funded).
- From February 2002 until June 2002 the module was to be taught and performance of students closely monitored with relevant feedback.
2. Problems
Both LTSU Newcastle and I were ignorant of the fact that the version of Blackboard at Newcastle could not be integrated with TransMATH as I originally envisaged and proposed to LTSU. I had rather naively assumed that the acceptance of the proposal by the Blackboard maintainers at Newcastle meant that it was feasible!
This integration was also subject to technical help from LTSU, and crucially dependent on information on how to access the underlying Database used by Blackboard. As is now obvious, this information was not in the end available in any useful form and time was lost in trying to extract this information despite repeated attempts. It became clear by November that the original aim to integrate TransMATH could not be achieved and that the project needed to be re-evaluated. Module preparation had already been put forward to start in October as these difficulties first emerged.
3. Project Changes
Given these problems it was agreed by LTSU and Mathematics and Statistics in November 2001 to modify the project as follows:
1. To abandon the close integration of TransMATH into Blackboard, but to use TransMATH in parallel and to report progress "by hand" to students via Blackboard.
This involved a great deal of extra work. However, as far as the students were concerned the effect was the same in terms of checking on their progress as if the original project aims had been achieved.
2. To use, as planned, Blackboard as the main site for the module: all worksheets, content and announcements were included on the Blackboard site.
3. To follow the same timetable as the original project in preparing the module, training assistants and teaching the module.
4. To include videos, explaining theory and going through examples, as part of the Blackboard content (calculus and differential equations). This entailed a large amount of editing and indexing of video files. The original unedited files were made available, after permission, from the Mathematics for Engineers project.
In addition, the Postgraduate Assistants made extra videos on Probability. We used the Media Centre at Newcastle, at the expense of the School of Mathematics and Statistics, to make these videos.
4. Implementation
The teaching of the module was organized as follows:
2. No lectures timetabled other than those below.
- Workshops were held in PC Rooms (two or three per week per student). These were supervised by 2 to 3 teaching staff (Teaching Assistants and lecturer). There were 20 to 25 students per workshop. These workshops were used by students to
a) Become familiar with the current TransMATH topic. An important aspect of this was the inputting of answers to the current TransMATH assessment.
b) Do the TransMATH assessments.
c) Obtain feedback and advice on their progress.
3. Lectures at suitable points on Integration and Statistics. Revision lectures also given.
4. Most module content and worksheets via Blackboard and videos – some content via TransMATH.
5. One worksheet per topic.
6. Feedback via:
a) Workshops
b) Using TransMATH database recording student attempts at assignments and then
c) Using the Blackboard Communication tool (e-mail in groups or by selection).5. Assessment
- There were 7 Computer Based Assignments using TransMATH covering the main topics in the syllabus. Each counting 5%
- Two short text based exams, each counting 25%, together with an opportunity to re-sit both exams.
- A text based statistics assignment (15%).
A small number of students had major difficulties – due to their mathematical background, or due to technical issues beyond their control e.g. not being able to enroll on Blackboard. They tended to fall behind quickly due to the continuous assessment nature of the module. For these students I supplied the opportunity to do a large number of supplementary exercises later in the module so that I could monitor their progress and enable them to catch up. These were on a one-to-one basis and students were awarded marks if they did the work.
6. Module and Project Outcomes
Student feedback was in general positive. A lot of good work was put into the Computer Based Assessments. The overall averages were satisfactory (55%).
The Project Outcomes
Despite the changes, the project achieved the main aims of the original project:
1. Student progression was closely monitored.
2. Students made significant use of the TransMATH assessments
3. Student evaluation of the teaching and the module was positive in the following respects:
- Use of TransMATH – although a small number did not like the software
- The self –paced nature of the assessments was appreciated.
- Use of Blackboard as a Communication, Coordinating and Information Centre.
- The Blackboard site was heavily used.
- Use of videos for worked examples.
- Feedback of their progress (via e-mail using Blackboard and workshops) and access to lecturers. Most students appreciated this and information was given to tutors at an early stage about students not attempting the course or doing badly.
4. Student evaluation was negative in the following:
- More conventional lectures were asked for. The majority of time-tabled hours were in Workshops manned by lecturers and assistants. However, lectures were given on Statistics, Integration and on Revision material (which were poorly attended!)
- The perceived bugs in TransMATH. This was mainly down to network problems in that permissions were re-written at random times. These problems were sorted by about week 5 in the Second Semester. Also a small number of answers given by TransMATH were incorrect – but these were all accounted for.
5. Technical issues. There were problems with:
- Enrolling of some students on Blackboard – this is extremely crucial. It is imperative that if VLEs are to be used that the process of registration is stable and timely. This was not the case for some students and they lost valuable time. This meant extra work for the lecturer and stress for the student in enabling them to catch up. The administration of Blackboard was not able to cope with these problems adequately at this time.
- Network problems. These caused some initial difficulties with the running of TransMATH as permissions had to be re-set. These were solved at an early stage of the module.
7. Conclusions
- The project showed that students will use and appreciate an e-learning environment. There was a call for back-up with lectures or tutorials at appropriate times – for example on areas that students perceive as difficult. But this has to be carefully managed as when these were provided there was poor attendance. The evidence is that if there is a mix of lectures and workshops, then students will prioritize these and opt for the contacts which will maximize their acquisition of marks – in this case workshops.
- CAA is possible and welcomed by students as long as
a) The assessment engine is reliable as a software tool
b) The method of input of mathematical answers is user-friendly.
c) Feedback is available on their answers and that progress is monitored.
d) Practising the questions is possible at any time and anywhere (subject to availability of machines etc.)
- Blackboard is appreciated and used as long as it is used dynamically by the lecturer/course leader with constant updates to react to students’ feedback and requests. It is important to have explanatory and useful content in the VLE. For example, past exam papers, videos etc.
- Videos are seen by students to be very helpful in demonstrating worked examples. These were made available through the Blackboard site and could only be accessed sensibly using the University Intranet – at workshops and outside those times in the various computer rooms scattered about the University. It is clear that the best method when using videos is to supply students with the appropriate properly indexed videos on CD ROMs. For example, Andrew Rae’s Discrete Maths Module at Brunel University has used this method very successfully. It is claimed that Blackboard can pick up the appropriate video from such a CD ROM thus enabling a web-based use of the videos. This has to be investigated further.
- If modules are to be run using Blackboard then all technical issues regarding registration etc. must be dealt with as lecturers will lose faith in the system and abandon it.
- You must have good access, and hopefully a good relationship, with the maintainers of the VLE. For example, to get warnings of downtimes, to ask for increased memory quotas for your module (especially if you are using videos), etc.
If you are thinking about integrating your favourite assessment tool or whatever into a commercial VLE such as Blackboard – then beware! You need:
a. To know if the version of the VLE you have will support this integration
b. Significant technical help.
c. Good knowledge of Java or some other programming language.
d. Support from the VLE’s maintainers at your institution (for example, you probably will need another server to run the experimental platform for your developments – which involves money and resource)
e. Probably yet more money on joining a Developer Network, buying Developers’ Licenses or variants on these themes. However, it is not clear that these are strictly necessary in some cases , and it is important to get more information (see next point).
f. To make contact with the company selling the VLE. My experience was that I had to make contact (via LTSU) with the US operation rather than the UK as there was no technical help available in the UK.8. The Next Phase(s).
Two projects are now under way building on this and other projects.
1. An internally funded University of Newcastle project is now looking at the integration of a powerful and well tested authoring assessment system into Blackboard using the Blackboard Building Block Initiative. The issues raised above on technical developments are being addressed and a report on these will be made available in summer 2003.
2. In parallel, INFORM-IT, a University of Newcastle based LTSN Maths, Stats & OR funded project is examining general issues in using VLEs to teach Mathematics in HE. This is in conjunction with the Universities of Birmingham, Brunel and Heriot-Watt. In particular, we will be reporting and giving guidelines on the practical issues and problems raised above concerning the use and development of VLEs in our discipline.
As a first step we have published an on-line survey on the use of On-Line Systems in MSOR. This can be found on this site at: http://ltsn.mathstore.gla.ac.uk/questionnaire/index.asp?quest=1
If you are involved, or plan to be involved in such systems please find the time to fill in this survey.