Maths CAA Series: May 2002
Page Guide: Home > Articles > maths-caa-series >
Printable PDF version of this article available from http://ltsn.mathstore.ac.uk/articles/maths-caa-series/may2002/highskills.pdf
Abstract: This article defines ``higher level mathematical skills" and details an important class: that of creating instances of mathematical objects satisfying certain properties. Comment is made on the frequency of higher level tasks in undergraduate work. We explain how such questions may be assessed in practice without the imposition on staff of an onerous marking load. Included are working examples which have been implemented on a free computer aided assessment system with some source code. Lastly we report an investigation of students' reactions to these questions and discuss their design and impact.
Keywords: high level mathematical skills, computer aided assessment, deep learning
Firstly in this article we examine and define what is meant by ``higher level mathematical skills" and elaborate on the ideas of [9] and show that constructing examples are but one part of the higher level skill of creating instances of mathematical objects satisfying certain properties. Secondly, by drawing on a recent analysis of current undergraduate work we examine and remark on how often, in practice, students are asked to perform higher level tasks. Thirdly we explain how such questions may be assessed in practice without the imposition on staff of an onerous marking load. Included are a selection of working examples which have been implemented on a free computer aided assessment system AIM with a technical appendix giving source code for some of these questions and explaining how such questions may be authored. Lastly we report an investigation of students' initial reactions to these questions during a group interview and discuss their design and impact.
A skill is defined literally as a ``practised ability, expertness, technique, craft, art, etc." Any attempt to elaborate on what is meant by mathematical skills must be based on an analysis of what in reality we ask the students to do. That is to say, the nature and purpose of the course are inherently influenced by the assessment criteria, which are in turn fulfilled by the successful demonstration of various skills. In contraposition, if we fail to ask the students to practice some technique, or to develop expertise in some craft - such as writing a mathematical proof - we can hardly complain afterwards that they have failed to develop this skill. If we genuinely covet other skills - use of computer technology, presentation, research and investigation, for example - our schemes of work, both formative and summative should reflect this. It is widely known that assessment drives what and how mathematics is learned and that students tend to adopt surface approaches to learning. Furthermore research [2] indicate that students who adopt a deeper approach to learning are more successful.
There have been various attempts to classify pedagogical tasks. Perhaps the best known general scheme is Bloom's Taxonomy [1]. This has been refined in [16] wherein mathematical examination questions are classified into eight categories shown in Table 1. The three groups broadly show an increase in ``level" of the skills. Of course, all such classification schemes are fraught: no attempt is made to classify difficulty (Smith, rather tongue-in-cheek proposed ``easy" = questions I can do and ``difficult" = questions I can't do). Similarly such classifications are entirely context dependent; demonstration of the skill 5. Application in new situations by definition may only occur once in each mathematical context. Furthermore we might not be assessing what we intend to assess: a proof requiring only verbatim transcription or minor modification may only test factual recall, not true understanding.
| Group A | Group B | Group C |
| 1. Recall factual knowledge | 4. Information transfer | 6. Justifying and interpreting |
| 2. Comprehension | 5. Application in new situations | 7. Implications, conjectures and comparisons |
| 3. Routine use of procedures | 8. Evaluation |
A recent BSc. project evaluated current level 1 undergraduate core mathematics material in the light of emerging hand held computer algebra technology [14]. Essential to this analysis was a classification of course work questions. Independent of [16], but with reference to [5], the question classes shown in Table 2 were identified. Elaboration of these classes is given, with examples in Section 3. Section 4 gives details of the question type on which we concentrate effort in this paper: that of creating instances. These two classification schemes show remarkable similarities, furthermore, when identical question sets were evaluated independently with the scheme shown in Table 2 the results showed a high degree of correlation, providing evidence of robustness in the procedure, despite its apparent subjectivity.
| 1. Factual recall |
| 2. Carry out a routine calculation or algorithm |
| 3. Classify some mathematical object |
| 4. Interpret situation or answer |
| 5. Proof, show, justify - (general argument) |
| 6. Extend a concept |
| 7. Construct instance |
| 8. Criticize a fallacy |
Given that we may classify the majority of course work tasks using one of the two schemes in Tables 1 or 2, one comes to the issue of defining ``Higher level skills". The classification scheme of [16] does this in the three groups A, B and C shown in Table 1. The TELRI project [15] differentiated between two forms of learning as follows.
They defined adoptive learning as an essentially reproductive process requiring the application of well-understood knowledge in bounded situations. Mathematically this may be typified by successful completion of tasks as laid out as 1-4 in Table 2. Proofs requiring verbatim transcription or minor alterations from a template are similarly adoptive in nature. This behaviour, they claim, would be typical of the competent practitioner.
They further defined adaptive learning as requiring higher cognitive processes such as creativity, reflection, criticism and so on. These attributes are typically tested by questions from classes 4-8 in Table 2. Such behaviour would be typical of the expert.
They introduce these terms to draw a distinction between deep adoptive and deep adaptive learning. The former learner may be adept at complex tasks requiring a number of separate skills, see Example Question 1 below for such a mathematical task. The latter learner will be able to generate examples and proofs more independently showing insight, creativity and a higher degree of conceptualization.
We define the skills which typify such an adaptive approach as the higher level skills. This clearly does not divide tasks into disjoint classes - there is room for subjectivity. The creativity needed to genuinely construct an example could manifestly be replaced by the appropriate factual knowledge. That is inevitable and presents one particular challenge for assessment design which discriminates between surface and deep learners.
This section provides brief explanation of the classification scheme outlined in Table 2. Some illustrative examples of questions are also given, more information may be found in [14].
1. Factual recall
All questions require factual recall to some extent. However, on occasion questions may
only require the recall, usually verbatim, of some knowledge.
2. Carry out a routine calculation or algorithm
Such questions may be complex and involved. They may require the recall of some facts,
such as the details of the algorithm itself at university level. They certainly
pre-suppose a certain level of algebraic fluency. Often such tasks may be performed by a
computer algebra system.
Example question 1 Solve the wave equation on the square two dimensional domain D subject to the boundary conditions... [Separation of variables followed by Fourier series methods]
3. Classify some mathematical object
There are essentially two parts to these questions. Firstly to recall the definitions,
which may be posed as a separate task in its own right. Secondly to perform any algebraic
tasks or provide justification to show the specific object satisfies this. Often this is a
question of the form ``Show that ... is a ...". Furthermore, correctly applying a
theorem, for example, tacitely assumes the case to which the theorem is being applied
satisfies the hypotheses. This latter example is a case of object classification.
Example question 2 Show that ( { 3,6,9,12 }, × mod 15 ) is a group, and identify the identity element.
Clearly there will be some factual recall, and possibly some routine calculation. The essential distinction between factual recall, use of routine procedures, and classification of objects is the presence of a specific object. The distinction between the latter is typified by the two questions
``Find a solution to the differential equation ..."
and
``Show that ... is a solution to the differential equation ..."
4. Interpret situation or answer
In this case a problem is posed with reference to a physical situation and requires
modelling prior to the application of routine procedures. Typically the solution will
require interpretation from the mathematical model to the terms in which the question was
posed. The student must communicate their solution in the terms used by the question.
Example question 3 [Kreyszig] Find the current in the RLC-circuit shown in Fig 60. when R=50 ohms, ..., I(0)=0, I'(0)=0.
5. Proof, show, justify - (general argument)
Such a question requires a general argument involving abstract or general objects rather
than specific examples. In posing such questions phrases such as ``prove",
``show", ``with justification" are often used. Although used interchangeably,
the solution to a ``prove" question typically requires Boolean logic, meta-logical
structures (such as a contradiction, induction, exhaustive cases) and formal definitions.
The solution to a ``show" question would, in general require an algebraic
demonstration of a general but routine result.
Example question 4 Show for all m Î R the matrix
has repeated eigenvalue 1.
This is a general argument in the sense that the l is an arbitrary parameter, but there is little in the way of logical structure in the simplest correct solution. Care needs to be exercised when ``with justification" is encountered as this may be code for ``we want to see your working" in an otherwise routine calculation.
Either direct factual recall and verbatim transcription of a proof, or adaptation of a known proof as template for the task in hand is not uncommon.
6. Extend a concept
Students are asked to evaluate previously acquired knowledge in a new situation. Such a
classification is highly context dependent, temporary and thus fraught. There may be an
overlap with other classifications, such as classification of some object. Nevertheless it
is a useful distinction, particularly for the course designer.
Example question 5 Show that the set of polynomials in x is a vector space over the real numbers. Show that the process of formal differentiation defines a linear transformation on this vector space. [In the context of a level 1 vector spaces course in which a limited number of examples of finite dimensional vector spaces have been introduced.]
7. Construct instance
Questions of this type will occupy the remainder of this paper and are addressed
separately in Section 4 below.
8. Criticize a fallacy
Finding mistakes in supposed proofs or criticizing reasoning does not fit within the
scheme above. Such questions are posed, and classified separately. Such questions require
higher level skills and a good understanding of the subject matter.
Example question 6 The following FORTRAN code will not compile. Explain why not and suggest corrections.
The task classification creating instances requires the student to provide an object satisfying certain mathematical properties. Typically there will be many correct solutions but no general method by which such a solution may be constructed. However an example may be only a matter of factual recall, as in the case of many famous examples1. In general the desired properties will generate a set of criteria and hence a mechanical procedure for testing whether a solution is correct. That no unique solution exists means that, in practice, such questions are difficult to mark, requiring close attention from the marker.
This category is mentioned by [16] who, differing from the usual pattern in that paper, provides no examples of such questions. In fact, the recent analysis of tasks set to level one undergraduates [14] revealed questions requiring the construction of examples are rare in undergraduate work.
In general exemplification is the process where something specific is taken to represent the general. Previous work [10] identified four classes of examples; startup examples, reference examples, model examples, and counter examples. Recent work [9] encourages the construction of boundary examples as a method of promoting higher level skills. We will argue that [9]'s boundary examples are but one part of a more general higher level skill of creating instances. Much of interest has already been conjectured and written about the rôle examples play in teaching and learning [10,13,17,18] (to name but a few) and in the development of the subject as a whole [4,8]. Research [2,pg 293] concluded that ``The generation of and reflection upon examples provided powerful stimuli for eliciting learning events", but that students show a reluctance to generate their own examples [11]. Absent from these, and other works, are practical methods for marking complex examples generated as solutions to course work problems at undergraduate level, some of which might involve significant calculations. This is the issue we address.
We concentrate on this question classification category for the remainder of this paper, and provide a number of sub-classification illustrating the sorts of tasks that may be classified under this heading. When carefully posed so that appropriate factual knowledge is neither pivotal or decisive, success with such questions demonstrates genuine creativity on the part of the student. It is precisely the lack of routine methods, or prototype worked examples, which require students to develop for themselves techniques to solve these kinds of questions.
Such questions ask students to provide objects satisfying explicit criteria. The boundary examples of [9] are an advanced form of this type in which a list of criteria are given which are progressively restrictive. This list is then reversed and the final condition negated to demonstrate the boundaries between concepts. For example given criteria A, B, and C:
It has been suggested that an effective way to pose such questions would be to reveal the criteria one at a time, but in such a way that a typical solution to part 1. would satisfy 1-3 inclusive. When part 4 is posed the real learning begins as the original answer becomes invalid.
Other more straightforward examples:
Example question 7 Find a cubic polynomial with a turning point at x=1 passing through the co-ordinates (-1,2).
Example question 8 Find a singular 3×3 matrix with no repeated entries.
In creating a counter example to a proposition or theorem such as
|
(1) |
there are two distinct stages. Firstly deriving the criteria, that
is stating and simplifying the criteria
|
(2) |
which the example must satisfy. Secondly providing the actual example itself and showing this indeed satisfies (2).
Example question 9 Prove that if n Î N then n2-n+41 is prime, or find a counter example.
Example question 10 Prove, or provide a counter example to, the following statement. ``If (an) is a sequence of non-negative real numbers that converges to zero then ån=1¥ (-1)n an converges."
Some theorems or techniques in mathematics require the use of facilitator objects. Given a question, the technique requires a complementary object to be constructed. These two must interact either, to satisfy a set of criteria, or to simplify the original question to some canonical form, to allow the application of a standard technique.
Typically this may be a substitution of some kind, or a change of co-ordinates. Where this is not provided or a hint is absent this question presents quite a challenge. In fact, questions that begin
Example question 11 Using the substitution u=... show that ...,
are ubiquitous. It is contended here that without providing an explicit substitution they would be classified as requiring the creation of an instance.
Example question 12 In dynamical systems theory proving the stability of a continuous dynamical system can be achieved by finding a Lyapunov function. This requires the generation of a facilitator object: namely the Lyapunov function itself.
Once the object has been found, the working is typically straight forward and relatively easy to mark. Factual knowledge about the general form of the required facilitator object is often indispensable but such knowledge does not immediately provide the solution. For example, while experience shows that ``almost all Lyapunov functions are quadratic-like" such questions can still be very difficult to answer. Furthermore, application of such knowledge demonstrates creativity, insight and the ability to transfer reasoning - all typical of higher level mathematical skills.
Some questions in mathematics require instances which are general. This apparent contradiction is perhaps best explained by the following example. When trying to prove, from first principles, that the sequence (an) defined by an: = [ 1/(n2+1)] converges to zero, the solution requires, in part, the solution to the problem
Example question 13 For each e > 0, find an example of an N=N(e) such that if n ³ N then | 1/(n2+1)| < e.
Notice that, in the context of the specific sequence, the solution requires for each e the generation of a specific N=N(e) that ``works". That is, the N must be an instance with certain properties. Compare this with the specific instance
Example question 14 Find an example of an N such that if n ³ N then | 1/(n2+1)| < 1/100.
Thus a distinction may be drawn between creating specific and general instances, as typified by the Example questions 13 and 14.
Current undergraduate course work has recently been evaluated, in order to better understand the effects of prototype hand held calculators equipped with MapleV2 [14]. This analysis generated the question classification scheme detailed in the previous sections. Results are given for two level one courses and their examinations. Course A is core algebra, calculus etc and takes place over two semesters. It was with this course in mind that the questions given in Appendix A were designed. Course B is a foundations of pure mathematics course. This takes place in the first semester in preparation for an analysis (sequences and series) course. In total 407 course work questions were analysed and details of the methodology adopted may be found in [14].
The results of this study show that 85.8% of course work questions, and 69.9% of examination questions are classified as either routine procedures or proofs. Many of these proofs, particularly in Course A are proofs chasing definitions via an algebraic demonstration. Some require verbatim transcription or only minor modifications. This is not intended as a criticism - these proofs provide ``proof templates" from which more complex proofs can be built. It is entirely appropriate for level one courses that students' confidence be build by requiring simple adaptations of prototype proofs. In course B the proofs are more substantial with additional logical structure, such as contradiction or induction. Questions identified as requiring students to use the higher-level skills (6-8) account for only 4.7% of the questions asked. The conclusion drawn from this preliminary analysis is that (i) the vast majority of current work may be successfully completed by routine procedures or minor adaption of results learned verbatim and (ii) the vast majority questions asked may be successfully completed without the use of higher skills.
It is evident that the type of questions of Section 4 generally do not have unique answers and so such questions are often difficult or time consuming to mark, particularly so at university level. Furthermore, questions which require the construction of an instance might appear to be necessarily advanced. This need not be the case. Consider the following example questions which test a simple understanding of complex numbers.
Example question 15 Given the
complex number z:=7+3i find:
When is your answer to (3) unique?
When a routine algebraic procedure exists to mark such a question this may be relegated to a computer algebra system, such as Maple. Recently [6] Maple has been combined with a web based assessment system to produce a genuinely sophisticated computer aided assessment environment. The key to this system is the marking procedure which must cope robustly with a variety of correct answers. This challenge can be met in some cases by computer algebra which can certainly:
Clearly this list is not exhaustive - only the surface potential of this system, in the light of the question design principles expressed here, has been so far explored. Take the following example question,
Example question 16 Give a differentiable function with a turning point at x=1.
If f(x) represents the student's answer the criteria we derive to check a solution is that f¢(1)=0. Thus to check the answer is correct we perform the following procedure
This may be done with the following (simple) Maple code:
Mark := ans -> evalb( subs(x=1,diff(ans,x)) = 0);
Briefly, evalb(·) evaluates the enclosed expression as a Boolean, subs is a procedure that substitutes x=1 into the second expression, and diff(·,·) differentiates the first expression with respect to the second. In this case Mark(x^2-2*x+3); and Mark((x-1)^4); both evaluate to true. Other conditions may be checked using Boolean connectives, so that to mark Example Question 7 we could use a Maple procedure
MarkAgain := ans -> evalb( subs(x=1,diff(ans,x)) = 0 and degree(ans)=3 and subs(x=-1,ans) = 2);
This uses the Maple procedure degree(·) to check the answer is a cubic and another substitution to ensure the function passes through the point (-1,2). Maple correctly marks the answer (x+1)*(x-1)2+2 for example.
In the case of Example Question 8 we could use a marking procedure such as
Mark := ans -> evalb (det(ans)=0 and nops(convert(ans,set))=9);
This uses the function det from the linalg package to check the matrix is singular and that the set of elements is of the correct size - the idea here is that sets do not contain duplicate elements.
In the case of multiple conditions, such as in Example Question 7, we would like to provide high quality tailored feedback and may want to give partial credit. If only one condition is violated, for example, the automatic procedure could indicate which. In the case of Example Question 8 this feedback might extend to giving a list of repeated entries using code such as
Feedback := ans -> FindRepetitions(Flatten(convert(ans,listlist)));
(which uses procedures from the Maple 7 package ListTools). This is possible in the AIM system, and has been done. A brief explanation of how some questions of this type were implemented on the AIM system are given in Appendix B. In Section 7 students' reactions to questions of this type as implemented in AIM are given.
It is not claimed that AIM is the only computer aided assessment system on which it is possible to assess in this way, nor that AIM represents the current best such system. However, competitors are not evident, nor does this question strategy seem to be encouraged.
The AIM system described is currently being integrated into level one (first year, UK university, mathematics degree) core material [7], with the help of an institutional Learning Development Unit (see http://www.ldu.bham.ac.uk) grant.
Analysis of the current work showed how rarely students were asked to think in creative ways, and how success could be achieved largely using mechanistic or rote learning. Since it was the intention to introduce questions into the course which departed from this, by demanding the creation of instances, there was concern that this was ``radical" and might alienate a large section of the student body. Such an outcome would be entirely counter productive. In an attempt to better understand students' reactions to questions of this type a group of six second year students were recruited to answer the trial questions detailed in Appendix A. This is a small study, never the less the results are revealing and interesting. Further work in ascertaining how students react and consequential refinement in posing such questions would no doubt be of great benefit.
The members of the group were selected and invited personally to join this study. These were students with a variety of experience but all were known to have been previously successful in the examinations for the core course. Furthermore, they were known to be comfortable talking in a group about their work. Two of the group had been previously employed to help with secretarial level coding work on the initial installation of the AIM system itself. The group was mixed gender.
The group was perhaps a-typical of the student body. This was deliberate. Firstly they could be better relied upon to report their thoughts honestly in a group - it was impractical to conduct individual interviews. Secondly, these students had succeeded on previous summative assessments - the material itself should present less difficulty than for other students. The experience of using the system removed, for two individuals at least, any barriers the system itself might present the students.
Students were recruited to ``test some questions I have written for the A core course. ... They are set using the computer aided assessment package AIM." Further, the students were reassured that ``I am testing the questions, not you. You may use any notes, books, calculators or other aids you like. However, please try to do the questions on your own, not as a committee!" This was to set students at ease, not patronize successful second year students by ``testing" them with level one material, and thirdly notes or calculators should in any case not help by providing an ``answer".
Setting the questions themselves presented a challenge. The characteristics sought were that questions should test simple core material and have non-unique correct answers which the student would have to create. Questions from a variety of topic areas, such as algebra, calculus, complex numbers, linear algebra, differential equations etc were set. Isolated from lectures this was found to be significantly more challenging than embedding questions in a coherent course. Discussion of the questions in Appendix A will be given below.
The web based nature of the AIM system allowed the students to take the tests when and where was most convenient. Two students worked from home over a weekend, the others used an open lab during the week. All students completed all the material, the vast majority (91%) correctly. Two students split their time into two sessions. By using the data logged by the AIM system itself, the time students took varied between an hour and an hour and a half. This information was subsequently confirmed during the interviews. All students met and discussed their reactions to these questions. With their permission this discussion was tape recoded, transcribed verbatim and analysed. Question 5 in Appendix A contained a bug in the computer code and correct answers were incorrectly marked by the system. Care also needs to be taken that a guess will not suffice. Unfortunately the automatic feedback given for question 9 promoted this behaviour.
CY: You actually get marks for guessing as well because it tells you its lower. `This needs to be lower.' If you get it wrong it tells you you need to be lower.
Students reported difficulties in recalling basic definitions. All took advantage of their notes, one student showed particular resourcefulness in this respect.
CY: When you said you could use anything, when I did that [Q8] I didn't even use my notes, I looked up symmetric matrix on the Internet.
Many of these questions exposed a lack of student confidence with concepts which resulted in over complex solutions to problems:
D: To begin with I thought you wanted a particular formula [for Q2], then I thought about it and thought, e just on its own - why not?
This student went on to say
D: At first I thought this [ie Q2] is a good question let's have a look at it, then I thought that if n isn't part of it, it won't matter so it won't depend on n at all.
J: It depends on your confidence of how you can put it in.
Interviewer: Why is that a cheat?
D: I just felt a bit tricked by it personally.
When asked why she had entered x3+x2+x+4 in answer to Question 1 one student responded
J: I don't view x3 as a proper cubic as a polynomial has more than one term as far as I'm concerned. I know technically it doesn't. But I assume a polynomial has a couple of terms.
That such questions have non-unique correct solutions should be
exposed. This could be achieved by asking students to give more than one different
solution to a particular problem. ``Different" here is taken to mean more than
trivial modifications. For example, the answers to Question 2
supplied by the students were
, [1student] 
, [2 students] 
, [1 student]
, [1 student] 
[1 student]
These students have responded with four identifiably different solutions: (i) the constant sequence, (ii) e+p(n) where p(n)® 0 as n® ¥, (iii) ep(n) where p(n)® 1 as n® ¥, and (iv) Euler's particular sequence an:=(1+ 1/n)n.
Asking each student to provide more of these, perhaps by asking ``Give an example, give a different example. Give a third example." would require them to think more deeply about the concepts. That is to say to expand the ``domain of variation" in which the student operates [18]. Example Question 5 expands the domain of variation of the concept of a vector space, for instance. At least one student responded
B: I don't know why I didn't put e, it didn't occur to me.
But, with hindsight, neither did the question encourage him to seek for examples other than his correct an:=e+[ 1/n]. Assessing what constitutes ``difference" in a particular circumstance could result in valuable discussion in tutorials and might generate the sort of discussions the students engaged in over Question 3.
JL: Ok, just take the parabola and shift it one.
...
B: I said, x-1=0, then integrated it.
These two approaches caused some friction in the group when the discussion moved to the general characteristics of the questions themselves. Student P was also `an integrator' while D was a `graph shifter'. Only reluctantly does P acknowledge an alternative method, which his voice intonation and manner revealed he had not considered.
P: Because if it says, give an example of a differentiable function which has a turning point at x=1 you have got to know what it means for a function to have a turning point at 1 and then infer back that means you can integrate this function so that it has ...
D: You don't have to do it that way! You could do it graphically like I did and consider x2 and move it along.
P: Yes, it is not a simple case of saying - this is a polynomial, where are its turning points? We know how to do that. You can differentiate it and set it equal to zero. This is saying `we are giving you a turning point what is the function?' You have then got to understand the process behind how you work out and apply it backwards ... or in this case you could sort of draw it and realize it is a shift.
The technique needed to succeed may pre-suppose fluency with the basic techniques. However, if a technique has not been mastered hitherto it is precisely the need to use the technique as part of a wider problem than will first expose any weakness and secondly encourage remedy. Question 9 exposed a weakness in understanding solutions to second order differential equations and generated much discussion.
JL: I approached the problem, ok I did the auxiliary quadratic equation then I thought ok its got to do something with the discriminant. And then is it less than zero, more than zero or equal to zero?
B: It requires you to have knowledge of what erm....
CY: ... what changes the general solution.
B: The form of the general solution you get from the auxiliary equation. So like, Acos(at)+Bsin(at)... I thought that was a prime candidate for the first graph. It looks like it could be described by that equation.
At the risk of exposing a failure on our part, students were adept at solving second order differential equations of a variety of types but uniformly had not considered the shape of the graph of such a solution.
B: Last year we didn't really look at graphs of differential equations did we?
CY: We were just told that if we manipulate the auxiliary equation we can get these different types of general solutions to these differential equations.
Interviewer: No attempt to relate the solution....
CY: Not to a graph.
D: The first graphs we did were Bessel's functions weren't it?
[General agreement - laughter]
Such behaviour is corroborative evidence that assessment drives learning: students had not been asked to consider the graphs of their solutions. With hindsight it would also have been interesting to ask them to ascertain the initial conditions from the graph.
Having discussed each question individually, the students were asked ``I designed these [questions] differently, that is why I've got you together. If I'd been doing normal things I wouldn't have done it right? But this is slightly different". To which B responded.
B: Question 9 was certainly asking us stuff that we had to think about.
Interviewer: In what way?
B: You didn't give us an equation and then say ``solve it". You have got to really think about what it means. You have to get a solution and then you think, ok that's the answer. Doing a question like that you think, argh, right, that is the shape of the graph.
As a group the students demonstrated a mature understanding of the purposes of these questions. The student generally agreed with P that such questions
P: ... test your understanding of the subject, rather than your ability to turn a handle.
This article has defined the higher level skill of creating instances of mathematical objects satisfying certain properties. It has been shown that students are asked to perform these tasks rarely. It has been explained how such questions may be assessed in practice without the imposition on staff of an onerous marking load.
There is great potential in the simple ideas contained in this paper: having the students create their examples is now feasible at undergraduate level in some cases where a computer algebra system may check their solutions automatically.
It is also conjectured that the skill of instance creation will aid students in checking conditions in other situations, such as when applying a theorem, when the hypothesis needs to be satisfied by the case in consideration.
The author would like to thank Joe Kyle, Grant Keady, John Mason and Anne Watson for valuable discussions and encouragement. Also thanks to the students who tested the questions for their diligence and permission to quote their comments. Introduction of the {\tt AIM} system described in this project has been funded in part by a grant from the Learning Development Unit (see http://www.ldu.bham.ac.uk)at the University of Birmingham, United Kingdom.
1. In the following questions we ask you to consider a polynomial p(x) and the following properties:
Give an example of a polynomial satisfying:
1. (a),
2. (a) and (b),
3. (a), (b) and (c),
4. (a) - (d) inclusive,
5. only (b) - (d).
2. By giving the formula for the general term an in
a sequence give an example of a sequence with limit
|
3. Give an example of a differentiable function f(x) which has a turning point at x=1.
Given the complex number z:=1+10i, [Random: real and imaginary parts in {1¼10}] In each case find a nonzero complex number w:=a+bi such that
Given a nonzero complex number z, is there a unique nonzero complex
number w that satisfies both z+w and z.w are real?
Multiple choice from:
[¯] No, there are more than one,
[¯] Not unique if z is real,
[¯] Yes, always unique,
[¯] Don't know.
5. Give a complex number z with nonzero imaginary part such that
6. Give an example of a nonzero singular 3 by 3 matrix.
7. This question tests that you can apply the definition of ``permutation matrix". (Reminder. A permutation matrix is a square matrix of 0s and 1s with precisely one 1 in each row and precisely one 1 in each column.) Enter a 5 by 5 permutation matrix P which has a 1 in its (1,2)-place [Random] and for which the smallest power of P which gives the identity matrix is P6. (Hint: Consider ``combining" a 2 by 2 permutation matrix with a 3 by 3 one.)
There are several correct different solutions. The question has been designed so that there should be a block diagonal permutation matrix solution.
8. Enter a 2 by 2 matrix A=[aij] which is symmetric and orthogonal, but which is not a diagonal matrix.
9. Consider the solutions to the differential equation
|
subject to the initial conditions y(0)=1, y¢(0)=0. Here the prime denotes differentiation with respect to t.
Examine the graphs below. For each give a value of a so that the
solution satisfies the initial conditions and has qualitatively the same behaviour as the
solution in the graph.
It is reiterated here, the AIM system is simply a proof of
technical feasibility. Other assessment systems may well exist which use computer algebra
to provide the level of sophistication provided by AIM. The system can be viewed,
and some public material tested live on
http://www.mat.bham.ac.uk/aim/
Questions are authored using a web interface to edit a plain text file containing lines of code. Each line begins with an AIM flag, such as line 1. of Figure 1 which begins with the flag h>, or continues from the preceding line which ends with a `\'. Full details of the AIM system are given elsewhere [6] but the following notes explain how such questions might be authored in practice.
Question 3 was encoded on the system as
t> Give an example of a differentiable function t> <i>f(x)</i> which has a turning point at <i>x=1</i> v> 2 ap> <i>f(x):=</i> s> [(ans)->`aim/Testzero`(subs(x=1,diff(ans,x))),x^2-2*x+3] end>
which is remarkably simple.
The t> flag generates text, as seen by the student. Note that HTML code can be embedded into this text. This could include HTML structures such as tables or internet links to notes, hints, images etc. In our case we use an italic text via the HTML code <i>.
The v> flag denotes the ``value", ie the number of marks for this question and ap> gives the ``answer prompt" that is displayed before the entry of an answer.
The s> flag contains a Maple procedure to mark the answers. Note the code performs algebraic calculations on the students' answer to check the actual criteria. There is no comparison to `the' answer. (Note in practice we use `aim/Testzero`, not Maple's evalb( ... = 0), see [6]) A student familiar with the system and Maple may choose to enter an expression such as int(x-1,x) in the knowledge that it will be evaluated. This may be prevented by forbidding answers containing certain expressions, such as int or Int
The twenty four lines of code shown in Figure 1 are sufficient to implement the following question which demonstrates some of the sophisticated potential of the system: randomization and feedback.
Example question 17 Give an example
of a cubic polynomial p(x) with the following properties
1. p(0)=1,
2. p(x)=0 at x=a and x=b
giving detailed feedback on the source of any errors. Note, the line numbers do not constitute part of the code itself. In this case there is a unique answer. However, these conditions could be used individually and this self contained question could be split, as has been done in Question 1 of Appendix A.
1. h> p_ :=rand(1..3)();
2. h> q_ :=p_+rand(1..2)();
3. t> Give an example of a cubic polynomial <I>p(x)</I>
4. t> with the following properties
5. t> <OL> <LI> <I>p(0)=1</I>
6. t> <LI> <I>p(x)=0</i> at <i>x=$p_</i> and <i>x=$q_</i>.</ol>
7. v> 4
8. ap> <i>p(x):=</i>
9. s> [ proc(ans) local marks_; marks_:=1;\
10. if not `aim/Testzero`(subs(x=0,ans)-1) then\
11. printf("Your polynomial fails to satisfy <I>p(0)=1</I>.<BR>");\
12. marks_:=marks_-0.25; fi;\
13. if not `aim/Testzero`(subs(x=p_,ans)) then\
14. printf("Your polynomial fails to satisfy <I>p(x)=0</i> at <i>x=%g </i>.<BR>",p_);\
15. marks_:=marks_-0.25; fi;\
16. if not `aim/Testzero`(subs(x=q_,ans)) then\
17. printf("Your polynomial fails to satisfy <I>p(x)=0</i> at <i>x=%g </i>.<BR>",q_);\
18. marks_:=marks_-0.25; fi;\
19. if not `aim/Testzero`(degree(ans,x)-3) then\
20. printf("Your polynomial is not a cubic.<BR>");\
21. marks_:=marks_-0.25; fi;\
22. marks_;\
23. end ,(1-x/p_)*(1-x/q_)*(1-x)]
24. end>
The h> flag pre-fix ``hidden" Maple commands. In the cases of lines 1. and 2. these generate random variables for later use in the question. Randomization prevents cheating by providing different examples to individual students. When used for formative assessment this can provide repeated different questions for practice purposes. This randomization can be very sophisticated using Maple's randpoly functions for example. Elements could also be selected randomly from lists and combined.
Lines 9.-23. (the majority of the question code) provide the marking regime and detailed feedback. This is Maple procedure, and in this case each of the four conditions are checked. If the student's answer fails then feedback is given and marks deducted. Line 23. ends this procedure and provides a correct answer, in terms of the random variables which will be substituted in before being displayed the system.
Maple plots can be included, as was done in question 9. Hints and sub questions for students struggling may also be included, as can detailed solutions, see [6].
1The harmonic series as an example of a divergent series generated by a non-negative sequence which converges to zero.
2MapleV