Maths CAA Series: February 2003

e-Grade: A Computer Assisted Assessment System

Index to article

	Abstract
	1. Introduction
	2. Overview: Working with Questions and Assignments
	3 Maintaining Question Banks and Creating Assignments
	4. Teachers' Gradebook
	5. Comments and conclusions

 

Abstract

This article will briefly describe eGrade, a web-based system that can process a wide range of math-based assessment, while supporting content across all academic disciplines, that has been developed by Professor John Orr at the University of Nebraska-Lincoln. In particular, the process of creating questions, maintaining question banks, setting assignments and maintaining a course will be described.

1. Introduction

The development of the WWW has seen an explosion in the resources which are available to academic staff to produce and maintain on-line assessment. (See http://ltsn.mathstore.ac.uk/articles/maths-caa-series/index.shtml) However the ‘Holy Grail ’ remains the development of a system which:-

1. Permits the rapid production of a variety of question types which display mathematics as students write it;
2. Is flexible enough for staff to author and maintain question banks and retain some measure of ownership of the material;
3. Provides a courseware management system to maintain a range of assignments and classes with associated marking and grading schemes.

This article describes eGrade, which has all of these attributes and has been used extensively for several years by many different US universities. It supports multiple learning models, such as immediate, formative assessment that includes comments on incorrect student responses, proctored summative assessment, reviewing of old tests to monitor progress, or adaptive mastery testing (customising each student's session based on his or her demonstrated learning needs), while repeatable testing allows for multiple tries and self-paced drilling of example problems. Each version of a test is different, and there are security features to ensure student identity for tests and exams. Comprehensive information is available at http://jws-edcv.wiley.com/college/egrade

2. Overview: Working with Questions and Assignments

The variety of question types (including sophisticated Math and Science question capabilities), the range of assignment modes, and the powerful tools and reports the system can generate make this system an ideal productivity enhancement for instructors while giving students an individualized learning experience.

The system enables an instructor to conduct self-scoring assessment assignments drawn from banks of questions and provide comprehensive reports on the performance of individual students, performance across a class section or across multiple sections of a class, and even statistical item analysis of questions.

In addition a number of textbooks are offering publisher-provided question banks that are distributed in conjunction with textbook adoptions and may be built from textbook problem sets. This licence also allows instructors to edit existing question banks or create their own questions from homework, test items, or other class problem material.

Question banks are the basis to construct assignments. Using the same question banks for source material, you can design assignments that range from low stakes self-study sessions to homework sessions to high stakes, proctored tests requiring student login and proctor validation. When you create an assignment you determine its content, rules and policies, and then publish it for use by students.

  2.1 Question styles available

eGrade permits a large variety of question types that can either be static or dynamic. These include:

• Numeric - questions that require a simple numeric answer or can be set to require Physical units and permit a margin of error.
• Formula - questions that require a mathematical formula which is checked algebraically for semantics and range, for example, is known to be different from .
• Click-on-image – point and click type questions.
• Sketch – questions that require a graphical input by the user.
• Multiple Choice – which are fully randomised.
• Matching questions
• Fill-in-the-Blank questions
• Choose all that apply
• Matrices and vector responses

In addition questions can be grouped together to provide multipart questions and allow for partial credit. All the above are graded automatically, in addition free response type questions, which may be graded using key phrases, are also available.

2.2 Authoring Questions

Although it is possible to author questions in native HTML it turns out to be easier to build them as LaTeX objects and then convert them, first to XML and then to eGrade’s internal format called EDU. In order to use LaTeX to author question banks for EDU, it is first necessary to download the relevant EDU LaTeX macro files. Typically the authoring process involves a number of steps:

1. Author any content in LaTeX (using a plain text editor).
2. Run LaTeX on the .tex files and preview the result to make sure it looks ok.
3. Convert .tex files it to EDU format. This step requires a password protected web service.
4. Upload and install it to a class website using a Question Bank Editor in the system.

Currently, this process is stepwise but later versions combine some of these steps.

In addition, given the source code, either in LaTeX or EDU, it is very easy to modify existing questions or generate new questions. All that is needed is a simple text-editor.

2.3 Typical Questions

A full description is provided at http://www.brownstone.net/support/edu/authoring/ so we will describe only a couple of question types. This link gives complete documentation that is very well organised.

2.3.1 Formula type questions

Figure 1 shows the screen image of a typical question where the student is expected to differentiate the given function.

Figure 1: A simple formula type question as seen by a student.

The code required to generate this question in standard LaTeX is:-

 

\begin{question}{Formula}

\qutext{\qnumber{Chapter 3. Section 1. Question 18.}

Find the derivative of

$$y=45t^2 + 72\sqrt{t} + \frac{47}{t}$$

}

\answer{90t+36/sqrt(t)-47/t^2 }

\comment{$$y'=90t+ \frac{36}{\sqrt{t}}-\frac{47}{t^2 }. $$ }

\end{question}

eGrade provides two alternative methods for the student to respond,

1. a simple command line,
2. a two dimensional format by selecting the option to Change Math Entry Mode. In this mode the student selects appropriate templates and then fills them in by point and click.

In either case the expression that is input can be previewed to ensure it is syntactically correct and semantically what is required.

 

Typically each question has 4 parts,

1. the text of the question,
2. the answer,
3. a comment that may or may not be displayed if the question is answered incorrectly

and

4. the code required to generate any random parameters.

A major advantage of eGrade is the way such questions can be randomised. For example, the following code might also produce the question in figure 1.

 

\begin{question}{Formula}

\qutext{\qnumber{Chapter 3. Section 1. Question 18.}

Find the derivative of

$$y=\var{a1}t^2+\var{a2}\sqrt{t}+ \frac{\var{a3}}{t}$$

}

\answer{\var{b1} t+ \var{b2}/sqrt(t)-\var{b3}/t^2 }

\comment{$$y'=\var{b1}t+ \frac{\var{b2}}{\sqrt{t}} -

\frac{\var{b3} }{t^2}. $$

}

\code{$a1=int(rint(50)+2); $a2=int(2*(rint(50)+2));

$a3=int($a1+2); $b1=int(2*$a1);$b2=int($a2/2); $b3=int($a3);

}

\end{question}

The code for this example generates 2500 different problems. In addition to supplying a full documented solution it is possible to supply only the answer or to provide help in the form of hints.

2.3.2 Sketch type questions

The system provides a useful sketching environment as shown in figure 2. Initially the student is presented with a blank set of axes and then selects points by clicking on the screen. Meanwhile the programme joins the points to form a curve. The students response is checked by using one or more criteria; such as increasing or decreasing, concavity, passing through a given point or the slope at a given point.

Figure 2: Sketch type question.

 

\begin{question}{sketch}

\qutext{\qnumber{ Chapter 1. Section 1. Question 6.} The

population of Washington DC grew from 1900 to 1950,

stayed approximately constant during the 1950s, and decreased from about 1960 to 2000. Graph the population as a function of years since 1900.

}

\begin{sketch}[5]{1900}{2000}{0}{1}

\example{1900,0.5 1950,0.81960,0.8 2000,0.75} \check{concave_down}

\check[increasing]{1900,1950}

\check[decreasing]{1960,2000}

\end{sketch}

\end{question}

 

3 Maintaining Question Banks and Creating Assignments

The production of assignments is very straightforward; eGrade takes care of all the presentation and course management.

Once all questions have been created and assembled into a question bank then the production of assignments is a matter of point and click, as shown below in figure 3. Note that in addition to randomisation within questions the order in which questions appear can also be scrambled. Furthermore, groups of questions can be selected, in which case only one of the group selected will be presented in any one assignment. The courseware management system also allow the instructor to set the policies associated with any assignment, dates when it is available, the mode in which it can be used and the level of feedback.

 

Figure 3: Building an assignment and allocating policies

 

To see this assignment go to  http://egrade.brownstone.net/dev/classes/quinney/ and select the assignment labelled "Sample Homework". No password is needed as this assignment has been set up as anonymous homework.

4. Teachers' Gradebook

When an assignment is created, the instructor can control assignment access, grading parameters, and how grades are recorded in a system gradebook. (See Figure 4 below.) After students complete an assignment, you can view the results in a gradebook or create downloadable performance reports in Excel. Students can always review the results of any previously-completed assignments

Figure 4: An extract from a grade book

Individual student work can be examined by following the hyperlink associated with their name. Figure 5 shows an attempt at a differentiation question. Note that the Instructor can modify the mark awarded and to respond to the student when next they log in. This level of flexibility provided by eGrade’s course management system gives instructors complete control but avoids the necessity of maintaining large paper archives.

Figure 5: Student attempt at an eGrade question.

5. Comments and conclusions

eGrade provides a content authoring environment, a delivery platform and course management system which are completely integrated and very easy to use. The web based platform makes accessibility very easy and is platform independent. The type of questions available is extensive though the emphasis is on mechanical type questions and at first sight may not be appropriate for assessing deep learning. However, quite substantial questions can be constructed by combining simpler types and by using mastery type assignments where access to an array of questions is only permitted on the satisfactory completion of gateway tests. The randomisation of parameters makes repeated practice a meaningful and useful exercise when immediate feedback is made available.