Competitive exams in India, particularly entrance tests for pursuing higher education, are mostly based on computer-based multiple-choice questions (MCQs). These tests have some advantages in terms of transparency and quicker evaluation. However, what ability these tests are intended to assess needs an understanding.
In recent times, performance in the 10+2 schooling has been used as a mere eligibility criterion. Only these entrance exam scores hold the golden keys to the higher education citadel. As these exams gain prominence and are projected to assess merit of the younger generation, they need critical scrutiny. We are going to examine these exams only from an academic perspective.
Mathematically, a multiple-choice question with n choices will have one correct answer and n-1 wrong answers. So every choice has an equal probability of 1/n to be the correct answer. For example, in an exam consisting of 100 questions with four choices, picking any one choice (called ‘guessing’ in the literature) for all the questions will fetch 25 marks.
This guessing can be prevented by giving negative marks for wrong answers. According to formula scoring, the negative mark should be equal to the ratio of the mark given for the correct answer and n-1. If the mark for the right answer is 1 with 4 choices, then the negative mark should equal 1/3. The logic is that even if the remaining 75 questions are marked wrongly, the 25 marks gained by guessing would be negated. Two situations can still benefit guessing: if the negative mark is less than 1/3, and when a particular choice among the 4 has more occurrences than the theoretical probability of 1/4.
Other intelligent guessing can also get a better score with negative marks. Consider a situation where a candidate is not able to choose the correct option but can identify one wrong option in every question; discarding it improves the probability of a choice being right to 1/3. After deducing the wrong choice, the same guessing can fetch a positive score even with negative marks. On the other hand, exams with no negative marks are simply flat, as no one can distinguish a score equal to the probability (1/n) times the number of questions as an outcome of guessing or factual knowledge.
The National Eligibility cum Entrance Test or NEET has 180 questions with 4 choices, where the correct answer gets 4 marks and a wrong one gets a negative 1 mark. Since the negative mark is less than 1/3, it cannot entirely negate the effect of guessing. Consider the recent NEET conducted in May, for which the answer key is available in the public domain. We observed that each of the four options renders a correct answer ranging between 21 and 30 percent, with a mean of 25 percent as predicted. If a candidate blindly chooses an option that is correct for 30 percent of the questions, she gets 54 correct and 126 wrong. So she can score 90 by mere guessing.
Apart from choosing one option for all the questions, finding different patterns to pick can also result in better scores. For example, a candidate who selects a pattern of one option for the first 90 questions and another for the remaining 90, gets a possibility of 56 questions being correct with a score of 100. In this scheme, there is another possibility for a candidate who prepares and knows the answer for only 25 percent of the questions and guesses for the rest. It can result in a score of 216 with a negative mark. It gives the opportunity to achieve the passing minimum merely by guessing, or by effectively preparing 25 percent of the syllabus and combining it with guessing.
The Common University Entrance Test or CUET for undergraduate admissions is in the same format, with 125 to 130 questions. A correct answer fetches 5 marks, and a wrong one gets negative 1. Since the negative mark is not optimal to negate guessing, the associated problems persist. As CUET is conducted across a broad spectrum of subjects for a vast number of candidates, it is conducted in multiple shifts and slots. So there is a need to normalise the scores for equivalence between the shifts and slots, assuming that the difference in the distribution of marks solely depends on the difficulty level of the papers in a slot and shift.
A statistical method called equi-percentile is used for normalisation. Any such statistical method suffers a severe drawback while interpolating a normalised score when there are outliers. In a slot and shift, a small group of brilliant candidates and a large number of average candidates write the exam, and the marked difference in their scores will definitely bring a considerable variation in the normalised score.
The very idea of exams like NEET and CUET is to set a minimum benchmark for entering the university system, as there is a problem with normalising the multiple boards’ school leaving exams. With the problem associated with multiple-choice exams, the purpose isn’t served.
One can counter that computer-based MCQ exams are the global standard, but there is a key difference. These international exams use a statistical approach to assess the student’s ability using the item response theory, taking into account the guessing parameter, item difficulty, and item discrimination.
One can also argue that a simple solution to the guessing problem in NEET and CUET is to increase the negative mark. But a much better solution would be to rethink the drawbacks of these tests and consider returning to the time-tested system of admission either with only the final school marks or a combination of them with entrance exam scores.
(Views are personal)
S Raja Sethu Durai | Professor of economics, University of Hyderabad
P Murugan | Assistant professor of management, University of Hyderabad