The controversies surrounding the National Eligibility cum Entrance Test (NEET) examination, the cancellation of the UGC-NET examination, and the postponement of several others call into question the ability of the National Testing Agency (NTA) to conduct large-scale examinations. While there are several issues relating to these examinations, one that has been of great concern to academics is the exclusive use of the multiple-choice question (MCQ) format.
Following a Cabinet approval in November 2017, the Union Ministry of Education created the NTA in 2018 as a registered society. It was provided a one-time grant of Rs.25 crore to begin its operations “as an autonomous and self-sustained premier testing organisation” in higher education. In the order notifying the NTA, the government said it would give policy directions that the NTA was bound to follow. The restraint on the NTA’s autonomy was underscored by the government’s role in the recent cancellation of examinations.
Also Read | The fallacy of one nation, one examination
The responsibility for conducting various all-India entrance examinations was transferred to the NTA from bodies such as the Central Board of Secondary Education, the All-India Council for Technical Education, the Council of Scientific and Industrial Research, the University Grants Commission (UGC), the Indian Council for Agricultural Research, and the National Council for Hotel Management. This was done so that “these agencies are relieved of the responsibility of conducting these examinations and can focus on their core activities”. Subsequently, Central universities—JNU, Delhi University, and Indira Gandhi National Open University—were made to transfer their entrance examinations to the NTA. The Common University Entrance Test (CUET) was floated as the single point for admission to undergraduate and postgraduate programmes in universities and colleges. In 2024, at the UGC’s behest, Central universities replaced their own written examinations for admissions to PhD programmes with a system of using the scores secured by candidates in the NTA-conducted National Eligibility Test (NET) examination.
How efficient is NTA?
The NTA’s mission statement aims to “improve equity and quality in education by administering research-based valid, reliable, efficient, transparent, fair and international level assessments”. It is unclear how the NTA, which as a financially self-sustaining organisation must generate its resources from the fees paid by those appearing in examinations, can develop the institutional capabilities this mission requires. The NTA’s RTI disclosures also show that six years after it was created, it has fewer than 15 regular employees, from the level of Director General to Senior Superintendent, and most of its staff are non-academic.
There is little evidence of the NTA having actually done any serious exercise or undertaken any wide consultation to determine assessment methods and procedures that would be most suitable to achieve the twin goals of equity and quality across the wide range of examinations it is supposed to conduct.
“Writing skills, the ability to synthesise material and concepts... cannot be tested in the MCQ format.”Ira BhaskarRetired professor, JNU
The exclusive use of MCQs has become the norm in the NTA examinations. Teachers familiar with this format say that each question is supposed to have a single, unambiguously correct answer that must be picked by the examinee from among the choices given. This is supposed to eliminate “subjectivity” from the process of evaluating ability, aptitude, and learning levels of students. Additionally, the checking of answer scripts and awarding of marks can be transferred to machines, through optical mark recognition sheets in the pen-and-paper mode or computer-based tests (CBTs). The need for qualified human examiners and a process to standardise evaluations done by multiple examiners is thus eliminated. There is also no need for a system of re-evaluation as part of a grievance redress process: this can be reduced to simply checking objections raised by candidates, or claims of errors in the questions, before the declaration of final results.
MCQ tests, therefore, offer an easy option to handle the huge administrative challenges that inevitably accompany any alternative mode of assessment in large-scale examinations. This is perhaps the single most important reason why it is used so extensively. The “MCQ syndrome”, as one eminent academic described it, is thus an automatic fallout of the “one nation, one test” approach. All the more so because the responsibility for several tests has been thrust on a thinly staffed organisation with a very short history.
Excessive reliance on MCQ is globally controversial
Excessive reliance on the MCQ format has been controversial the world over, including the US where it originated and came to be used extensively. It is well recognised among educators that preparing a good multiple-choice test is not a simple exercise. Academics contacted by Frontline said that when the truth is a matter of interpretation and where there are contending interpretations, as is the case in several fields of knowledge, it is not easy to formulate purely “objective” questions that do not reflect in any way the perspective of whoever sets the questions. In the current Indian context, this can even lead to a solution worse than the problem, namely, the imposition of a particular version of the truth as correct if examinations are conducted under the control of a government with its own ideological agenda.
Students being checked before entering the NEET examination hall in Mangaluru’s Canara College. | Photo Credit: Manjunath H.S.
Constructing questions that are free of bias is even harder if they are to have a single correct answer and yet test the examinee’s abilities of reasoning and deduction. Testing the ability to reason is different from merely testing the ability to remember facts. Even the ability to recall the correct answer can be imperfectly tested in an MCQ test because right answers can be “recognised” in the options given, which is not the same as remembering them. Since the MCQ format also opens up the possibility of “guessing” the answer or arriving at it by elimination, the incorrect choices also have to be appropriately framed to serve as effective “distractors”. Getting a correct answer by pure chance also has a high probability in the MCQ format, but this can be reduced by methods such as negative marking for incorrect answers. These are all well-known challenges of using MCQs. How the NTA has fared so far in meeting these challenges has never been systematically examined. It is doubtful if the organisation has developed appropriate mechanisms to ensure that its MCQ tests are of the highest quality.
Limits of MCQ as a method of assessment
The second widely accepted problem with the exclusive use of the MCQ format consists in its limits as a method of assessment. Questions that cannot have a single correct answer have to be excluded. Several fields of knowledge, however, involve a scientific approach—a combination of reasoning and assessment of evidence to arrive at conclusions—without single and unambiguous answers being the norm. This is not simply the case in the social sciences and can hold true in fields like medicine. The MCQ format also renders invisible to the assessment process the steps through which the correct answer is arrived at by the student. This has implications for assessment across all fields, including the natural sciences and even mathematics where a single correct answer to every question may actually be the norm.
Nandita Narain, former president of the Delhi University Teachers’ Association, and for several decades a celebrated teacher of mathematics at Delhi’s St Stephens College, describes the MCQ format as a shallow and unsatisfactory method of testing. She says that “what is most important in higher mathematics is the process of reasoning and logic applied in arriving at a solution to a mathematical problem, and it is entirely possible for the answer to be correct even when the logic used is flawed. This reasoning ability is not tested in the MCQ format.” She also asserts that performance in MCQ-based CUET does not reveal to teachers “how solid are the school basics of students admitted to undergraduate programmes”.
Ira Bhaskar, a retired professor of cinema studies at JNU’s School of Arts and Aesthetics and for many years previously a teacher of English literature at a Delhi University college, was extremely concerned when JNU made the shift in 2018 from its own entrance examination to the MCQ-format CBT conducted by the NTA. She became the lead petitioner, along with several other JNU faculty members, in a case in the Delhi High Court challenging that shift. “Writing skills, the ability to synthesise material and concepts, and the capacity to develop arguments and articulate them cannot be tested in the MCQ format. These are, however, the most important abilities needed in fields that study creative expressions—like literature, cinema, the arts, and performance,” says Bhaskar.
How MCQ promotes surface learning
The exclusive use of MCQs has become the norm in the NTA examinations.
The MCQ format requres each question to have a single correct.
Constructing questions free of bias is hard when there must be a single correct answer.
Such a format is not useful in testing the critical and analytical faculties of aspirants.
MCQ format also does not test writing skills.
MCQ is imperfect even in testing memorising ability.
MCQ often allows the correct answer to be arrived at through elimination.
It is doubtful that NTA has developed mechanisms to ensure its MCQ tests are of the highest standards.
Promoting surface rather than “deep” learning
International studies have highlighted the fact that evaluation formats tend to have an effect on a student’s approach to learning. Excessive weightage to the MCQ format tends to incentivise a focus on acquisition of a narrower range of abilities and to promote surface rather than “deep” learning. This becomes a serious issue even if the method is used only for entrance examinations and not to evaluate performance in the academic programmes to which students are admitted. A broad-based learning in such programmes, in fact, gets relatively devalued when all admissions from the undergraduate stage to research programmes, and the securing of fellowships and eligibility to be teachers in higher education institutions, depend critically on performance in MCQ-based competitive examinations. Such limited learning can have a cumulative effect through successive stages of higher education.
On the other hand, the abilities that may not be tested in the MCQ mode—the ability to think independently and critically, to be innovative and creative—become more and more important in every step of the transition from the final years of schooling to PhD. This is true for all areas of knowledge. Even the current Vice Chancellor of JNU appointed by the Modi government has made several public statements questioning the appropriateness of this format for admissions to postgraduate and research programmes.
The exclusive use of MCQ-based testing for judging abilities has implications for equity and fairness of those examinations. That they can be more amenable to the use of unfair means was brought home by the case of the NEET examination, where candidates were able to memorise overnight the correct answers to leaked questions.
Also Read | Plans to overhaul higher education afoot despite Bill awaiting Parliament go-ahead
The petition by the JNU teachers also highlighted how differences in access to computers and the Internet and consequent differences in computer literacy can work to the disadvantage of those from less privileged backgrounds in a diverse country like India. The narrowness of the abilities tested and the greater role that access to expensive coaching can play in developing those are also aspects that have been flagged as inimical to equity considerations. Narain describes coaching centres as “hotbeds of corruption whose success can depend more on their ability to access question papers”.
On the other side, however, it has also been argued that the MCQ format is relatively advantageous to underprivileged students as it removes the handicap of less developed language and writing skills that are the result of their circumstances and unequal access to education. The counter-argument to this has been that equity and quality are not incompatible with each other, and there are other methods of “equalising” performances of students with diverse backgrounds in examinations. In addition to reservation of seats for designated disadvantaged groups, the system of “deprivation points” pioneered by JNU, which involves criteria-based award of such points over and above the marks secured in the examinations, has been referred to in this regard.
The JNU Teachers’ Association has highlighted the correlation between the shift of the entrance examinations to the MCQ-based CBT and the marked decline in the proportion of women students in the university, reversing the previous trend. The JNUTA has also highlighted how the examination fees that students had to pay increased significantly when the shift was made from its own entrance examination to one conducted by the NTA. Narain pointed to her experience of a significant reduction in the regional and cultural diversity of the classroom after Delhi University’s shift to the MCQ format and the CUET. This, she argues, has adverse consequences for learning for students by narrowing exposure to diversity.
The qualitative implications of the MCQ format in higher education are many, as are the challenges of equity and diversity. These, which academics say are an integral part of the process of acquiring an education, deserve greater attention than they may yet have got.
COMMents
SHARE