DA queries ‘artificial inflation’ of matric results

The Democratic Alliance on Friday expressed concerns about the standardisation of matric examination results for 2016, pointing to what the party said looked like artificial inflation of results.

In an open letter to the head of the Council for Quality Assurance in General and Further Education and Training (Umalusi) DA MP Gavin Davis raised questions about the process where many marks that were lower than the historical mean were raised, while marks that were higher than the mean were left as they were, and welcomed as a sign that the system was improving.

Davis noted in his letter to Dr Mafu Rakometsi that his intention in raising concerns and asking for evidence was “to safeguard the integrity of our examination system, and to ensure that all learners get a fair deal”.

At the standardisation meeting in Pretoria last Friday, Davis said, it was learned that marks were adjusted in 32 of the 58 subjects, compared with 29 a year ago. Of those 32 adjusted subjects, 28 had their marks adjusted upwards and only four downwards.

Matric marks are standardised in a joint exercise by Umalusi and the Department of Basic Education (DBE) in an effort to create consistency.

In an ideal world, students who graduate from high school in different years can be measured against each other according to the marks they achieved despite having written different exams.

Standardisation is effectively a control for an examination that is tougher than the previous years.

Davis wrote in his letter that, according to Umalusi and the DBE, adjusting the raw mark upwards is justified if the exam paper was demonstrably more difficult (ie more cognitively demanding) than previous years.

“However, no evidence has been put forward to demonstrate that these papers were of a higher standard.”

Davis pointed out that the starting point for adjusting the marks was not the papers themselves, but the results.

“In cases when the raw mark was worse than last year’s, the DBE went back to the paper and found difficult questions to explain the drop in the raw mark. The DBE then motivated for the raw mark to be adjusted upwards accordingly.”

Davis said this methodology seemed incorrect.

“It would seem that – as a general principle – the cognitive demand of the papers should be assessed independently of the marks. This should preferably happen before the papers are written so that the need to adjust the marks afterwards is minimised.”

The obvious problem of using the raw marks as an indicator of a paper’s cognitive demand, he added, was that the paper might be of the appropriate standard, but the learners were below the standard of previous years for whatever reason.

“Adjusting the marks upwards in such cases would therefore mask systemic problems that need to be addressed,” he said.

As a case in point, Davis tackled the related, and thorny, issue of progressed learners and how they might distort the picture. Progressed learners, in this case, are those who had failed the penultimate year of high school but had nonetheless been ‘pushed up’ to matric. This group, 109,400 learners, represented 13.4 percent of the total enrolment in the National Senior Certificate (NSC) examination in 2016, a significant increase from 66,088 in 2015.

“It follows, therefore, that that there was a significant increase in the number of weaker students who wrote the NSC this year. This raises the question of whether the inclusion of progressed learners in the standardisation process leads to certain anomalies,” said Davis.

He made the point that where raw marks were lower than previous years (when there were fewer progressed learners), the deterioration could be the result of the inclusion of the weaker students rather than the papers being more difficult.

“Would this not mean that the marks of non-progressed learners will end up higher than previous years when there were no progressed learners?”

Davis noted that the lack of intervention in a number of cases on the other end of the scale, ie where the marks were higher than the mean, was another cause for concern.

“There was little interrogation of why the raw mark was better than last year’s, and whether this could have been because the paper was ‘too easy’.”

Instead of applying the same methodology, which would have meant adjusting the marks downwards, the good marks were merely welcomed as a sign that the system was improving.

A glance at the data, however, said Davis, shows there may have been good reason to adjust these marks downwards to bring them into line with the historical mean.

“At the very least, there needs to be an explanation (based on an assessment of the cognitive demands of the papers) as to why such a downward adjustment was not made.”

In Physical Science, for example, the raw mark (mean) was 35.48 percent. The computer recommended a score of 34.45 percent to bring the final mark in line with the historical mean of 34.47 percent. However, Umalusi and DBE rejected this.

That was one of eight cases where the computer recommended an adjustment that would bring the final mark closer to the historical mean, but this recommendation was rejected and the higher raw mark was used instead.

Davis asked why the computer’s recommendation of downward adjustments was rejected, but its recommendation of an upward adjustment in 28 cases was accepted.

He said there was reason to believe that the standardisation process might have lead to an artificial inflation of the matric marks this year. He urged Umalusi to publish evidence that the exam papers for all 28 subjects that had been adjusted upwards were more cognitively demanding than previous years.

He noted that artificially inflating the marks could result in “a number of matriculants with passes and bachelor passes on paper, but lacking the actual cognitive skills to succeed in the real world of tertiary study and work”.

Davis told African News Agency (ANA) on Friday afternoon that there might be perfectly good explanations for all of the issues he was querying and, if so, he would like the public to hear them.

He is seeking reassurance that standards are being properly maintained and “the integrity of the 2016 matric exams is intact and that no learners have been unduly advantaged or disadvantaged in any way”.

Lucky Ditaunyane, communications manager for Umalusi, said he was unable to comment on Friday afternoon. He said they had received Davis’s letter, were studying it and would respond in due course.