This article contains many quotes from others (follow the links). All such quotes are in italics. The blog site ‘Reclaiming Schools’ has published another well-argued article about Progress 8 .
Progress 8 was supposed to be a fair measure of secondary school ‘effectiveness’. New research confirms that it is seriously biased against schools with more disadvantaged pupils. It is vital to expose this injustice because scoring ‘well below average’ on Progress 8 triggers consequences such as an early Ofsted, forced academisation or transfer to a more ruthless MAT. Poor Progress 8 scores can also lead to families avoiding schools they assume to be of poor quality.
The Progress 8 methodology has also been used by the Sutton Trust and the Social Mobility Foundation as the basis of flawed but persistent claims of a north/south ‘Attainment Gap’ allegedly caused by the failings of northern schools. I wrote about this extensively throughout 2018 starting with this article.
Cognitive ability (CATs) data show that the Sutton Trust and Justine Greening have got social mobility all wrong. The north/south attainment gap disappears when cognitive ability differences are taken into account. Failure to do this is resulting in the impoverishment of the curriculum of primary schools and invalid judgements of secondaries. The victims are children of all abilities that are denied the rich, developmental, inspirational state schooling that should be a human right, all sacrificed on the altar of free market ideology.
Progress 8 will have perverse consequences, as market-based incentives always do, because although he ended the curriculum and exams scam of the Blair era, Michael Gove failed to recognise that the marketisation paradigm is still corrupting, stultifying and degrading the school experiences of our students and their teachers. Progress 8 will fail to capture the essential qualities at the core of high quality, developmental education.
The ‘fit for purpose’ doubts of Progress 8 were raised some time ago by Tom Sherrington on the headguruteacher website. Fundamental statistical issues that he raises do not appear to have been addressed. I quote from his article as follows.
It’s all so convoluted; so removed from what learning looks like, turning ‘Progress’ into some kind of absolute metric. To begin with, there is an input measure – a fine sublevel – that is derived from the raw scores on two tests in different subjects. If you read my posts The Data Delusion or The Assessment Uncertainty Principle, you will see how far we move away from understanding learning even with raw marks. However, it appears that raw marks in different subjects are to be put through a convoluted mincing machine where 74 and 77 become 5.1. One number representing EVERYTHING a student has learned at KS2. On average. But then we get to the crux. Despite all the four sig fig nonsense, we actually end up with an outcome, in the worked example, where Progress 8 is 0.3 +/- 0.2. In other words; 95% certain to fall somewhere between 0.1 and 0.5. (Coincidentally, these are the same numbers for my school.). What we end up with is a super-crude 1 significant figure number falling somewhere within a range that is bigger than the number itself. Essentially, the whole palaver divides the Progress measure into three categories: Significantly above; average; significantly below. That’s it. The numbers actually don’t tell us anything significant at all.
Coincidentally, on the very day that I am writing this article, our local newspaper carried an article about the alleged failing of a local Academy School.
Schools fall below the government’s performance threshold if pupils fail to make enough progress across eight subjects, with particular weight given to English and maths. A secondary is considered to be below the government’s floor standard if, on average, pupils score half a grade less (-0.5) across eight GCSEs than they would have been expected to compared to pupils of similar abilities nationally. Four other schools in Cumbria; the Richard Rose Central Academy in Carlisle, Netherhall School in Maryport, Whitehaven Academy and Dallam School in Milnthorpe, were also included.
But the DfE does not measure the ‘ability of students’ nationally – it uses only the high stakes KS2 SATs scores. Where Cognitive Ability (CATs) data are available my colleague John Mountford has shown that SATs scores are inflated generally, and especially so for FSM and SEND pupils, who are always over-represented in schools that serve socially disadvantaged catchments.
Geoff Barton, general secretary of the Association of School and College Leaders, said: “Performance tables can never tell the full story of a school and we urge parents and governors not to place too much weight on them.
“The secondary school performance tables are inherently flawed in that the headline measure of Progress 8 which is used to judge the performance of schools effectively penalises schools which have a high proportion of disadvantaged children. The effect of this is to stigmatise these schools, making it more difficult to recruit headteachers and teachers and demoralising pupils, parents and communities.”
I happen to know quite a lot about the recent history of Walney School, which is situated on Walney Island, separated from mainland Barrow-in-Furness by a bridge across Walney Channel. In 2009 the three mainland Barrow non-religious secondary schools, Parkview, Thorncliffe and Alfred Barrow were closed as a result of a locally very unpopular and ultimately disastrous Academisation scheme supported by both the Conservative and Labour groups on Cumbria County Council. The driving force for the closures was the allegedly very poor GCSE results of Alfred Barrow School, where I had been head since 1989, retiring in 2003. Despite the school receiving a good HMI Report (1990) and successive good OfSTED reports in 1995, 1998 and 2004 (under my successor), it failed its OfSTED in 2007. Curiously, Parkview and Thorncliffe schools also failed OfSTEDs around the same time. The County Council Education Department along with all the pre-2007 inspectors, were well aware that the reason for its low GCSE results, which based on intake CATs scores, were actually remarkably good with lots of A and A* grades and good progression to further and higher education.
The Alfred Barrow admission cohorts never varied much from a mean CATs score of 85 (-1 SD). This placed the average pupil admitted at the 16th percentile. In a year group of 100 there would typically be only 10-12 pupils with above average (100+) CATs scores. Despite that pattern we always had, scattered throughout the year-groups, a small number of very able pupils in the 120 -130+ CAT score range. The senior Education Welfare Officer and a high proportion of the school’s employees, including teachers, admitted their own children to the school. This included our youngest child.
Cumbria Education Department realised that the school could not be closed with any realistic hope of any successor on the same site getting better results. Alfred Barrow had a roll of about 550, with Parkview and Thorncliffe having about 1000 each. So the plan was to close all three schools and replace them with a single new build Academy on part of the huge Parkview site, most of which could be sold off for lucrative executive housing. Thorncliffe School met the same fate. The assumption was that if the 550 very low average ability Alfred Barrow pupils were diluted by the much higher numbers from Parkview and Thorncliffe, then the new Academy should be able to rise to the challenge. This hope may have had stood some chance of being fulfilled had a significant proportion of the Alfred Barrow teachers been taken on by the Academy. This did not happen. None of the senior Alfred Barrow team were willing to transfer and neither did most of the other teachers, so virtually nothing of the successful Alfred Barrow practice was taken on by the Academy.
The new Furness Academy turned out to be a failing school disaster from the outset, about which a book could be written. This was despite an innovative proximity based admissions policy approved by Secretary of State Ed Balls. The innovation was that admission to the new Academy was based not on proximity to the Academy but to Walney School. Pupils that lived nearer to Walney School were not to be admitted to the new Academy. The effect was to transfer most of the former town centre Alfred Barrow, very low CATs score catchment area, to Walney School, which itself soon found itself in prolonged Special Measures eventually resulting in Academisation that has not prevented its (almost certainly unfair) new classification of being one of the worst schools in England despite having recently emerged from its Special Measures status.
As for Furness Academy, it was swiftly deserted by most parents living in the posher (higher CATs scores) parts of the former Thorncliffe and Parkview school catchment areas. These pupils now travel daily by bus and by train (when the Northern Rail services are running) to the very successful LA comprehensives in the nearby towns of Dalton and Ulverston (that have greatly benefited from this influx). More than 200 pupils per day make this journey, while the Furness Academy roll has drastically dropped. The original Furness Academy sponsors (University of Cumbria, Furness (FE) College and Barrow Sixth Form College) were removed by the DfE and replaced by nuclear submarine manufacturers, BAE Systems.
So marketisation, Academisation, floor targets, Progress 8 and league tables continue to wreak devastation not just on the now fragmented Barrow-in-Furness school system, but also arguably on the town itself following the destruction of its former popular, coherent and non-competitive LEA managed education provision.