I have recently written a number of articles on the subject of the ‘Attainment Gap’ and the ‘Pupil Premium’. Having discovered the email address for Amanda Spielman from her letter to the Public Accounts committee, I have tried to engage her on these issues in relation to her recent statements about shifting the focus of inspections away from KS1, KS2 and KS4 performance data.
These attempts are invariably intercepted, eventually resulting in a bland restatement of current policy that fails to address the serious, fundamental issues that I, along with many others, are trying to raise. Progress, however, has now been made, as the latest of these replies is signed by HMI Sean Harford, National Director (Education). His letter to me was received as an uncopyable pdf email attachment. In it he states:
“During Inspections, inspectors will ask schools how they are spending the Pupil Premium and its impact on outcomes for pupils“.
There is a link to the current DfE guidance for schools on school attainment and progress data, but no engagement whatever with the issues raised with Ms Spielman in this article, to which I had requested a response. So here is another try in the form of an open letter published on my website.
Dear Mr Harford,
Thank you for your letter of 13 December 2018.
The central issues are the true nature of the ‘disadvantage’ that the Pupil Premium (PP) is designed to address, how this can be validly quantified and the nature and validity of the data used by OfSTED inspectors to judge whether a school has been successful in using the PP to overcome the assumed disadvantage.
The nature of disadvantage that generates the Pupil Premium
The DfE criteria are as follows.
In the 2018 to 2019 financial year, schools will receive the following funding for each pupil registered as eligible for free school meals ( FSM ) at any point in the last 6 years: £1,320 for pupils in reception to year 6. £935 for pupils in year 7 to year 11.
It is therefore clear that the assumption of the DfE is that the disadvantage arises from parental poverty. However Professor Allen points out that degree of parental poverty is not reliably indicated by FSM.
The OfSTED criteria for judging the effectiveness of school’s interventions
The pupil-based data stored and used by OfSTED to judge a school’s use of PP is based on attainment in KS1 and KS2. These ‘prior attainment’ data generate ‘expectations’ for improved individual pupil attainments needed in the next Key Stage to ‘narrow the gap’ with non-FSM pupils. Despite the fact that DfE data has failed to produce national evidence of any ‘gap narrowing’ as a result of this PP policy (despite false DfE claims to the contrary), schools continue to be subject to harsh OfSTED judgements and enforced interventions should PP pupils not reach their prior attainment-based targets.
DfE has made it clear that it does not collect or have any regard to ‘ability’ rather than ‘attainment’ data and has shown no interest in the work of my colleague, John Mountford, who has compared Cognitive Ability Test (CATs) with SATs data for cohort and FSM pupils in a number of secondary schools along the M4 corridor.
CATs results are statistically standardised and reported on the IQ scale that has a mean of 100 and a Standard Deviation (SD) of 15.
SATs results are now reported on a scale of 80 – 120 with a ‘minimum expected’ score of 100 for all pupils. This looks like a standardised IQ scale, but it can’t be, because DfE reported that in 2017, 61% of pupils exceeded the ‘expected’ score of 100. This being the case, the mean national attainment could not have been 100 (the mid point of the 80-110 scale)
Despite the lack of statistical comparability between SATs and CATs scores, the patterns found by John Mountford still point to unavoidable problems with the DfE assumptions.
CATs are now marketed by GL Assessment and were previously by NfER-Nelson. They have long been used by many Academy schools and MATs as the basis of ‘fair banding’ admissions policies approved by DfE to ensure cognitive ability balanced intakes. 11-plus grammar school admission tests have long been constructed on the same basis, which is to assess levels of cognitive function/IQ rather than recall of taught knowledge.
Unlike SATs, CATs are reliably predictive of GCSE performance based on huge amounts of data collected over many decades. Secondary schools that do not use them for admissions purposes are increasingly screening all their Y7 pupils with CATs because of a widespread belief by secondary school leaders that SATs are often inflated as a result of their high stakes nature incentivising ‘gaming’ and shallow ‘cramming’, rather than teaching for deep learning and that they therefore lack validity as base-line measures for judging pupil progress.
This is currently being raised as a serious issue by Amanda Spielman, the OfSTED Chief Inspector with widespread support across the education system.
In his study, John obtained data from 11 secondary schools where the intake FSM ranged from 4% to 44%.
In every school, the SATs scores were significantly higher than the CATs scores.
Both SATs and CATs scores for FSM (and therefore PP) pupils were on average significantly lower than the Y7 cohort mean with no clear relationship with the overall FSM percentage in the intake.
The FSM pupil SATs scores were on average 3.19 points below the cohort means.
The FSM pupil CATs scores were on average 7.56 points below the cohort means.
Further support for this pattern can be found in the national CATs data on p10 of this document published by GL Assessment.
These are reproduced below with the corresponding percentiles (percentage of population scoring below) in brackets.
Not FSM pupils
Verbal Reasoning 102.0 (55th)
Quantitative Reasoning 101.2 (53rd)
Non Verbal Reasoning 102.1 (55th)
Verbal Reasoning 91.7 (28th)
Quantitative Reasoning 93.2 (32nd)
Non Verbal Reasoning 94.1 (34th)
It is therefore clear that there is a huge gap between Non FSM and FSM pupils, but it is a cognitive ability, not an attainment gap.
This being the case, not only is the DfE/OfSTED attainment/ pupil premium gap assumption based on a completely false premise, but so therefore is the OfSTED inspection policy in relation to SATs data in general and to PP in particular.
On 14 December, an important and highly relevant blog was published on the ‘Reclaiming Schools’ website from which the following is quoted (in italics).
A major government-funded study [Influences on Children’s Attainment and Progress in Key Stage 2: Cognitive Outcomes in Year 6] run by Oxford University tracked large numbers of children from nursery school to leaving school.
Children of mothers with GCSE as their highest qualification had Key Stage 2 scores around the national average (i.e. the 50th percentile rank). The average score of mothers who had no qualifications was well below average – around the 30th percentile. Few of these children score above average in SATs, and some are right at the bottom.
The average score for children whose mothers had university degrees (or NVQ level 4) was very high – averaging at the 78th percentile. Very few of these pupils score below average, and many will leave primary school with SATs scores near the top.
Well educated parents are able to pass on many educational benefits to their children. [my bold]
The hidden differences between schools and areas
There are enormous differences between different parts of England in terms of adult qualifications. Partly this is the result of the brain-drain south – graduates moving towards London for suitable work. Two areas can even have similar Free School Meals levels but differently qualified adult populations. Imagine for example two areas showing 20% FSM and 80% non-FSM. Maybe very few of the FSM children have graduate parents. But suppose in one area the 80% non-FSM includes many graduates, and far fewer in the other. This is not unusual.
Compare for example Kensington and Chelsea (London) with North East Lincolnshire (Grimsby). The free meal data is almost identical, since affluent Kensington and Chelsea contains some areas of extreme poverty.
Kensington and Chelsea – 49% disadvantaged, 21% current FSM
North East Lincolnshire – 41% disadvantaged, 19% current FSM
In the former, 64% of adults have degrees, but in the latter it is only 22%. This more than explains the proportions of pupils passing their KS2 SATs (70% and 51%). Nationally 39% of the adult working-age population have university degrees (or other qualifications at NVQ4). In most of London it is well above average, even in areas which have traditionally not been regarded as affluent and which include serious pockets of poverty:
In many poorer northern areas with rundown coastal towns and de-industrialised cities it is in the 20s:
South Tyneside 29%.
No account is taken of this very important factor when local authorities and schools are judged by league table position or Ofsted. The teachers serving the poorest populations are simply told to try harder. [my bold]
Of course this doesn’t mean it’s right for children growing up in Grimsby to achieve less than those in Chelsea, but it’s pointless and offensive blaming their teachers. To change this we need a dramatic political change.
There is nothing new about this. The fact that school attainment is very strongly correlated with parental level of education has been established for decades.
Well educated parents are indeed able to pass on many educational benefits to their children. But what is the mechanism of this?
Is it that better qualified parents are more able to help their children with their homework, or that more cognitively able parents produce more cognitively able children? It seems likely that the latter is the case and that both ‘nurture’ and ‘nature’ play a part.
The key fact is that it is parental education, not relative affluence/FSM eligibility that is the main driver. The children of less well educated parents consistently fare worse in the education system regardless of relative affluence. Child development experts point to the amount and quality of conversation that takes place in the family home as the key driver of cognitive development. As we have noted, this has only a tenuous link with FSM eligibility, so it is not surprising that an FSM-driven PP policy is likely to be ineffective, provide poor value for money and victimise schools serving communities with low levels of parental qualifications.
In KS1 and KS2 an effectively targeted PP driver should surely be related to the developmental level of the child. Experienced teachers are far more expert in forming diagnostic judgements about the developing abilities of their pupils than OfSTED Inspectors, pre-armed with invalid data, spending a short time in a school. Where a developmental ‘gap’ is identified then PP funding can support appropriately targeted development enhancing interventions. Of course accountability for the outcomes of school approaches to teaching and learning is required. This article explains how this can be achieved.
In KS3 and KS4 the PP driver cannot be SATs, because these are flawed ‘high stakes’ measures primarily devised to drive competition between schools in a marketised education system that perversely incentivises inflated scores. Here too, it is clear that criteria based on dodgy measures of attainment targeted at FSM pupils will result in the misdirection of resources to meet high stakes, statistically flawed ‘attainment’ targets, when it is cognitive development where the fundamental ‘gap’ lies.
The obvious solution is to scrap SATs and replace them with CATs screening of all pupils at the start of Y7. This was the Cumbria system in the early years of Local Management of Schools when LEAs were able to distribute considerable resources and funding under their ‘Non-Statutory SEN’ budgets. The Cumbria ‘PP’ at ascending funding levels was driven by CATs score boundaries at various levels below 85 (-1SD). The Verbal, Quantitative and Non-Verbal Reasoning CATs sub-tests informed further diagnosis of the developmental deficit of each individual pupil. I discuss this here.
Nature, however, also has a role. Although the ‘Reclaiming Schools’ blog authors are uncomfortable with inherited cognitive ability/IQ, it would be unsurprising if cognitively able women tend to seek out their cognitive equals when selecting the fathers of their children. Recognising that our genes make a significant, though currently unquantifiable contribution to educational outcomes has to be viewed as a factor that helps explain why the children of cognitively able parents tend to do well at school. I have always taken the view that it is best for schools to concentrate on things they can influence rather than those they can’t.
Given what we now know about the scope for cognitive enhancement of all pupils through appropriate approaches to teaching and learning, this surely represents life chance enhancing opportunities for all pupils of all abilities, that teachers should and would be excited about, rather than threatened by.