Evidence re higher floor standards has “limitations” and can’t be easily disentangled from simultaneous changes, says DfE

Janet Downs's picture
 0
“There is strong evidence that higher ambition and higher floor standards lead to higher standards.”

DfE Press release, July 2013.

In August the Department for Education (DfE) provided what it claimed was evidence supporting the view given in the press release. However, this didn’t show higher floor standards raised results.  And two pieces of evidence actually criticized the accountability regimes in England (see here).

I asked the DfE to produce more evidence. This is the response:

The DfE says the number of below-floor schools has fallen since floor targets were introduced. However, it was “difficult to establish direct causality as a result of the introduction of other changes during this period”.

The next two pieces of evidence were from the United States. But the USA is not directly comparable with the UK. The USA has no standardized tests like Sats. Participation in the National Assessment of Educational Progress tests was voluntary for individual states until recently. The floor standards, or “thresholds”, weren’t standardized benchmarks as they differed between states

That said, what did the US evidence* reveal?

1 Stick-and-carrot accountability regimes were more effective than “report card” accountability. The latter resulted in no significantly different achievement levels from states without accountability programmes.

2 The “more students excluded, the better the results.” This is hardly a positive recommendation. However, the researchers said this finding “does not affect the estimated importance of accountability”.

3 Accountability closed the achievement gap between Hispanics and White but widened it for Blacks.

4 “Simple averages of annual test scores” can change over time when school intake changes. This was a “real problem” for schools with high pupil mobility rates.

5 Different ways of ranking schools resulted in many schools moving league table position. This meant good schools could be ranked as poor schools and vice versa depending on what indicators were used.

The discussion following the research said:

1 Accountability systems will only be successful if they are based on “actual quality” and not what the pupils themselves bring to school.

2 Flawed assessment would fail to judge schools “based on true quality”. This would discourage teacher recruitment and retention.

3 Although the results suggest that “high-stakes accountability” can raise achievement, findings should be interpreted with caution because other variables may skew results.

The other piece of evidence** said accountability systems introduced in the US had “a clear positive impact on student achievement.” The policy narrowed the Hispanic-White achievement gap but not the Black-White gulf. And there were negative outcomes – these included high exclusion and drop-out rates.

CONCLUSION: The evidence supposedly showing higher floor standards raise achievement doesn’t relate to floor standards except in so far as the number of below-floor schools has reduced since floor standards were introduced. But it’s difficult to disentangle floor standards from other factors. The US evidence mentioned “thresholds” but these were descriptions (“proficient”, “below proficient”) not numerical targets (eg 85% of 11-year-olds must exceed the “stretching threshold” in 2016). Targets were not standardized but differed from state to state. The accountability systems didn’t narrow the Black-White achievement gap and had unforeseen negative consequences.  And the findings should be interpreted with caution.

The DfE, therefore, has produced no “strong evidence” that higher floor standards raise achievement. Another myth to add to the list.

This thread is part of the “DfE Myths” series. Companion pieces are available here and here.

 

*Hanushek, E.A., Raymond, M.E (2006)

**Hanushek, E A, Raymond, M E (2005) available only in abstract except to subscribers.

 
Share on Twitter Share on Facebook

Be notified by email of each new post.





Add new comment

Already a member? Click here to log in before you comment. Or register with us.