Nuffield-funded research paper on England’s school accountability system appears riddled with errors

Analysis by this website reveals multiple apparent mistakes, references which appear not to support the claims made and sentences which seem lacking in clarity, in a paper which may be a case study in the dangers of over-relying on AI. However, UCL, under whose banner it was jointly published, has defended it.

 

A seemingly influential piece of research on how England’s school accountability systems work appears to be riddled with basic factual and conceptual errors, analysis by Education Uncovered can reveal.

In what may be a cautionary tale for the use of artificial intelligence in academic investigations, the paper, funded by the Nuffield Foundation, features sentences which appear to make little sense and references which do not relate to the topic to which the paper links them, among multiple other apparent mistakes.

The paper, published at the end of last year and entitled “Accountability policy in secondary education in England: Evidence from policy documents and research,” has links to major institutional and individual names in research, including University College London (UCL) and the Education Policy Institute. UCL said the project’s “principal investigator” was another organisation: the Centre for Education Systems.

It was published as one of a host of papers contributing to a research project, backed by some big-name education policy thinkers, which aims to compare school accountability, curriculum and special educational needs and disabilities policies around the world. The project received £390,000 in total from Nuffield.

In the piece below, Education Uncovered analyses 46 statements within this paper which seem problematic, many of them in multiple ways.

However, UCL has defended the research as a “careful, robust, and transparent presentation of the available evidence.” The Nuffield Foundation said: “We are working with the research team to understand and address the issues raised.”

The paper is no longer available on CES’s website. This says instead that its publication or publications on accountability in England are “currently undergoing revisions”.