Skip to main content

Collapse the levels of the system: the local authority role in London Challenge

  • By Jon Coles

London is a great and complex city. Globally, perhaps only New York rivals it for diversity, cultural wealth, influence and its extremes of wealth and poverty. It is at once the country’s centre of politics, government and the legal system, a major global financial centre and a centre of the creative industries. It has at the same time disproportionately many of the poorest wards in the country and particularly of children growing up in poverty.

The impact of place on the nature of public services is under-discussed and under-analysed. London creates a unique challenge. In the first place, it has all the complexities and problems of any large UK conurbation – the inner urban issues of deprivation, worklessness, crime, family breakdown and substance misuse, for example. Simultaneously, it is the powerhouse of the national economy – the wealth creating centre of the wealthiest part of the country.

If families in the top quintile of the income distribution are disproportionately likely to opt out of public services – to use private schools and hospitals, for example, as many do, so the already larger than average group of poorer families are even more concentrated in the capital’s state-funded schools. And such is the level of residential intermingling of rich and poor in inner London (with very valuable owner occupied houses directly adjacent to very deprived estates in many places) that the ‘middle class comprehensive’ seen in most other cities was largely absent there – reducing further the likelihood of some wealthier families choosing the state system.

Perceptions of London schools in 2002
When we started London Challenge, this contributed to an already well-established narrative about local London schools. Headlines had been generated repeatedly by the decisions of senior London-based figures in the government to send their children to private schools, grammar schools or other schools far from home in preference to local state secondary schools. These headlines resonated naturally with a national view that London schools were particularly poor. And this perception was easily reinforced by a largely London-based media.

On a range of measures, London schools were indeed doing worse than those elsewhere. Results were lower in Inner London than anywhere else; Ofsted ratings of schools, of their leadership and of behaviour were worse than elsewhere; there were proportionately more schools below government floor targets than anywhere else; and more in special measures than anywhere else.

As we look at today’s schools London, it is undeniable that there has been dramatic change. Its secondary schools are the highest performing in the country; Ofsted judges schools, their leadership and behaviour, to be better than in any other region; there are fewer below floor target than anywhere else; and fewer in special measures than anywhere else. Too often the word ‘transformation’ is used; but in relation to London schools, it is used correctly. London schools have genuinely been transformed.

However, even in 2002, the schools were not quite as much worse than others nationally as was commonly thought. In fact, close analysis of the evidence showed something perhaps more alarming than the story that ‘London schools are awful’. In fact, London schools were not all that much worse than similarly deprived schools nationally – there were just many more schools with high deprivation and low achieving groups in London than elsewhere.

It is easy to forget that before 2002, we did not have pupil-level data, so couldn’t see detailed results by gender, poverty and ethnicity. The 2002 dataset was treated as experimental and never published, though the patterns it showed were confirmed by the coming years’ data. What it shows is that London pupils didn’t do all that much worse than pupils with similar characteristics in the rest of the country. There were just more of the lowest achieving groups in London than elsewhere.

The critical issue to address was the shocking and indefensible difference in educational outcomes between different groups of young people. While more than four out of five girls of Chinese origin not eligible for free school meals achieved five or more good GCSEs, fewer than one in five white and Black Caribbean boys on free school meals did so. London wasn’t so different from the rest of the country on this measure; but it was particularly unacceptable in the most diverse city on the planet. Changing this pattern for the better would change the performance of schools.

And so, we formulated our goal: that London should become the world’s leading city for learning and creativity, and that critically, it should break the link between deprivation and low educational attainment.

On examining the evidence, talking widely to people involved in London schools and visiting schools, some things stood out very strongly.

First, there were indeed schools facing acute problems. At the extreme, schools where a visitor, let alone a pupil, might not feel very safe, and where the quality of education was very poor.

Second, that there were parts of London which had concentrations of schools with low attainment; often in very central parts of London and in places where ‘opinion formers’ disproportionately live, whose views of London schools (and schools nationally) were strongly influenced by what they saw around them.

Third, though, that the individuals leading those schools were often struggling to control much bigger forces which also affected other schools, albeit to a lesser extent. Schools at the bottom of a local ‘pecking order’ were unappealing to both pupils and teachers because of their reputation. With continuing teacher shortage in London, attracting good staff was particularly difficult for these schools. Retaining them was just as hard, when success in a tough school might easily lead to opportunities nearby in a less tough one. ‘Middle leadership’ appointments were often ‘battlefield promotions’, as able but inexperienced staff took on leadership tasks before they were really ready, in the absence of any alternative. Meanwhile, unattractive to pupils and therefore not full, these schools faced high levels of turnover and of in-year admissions – taking in many new arrivals to the country, excludees from other schools and other vulnerable children.

If these were some of the challenges, then the fourth key insight was critical. There was extraordinary practice in London. In a vast city of extremes, it was perhaps unsurprising that we ended up concluding that everything we needed was already there – all of the ideas and the energy and the extraordinary capability of London was enough. We just needed to create better conditions to allow it to flourish.

The strategy
The strategy that we developed was intended to work at three levels simultaneously: addressing the London-wide factors which had an impact on all schools, in order to enable all schools to succeed better; working across London with those schools facing the most difficult challenges; and working to create dramatic change in areas which faced the most difficult combination of low historic attainment and deprivation. 

The language of ‘Key Boroughs’ and ‘Keys to Success Schools’ was carefully chosen. It reflected a general stance of wanting to ‘get behind’ schools, rather than join the chorus of criticism. The words ‘Keys to Success’ were Tim Brighouse’s, reflecting his personal commitment to an ethos of energising rather than problematising. But these were not empty words – they also reflected the real ambition of the strategy.

If the key goal of London Challenge was to break the link between deprivation and low educational attainment, then the schools facing the greatest challenge of deprivation and historically low attainment had a critical role – they genuinely were the keys to the success of the strategy.

Likewise, schools in the five boroughs initially chosen for intensive focus needed to succeed in ways in which few schools in those circumstances succeeded nationally if London Challenge’s goals were to be reached.

At each stage, an important consideration was how to use the knowledge and resources which already existed within London. If what was currently seen as unusually good practice could become the norm, then improvement could be rapid. If the wider wealth of resources of London could be brought to bear, then there was great potential for something exceptional to happen.

London local government
Nationally, local education authorities had been facing many years of policy ambivalence about their role. In the preceding Parliament, the post-1997 Labour government had abolished Grant Maintained schools, replacing them with Foundation schools, funded on the same basis as other maintained schools. At the same time, however, the Code of Practice on LEA-School relations had made clear that the local authority role in running schools was to be sharply curtailed, restricted to ‘intervention in inverse proportion to success’. Meanwhile, a policy of intervening in the weakest local authorities had been developed, using private sector providers to replace poorly managed authorities. And the first ‘City Academies’ had been proposed, though none were yet open.

In London, it was visible that the local authorities were contending with some of the same problems as the schools. Ofsted assessments of London LEAs had been much more polarised between the very strong and the very weak than was normal nationally – there had been government-enforced external intervention in four of the five Key Boroughs. Specific local authorities had repeatedly been singled out for criticism.

Meanwhile, the cost of living in London and the pressure created by pay increases in other parts of the public sector (including for head teachers) meant that some local authorities faced difficulties in attracting and retaining the best staff. Once again, the most challenging authorities faced the most acute problem, as talented staff could be attracted by a nearby authority with equally interesting roles but slightly less intense problems.

At the same time, there were excellent examples of local authorities which had led significant improvement in schools after a period of severe criticism. By 2002, Tower Hamlets, for example, was already being recognised nationally for the quality of its work to engage the community and improve standards in one of the most deprived parts of London. As with schools, however, it was very easy for excellent practice in one local authority to be trapped within its borders, unknown even to its neighbours.

Collapsing the levels of the system
I came to characterise one aspect of the approach of London Challenge as aiming to ‘collapse the levels of the system’. I should explain what I mean by that.

In debates about education reform in this country and abroad, it has been common to talk about a ‘middle tier’ – what lies between the ‘top layer’ of the jurisdiction – the national, state or provincial government –and the ‘operating layer’ of the schools and other educational institutions where children and young people actually learn. Indeed, the legal framework of the English education system had been predicated on precisely this model since the 1944 Education Act put the relationship between church and state in the provision of schools on something close to its current footing. The Minister or Secretary of State had certain key powers to set the overall framework, within which local education authorities secure sufficient school places. Schools established by local authorities were then ‘maintained’ by them to provide ‘efficient’ (understood in the jargon of the Act to include ‘effective’) education.

A system with multiple levels is a standard part of the design of education services across the world – the most and the least effective. The specific features of the system, including precisely what is done at what level, differ in different jurisdictions, but the ‘levels’ can be seen even in relatively small countries or jurisdictions. A similar approach can be seen in many other public services, in this country and abroad.

The rationale for this approach and its advantages are fairly clear. There are things about which we might want clear national policy: funding of schools and their governance, rules about who may and may not teach in schools, what is to be taught, the qualifications system and so on. Equally, there are aspects of a school system which cannot easily be determined at a national level – where new school places are needed, how local arrangements for admission and exclusion are to work in the interests of all, for example – but which are not matters which can be decided by individual schools.

However, there are system effects which recur very persistently wherever there is a system with a ‘top’, ‘middle’ and ‘bottom’. This is as true of an organisation with senior and middle management layers as of a large system with multiple layers of organisation. Those at the top tend to have moments of frustration – ‘why can’t they see what we want them to do?’ – those at the bottom tend to feel an equal and opposite frustration – ‘why don’t they know what the real world is like?’ Those in the middle are consistently pulled in both directions and blamed from both sides.

In a range of ways, in a complex system of this sort, the layers of the system can get in the way. In the first place, if agreements need to be passed up and down a chain, the process can be slow. If each layer deals only with the adjacent layer or layers, it can be difficult to get a contribution from all the relevant actors and difficult to develop creative solutions which have the ownership of all parties. And where a solution would require input from every layer, it can be difficult to achieve this.

We wanted to take an approach which recognised and took advantage of the fact that government has some unique capabilities – it has resources, access to knowledge and skills, and the ability to convene, for example, which no other body has. On the other hand, as Hayek points out, it suffers from information asymmetries – it knows much less about local circumstances and conditions than other people do. As a result, it easily does stupid things. Central government needs to draw in other parts of the system in order to use its powers wisely.

So, the approach we took consistently was to try to collapse the levels of the system: to get in a room with local authorities and schools and sort out joint solutions which were better than any of us could achieve acting alone. For government, this requires a dose of humility and therefore a high level of confidence: to be able to acknowledge what it doesn’t know and cannot do alone, while remembering its unique contribution. It requires high levels of trust, and it relies on partners who themselves are unusually open about their strengths and weaknesses in the same way, unusually willing to admit fault and to challenge government constructively in areas where government’s performance isn’t good enough.

Establishing the necessary trust requires a strong sense of shared purpose and commitment to a higher goal than the institutional interests of any organisation. It was this shared determination to achieve something together which was most important in opening up a different way of working.

In the remainder of this piece, I describe why this approach was an important ingredient of success.

Developing strategy jointly
Early in the life of the London Challenge in 2002, we held a big event at the Renaissance Hotel, to which all London’s Directors of Education were invited. Tim Brighouse and I presented – Tim inspirationally, I at great length with a lot of data. But the main event was the discussion which followed at tables, between the boroughs and the central team.

We looked hard at the themes and the evidence, the key areas for action and the impact of these on schools and local areas. We considered the impact of pupil mobility, the patterns of pupil migration, the diversity of schools and the achievement of different groups, the very steep hierarchy of schools, the imbalance between boys and girls in mixed inner London schools, the demands of vocational education, the performance of schools serving the most deprived, the impact of admissions and exclusions policies and a whole range of other issues. We debated our proposed prioritisation and refined it. We agreed in principle that we would identify Londonwide projects which small groups of local authorities could lead on behalf of the whole of London.

At a subsequent meeting of the Association of London Chief Education Officers (ALCEO, later ALDECS), we put forward a list of projects which could be led in that way and agreed it. Within a few months of starting London Challenge, and before the strategy was even published, groups of three London Education Directors were leading projects designed to contribute to the London-wide strategy. Working with Anna Paige in the central team, serious contributions were made over time to improving transition from key stage 2 to 3, transition to post-16 education, pupil mobility, education of the lowest achieving ethnic minority groups and other key London issues.

The Renaissance Hotel event began a broader pattern of collaboration across the city. A crucial part of this collaboration was the routine meeting of ALCEO, chaired for a long period by Paul Robinson of Wandsworth, who provided a crucially measured and effective conduit, never shirking the difficult conversations, but always making sure that they took place in the right way. An early ALCEO meeting after that event was turned over more or less entirely to discussion of London Challenge, as the range of concrete actions to be proposed in the strategy became clearer. Those discussions were an important factor in shaping action, making sure it was tailored to the different circumstances and contexts of different parts of the city.

From then on, I was invited to every ALCEO meeting and (although London Challenge was a standing item on every agenda) included in all the wider discussions of the group on a very open and trusting basis. It meant that projects already being led by local authorities (such as the London Grid for Learning and the pan-London Admissions System) could be factored into our thinking – and we could be brought in to support as necessary. The rhythms and routines of collaboration may be mundane but become an essential part of communicating, monitoring progress, hearing feedback and making sure that concerns are addressed.

The equality of the relationship was, in my view, important. I was included in discussions which ordinarily I wouldn’t have been, because it was understood that my interest was in improving education in the city. I likewise openly shared things we were doing which weren’t directly affecting the boroughs’ responsibilities or which wouldn’t always have been shared with local authorities at such an early stage, because I felt a similar level of trust in colleagues.

As a result, our strategy from the start was informed by feedback and the involvement of local authorities and could constantly be refined in the light of experience and feedback. Likewise, Directors of Education were as well placed as possible to take advantage of things we were doing. This approach was mirrored and intensified in the five Key Boroughs.

The Key Boroughs
In the Key Boroughs, we had a simple idea of what we wanted to do: establish a vision for the next five years of how the pattern of education would be transformed in each area, publish it and then implement it. Simply stated, but impossible to do without the full involvement and engagement of the people concerned; and without that participation being committed and wholehearted, the chances of it being wrong, damaging or ineffective were high.

We sat down with each of the boroughs in turn – directors and key members of their senior teams. Tentatively at first, we began to explore the options for the future. We talked about government programmes that could be used locally and encouraged expansive thinking. We began to hear about what would and wouldn’t land well locally, about why some programmes were working for them and others weren’t. We talked about how it could be possible to make major changes to the building stock and about how we could most rapidly change the underperforming schools.

We began to talk about how government programmes could be adapted to fit local circumstances. We encouraged more expansive thinking and tried to introduce a governmental perspective of ‘what could be’, going beyond normal expectations. At the same time, we found ourselves becoming better grounded in the operational realities – how things are now, what could and couldn’t be done in what timescale. We were, in short, working together at the same level, bringing together our unique capabilities to achieve something that neither of us would have managed alone.

Less intensively, but equally importantly, we were keeping in touch at political level, ensuring that our messages remained consistent and political problems were addressed. Stephen Twigg’s visibly open, listening, trustworthy approach played a vital role in reassuring politicians and making sure that everyone had a shared understanding of the issues.

We produced a series of published vision documents – one for each of the five boroughs – setting out a five year vision and some concrete milestones for what would be achieved within one, three and five years. Uniquely in my experience, these were joint publications between the DfES and the borough concerned, with the leader or lead member of the council alongside Stephen Twigg in the foreword and at the launch.

This model of joint working intensified in the implementation phase. The arrangement we wanted to establish was to agree a joint implementation plan and timetable, and then monitor its implementation through half-termly ‘round tables’ held in the Department, through which we would look at what was and wasn’t on track and course-correct as necessary.

More controversially, we wanted there to be a dedicated project manager in each borough, whose role it was to ensure that the plan was on track, reporting to the round table on progress internally and externally. We would pay for the project manager, and there was understandable unease from all of the boroughs – was this a spy in the camp? The level of resistance to the idea ranged from moderate to very high. It took all of Hannah Woodhouse’s considerable diplomatic skills, as lead official in the team, to persuade the last two boroughs to agree.

I felt that we’d used up all our capital with the boroughs in one go in getting this model agreed. We now had to make it work. In the first implementation round table post-launch, we looked very honestly at progress. In each borough there was progress and there were surprising breakthroughs; in each borough there were unexpected problems: some in the authorities, some in schools, some in the Department. We dealt with each honestly, practically and with as much imagination as we could muster. I fear I began a pattern of being at least as hard on my colleagues from other teams in the Department as on the local authorities.

Hannah with Anfal Saqib and the project managers made sure that actions were followed up rapidly, even where complex, difficult or requiring policy adjustment by the Department. And so we established a routine of half-termly meetings, honesty and directness, constant action-focused communication with project managers, Directors and senior teams, and genuine sharing of problems and solutions. On each issue we shared the problem and considered equally what we could or should each do differently to change things.

The result of this working arrangement was a relationship of honesty, trust and productive impact as good as any I have ever known between government and local authorities. Bill Clarke (then leading Islington’s education service) later said to me: “When you next see Hannah Woodhouse, tell her from me that although I gave her a hard time about the project management arrangement, it was the best thing we ever did.”

I don’t think I appreciated at the time the level of courage it took for local leaders – particularly Bill, Alan Wood, Phyllis Dunnipace and Sharon Shoesmith to work with us in the way that they did. It was so typical of their outstanding and committed local leadership that I never doubted that they would. Each of them, and their teams, deserves great credit for the very rapid improvements to schools in their areas on their watch. Of political leaders who made important contributions, I would single out James Kempton’s impact over a long period in Islington as particularly important.

Their collective work and the work of some extraordinary schools leaders and teachers in those areas has seen some of the most dramatic improvement in standards in any area of the country – consistently being identified in the ‘most improved’ local authority areas by DfE for several years in succession. Each has GCSE results above national average – having been on average 15 percentage points behind in 2002.

The Keys to Success
The second element of the strategy – the Keys to Success schools – has been much more widely discussed. The impact in moving the number of schools in London below floor target from the highest in the country to the lowest, and the proportion of schools in special measures from the highest to the lowest, are well-documented.

The model was developed and led in the Department by Anna Bush and then successively Amy Collins, Hannah Sheehan and Natalie Abbott (now Yeo). It relied on very high quality London Challenge Advisers (a team led throughout by the brilliant David Woods), many of whom were attracted to the role by the prospect of taking a very different role in the final years of their career, working with government and particularly working with Tim Brighouse. Former heads, senior local authority staff and HMI like Victor Burgess, Hazel Taylor, Andrew McAlpine, Kate Myers, Doug Trickett and later Heather Flint brought a depth of recent, relevant experience and insight in a way which won instant credibility with colleague head teachers.

The role of the London Challenge Advisers was to diagnose (understanding the issues), prescribe (understand what needed to be done to address them) and coach (recognising that school leaders need to lead). They worked closely with excellent colleagues in the Department, who developed a series of relationships, interventions and resources which could be deployed rapidly to support schools and address different issues. One of the strengths of this model was the pace at which resources could be deployed and solutions implemented. Another was the bespoke nature of intervention – focused on the precise diagnosis of the issues, not on a generic solution to a specific problem.

In theory, all this could be done without local authority involvement. We took the view from the start, however, that it would be counterproductive to ignore existing local knowledge and expertise or to risk having different messages going to schools from different sources.

So we decided to begin in 2002 by holding ‘case conferences’ with each local authority, looking at their schools in detail – considering the ones we were concerned about, sharing knowledge and trying to reach agreement about which most needed the support we could offer. Counterintuitively to my mind, but very cleverly, Tim Brighouse persuaded us that doing this with small groups of local authorities together would get a more open, less pressurised, less defensive conversation.

It was another moment when local authorities could have chosen not to engage. One local authority did make that choice. In truth, we worked around them, went in to support their schools anyway (who were grateful) and waited for a new Director of Education a year or so later in order to engage more formally. Every other local authority came – some very openly, some warily. All spoke with knowledge of their schools, helped us to get to the right list of schools which needed support, understand the priorities and identify what forms of support would be needed. In each conversation, we got into joint problem solving mode.

Overwhelmingly, over the time of the London Challenge, we managed to stay in this mode. Difficult issues arose – there was sometimes a shared recognition that there needed to be a change of leadership, for example. Contrary to general perception in the Department, we found in several cases that it was neither lack of money nor lack of will that led Directors of Education to be cautious about addressing weak heads – rather, concern about the availability of better replacements. Being close enough to local authorities to see what was really happening allowed us to open up routes to appointing new heads which a single local authority didn’t have access to.

We aimed to be in the same mode with Keys to Success schools. By being alongside them, not in an accountability relationship, London Challenge Advisers had more open access to the real concerns and problems of school leaders than the Department had probably ever had before. The practical, on-the-ground information we received allowed us to see patterns more clearly, both within a locality and across the city. We could sometimes resource an intervention at school-level, local authority level or across the city which the LEA couldn’t resource itself. In several critical examples at school level, joint action by central and local government with the school itself fixed problems easily and elegantly which probably none of the parties could have solved alone.

There is not space here to discuss at length the other aspects of the strategy – some of which, like the ‘London Teacher’, ‘London Leader’ and ‘London Student’ aspects of the pan-London part of the strategy, were absolutely crucial to the success of London Challenge. There are many other lessons from that work and indeed from the work described above.

The focus of this piece has been on what I see as one key lesson of the London Challenge: collapse the levels of the system. By working with and respecting the other key actors in the system, government can add significant value and dramatically improve its own impact. Government can never hope to have perfect information and a single local authority or school cannot have the same access to resources or expertise as government, but by working in genuine and respectful partnership, they can significantly improve the impact they each have.

There is a risk that this message – and indeed other messages about London Challenge – could be seen as ‘soft’, compared to a hard-nosed world of direct intervention. The opposite is true.

Working in this way requires a willingness to stay in the difficult conversation, to hold others to account while looking them in the eye and to be held to account in the same way. It assumes that success against the measurable indicators is necessary and cannot be ducked – but refuses to accept that this alone is sufficient. It is an approach which requires risk taking and trust, and therefore courage and integrity. It involves commitment to a higher, shared goal; and agreement on a mechanism for holding one another to account and solving problems, an operational approach and a set of behaviours. In short, it entails discomfort as everyone has to move away from their established way of doing business – so cannot simultaneously be done in every area of activity. It is justified by a great enterprise of the highest public interest.

It was our great good fortune that underneath many differences of view and feeling, many practical problems and a multitude of political differences, central and local government found sufficient common cause to seize the opportunity of London Challenge and do things very differently for a time.


Jon Coles Jon Coles

Since 2012, Jon Coles has been Chief Executive of United Learning, a group of independent schools and academies established 130 years ago. Prior to this, his career in the Department for Education included four years on the Board as Director General, first for Schools and then for Education Standards, responsible for a wide range of policy areas, such as the review of the National Curriculum, the Academies Act 2010 and National Challenge. As Director of 14-19 Reform, he led the drive to raise participation post-16 and attainment at 19, reduce NEET numbers and reform curriculum and qualifications. As Director of London Challenge, Jon was responsible for developing and implementing the strategy to improve secondary education in London, which also led to similar approaches in other parts of the country. Jon was responsible for various Green and White Papers and took the 2002 Education Act through Parliament. A qualified secondary teacher, his previous jobs included implementing the infant class size pledge, and a period of cross-government strategy work at the Cabinet Office.