Summary

There is a long tradition in the UK of using area based initiatives (ABIs) to attack problems of urban deprivation.  In 1998 the government launched an especially ambitious ABI: New Deal for Communities.  In 39 areas local Partnerships are driving forward ten year programmes to narrow the gaps between these neighbourhoods and the rest of the country in relation to crime, education, jobs and so on.  Change data indicates that there has been continuing progress in NDC areas.  But change has been more evident in relation to place based indicators, such as fear of crime, rather then people based outcomes such as fewer jobs, better health and so on.  The Programme confirms that regenerating deprived areas is a complex process not least because of continuing demographic ’churn’ in these neighbourhoods.

The urban context in the UK

For four decades UK governments have sought to tackle problems of urban deprivation through the designation of Area Based Initiatives (ABIs).  Typically ABIs are time limited programmes designed to address problems impacting on defined urban localities.  Previous ABIs have been subject to considerable evaluation and debate (Department of the Environment, 1994; Gripaios, 2002; Shaw and Robinson, 1998).  Much of this has focussed on ’process outcomes’ notably ’community engagement’ and the apparent need to embed ABIs in ‘partnership working’.  This is not the place to engage with either of these well trodden debates.  The key issue here is not so much what has proved central to the ‘ABI agenda’ but rather what hasn’t.  In particular there has been a dearth of impact studies.  This emphasis on process and not on longer term impact is not surprising.  Central government has provided a robust theoretical and practical framework within which ABI evaluations should take place (ODPM 2004).  However in practice evaluations have often proved of limited value (Rhodes et al, 2005).  One immediate practical problem is limited resources.  Even when evaluations have been more generously funded many have been commissioned when the initiative concerned was well underway, indeed sometimes after ABI funding had actually ceased.  Not surprisingly, the 2001 Review of the Evidence Base for Regeneration Policy and Practice concluded that ‘there remains widespread neglect of issues such as the impact of intervention on both beneficiaries and anticipated outcomes’ (DETR 2001, p. 15).  However evidence emerging from the New Deal for Communities (NDC) Programme wide evaluation offers an unprecedented opportunity to explore the longer term impact of an intensive ABI.

The New Deal for Communities Programme

NDC Partnerships, launched in autumn 1998, were given the challenging target of helping to ‘turn around the poorest neighbourhoods’ (DETR, 1998, p.1).  The origins to the Programme lay in the 1998 Comprehensive Spending Review which announced a New Deal for Regeneration, one central element of which was to be NDC.  This new ABI was in turn informed by the Social Exclusion Unit’s Report ‘Bringing Britain Together: A National Strategy for Neighbourhood Renewal’ (SEU, 1998) which argued that, despite many years of area regeneration policy, there remained at least 4,000 multiply deprived neighbourhoods in England.  NDC was to be an instrument through which ‘to reduce the gaps between some of the poorest neighbourhoods and the rest of the country’ (DETR, 2001, p. 2).  Partnerships were established in 39 locations across England, to devise and implement ten year regeneration strategies.  17 Round 1 Pathfinders were announced in 1998, a further 22 Round 2 NDCs a year later. Ten are located in London, two in Birmingham and the rest in cities and towns across England. Programme wide funding was to amount to about £2 billion over ten years, although it was always anticipated that there would be additional ‘matched’ investment from other public agencies.  In broad terms therefore each NDC would have about £50 million to invest over ten years. This amounted to substantially more than had been made available to any previous English ABI.

The NDC Programme: The evidence base

In 2001, the then Neighbourhood Renewal Unit (NRU) commissioned a consortium of some 14 organisations headed up by the Centre for Regional Economic and Social Research (CRESR) at Sheffield Hallam University to undertake the initial 2001-2005 phase of a national evaluation.  This culminated in the publication of a 2005 Interim Report (NRU/ODPM, 2005). In 2005 CRESR was again commissioned with a smaller group of partners to undertake phase 2 of the evaluation which is due to run through until 2008/09. The national evaluation team has used various data sources of which two are of particular significance.

(i) The 2002, 2004 and 2006 Ipsos MORI Household Surveys

A household survey was initially undertaken in 2002 to establish a ‘baseline’ in all 39 Partnerships. The survey questionnaire addressed socio-demographic, status and attitudinal considerations across key outcome areas such as health, education and crime. The survey was based on a random sample design and culminated in approximately 500 responses from all 39 areas. This survey was repeated in 2004 and 2006. In 2004 MORI/NOP revisited 2002 addresses and in 2006 those interviewed in 2004. In essence this provides two types of data: area based cross sectional evidence for 2002, 2004 and 2006; and longitudinal data based on individuals who stayed in the areas between 2002 and 2006. Sample sizes of this magnitude have a very high level of statistical reliability.

(ii) Administrative data

The Social Disadvantage Research Centre (SDRC) at Oxford University has headed up the collation and analysis of NDC level administrative data.  Data is gathered from a number of sources: the Work and Pensions Longitudinal Study (WPLS) from Department of Work and Pensions; house prices from Land Registry; individual pupil level attainment data from the Pupil Level Annual Schools Census (PLASC) provided by Department for Education and Skills (DfES); and recorded crime data sourced from all 39 police forces in England.

The evidence base: an overview

Reflecting on the evidence base available to the national evaluation it seems reasonable to conclude that this is one of, if not, the best sources of data ever available to any ABI evaluation ever commissioned in England and quite possibly anywhere.  But there are weaknesses too, some of which reflect difficulties inherent to the Programme, others which come with the ABI terrain (Department of the Environment, 1994).  To give a flavour of just two of these problems:

• It is difficult to identify the  ‘NDC model’: a fundamental requirement of any evaluation is that the ‘programme’ can be defined; but here there are 39 different schemes, designed to achieve different outcomes, operating in contrasting contexts.
• there must be doubts as to whether it is ever possible to define the counterfactual: it is possible to assess NDC level change against national and local authority benchmarks; but these are ‘distant’ from the NDC experience; the most appropriate benchmark is that drawn from other similarly deprived ‘comparator’ neighbourhoods; but even then there are problems: NDCs tend to be more deprived than comparator areas; and  there are often other ABIs operating either in the 39 NDCs and/or in comparator areas, hence neither NDCs nor comparator areas are ‘clean’.

Despite these caveats the NDC data base still provides an extraordinarily rich evidence base against which to assess change.  This paper explores cross-sectional area based data as of late 2006. During 2007 further analysis will be undertaken on the longitudinal individual level data. This is an important distinction to make. The cross sectional area based evidence explored below generally points to only modest changes across the Programme. However, it should be stressed that when the national evaluation team last explored change, from 2001/2 to 2004, the individual level evidence pointed to more positive outcomes than was true for area based data (NRU/ODPM, 2005).

Cross sectional area based change data: 2001/2 to 2006

Reflecting across the 2002, 2004 and 2006 surveys, and available administrative data, four overarching conclusions can be drawn about area based change in the 2001/2 to 2006 period.

First, there has been continuing change across the 39 areas.  Initial analysis focussed on 63 key indicators covering issues such as housing, quality of life, community, crime, demographics, work, education, health and finance.  59 of these moved in a positive and statistically significant fashion at the five per cent level.

Second, an overview of the 16 indicators achieving greatest change between 2002 and 2006 (Table 1), points to more obvious signs of positive change in relation to place (fear of crime, attitudes to the area, thinking NDC improved the area, and so on) rather than people based outcomes including more jobs, or improving ‘ultimate’ health indicators such as mortality and morbidity. This is not an absolute distinction. Some people based indicators improved. The proportion of NDC pupils achieving 5 GCSEs at Key Stage 4 rose from 28 per cent in 2003 to 37 per cent in 2005.  But in broad terms indicators of place improved at a faster rate than did people based outcomes. There are a number of reasons why this might be so. Some people based outcomes, notably in health and education will take years, even decades to become apparent.  In addition the NDC Programme may well in the long run confirm the widely held assumption that the neighbourhood is an ideal locale within which to achieve place based renewal outcomes. Problems surrounding the environment, crime, liveability, community cohesion and so on are of areas, and can be resolved within them. People based outcomes are just harder to achieve at the neighbourhood level.

Table 1: Survey data 2002-2006: indicators showing greatest change

						Change
	2002	2004		2006	2002-06		2004-06
NDC improved area (a)	33	51		57	24		6
Heard of NDC	63	79		80	17		1
Car crime a serious problem	38	27		21	-17		-6
Have Internet at home	25	32		41	16		9
Abandoned/burnt out cars a serious problem	21	11		5	-16		-6
Have a PC at home	35	42		50	15		8
Vandalism a serious problem	33	26		18	-15		-8
Household burglary a serious problem	25	16		11	-14		-5
Very worried about burglary	34	25		20	-14		-5
Very worried about being mugged	30	22		18	-12		-4
Satisfied with area as a place to live	60	66		71	11		5
Litter a serious problem	37	33		26	-11		-7
Very worried about vandalism	28	21		17	-11		-4
Feel very/fairly unsafe walking alone after dark	55	49		45	-10		-4
Run down or boarded up properties a serous problem	19	15		9	-10		-6
Very worried about being physically attacked by strangers	27	20		17	-10		-3

Base:  All; (a) All heard of local NDC, (12,661), 2004 (15,749), 2006 (13,008) Rates of change: 2004-2006 compared with 2001/02-2004

Third, a disproportionate amount of positive change occurred between 2001/02 and 2004, rather than in the following two year period. It might have been assumed that the opposite would occur: change would accelerate through time. However there are several possible explanations for this apparently counterintuitive finding. It could be, for example that initial positive effects arising from ‘quick wins’ implemented by NDCs in their early days have diminished through time. It may be too that further analysis points to outcomes being more positive for particular groups of people defined by age, gender, ethnicity and so on, than these Programme wide averages suggest. In relation to some attitudinal indicators such as fear of crime and perceptions of the area it may be  easier to make  bigger, earlier  shifts because there is more ‘headroom’ for change. Perhaps the pattern of change in areas subject to long term renewal programmes is one of relatively rapid initial movements followed by a longer period of consolidation.

Fourth, change in NDC areas is not massively different from what is happening elsewhere.  For some 40 indicators it is possible to compare change in NDC areas with national benchmarks.  Change in NDC areas proved to be at least five percentage points greater than national benchmarks for about a quarter of these indicators.  These mainly relate to fear and incidence of crime and perceptions of local environmental problems.  But for most indicators change in NDC areas is often very similar, if generally slightly better, than national figures would suggest (CLG, 2007a).  The pattern of change in NDCs is also similar to, although perhaps only slightly greater than, that occurring in the Neighbourhood Management Pathfinder areas (NRU/CLG, 2006).  This is perhaps somewhat surprising in that the Pathfinders are not as generously funded as are NDC Partnerships.

Reflecting on area based change data available to the national evaluation team as of late 2006, the key headline is that, whilst change has continued to occur in the 39 areas, the scale of this transformation is often relatively modest, is frequently not out of line with what is happening nationally, and if anything appears to have moderated through time.  This raises an intriguing dilemma.  By historic standards, this Programme is an exceptionally well funded, intensive ABI seeking to effect change over fully ten years.  If any ABI was ever going to work it was surely this one.  It would be wrong to suggest the Programme has ‘failed’: there is, for example, no evidence of any indicator moving in a negative or perverse manner.  It should be stressed too that analysis of longitudinal individual level data to be carried out later in 2007 might well point to more positive change than is evident from area based data.  It may be too that further analysis points to more positive changes for some groups and some types of NDC area.  Nevertheless, it would be hard to argue that evidence presented here points as yet to major positive changes occurring across these areas.

Discussion: why is neighbourhood level regeneration so difficult?

A number of factors which might help explain the apparently modest rate of change across the Programme.  Five considerations are explored below.

First, this is a very ambitious Programme.  The original architects clearly reflected on weaknesses apparent in previous ABIs, such as their relatively short time horizons, limited community engagement and lack of clarity in relation to outcome objectives.  But the pendulum perhaps swung too far with NDC.  The Programme was premised on several key principles: the creation of 39 separate Partnerships, the community being at the ‘heart of the Programme’, extensive engagement with other delivery agencies, and the implementation of ten year renewal schemes.  This was always going to be challenging. As the Programme evolved, it became apparent too, that there were tensions across some of these principles.  For instance renewal driven by ‘community engagement’ was not always the best mechanism through which to plan ten year strategies.  Community representatives on NDC boards were generally more interested in immediate issues of the area, crime and environment in particular, and less on jobs, health and education.

Second, Partnerships have always struggled because there is no comprehensive evidence base to guide regeneration agencies from baseline problems, through suites of interventions, to outcomes.  To take one obvious example: are high rates of crime best addressed through say more investment in crime prevention such as additional police and/or initiatives in other outcome areas notably education and worklessness?  Because NDCs operate in very different local contexts it will never be possible to tabulate the most appropriate suites of interventions to move Partnerships from identifiable problems through to realistic ten year outcomes.

Third, an increasingly critical problem is that of demographic change.  This is a complex arena.  But in brief, there has always been a tension in ABI policy between place, and people, based outcomes.  The former have the huge benefit of ‘staying put’.  However interventions designed to achieve people based outcomes such as training, job mentoring, improving health and education are perhaps more likely to transform lives. But the argument has always been that those receiving people based interventions will tend to leave the area: benefits are thus lost to the ABI.  In fact evidence is not clear cut here (CLG, 2007b).  A survey of some 330 people who left NDC areas between 2002 and 2004 suggests that they moved overwhelmingly because of area and housing related factors, and not generally for say job related reasons.  There was nothing to suggest that this group was more or less involved in (and hence potentially benefiting from) NDC interventions than were those who stayed.  But mobility does have an impact on people related outcomes.  Those who left NDC areas between 2002 and 2004 were more likely to be owner-occupiers and in employment than those who moved in.  These patterns of mobility suggest NDC areas increasingly accommodate those who are most likely to need investment in people based outcomes and are losing those who are more likely to help NDCs achieve their person based outcomes.  The scale of this population churn is startling: 40 per cent of residents wanted to move from NDC areas in 2006 (a slight increase on 2002 and 2004); and by 2006 English was not the first language for 21 per cent of NDC residents a five percentage points increase on 2004.  Population churn is making it increasingly difficult for NDCs to achieve person based change.

Fourth, although it is often argued that this is a well funded ABI Programme, in practice NDCs have relatively limited resources: less than £100 per person, per theme, per year.  It was always assumed that genuine change would depend on NDCs securing alliances with other agencies in order to enhance the quality and scale of services. This was always an ambitious goal: why should delivery agencies ‘bend resources’ or improve the quality of services to relatively well funded NDC areas? In the event as NDCs have matured many have been able to secure mutually beneficial alliances with a range of organisations including the police, Primary Care Trusts (PCTs), and various local government departments. But this process of ‘partnership’ working has not always run smoothly. National policy demands can undermine attempts by other delivery agencies to effect a ‘neighbourhood agenda’. For instance central government instructions to the police to reduce rates of burglary has lead to fewer resources being made available to address the issue which NDC residents invariably prioritise: anti-social behaviour.  Some agencies, PCTs being a classic example, have also been reluctant to enter longer term agreements because of financial uncertainties.  Although many NDCs originally secured the engagement of senior agency personnel, there has been a tendency too for these key players to move on to be replaced by middle managers unable to make strategic commitments.  In part this, perhaps, reflects the sense that NDC is something of an historic oddity.  There is widespread belief that the future of ABI policy is generally cloudy.  Hence, the NDC experience might have only limited applicability to other ‘normally funded’ deprived neighbourhoods.

Fifth, the Programme’s ten year horizon has raised a perhaps unexpected dilemma.  There is widespread agreement that the renewal of these areas will be a long term process requiring at least a ten year time span.  But in order to plan ten year strategies, NDCs would ideally have preferred an institutionally stable context.  It is simply much easier to plan longer term programmes when there is a reasonable degree of certainty in relation to institutions, funding mechanisms and policy agendas.  But that is exactly what they have not had.  There is a strong argument that the last ten years has seen greater change in relation to the governance of urban regeneration than in any previous era.  NDC Partnerships have needed to familiarise themselves with, respond to, and if necessary secure alliances with, new agencies such as Local Strategic Partnerships, new funding mechanisms notably Local Area Agreements and new policy priorities including social cohesion, citizen governance, social capital, and social enterprise. In the light of this scale of institutional flux the real question may be not why NDC achievements appear relatively modest, but rather why they have done so well.