Data journalism reading the riots: what we know. And what we don't

In the Media

Data journalism reading the riots: what we know. And what we don't

PUBLISHED December 9, 2011

What does the data tell us about the riots and what role did data journalism have in reporting the disorder?
? Get the data
? More UK riots data

It took the world by surprise. For five days in August 2010, following the shooting of a young man in London, riots spread from Tottenham, across the capital, as far as the north of England, Manchester and Liverpool.

They were an echo of the disturbances of two decades earlier in 1981 in Brixton, London, Moss Side in Manchester and Toxteth in Liverpool. But they took place in an age without instant communications ? and without access to the data which allowed us to tell a new story about the riots in 2011.

In 1981, an official board of inquiry was set up into the causes of the disturbances under Lord Scarman, who produced a report that influenced social policy for a decade. In the aftermath of 2011, there was to be no official inquiry; instead the causes have been pronounced on by every politician and commentator. That environment is one where data journalism can start to provide real answers, based on evidence rather than assertion. And, this time around, there is no shortage of that data.

For the Guardian it meant instant data journalism. Firstly, it was filling the hole of knowledge for anyone wanting to know what was happening where. We compiled a list of every incident where there was a verified report, then mapped it with Google Fusion tables, and allowed people to download the data behind it ? possibly the simplest but most popular thing we did.

Now, this was just a small part of a much bigger project: Reading the Riots ? a unique look at the riots as experienced by those who were there. Under the project ? led by Paul Lewis at the Guardian and Prof Tim Newburn at the London School of Economics - a specially-recruited team interviewed around 270 people about the riots and why they had been involved. The project was the first time such a major attempt had been made to forensically examine the motivations behind a riot since the work in Detroit in 1967. In addition to the qualitative data of the interviews, we would also have quantitative responses to a set of questions each person interviewed would answer.

Plus, for the first time, we would have access to 2.57m tweets sent around the riots themselves to help us try to establish the truth about the role of social media.

Inside the court records

As the riots wound down and more were arrested and charged by police, hundreds of people started appearing in magistrates courts charged with riot-related offences. Security vans were queuing up outside courts across the country, judges were drafted in to man special night courts as they struggled to process all the defendants.

Under UK law, the name and address of someone appearing in court is public information ? it just can't be revealed by the media if the defendant is aged under 18. Each day in every court a detailed record of each case is compiled, extracts of which are given to court reporters after the cases are finished. This is the magistrates' court register and, although it's stored on computer, it's given to reporters normally as a printout ? and not necessarily for every case, either ? just the ones they're interested in. In the aftermath of the riots this meant the only court cases that were being reported were the unusual, newsworthy ones, not the everyday typical cases. If we were to work out what the riots meant and what happened, we would need those records.

When we requested these registers we were met with stony silence by the courts themselves. "I don't understand what you need it for," said one court. "You'll have to go to the Ministry of Justice," said another. It was only when the Ministry of Justice sent an instruction to every court in the country that they released the data ? as PDFs. These were particularly tricky to extract, except by hand ? and we wanted to record addresses and case outcomes. Literally thousands of defendants' details were involved so we persisted. We wanted to be able to create a picture of who was in court, what happened to them and where they came from, and this data would tell us addresses, ages, gender, what they were charged with ? and where it happened. Soon we had over 1,000 cases ? an unprecedented criminology database which has given us a unique set of data.

Our quick analysis of 1,000 cases ? and one of the first detailed studies of the data - showed us what many lawyers had feared: the courts were handing down prison sentences to convicted rioters that were on average 25% longer than normal. As more cases came in, even this stark figure proved to be an underestimate of the justice facing people involved.

What do we know about the riots?

The response to the riots was so strong that three major pieces of research were also carried out inside government. Since August, the Ministry of Justice, Home Office and the National Centre for Social Research have all published comprehensive guides to what went on and who was involved ? but only of those arrested and processed by the courts. This is what we have learned so far from both our research and the official figures.

What do we know about the five nights? The most comprehensive research has been conducted by the Home Office and it found that there were a total of 5,112 individual riot-related crimes across 10 police force areas. Most were recorded by London's Metropolitan police with 68% (3,461) of the total, followed by Greater Manchester police (11%, 581), West Midlands (10%, 495) and Merseyside (4%, 195). There were over 4,000 arrests as a result of the riots.

Some types of crimes happened more often than others. Just over half (51%) of all crimes were committed against commercial premises ? shops being the main target, especially electrical and clothing shops. Cars, assaults and robberies also happened. But the picture varied across the country. In London, Manchester and the West Midlands, what the Home Office calls "acquisitive" crimes were the most common. Everywhere else, criminal damage was the first type of offence. The worst violent crime was in London and the West Midlands where collectively four men were murdered.

? Read more: which shops were looted?

Who were the rioters?

What we do know is that people accused were predominantly young and male (the official figures are that 90% were men or boys). Just under half were aged 18 to 24 ? with 26% aged between 10 and 17 years old ? children, in the eyes of the law. In West Yorkshire, 44% of those arrested were kids. Interestingly, men were more likely to be arrested for disorder and violent crimes; women for theft and burglary.

We also asked our sample about their educations. Of the adults, 33% had no qualification higher than GCSE level, while the highest qualification of another 15% was A-level or similar. Only 5% of rioters said they had a degree, compared to around 20% of the UK population as a whole (based on 2001 census data) ? though around 44% of rioters were in education.

Race was certainly an issue for the media, with the assumption that a huge number of rioters were non-white. In fact, numbers were more evenly split than that. The Ministry of Justice data revealed that where ethnicity was recorded, 33% of those appearing in the courts on riot-related charges were white, 43% were black and 7% Asian. However, these figures varied significantly from area to area, often closely resembling the ethnic make-up of the local population: in London, 32% of defendants were white. In Merseyside, the figure was 79%. Similarly, the data from Reading the Riots found the majority of interviewees in London were black or of mixed race, while in Manchester, Salford or Liverpool, interviewees were overwhelmingly white.

? Read more: Were the riots about race?

Poverty and the riots

One of the defining features of the response to the riots is the pronouncement of causes before there was any data. One example is poverty. "These riots were not about poverty," said David Cameron. "That insults the millions of people who, whatever the hardship, would never dream of making others suffer like this."

But the question is: how do we know? If poverty affects health, education and crime, could it be a factor?

The Home Office research found that those appearing at court tended to be from more deprived circumstances than the wider population of England: 35% of adults were claiming out-of-work benefits (compared to 12% of the working age population); 42% of young people brought before the courts had free school meals, only available in England to the 16% of secondary school pupils from the poorest backgrounds.

Firstly, we showed where people accused of rioting lived and compared it to where the riots took place. This showed how Manchester's arrested all came from the suburbs, while in London the riots were closer to home.

We wanted to know what would happen if we overlaid those addresses with the poverty indicators mapped by England's Indices of Multiple Deprivation (IMD), which map poverty in very small areas across the country. We had already done this with the riot locations themselves, but knowing where people came from seems a better indicator, especially if some were travelling.

The map above is centred on Manchester, where there seems a particularly strong correlation between suspects living in poor areas. Liverpool University urban planning lecturer Alex Singleton took a look at our data to work out exactly what links can be shown so far by mapping people to IMD areas. He found

? 58% of those appearing in court identified their residential location as being within the 20% most deprived areas in England ? which matches what the Home Office found.

? For 60% of those addresses appearing in the sample, these areas had not changed; however 14% had got worse.

Of course, riots are complex things and all sorts of things play a part, shown by the cases where reasonably well-off people took part. But what if poverty matters, whatever the prime minister says?

As Singleton wrote for us at the time:

Rioting is deplorable. However, if events such as this are to be mitigated in the future, the prevailing conditions and constraints affecting people living in areas must form part of the discussion. A "broken society" happens somewhere, and geography matters.

The riot commute

Did rioters commute in for the looting? That is a theory being explored by police and politicians. And as hundreds of suspected looters are pushed through the magistrates courts, what information does it tell us about how the events of those nights in early August unfolded?

An analysis of one day's court hearings found 70% of those accused of riot-related crimes had travelled from outside their area. Communities secretary Eric Pickles has called it "riot tourism", and it is part of the argument used by those who want to withdraw benefits and council housing from those accused of rioting, even in a different borough.

We wanted to know the answer to this and brought in transport data specialists ITO World to help show the distances between people's home and the riots themselves.

How far did people actually travel? Now we have the first answer: 2.2 miles

According to analysis by the UK's top transport data mapping company, ITO world - based on the Guardian's database of riot-related court records - the average distance from home to where defendants were accused of a riot offence was just over two miles, or a half hour walk.

If the most likely road route was taken into account, that distance rose to 2.6 miles.

That varies between cities - in Manchester, the average from home to offence location was 2.8 miles. In Birmingham, the average was 2.9 miles and in Nottingham, 2.6.

In London, people were closer to home: 1.5 miles in Peckham and 2.2 miles in Brixton. But those accused of riot-related offences in suburban Ealing and Croydon were 2.7 miles and 2.3 miles.

? Read more: the riot commute mapped

The riots and gangs

The Home Office also looked at how significant gangs were in the riots. They found that 13% were "reported to be affiliated to a gang" in London. Outside London, less than 10% of those arrested were identified as gang members. But in two areas, West Yorkshire and Nottinghamshire (17%) the figures are a lot higher.

Many believed the rioters to be existing criminals and the official stats do back up the idea of an "underclass of convicts". According to the MoJ, 76% had a previous caution or conviction; 26% had more than ten previous offences and 26% had been in prison before.

The Guardian/LSE's own statistics supported this to an extent, but suggested that among rioters not caught by police previous conviction rates may be lower. 63% of Guardian/LSE respondents had previously received a caution or reprimand and over 70% had been previously arrested ? significantly higher than the population as a whole, but lower than the averages for typical court defendants.

? Read more: the four day gang truce

Social media and the riots

But w
hat about the role of social media? Did Twitter or Facebook actually incite the riots themselves? The Metropolitan police considered switching off social networks during the worst events in London and politicians called for censorship of Twitter, Facebook and the BlackBerry messenger system. But the first proper analysis of Twitter ? thanks to 2.57m tweets seen by the Guardian's data team - seemed to show that the opposite happened, and that it was mainly used to react to riots and looting.

In fact, it showed how extensively Twitter was used to co-ordinate a movement by citizens to clean the streets after the disorder. More than 206,000 tweets ? 8% of the total ? related to attempts to clean up the debris left by four nights of rioting and looting.

James Ball wrote on Comment is free that

The lessons of the experience are several. The most obvious is that social networks are simply tools: when used for activities of which we approve ? like riot clean-up, or the Arab spring ? their power seems unambiguously positive. When used for other causes, it is portrayed as sinister. There's no way to embrace the immense good such tools can use without learning to live with, and mitigate, their downsides

Data and the riots

If data journalism is traditionally a long drawn-out affair - resulting in analysis of only marginal interest ? the riots highlighted a new type of data work. Initially, instant data journalism, which helps to make an evolving news story clearer.

But what made this compelling is what came next: a rigorous, academically significant piece of social research, which brought together the Guardian's best journalists and dedicated researchers to create a comprehensive picture of what really went on. It had data behind it, surely, but at its heart were real stories told by real people.

Data has too often been seen as abstract to real lives. Not any more.

Download the data

? DATA: download the full spreadsheet

More open data

Data journalism and data visualisations from the Guardian

World government data

? Search the world's government data with our gateway

Development and aid data

? Search the world's global development data with our gateway

Can you do something with this data?

? Flickr Please post your visualisations and mash-ups on our Flickr group
? Contact us at data@guardian.co.uk

? Get the A-Z of data
? More at the Datastore directory
? Follow us on Twitter
? Like us on Facebook

Simon Rogers