It started with a minor breach of prison rules, caused by accident. The retribution that followed was as deliberate as it was savage. While waiting in the queue for his Sunday breakfast, an inmate at Wormwood Scrubs rinsed his cup out under a tap and poured the contents into a waste pipe. Some of the liquid spilled on to the floor, annoying a watching prison officer. "I'll be up for you at 10," he shouted.

He was as good as his word. Shortly afterwards, three officers entered the man's cell. One told him to wait outside while the other two conducted a search. After about five minutes, he was called back in and ordered to stand, barefoot, on a blanket and take off his T-shirt. As he was removing the garment, he suddenly received a blow to his chest, followed by further punches to his back and sides. He was then pushed on to the bed and subjected to further blows to his back and ribs. Face down on the bed, with his head turned slightly to one side, his eyes began to bulge as he gasped for breath under the pressure of hands tightening around his neck while the blows continued to rain down on his body. The attack was accompanied by racist taunts. He was called an "Irish bastard" and told to beg for mercy from his "English masters" for spilling water on their "fucking floor".

He was placed in handcuffs while his head was forced towards his knees as the staff began to drag him out of the cell and along the prison landing. He was taken to a strip cell and told to remove his clothes and stand against the wall. The prisoner was then moved to another wing and placed in a punishment cell, where five or six uniformed staff gathered around him. By now, pain and fear had taken over, and he began to shake violently. Two more officers entered, one punched him in the face, causing him to topple on the bed. He was called "terrorist scum" (he was on remand for burglary) and warned that, unless he pleaded guilty to assaulting an officer, "We're going to fucking hang you. We hang shit like you." "Get the fucking sheet," said another of the officers.

They did not carry out their threat and he lived to tell the tale in court, where the judgment in his assault case was in his favour. But the attacks left him badly bruised, cut and inwardly scarred by the attack.

The man was one of 56 prisoners and former prisoners compensated for their suffering at the hands of prison officers who operated a reign of terror at the Scrubs in the 1990s. Six officers were jailed for their part in the assaults (though three later had their convictions overturned), and the prison service claimed to have learned from mistakes made at the west London jail.

But did it? The claim is now seen to be deeply flawed, as evidence has come to light showing that, while the uniformed officers responsible for the violence were pursued through the courts, their managers - many of whom were severely criticised in official reports - escaped any form of disciplinary action. Some senior staff were moved sideways; others were promoted. Peter Quinn, a former prison governor and author of one of three reports into the affair commissioned by the Home Office, declared that "Far from any of the senior managers being investigated or disciplined, they emerged unscathed, or even appeared to benefit from the experience."

Lord Ramsbotham, the former chief inspector of prisons, is also critical of senior managers. He says he never understood why ministers and senior prison service managers failed to discipline those who were responsible for allowing the maltreatment of prisoners, and that those responsible for the day-to-day management must have known, or at least suspected, what was happening. "Either way," he says, "they should have been removed for at best incompetence and at worst connivance at what had disfigured the prison service for so long." Last year, in the House of Lords, Ramsbotham asked the minister for prisons how many of the senior staff directly involved in the running of Wormwood Scrubs at the time of the abuse still worked for the prison service. Last month, he was told that six people were still employed. The prison service has been shown to be an institution that routinely rewards failure.

It is the prison service's apparent unwillingness to deal with mismanagement - not just at Wormwood Scrubs, but at other jails - that fuelled Monday night's defeat in the House of Lords of the government's proposal to insert a clause into the corporate manslaughter bill excluding prisons and other public bodies from the remit of the legislation. The proposed exemption had been criticised by prison reformers, lawyers and parliamentarians, with the shadow attorney general calling it "scandalous" and reform groups declaring there were no logical, legal or moral grounds for excluding deaths in custody from the bill. The defeat of the government's proposal - by almost a hundred votes - was orchestrated by Ramsbotham, who received support from all sides of the house.

At the Scrubs, most of the assaults followed a similar pattern: a prisoner who was perceived by staff to be a problem would be assaulted and dragged to the segregation unit, where further beatings took place. The segregation unit - a jail within a jail - was virtually a no-go area for senior staff, who were asked to phone and make an appointment to visit. According to prison service orders, governors, along with doctors and chaplains, are supposed to visit these areas, unannounced, on a daily basis.

In 1998, Daniel Machover of the London solicitors Hickman and Rose handed a dossier to Ramsbotham, then chief inspector of prisons. A police investigation followed; more than 150 members of staff at the Scrubs were interviewed, 27 officers were charged with assault, and 11 were sent for trial. Yet of the dozens of managers connected with the prison during the years of abuse, not one was sacked or even disciplined. Quinn argues that while the prime responsibility for the abuse lay at the hands of the "thuggish officers" who carried it out, these men did not operate in a vacuum. He says that, according to standing orders, the governor or his deputy should have visited the segregation unit on a daily basis, and twice fortnightly at night or in the early morning. "In the absence of robust remedial action taken to prevent abuse," he said in his report, "staff no doubt believed that their actions had tacit approval." Speaking this week, he said managers at prison service headquarters should also have exercised greater responsibility. "The governor was supervised by his area manager and operational director, both working three or four miles from the prison, and they would, or should, have visited the Scrubs on a regular basis."

The Guardian gave the prison service a list of managers at the Scrubs during the years when the abuse of prisoners went undetected. It declined to give specific information on their current status, but it is understood that all of the 11 managers criticised in the three Home Office reports have either since left the service, with full pensions and a lump-sum payment, or been promoted. Machover, the solicitor who coordinated the Wormwood Scrubs Litigation Group and represented 44 of the 56 prisoners compensated, says the prison service was incapable of grasping the extent and effects of the maltreatment. He adds that internal reports on the affair, only disclosed through the Freedom of Information Act, have revealed that mismanagement at the jail was far worse than the victims' lawyers realised at the time. "There was poor practice and a general malaise at the prison which was either encouraged, or tolerated, by management," he says. "Virtually no action was taken against the managers at fault. Instead, the prison service has generally rewarded failure."

In 2003, the prison service commissioned a report on the management of the jail between 1994 and 1998. Its author was Brodie Clarke, another former governor. Clarke reported that rumours of assaults first emerged in the ear
ly 90s, but "there was a failure to recognise the seriousness of the information coming forward". Clarke noted that warnings about abuse from the inspectorate reports in 1993 and 1996 and information from the board of visitors (now the Independent Monitoring Board) were "not seen as particularly significant" by senior managers, and the "headquarters' handling of the responses did not give these matters their due weight".

Jim Perris was governor of the Scrubs from 1989 to 1996. He was seen as weak and ineffective, and the Clarke report said that, in 1995, at least one middle manager expressed concerns about assaults to Perris but that he was "apparently unconcerned". The report concluded that "it was inconceivable that Perris was unaware of assault allegations and the use of the treatment and training cell". In 1996, Perris was promoted to area manager, overseeing several prisons. He later left the prison service on full pension, plus a lump sum in excess of £100,000.

Mike Gordon worked at the Scrubs at a governor grade from 1995 to 1998, rising to deputy governor and with a spell as acting governor during his last year at the jail. Gordon, who declined to be interviewed by Clarke, was singled out for criticism in the report. "We do not believe that he was unaware of the assaults that occurred during his time as deputy governor, or of the use of the training and treatment cell," wrote Clarke. "We have concluded that Gordon was viewed by certain staff as someone who would be unlikely to take any formal action if prisoners were assaulted or mistreated." In 1998, Gordon was moved to a staff job at prison service headquarters. The Home Office declined to give details of his current status, but the Guardian has learned that he was retired on medical grounds on full pension and with an additional lump-sum payment. It is believed that, even after the publication of the Clarke report, he continued to work at HQ.

From 1995 to 1998, Alan Walker was the operational director, south, overseeing the area managers responsible for the Scrubs and other jails. In 1996, Gordon was made acting governor, an unfortunate move given Clarke's later appraisal of Gordon's performance. Walker played a part in Gordon's appointment, and Clarke's report found that Walker, along with the area manager, Peter Kitteridge, "should have done more to address management weaknesses during Jim Perris's time. The area manager and operational director did altogether too little to ensure that assaults did not resume when Mike Gordon was acting as governor." Clarke concluded that this failure by senior management was "more than a mere setback ... it sent out the wrong messages" to prison staff responsible for the abuse. The area manager left the service with the standard pension/lump-sum deal in 1998. Walker remained as a senior operational director, answering only to the director general and his deputy, after the publication of the Clarke report, before leaving on the usual terms.

Asked why no managers had been disciplined following the abuse at Wormwood Scrubs, a prison service spokesperson said: "An investigation began in March 1998, but was curtailed to allow for police inquiries. In December 2002, the then director general commissioned an internal investigation which reported in April 2003. By then the operational managers with the most direct management responsibility for Wormwood Scrubs had left the service. The Government does not believe that there is a need for any further inquiry. The management of Wormwood Scrubs has vastly improved, the internal investigations were thorough and critical, and the vast majority of the recommendations have been implemented. The prison service totally condemns the mistreatment of prisoners and will continue to ensure that any allegations are quickly and rigorously investigated and, where appropriate, staff are disciplined."

Quinn fears that, because the circumstances of the abuse at Wormwood Scrubs were hidden from public view, "the public should have no confidence in the prison service preventing something similar happening again". An exemption would have added to public concern. Ramsbotham says he is pleased that the proposal has been defeated in the Lords. "I hope the possibility of prosecution under this bill will alert ministers and senior management to their responsibility for the humane treatment of those committed to their care," he says. "That is their duty, according to the prison service's statement of purpose."

The system of promoting failure was by no means confined to Wormwood Scrubs. Last year, the Guardian revealed that Niall Clifford, the governor of Feltham at the time Zahid Mubarek was murdered by his cellmate, had been promoted to area manager following the teenager's death. Clifford, who was heavily criticised by the Mubarek inquiry, was later allowed to take early retirement. It was believed that Clifford, who did not leave for medical reasons, received a lump-sum payment of £175,000. The Guardian has now learned that the figure was closer to £250,000, with the extra money paid to compensate for the years of salary that he missed.

Deborah Coles, co-director of campaigning group Inquest, describes the attempt to exclude prisons from the bill as the "ultimate in evading responsibility at a time when the need for scrutiny and accountability has never been greater". Since 1995, Inquest has investigated more than 2,000 deaths in custody. It says that many of those deaths raised issues of negligence, systematic failure to care for the vulnerable, institutional violence, racism and abuse of human rights. Yet, despite a pattern of cases where inquest juries have found evidence of unlawful or excessive use of force or gross neglect, no prison officer, at individual or senior management level, has been held responsible. This even applies to the 10 cases, since 1990, where "unlawful killing" verdicts have been returned.

In March 2002, 16-year-old Joseph Scholes was found hanging in Stoke Heath Young Offenders Institution. Before returning a verdict of accidental death, the jury at his inquest heard that Scholes had undergone nine days of dehumanising treatment at the Shropshire jail, and the coroner called on the Home Office to hold a public inquiry into the teenager's death. Scholes's death was referred to by Ramsbotham in his Lords speech against the proposed exemption. He argued that had there been a risk of a charge of corporate manslaughter, they would have taken a great deal more care and Scholes "might still be alive if that care had been properly exercised".

Scholes's mother, Yvonne, says she was appalled, though not surprised, at the attempt to exclude prisons from the bill. She says her son's death could have been prevented, yet nobody has been held to account, and she believes there were issues of negligence in the cases of all 20 of the children who have died in custody since 1990. She argues that the government's response to calls for public inquiries into the deaths is to claim the current system of investigation works. "The system has worked very successfully in ensuring that not one prison officer or Home Office official has been prosecuted or held to account following the death of a child in prison," she says. "Why change a system that, shamefully, works for them?"

For proof that the prison service has a long track record in seemingly rewarding failure, go back to May 1990, shortly after the riot at Strangeways prison in Manchester over conditions. The board of visitors at Hull prison asked the then home secretary to stop sending unconvicted 15- and 16-year-old boys to the Humberside jail. They reported "intolerable" cramped conditions, where large numbers of cell windows had been without glass for almost a year. This left inmates exposed to the elements and disposing of urine and faeces into exercise areas. The board reported that "Hull B-wing was no place for an unconvicted 15-year-old to spend a year or more". Judge Stephen Tumim, then chief inspector of prisons, said that the unconvicted teenagers held at Hull were "subjected to t
he most rigorous form of punishment", and MP Barry Sheerman, then Labour's deputy home affairs spokesman, described the jail as having "the most ghastly conditions in the entire prison system". The governor of Hull at that time? Phil Wheatley, the current director general of the prison service.

Eric Allison, who writes on prisons for the Guardian, spent 16 years in prison, all for theft-related offences.