It's perhaps not the biggest come-hither to the audience for a lecture if the lecturer confesses at the outset that the subject upon which he is going to speak is one that in part he is fairly ignorant of, and in another part quite bores him - yet that is exactly my position. Many of you will be either professionals working in the prison service and its satellite agencies, others of you will have more than an accidental and cursory engagement with penology. My position is this: as a writer I am interested in just about everything under our febrile sun, and my knowledge of the British prison system, such as it is, is partly a function of my work as a writer. In the mid-1990s I was invited to judge a short story competition organised by the Koestler Trust. When it became time to present the awards I went to HMP Wandsworth - it was only the second time I had been inside a prison.

At the little ceremony - which was held on F Wing, at that time the clearing house for Rule 43 prisoners in the system - I met Stephen Tumim, the then Chief Inspector of Prisons; and afterwards he invited me to join him on a spontaneous tour. Those of you who ever encountered Stephen Tumim will be aware of the fearlessness and brio which he brought to his job: we stalked the corridors and landings, with him barking at prison officers and gubernatorial staff to open this or that door, then he would poke his tousled and snowy head inside cell or canteen or association area, and enquire civilly of the inmates how they were getting on. As for the staff, where he felt they were being obstructive Tumim was critical, where he found conditions not to his liking he was scathing.

It didn't take too much of this for me to grasp that Tumim understood his remit as being vitally concerned with the wellbeing of those the state had deprived of their liberty. His standpoint regarding the morality of custodial sentencing as a punishment was not an issue - and possibly even an irrelevance - clearly, what he felt himself to be responsible for was ensuring that prisons worked in this sense: that their inmates were not subject to the arbitrary fiat of their guards, that their rights - such as remained - were observed, and that they were provided with an effective recourse for perceived injustices. I can safely say that in many years now of observing the machinery of the British state, that hour spent with Stephen Tumim remains as a strange bright interlude, illuminated by the sheer force of his integrity, in the murk, muddle, bureaucratic obfuscation, and even outright malfeasance that encompasses incarceration, making of it, if you will, a jail within a jail.

I decided to write a short story based on my experience of judging the award, but realised that I needed to know more about Wandsworth from the point of view of its inmates. To that end, through the former criminal John McVicar - who I was friendly with - I got in touch with Noel Smith, a career bank robber who had just been released from the prison. Smith - who had literary ambitions himself - provided fascinating insights and telling details, and in due course I wrote and published the story. I tried to help Smith get some of his own stories accepted, but after a few weeks I stopped hearing from him, and it wasn't until 1999 that he got in touch again - writing from HMP Whitemoor to say that he had just begun a life sentence for a string of bank robberies, and under the circumstances he'd quite like to restart his literary career.

In the intervening period I too had undergone some life changes - which I'll return to later, but suffice to say, as a by-product of these, any residual sense of glamour or excitement I might once have felt attached to the criminal lifestyle had been utterly purged. When I replied to Noel it was to say I'd be happy to assist him (his writings exhibited both talent and ability), strictly on the understanding that he was abandoning crime forever. He told me that this was indeed his intention, and I subsequently went to see him in Whitemoor, where we sealed an agreement that I would act as his literary agent. Over the next decade I visited my unusual client in a variety of institutions - Whitemoor itself, then HMP Downview in Surrey, followed by HMP Grendon in Berkshire where he attended the therapeutic unit for violent offenders, and finally HMP Blantyre House in Surrey, where Noel experienced open conditions for two years before being finally released on licence in 2010. During this period he wrote three books for publication, all of which received considerable acclaim, and I - by reason of three or four visits every year and the many, many hours of conversation that ensued - learnt most of what I know about the British prison system.

• Doping prisoners harms them - and us too

  21 Nov 2012

• Heroin addict awarded £3,000 payout for being denied methadone in prison used it to buy drugs

  30 Apr 2012

• Drug-free inmates put on methadone

  06 Apr 2010

• Government spends £4m on methadone vending machines for prisons

  16 Jul 2009

• Prisoners could be banned from watching Sky TV in jail

  20 Nov 2012

• Locking up prisoners for life does not breach human rights

  21 Nov 2012

The strange paradox of my relationship with Noel Smith, was that while it was he who was confined to a series of institutions where - according to the best current estimates - as many as half the inmates have drug or alcohol addictions and related mental illnesses, he himself was not a problem drug or alcohol user. No, if anyone experienced a frisson of fear in the visiting room when the sniffer dogs came snuffling and loping and straining at their handlers' leashes, it was me. It's difficult to place precise limits on addiction - given that, by its very nature, intoxication tends to the derangement of the senses - but my own best estimate is that I spent something like twenty years of my life as a poly-substance abuser: cannabis, heroin, cocaine, amphetamine, major hallucinogens and a steady ebb and flow of alcohol - I was what my counsellor in rehab' called 'a garbage head'. During those decades, as a writer, I was only too prepared to comment on what I saw as the follies of drug policy in this country - just as I also contributed, in some measure, to supplying what I think of as 'drug porn', that weird exaltation of intoxicated abandonment that provides a frisson of excitement for its consumers, most of whom would never dream of using hard drugs. But after I finally cleaned up from drugs and alcohol in 1999, for the most part I fell blissfully silent.

I am still asked to comment on the so-called 'drug problem' most weeks, but in response to these requests I usually offer an analogy: imagine if for twenty years of your life you were obsessed by chives, you woke up thinking about chives, and you went to bed still preoccupied by them. From time to time - usually without any warning - you found yourself compulsively buying and consuming chives, often neglecting your responsibilities to do so, and in the process frequently abusing those who had placed their trust in you
. Not only that, but you were also compelled to listen to chive music, read books and watch films featuring chive consumption, and even affect chivish clothing. Well, at the end of that time, suppose you were miraculously relieved of your obsession with these frankly laughable herbs - surely the last thing you'd want to do is set yourself up as some sort of authority on chives then spend the rest of your life thinking and talking about them, albeit in a different guise and from another perspective?

My analogy is flawed of course: we may, as a society, be rather more taken by what's on the end of our forks than is healthy for us, but chives - unlike drugs and alcohol - are entirely benign garnishes on the dish of life. Be that as it may, I do think one of the major difficulties we have in coping with intoxication in Britain today is precisely that - to pursue the analogy a little farther - most of the cooks who serve up our drug and alcohol policies have either never tasted a chive in their lives, or else they've had a surfeit of the things. The same can be said of the whole of the criminal justice system; a Manichean simplicity prevails in the arena of policymaking, with to one side a baying crowd giving the thumbs up to more laws, offences - and hence incarceration, while to the other side a rather more plaintive assembly bleats for restraint and understanding.

It would be too simplistic to characterise all who pronounce on criminal justice as having had either too much of it, or none at all; but just as with drugs and alcohol policy, there is a savagely catalytic effect to be observed among the opinionated - whether they be liberal or authoritarian - when they interact with the largely indifferent majority on whose behalf they claim to speak and legislate. I say indifferent rather than silent, because here too drug-taking and imprisonment seem in conformity with one another, both being, almost by definition, activities that take place behind closed - indeed locked - doors. Just as the indifferent majority have never taken drugs, so they neither have knowledge of, nor are much concerned with what happens behind the razor-wire-garnished walls of our prisons. Yes, yes, the occasional media-hyped story about the ping-pong-playing and sybaritic conditions under which malefactors bask will exercise the notional tax payer; just as the occasional editorial quoting Dostoevsky's aphorism that 'The degree of civilisation in a society can be judged by entering its prisons', will prick a community-minded conscience - without in any way encouraging its possessor to actually enter those institutions.

This is the problem we face: with criminal justice and drug and alcohol policy the political class operates within a forum where the adage 'out of sight is out of mind' is not simply a given but a desideratum. In such an apathetic atmosphere is it any wonder that those charged with locking people up and those with palliating people's addictive illnesses engage prodigiously in what sociologists term 'professional closure'. If your livelihood depends on doing things most would rather have no cognisance of, it's that much easier to pursue agendas that are both proximately and ultimately self-serving. Therefore, if I can use this platform to do anything worthwhile at all it will be by following the examples of the late Stephen Tumim - and also the late Frank Pakenham, in the ennobled form of whose name this lecture was established - to, in a strictly non-professional way throw open the locked doors and lead us all into the crepuscular realm inhabited by jailed addicts.

You may have noted that in what I have said so far I have been at pains to consistently group drug and alcohol addiction together; this is because I believe that both are simply aspects of the same core malaise. The use of drugs, being illegal in our society, leads those addicted to them to commit vast numbers of acquisitive crimes, and so for some years now there has been a substantial budget - in excess of £100m per annum - allocated to providing jailed drug addicts with treatment for their illness. By contrast, booze is cheap each and every unhappy hour of the day; moreover the use of alcohol, while subject to some controls is not simply tolerated in our society, but often celebrated and condoned by our culture. As for the crimes it gives rise to, being largely those of violence, or involving the destruction of insurable property, these are not so easy to calibrate in the sort of cost-benefit analysis beloved by politicians and professionals. So it is that there is no defined budget to speak of for the treatment of alcoholics in British prisons.

Moral relativism has become one of the dirtiest couplets in our language; a portfolio term baggy enough to encompass everything from a complacent acceptance of minority communities' misogynism, to a sort of rudderless non-progress through the choppy waters of change. But it needs only a cursory examination of social history to conclude that different things have been considered wrong at different times - I doubt that many of you present would've found the late Frank Pakenham's views on same-sex relationships to be at all congenial, let alone right, although for much of his life they were in lockstep with established social mores. So it is with our attitude towards intoxication, which is continually evolving so that some things - such as young women drinking to the inevitable and emetic conclusion, in public - become acceptable, that were heretofore absolutely not so.

The most recent body to recommend the decriminalisation of the possession of small quantities of currently illegal drugs is the UK Drug Policy Commission, but its members join a long queue of would be reformers that have been for the most part ignored. It's not that the social mores surrounding drug use haven't changed - it's that there remain no votes in acknowledging these changes. Britain remains, as Oscar Wilde said of it over a century ago, the native land of the hypocrite. And so the public debate on drug policy is as boring as debating chives, what with experts' opinion being wheeled out to condone or condemn this or that minute calibration of what pseudo-scientific classifications of the harmfulness of this or that substance. The tedious to and fro over whether cannabis should be regarded as a Class A or a Class B drug would probably seem as ridiculous to Churchill as the notion that he should drink less that the state-directed 28 units per week. If you want one single statistical correlation that makes a mockery of all attempts to impose such crude quantifiers on the complex pattern of our collective interaction with intoxication, you can try this one: the amount of alcohol drunk per capita in this country has doubled in the past fifty years, a half century that has seen a progressive relaxation in licensing laws and a reduction in its real price. If you want to extrapolate from this that control works, then you only have to consider the counterpoint: over the same period there has been a shift from a medical model of treating heroin addiction to a legal one of punishing its use, and during that same period we have seen an explosion in the numbers of addicts from a few hundred to in perhaps excess of 200,000.

What I wish to get across here it isn't even worth talking about the problem of drug addiction in British prisons unless we're prepared to consider the issue of drug addiction in the wider society - moreover that problem cannot be disentangled from our equally vexed relationship with alcohol, while addiction in general - to illegal drugs and to alcohol - cannot be regarded as walled off from less obviously harmful forms of intoxication. The magistrate who sits nodding off on the bench after one too many Sherries at lunch is not only physically close to the crack-addicted shoplifter she is about to sentence; she's psychically adjacent as well. Until we are prepared to regard our relationship with intoxicants as something that we not only hold a personal responsibility for, but also as a practise that is crucially and constan
tly socially mediated, will we continue to fall prey to delusional thinking; the crude mad/bad dichotomy that underpins so much of the argufying can be exposed by a simple act of self-examination: sometimes we take leave of our senses, at other times we abandon our morals, but in all instances we choose to regard ourselves as the same agent. Surely all our treatment of drug and alcohol addicted offenders should be directed at fostering the same overarching sense of responsibility?

All of this being acknowledged; let us look more closely at what happens to convicted drug and alcohol addicts who receive a custodial sentence. We have noted that of an annual throughput of some 140.000 inmates as many as 70,000 have some form of addictive illness. They move from one environment in which drugs are both sustenance and currency while crime is the means to pay for it, to another in which exactly the same is the case - only with greater intensity. I don't wish to anatomise this ridiculous - if not Kafkaesque situation - in any great detail, but let's just assume, for the sake of my own argufying, that each of these inmates procures just a single gram of heroin while inside; this would imply that 70 kilos of heroin are smuggled into prisons during that year. In fact, as any reasonably dispassionate professional would tell you the quantities are far larger - while any inmate, having no reason to dissemble, would readily concur. In the past illegal drugs were brought into prison by visitors - and this continues to be the case; however, in the past decade or so, the use of sniffer dogs and searches has considerably constricted this flow, and the shortfall in supply has been made up by corrupt prison officers and other staff.

How do I know this? After all, the numbers of prison officers being convicted for drug smuggling are paltry. The Under Secretary for State with responsibility for Prisons and Probation, Crispin Blunt, was asked about this in Parliament as recently as March, and he replied that a total of 18 officers had been convicted since 2008. Unless we are to assume that these individuals were not simply mules but actual packhorses, so great were the quantities of drugs they were taking in, we can only conclude that they represent a fraction of the total - perhaps a similarly sized fraction to our modest 70 kilos of heroin. I spoke of professional closure earlier, but perhaps the existence of groups of prison officers prepared to flout the law so egregiously should be seen, charitably, in this light: they have the unenviable task of dealing with Britain's ever increasing prison population, some of whom are potentially violent, the majority of whom are mentally ill; their pay may be adequate - but it's hardly generous, while the roiling inmates - who are often confined to their cells for as much as 23 hours a day - quite obviously require some form of sedation, and are going to get it by any means necessary. To turn a blind eye to colleague who happens to be smuggling drugs - even to accept a backhander from one such - is surely a minor dereliction in such an environment, especially given that even if the addict criminals are denied access to drugs, they simply go right back to using upon their release?

Actually, the attitude of my hypothetical corrupted prison officer isn't far removed from that of other professionals who work in the prison service: psychiatrists, doctors, and so-called drug workers who actively support the current widespread prescription of maintenance methadone or other psychoactive drugs as a 'treatment' for addict inmates. These people may be perfectly well-meaning; they point out - entirely reasonably - that the previous regime under which addicted prisoners were denied heroin substitutes or other medication, was altogether inhumane. They often also argue that by prescribing drugs legally they are removing addicts from the criminal subculture and normalising their lives - at least they do when they're prescribing such drugs outside prison, despite the fact that an open market in methadone and other bromides exists in all addict subcultures, the dull, prescribed drugs being traded for more exciting and illegal ones.

But whatever the arguments advanced for maintaining addicts on drugs, the fact remains that most professionals have no alternative, because it's all they know how to do. And can we blame them for this, surely they are just a small part of a much wider cultural acceptance of the idea that the way to treat malaises of the psyche is through some form of psychopharmacology? The prison doctor, doling out methadone to addict inmates is not that far removed from the GP who hands out antidepressants to patients who feel themselves to be incarcerated in dead-end jobs, unemployment or failed marriages. I've spoken to both kinds of practitioners, and they advance exactly the same sort of arguments for what they do, arguments that range from the strong assertion that such drugs simply work, to the weary plaint that it's the best they can do given the calls upon their time, to a frank admission of impotence accompanied by a shrug of their shoulders. Ordinary GPs may face simply the expectation that they will prescribe antidepressants for commonplace unhappiness - prison doctors, on the other hand, may be facing angry inmates who say, 'Give me another ten millilitres of methadone or I'll smack you in the mouth.'

Stephen Tumim's lineal descendant, the current Chief Inspector of Prisons, Nick Hardwick, recently spoke out about the widespread 'abuse' - his term - of prescription drugs in prisons, saying that in most prisons they were being 'diverted to those who didn't need them'. Again: as it is to the wider society, so it is to the prison system, which is surely only a savagely compressed synecdoche, expressing the whole problem, while walling off a mere part of it. In the past few years there has been something of a sea change in thinking; more and more people seem to be realising that the solution to drug problems may not, in fact, be more drugs. In the wider culture the current Government has pushed for psychotherapy to be offered as an alternative to antidepressants; and within the prison system the message has begun to trickle down from on high that there are other options besides the oxymoronic practise of doping prisoners so that they won't want dope.

Among policymakers and professionals it's well understood why this change in heart has come about - in part it's a generational shift: the current political leadership are mostly young enough to have had unproblematic experience of illegal drugs themselves, and to have seen addiction at close hand among friends and family. They have also had the opportunity to witness therapies that proceed from the assumption that addiction is treatable malady, from which recovery is possible, although not complete cure - if that is taken to be a return to socially acceptable alcohol use. These treatments take as their foundation the complete abstinence from all mood-altering drugs, including alcohol. It is because they have seen long term sobriety achieved by those who follow these regimes that Coalition ministers, from the Prime Minister down, have become prepared to champion their adoption in the prison service.

This is not to suggest that these programmes - which base their approach on the 12 Steps of the anonymous fellowships - have not had their supporters in the prison service for some time. The first intensive, 12-step-based, complete abstinence programme was opened in 1992 in HMP Downview - I know, because I, in turn, wrote the first article on it to be published by a British newspaper. My visit to Downview was the first time I had ever been inside a British prison. That I myself was then floundering back into active addiction after a period of sobriety is I suppose an example of what - in the strictly modern idiom - is termed 'irony'. (For me it was not remotely ironic - merely tragic.) Now, twenty years on, there are some fourteen of these programmes throughout the prison system, twelve run by the Rehabilitation of Addicted Prisoners
Trust, and two further ones run by the Phoenix House charity. The abstinence-based programmes, in conjunction with regular attendance at the anonymous fellowships, have allowed hundreds of addict and alcoholic offenders to achieve and drug- and alcohol-free lifestyles, and upon release to become responsible and productive members of society. Yet as things stand, no more than 1% of inmates are given the opportunity to attend these programmes, and of those that do the bureaucratic obstacles that stand in the way of their being able to depend on a drug-free environment within which to complete their sentences once they have undergone treatment, remain formidable. All too often a prisoner will complete her therapy on a rigorously maintained drug and alcohol-free wing, only to be transferred the next day to one where she's exposed to a dangerous environment dominated by the use of drugs.

For the reasons stated above, the maintenance of drug-free environments in our prisons is currently well nigh impossible - while as for alcohol, the brewing of prison hooch is endemic: wherever there's a vessel, some warmth and some fermentable material, alcohol will surely soon ensue. No doubt those who support the prescription-based policy - often subsumed under the heading 'harm minimisation' - can produce plenty of statistics to support the efficacy of their approach; but it strikes me that we should be absolutely clear about what we're trying to achieve here: just as prison, whether conceived of as punishment, or rehabilitation, or both, must surely aim to increase an individual's capacity to take responsibility for his or her actions, so drug and alcohol treatment must aim to do the same. At an existential level it is quite simply a contradiction in terms to imagine that either the carrot of being prescribed drugs, or the stick of having them forcibly withdrawn, will lead to greater autonomy.

The idea that drug and alcohol treatment depends vitally for its success on the addict making a free choice to be abstinent, and then being encouraged, supported and educated in the furtherance of this goal, is furthermore an anathema to a system that proceeds on the basis that it must know the cost of everything, and by implication remain wilfully blind to the value of anything. While competitive tendering for contracts covering everything from canteen supplies to transportation may possibly allow for transparency and cost-effectiveness, when it comes to the provision of rigorous therapies for complex malaises it seems doubtful that the pseudo-privatisation that has spread, virus-like, through our public services, has much to recommend it. Talk to any chief executive of any NGO-cum-charity charged with delivering social care and they'll tell you that a large proportion of their time is taken up with the bidding round for government contracts, negotiations in which their only possible bargaining counter is a reduction in the efficacy of the services they offer. So, while it is true that there is now an inclination to provide some form of non-drug therapies to incarcerated addicts, all too often what is available is the cheap option: the same sort of cognitive behavioural therapy sessions that are also being offered to depressed and neurotic patients in the general population. It may well be that CBT offers at least some help to jailed addicts and alcoholics, but what seems indisputable is that it's far easier to process the live bodies through such courses - which simply require bums on seats - rather than through the sort of intensive abstinence-based therapy that may lead to long-term recovery.

We are currently in a state of flux so far as the provision of even the CBT options are concerned; since April of this year all drug treatment services in prisons have been handed to a new quasi-autonomous directorate, prior to their being devolved through the new NHS commissioning bodies to a local level. Thankfully, the overall budget for drug treatment in prisons remains ring-fenced pro tem, but what may happen in the future remains in the lap of gods, who, so far as the vast majority of health professionals in this country are concerned, already have feet of clay. As things stand, my hunch is that the current Government's drive towards further privatisation of public services can only have a negative impact on drug treatment in prisons, with each provider competing in a race to the bottom.

So, what should happen? Since our prisons are meant to be entirely drug and alcohol free, and we estimate that at least half of inmates are currently problem drug and alcohol users, the first thing, surely, is to ensure that at least half of these institutions are maintained in that state. If this were the case it would become possible to offer those addicts and alcoholics who receive a custodial sentence with a genuine choice: either go to a prison which, to all intents and purposes, is organised as an intensive rehabilitory regime, or take your chances in those other jails that are subject to the arbitrary dictates of drug barons and corrupt staff. Choose addiction and more crime - or choose recovery and less recidivism. With half the prison system organised in such a way it seems to me fairly obvious that there would be a profound reorientation so far as staff are concerned, with those whose loyalty is to the status quo ante trotting goatishly one way, while those who woolly-liberally wish to see the cycle of addiction and reoffending genuinely broken, will baa in the other.

I realise my proposal sounds radical; and as I said at the outset, why would you be inclined to listen to a recovering addict and occasional prison visitor who confesses himself bored by the whole issue? Well, it's because I have the temerity to believe that my boredom reflects the wider inertia that surrounds this problem: we're all bored by alcoholics beating up their partners - we're all bored by drug addicts' petty thefts. We're all also bored by politicians who come into office, often with the best intentions of cleaning out this Augean stable, only to find themselves quickly bogged down in institutional dung, with the stable boys and girls telling them that they can't do this, or that, or the cattle will stampede. But what isn't boring is the idea that far from our prison system being a diseased and useless appendix to the rest of society, in which all of the drunks and junkies are left to fester in their own foetor - as well as their fetters - it could, on the contrary, become the test bed for a new and enlightened social experiment. If this were to be the case, given the parlous situation as regards accessibility to drug and alcohol treatment in the outside world, we could see a bizarre reversal in the current direction of travel, with addicts and alcoholics desperate to receive treatment happy to be incarcerated, while those who formerly looked to a spell inside to temporarily ameliorate their chaotic lifestyles, only too keen to avoid the rigours of properly applied rehabilitation from addictive illness.

I'm not being facetious about this; frankly, were our jails to be these exemplary institutions, it would of necessity imply a wholesale change in our attitude to intoxication. I've stressed the equivalence between alcoholism and drug addiction throughout what I've said, and this understanding alone would impact constructively on our legislation, making the unproblematic use of all drugs a matter for individual preference, while the addict and alcoholic would be receive treatment early on for what most authorities now agree is a progressive illness. I stress: the notion of drug and alcohol-free prison rehabilitation units is not a soft option. I speak from bitter and lived experience on this question at least: if you had asked me at almost any time during the twenty years I spent in active addiction which I would have preferred, being deprived of my liberty, or deprived of my drugs, I would have opted for incarceration every time. Had I known that incarceration would, of necessity, forced me into making a real and fundamental decision about whether I wanted to re
cover or not, then perhaps that decision would've been arrived at earlier - as it is to the individual, perhaps it could be to the society as a whole.

It took me a long time, and occasioned a lot of suffering to both my family and myself, before I reached that decision - and I am someone with highly privileged life circumstances. How much lower still is the rock bottom most addicts and alcoholics in our society have to reach before they have a moment of clarity; we could make the stony floor of a cell an unavoidable rock bottom for incarcerated addict and alcoholic offenders, dispelling the denial of the problem that is the chief symptom of their illness. Unfortunately, we suffer from a collective form of denial about this problem as well, and as things stand we're unlikely to reach our rock bottom and make a decision to abandon counter-productive policies on drugs, alcohol and incarceration, before we have a prison population as large as that of the United States, and a comparably savage and destructive illegal drug culture.

Details of the work of the Longford Trust in social and prison reform can be found at longfordtrust.org. An abridged version of his speech ran in the paper on Wednesday.