Nick Herbert will say in a speech in Washington DC that a "damaging message" is sent to criminals when they are given cautions and easy community sentences or allowed to avoid paying fines.
Instead, he will say the criminal justice system must be like a "good parent" by setting clear boundaries from the start and making sure there are "consequences" if rules are broken.
Mr Herbert will also accuse the previous government of "hollow rhetoric" and a wasted "flood of public spending" on crime, warning that today's offenders grew up under New Labour.
His comments come as the Coalition tries to maintain the Conservatives' traditional position as the party of law and order in an era of austerity, by looking at crime prevention and the rehabilitation of offenders as well as tough sentences.
A minimum price on alcohol will reduce drink-fuelled violence and vandalism, it is hoped, while a taskforce is focusing on 120,000 troubled families linked to addiction, anti-social behaviour and truanting.
Meanwhile community sentences will become more punitive, bailiffs will be used to recover court fines and a "rehabilitation revolution" will help offenders addicted to drugs or who suffer mental health problems.
The first Police and Crime Commissioners will be elected this November, allowing them to focus on what matters to local residents and businesses.
Mr Herbert will tell the Jerry Lee Crime Prevention Symposium in the American capital on Monday that on both sides of the Atlantic, politicians are questioning the "decades-old mantra that an ever higher rate of incarceration is the best way to fight crime".
He will say that despite the crime rate falling, the prison population in England and Wales is at a "record level", having doubled since 1993 to reach 87,165 on Friday.
Half of all offences are committed by those who have already been through the courts, the minister will say, so re-offending must be tackled as well as crime prevention.
Mr Herbert, the Conservative MP for Arundel and South Downs, will point out it is almost two decades since Tony Blair promised to be "tough on the causes of crime".
"But his remedy - a flood of public spending - meant expensive but poorly targeted programmes.
"The Left's faith in an ever larger and more expensive State also failed to understand the importance of social structures and order. Family breakdown, welfare dependency and school discipline were largely ignored. Drugs policy emphasised maintenance, not abstinence, and violence was fuelled by the liberalisation of alcohol licensing. Today's offenders grew up in New Labour's years."
He will say that successful programmes must be free from Whitehall "micromanagement" as well as targeted at specific problems.
Mr Herbert will go on: "If offending occurs, it must always have consequences. The first instances of wrongdoing - very often nuisance or antisocial behaviour - must be dealt with effectively.
"When cautions are handed down repeatedly, fines aren't paid, or community sentences aren't rigorous, a damaging message is sent to offenders. The State too often acts like a bad parent, neglectful in repeatedly tolerating bad behaviour, then inevitably harsh.
"Like the good parent, the State should set clear rules and boundaries from the start, dealing with transgression swiftly and surely to prevent escalation."
He will add that being "smart on crime" should not mean being "soft-headed", however, and that offences should not be excused nor offenders seen as victims.
"Getting them back onto the straight and narrow should be a rigorous task where we demand results, not a misplaced act of compassion."