Helen Grant said inmates are "literally paying for their crimes" as money deducted from their wages has gone to providing counselling for those who suffered traumatic offences or providing security for pensioners' homes.
On the anniversary of the introduction of the Prisoners' Earnings Act, she said she hoped it would raise up to £1million a year in future.
Ms Grant, the Conservative MP for Maidstone and the Weald, added that increases in fines and the proportion given to victims would see £50m extra a year diverted from offenders to those affected by crime and anti-social behaviour, on top of the £66m already provided by Government.
The cost of Penalty Notices for Disorder will rise by £10, to reach £60 or £90, while the Victim Surcharge will now increase to a maximum £120 from a flat rate of £15. Even those given conditional discharges or community sentences will have to contribute, for the first time.
Ms Grant told The Daily Telegraph: "For too long, victims have felt overlooked by the 'system'. Help has been given in an outdated, one-size-fits-all fashion. And hardworking taxpayers have been left to foot the vast majority of the multi-million pound support bill - which is simply scandalous. This must end.
"My role, as Victims' Minister, is to put this right. It's a vital role and one that I am honoured to be asked to do, and am equally determined to get right."
She added that those who have to give evidence in court about their ordeals often undergo a "confusing and sometimes frightening experience", and that information will be improved to help them cope.
A new Victims' Commissioner is to be appointed soon to "champion their needs" and ensure ministers are delivering on their promises, following the resignation of Louise Casey a year ago.
Services for victims will also be decided upon more locally following the election of the first Police and Crime Commissioners, covering 41 police force areas in England and Wales, in November.
Ms Grant concluded: "I am passionate about this exceptionally important position and making sure that the criminal justice system supports, protects and defends the rights of victims, as well as punishing and rehabilitating offenders."