Employment Minister Chris Grayling said the company's Mandatory Work Activity contract to help up to 1,000 jobless people in the South East find work was being ended.
The Department for Work and Pensions (DWP) has been auditing its commercial relationships with A4e after receiving an allegation against the company earlier this year.
"While the team found no evidence of fraud, it identified significant weaknesses in A4e's internal controls on the Mandatory Work Activity contract in the South East," Mr Grayling said.
"The documentation supporting payments was seriously inadequate and in a small number the claim was erroneous. There was also a high incidence of non-compliance with other relevant guidance (including A4e's own processes).
"The process established prior to March fell significantly short of our expectations. As a result, the department has concluded that continuing with this contract presents too great a risk and we have terminated the Mandatory Work Activity contract with A4e for the South East.
"Contingency plans are in place to ensure there is continuity of support for participants in the Mandatory Work Activity programme."
A4e held the prime contract for delivery of Mandatory Work Activity in the South East, covering Hampshire, the Isle of Wight, Surrey & Sussex, Thames Valley, Bedfordshire, Buckinghamshire and Oxfordshire.
The programme is aimed at helping jobseeker's allowance claimants identified as most in need of support. Participation is mandatory, with sanctions against those who fail to take part, or complete a placement.
A4e is keeping other contracts it has with the DWP, including those under the Work Programme, which tackles long-term unemployment.
An A4e spokesman said: "Following the DWPs review of A4e's small Mandatory Work Activity (MWA) contract, some specific issues were highlighted which relate to a period when A4e dealt with an unexpected volume of work which exceeded the anticipated monthly contract volume.
"During this time A4e's focus was for staff and service partners to ensure customers swiftly found effective work placements. However, in the same period, our administrative processes fell short of our own standards and those required by DWP, and to this end we have accepted that the MWA contract will be terminated."
