In one of the biggest vetting operations in 70 years, the backgrounds of half-a-million people have been screened amid concerns the Games remain threatened by terrorism.
The Home Office has so far refused more than 100 applications for a range of reasons, but mostly due to concerns about extent of an individual's criminal records.
Some of those denied accreditation came after advice from the Security Services, including MI5, which assesses if there is a significant threat to national security posed by approving any application.
On Tuesday night, it was suggested the "threshold" for refusing accreditation has become more rigorous in the lead up to the Games, which starts on July 27.
Security officials suggested this meant some people attempting to work at the Olympics would have "come to the notice of" police or MI5 previously.
Both MI5 and Scotland Yard have hundreds of investigations "live" at present with that number likely to increase as the Games approach, The Guardian reported.
The security vetting process, which began in October last year, is now two-thirds complete and is said to be the biggest such operation since the Second World War.
Among those who have undergone vetting include potential Games workers - including about 10,000 security guards- athletes, coaches and other officials from more than 200 competing nations.
Home Office officials have dismissed concerns on most of the 10,500 competing athletes, who have been deemed unlikely to pose any security problems.
The 70,000 volunteers recruited by the London Organising Committee of the Olympic and Paralympic Games (Locog), have also been screened.
Authorities have been reportedly closely scrutinising those who have expressed an interest to work at Olympic sites, especially at Stratford, east London.
The main Olympic Park will be protected by the biggest peacetime security operation ever seen in Britain.
The Daily Telegraph has reported that senior security officials have become concerned that other sites around the country could also become targets.
The Ministry of Defence has confirmed that a ring of ground-to-air missile launchers could be deployed around London to protect Olympic venues.
The security service is reported to be braced for a potential deluge of information from foreign police forces and intelligence agencies.
In March, the director-general of MI5, Jonathan Evans, took the rare step of briefing the whole Cabinet on the terrorism threat to the UK in the run up to the Olympics.
A Home Office spokesman said on Tuesday night: "We are undertaking stringent checks on all those seeking accreditation.
"This rigorous process has been designed to ensure those working at the Games are fit to do so.
"We will leave nothing to chance in our aim to deliver a safe and secure Games that London, the UK and the whole world will enjoy."
