The trio, whose identity or nationality have not been released, were drafted in to work as security guards at St James' Park, Newcastle.
Staff from the UK Border Agency made the arrests alongside officers from Northumbria Police after acting on intelligence and carrying out routine checks.
The three were employed by a private security firm, but were not G4S staff. Newcastle City Council replaced G45 with local firms earlier this week, amid concerns over the company's staffing.
The Home Office said they were were arrested on Wednesday, just 24 hours before the Mexico and South Korea match at the stadium.
''These arrests show that UK Border Agency security checks are working," A spokesman said.
''The responsibility for ensuring that employees have the right to work in the UK rests with the employer.
''The UK Border Agency is working closely with employers and contractors at Olympics venues to prevent illegal working.''
The Home Office were unable to release the nationalities or ages of those arrested.
A Northumbria Police spokesman said: "On Wednesday, July 25, officers carried out a number of arrests in conjunction with the UK Border Agency in connection with suspected immigration offences."
He declined to comment further.
It comes as authorities launch the biggest-ever peacetime security operation.
British Prime Minister David Cameron has stressed that safety would be his top priority.
"I think we've made as many preparations as we can. I think we have very good contingency plans in place," he told reporters at the Olympic Park.
A force of more than 40,000 military and civilian personnel, backed by a huge intelligence operation, has turned the British capital into a fortress to protect venues, athletes and millions of visitors.
But security has been under the spotlight after the government was forced to deploy an extra 4,700 troops to guard the Games because of a recruitment shortfall at private contractor G4S.
