The alleged incident took place in North London on Wednesday morning and the officer was suspended when a colleague reported the matter to a supervisor.
No members of the public were involved, but the case is the latest in a string of race allegations to hit the Metropolitan Police.
There are currently 13 separate allegations of racism under investigation by the Metropolitan Police and the Independent Police Complaints Commission involving 27 officers and staff.
The Metropolitan Police said in a statement: "The incident happened at around 8.20am on Wednesday May 9 in the north London area.
"The male officer is alleged to have made a racist comment in the company of other officers. One of his colleagues subsequently reported the incident to a supervisor.
"The police sergeant has been suspended from duty while inquiries continue.
"No members of the public were involved in the incident. The incident has been referred to the IPCC.
"Racism and racist language is totally unacceptable. The action taken in response to this allegation demonstrates the MPS's determination to act swiftly and to support those that challenge others when alleged racist language is used."
Last month Scotland Yard vowed to get to the bottom of "very damaging" footage of one officer apparently racially abusing a man and another allegedly assaulting a teenage boy last summer.
Since then a number of other allegations have surfaced against officers.
The Met is expected to have completed its investigations into the allegations by the end of this week.
