The chief inspector of prisons has raised concerns about the treatment of Muslim inmates at Belmarsh maximum security jail in London.
Anne Owers says there is evidence of bullying and the prison is struggling to deal with the large proportion of Muslims held on terrorism charges.
She said prison officers did not understand the social and religious behaviour of Muslim inmates.
The Prison Service said Belmarsh had made huge progress in inmate care.
Prison Service director general Phil Wheatley said: "The chief inspector acknowledges some of the positive strides Belmarsh has taken since the last inspection.
"Excellent working relationships between the multi-faith chaplaincy team, staff and managers enables them to work together to ensure the diverse faiths and cultural needs of prisoners can be met."
It is important for Belmarsh not to be diverted by issues specific to a small number of prisoners, he added.
Around 100 of the 900 inmates in the maximum security jail in south-east London are Muslim.
Belmarsh: A 'complex' jail
In her report Ms Owers also highlighted concern that four remand prisoners on terror charges were only allowed to associate in pairs, and were banned from communal Friday prayers.
She also revealed that another inmate facing terror charges was kept in a separate secure unit with a "dedicated group of staff".
"The prison's high security and special security units were at full stretch, holding among others those suspected of the failed suicide bomb attempts of July," she said in her report.
She said it was "vital" for prisons to understand their inmates and the different relationships within and between them in order to manage them safely.
"We did not believe this was the case for staff in relation to Muslim prisoners at Belmarsh, in spite of the efforts of a competent and trusted imam."
Ms Owers added there was "insufficient staff understanding of (Muslims') interactions and needs".
She gave the prison 127 recommendations for improvement but said overall Belmarsh was "making progress".
Speaking on facilities for Muslim inmates, Prisons Handbook editor Mark Leech said there was a need to be vigilant to avoid creating "our own Guantanamo Bay inside Belmarsh prison."
A solicitor who has acted for current and former inmates told BBC News 24 ignorance of Islam by some prison officers was to blame.
"They can't understand and accept that religion is very important to them, for them to be able to pray is very important," said Mudassar Arani.
"So it's different cultures not being able to understand each other."
But the Home Office said Belmarsh has 17 different religious faiths represented and staff are made aware of the cultural and religious issues.
"HMP Belmarsh takes the issue of faith very seriously, as does the Prison Service at every level," it said in a statement.
At the time of inspection, four category A prisoners were considered to be an exceptional risk and were undergoing an assessment process, she said, after which they were allowed to take part in communal worship.
It added: "We will, however, ask prisoners on health and safety grounds to conduct prayers where they do not obstruct other people.
"Prisoners are able to pray during work and education as well."