Kryemadhi, 37, first came to Britain in March 2001 in the back of a lorry. He claimed asylum on the grounds that as an ethnic Albanian in Kosovo, then under Serb control, he was at risk of persecution. Three months later he and his wife registered the birth of their first child in Portsmouth - they now have three children - but his asylum application was turned down.
While he launched an appeal against that decision he was convicted of wounding with intent to cause grievous bodily harm at Southampton Crown Court.
The incident involved a violent row between Kryemadhi and another man over "unwanted attentions directed to his wife by the victim".
He was jailed for 30 months in October 2002 and the Home Office began legal proceedings to have him deported. Eventually in February 2007 he was removed to Kosovo.
Just a month later he returned to Britain - it is not known how - and set up home with his family again in Portsmouth.
However, in 2008 Mrs Kryemadhi wrote to the Home Office, claiming that she and her husband were now living apart, saying that she wanted to make an application for indefinite leave to remain (ILR) in Britain for herself and her children.
Her ILR application was granted and she and her children were permitted to stay and now have citizenship.
In 2009, Kryemadhi wrote to the Home Office asking to stay here legally, told he was liable for detention and deportation, then appealed and lost his case.
The appeal hearing revealed the truth about his wife's claim to have separated from him was false and concluded that she had won citizenship "by way of a fundamental untruth of which she was fully cognisant".
It found that Kryemadhi and his wife "were prepared to lie before us to suit their own ends such that we found them unreliable witnesses" and rejected Kryemadi's case, opening the way for deportation.
Then Kryemadhi launched another appeal which saw his barrister, Ronan Toal, claim the lower court had failed to attach importance to the fact that the children are British citizens.
He argued that under Article 8 of the European Convention on Human Rights, which protects the right to family life, Kryemadhi should not be deported again - even though the children's citizenship had been granted on the basis of their mother's "fundamental untruth".
Lord Matthews, in the Upper Tribunal Immigration and Asylum Chamber, ruled: "We consider that insufficient weight has been given to the status of the wife and children such as to amount to an error of law.
"The only real issue before us is whether or not deportation would be disproportionate in the circumstances. Having weighed up all the evidence we are satisfied that it would be. In these circumstances we allow the appeal."
At their home in Portsmouth, Mrs Kryemadhi said the family had been "lucky".
"We were very lucky to win after he was deported," she said.
"He won his case finally after living in this country for a long time.
"Neither of us feel like Serbia is our home now. It's nice to be a family now and we are grateful for the decision."
The Home Office is able to remove British citizenship if it has been obtained by deception, but the court heard this had not been done in the case of Mrs Kryemadhi and her children.
Sources said Mrs Kryemadhi's British citizenship is now being reviewed.
