The father-of-three allegedly gave the threat after the pair had met in a car park on New Year's Day for Mrs Hay to deliver the news she was ending their relationship.
But the break-up was one of many as Mrs Hay fought with her conscience over her affair as she tried to make amends in her own 36-year marriage, which the court heard was bowing under the stresses of Mr Hay tending to his seriously-ill mum.
Bradford Crown Court heard on Wednesday from widow Mrs Hay on the third day of Bill's trial.
A jury was told Mrs Hay and Bill had first had a five-year relationship in the 1970s when she was just 19, but that had broken off when she met John Hay.
Bill came back into her life in October when she was speaking to old friends at a party and they talked of Bill and life back then.
Mrs Hay had decided to write a letter to Bill in which she asked if he wanted to meet up for a cup of tea, saying she had "fond memories of that time", adding "for old times sake - no ulterior motive."
But the court heard their relationship soon progressed. "He was very affectionate, very complimentary. He told me that he loved me, that he'd never stopped loving me and that we should never have split up," Mrs Hay said.
But it was when Bill invited Mrs Hay to his luxury detached home in Holmfirth, West Yorks., "to see his animals" that their relationship became more "intimate".
He had given her a mobile phone so they could keep in touch without being detected.
The jury heard evidence from text messages show how Mrs Hay was getting cold feet about their relationship as early as November with Bill talking her round, saying she owed it to him to give him a chance.
She said: "I tried to break it off a couple of times and he would be overly charming and persuasive and he would say we could keep it platonic."
But the court heard Bill became more "intense" after Christmas when the Hays' marriage was showing signs of improving and Mrs Hay continually tried to end their affair.
The jury heard that once, when she refused to answer his calls he had turned up at her daughter's home, and on another occasion he had sent a letter to her home with the acronym SWALK (sealed with a loving kiss) written across the back.
She said: "He was intense about everything, it was becoming more and more uncomfortable. I had a feeling he would tell John everything, I was becoming more and more worried."
She had told him: "Tea once a week is not enough for you and I can't give you anymore."
"I said to my sister, 'it's got to end, no more cups of tea or friendship'. I didn't want to see him face-to-face because I knew he'd talk me round," she said.
The court heard she finally ended it in February this year when he sent her a final text message saying: "I'm here if you need me."
She said: "I thought it had ended on good terms."
In his opening to the court on Monday Robert Smith QC, prosecuting, claimed Bill lured Mr Hay to an industrial unit by pretending to be a prospective customer for the self-employed builder before pushing or knocking him down a steep flight of metal steps.
It is then claimed he attacked him and used a horse trailer to transport his body from the industrial unit in Meltham to land near his home where it was burned. The jury visited the site in Meltham on Tuesday.
The court heard that as Mr Hay was declared missing after failing to return home from his meeting with Bill on March 15, Mrs Hay had sent texts to Bill asking what he had done, as she was looking for someone to blame and he was the only person she could think had anything against him.
One read: "For God's sake... if you care about me at all you will tell me where he is, I'm begging you. My children are in bits."
But she said it was only when she recognised the breathing on an unheard voicemail that she knew the unknown caller who had arranged to meet her husband belonged to domino and snooker-fan Bill.
She said: "I didn't recognise the voice but I listened to the phonecall all day and something about the way he breathed, quite heavy and through the nose, it sounded like Kenneth Bill. That's how he breathes."
The court also heard from Bill's estranged wife, Carole Bill, who told the court she had seen her husband on March 18 as they showed prospective buyers round their home and she noticed he had scratches on his face.
She said he was "squeamish" and had previously fainted at the sight of blood.
Mr Smith said to her: "You said you were stunned by the arrest, what you know of him he wouldn't hurt a fly and he wouldn't kill anything, let alone a person," to which she replied: "Yes."
Mr Hay's son, 33-year-old joiner John Hay junior, told the court how his dad had asked him to go along to the meeting set up with 'Eric Johnston' on March 15, but how he had decided to stay behind and get some other work done instead.
Bill denies murder, saying he wanted to tell Mr Hay all about the relationship with his wife and had set up the meeting. Bill said he gave Mr Hay a letter and after reading it Mr Hay had gone for him.
He claims that as the two men were struggling Mr Hay had collapsed and died.
The trial continues.
