Cowell, famed for his no-holds-barred nastiness, believes the rock matriarch behaved very badly towards him, despite her warm-hearted reputation.
But his publicist Max Clifford refused to stop Osbourne's behaviour, insisting their battles made for compulsive viewing.
Cowell says, "Max always said it was very good for the show.
"But there were times I needed help in dealing with her, particularly when she launched a prolonged attack on me that lasted about four weeks.
"After the third week, I told Max we had to put a stop to it and do something publicly in my defence. But he said, 'Ignore it, drop it.' He was right.
"I asked myself, 'Are her accusations life threatening?' They weren't. 'Was the whole thing designed to p**s me off?' One hundred per cent.
"I think she was using me to get herself publicity. It was a shame because I thought we had a good relationship."