The Fight For This Love hitmaker was invited to join Cowell for the American take on his talent show in 2012, but she was fired after filming just one episode.
The career low came after several turbulent years for the star, who had gone through a divorce and a near-fatal bout of malaria, and Cole now reveals she was on the brink of a meltdown.
She tells Britain's Elle magazine, "It only hit me at the beginning of 2013 that I'd completely lost myself. I'd look in a mirror and think: 'Who the hell is she?'
"I didn't know what I thought or felt about anything, couldn't make decisions - I didn't even know what to eat or drink. Something was seriously wrong: it felt like a nervous breakdown. I had to stop, get out of my life I'd been living and find myself again. The consequences if I didn't were just too frightening.
"I can talk about this because I'm Ok about it now. But I went through hell and it literally drove me mad. I became so desensitised, you could say terrible things to me and I wouldn't even think about it."
The star now feels Cowell was right to axe her from the show because she was not well enough to be working on such a high-profile project.
She adds, "Now I can say that Simon was right to get rid of me. I wasn't well in the head when that was going on, so it wasn't going to work. But he wasn't right not to tell me to my face."
Cole and Cowell have reignited their friendship and are working together again on the upcoming series of The X Factor in the U.K.