The 31-year-old singer was going through a difficult time emotionally when she signed on to appear on the Stateside pop star search show and because she ''wasn't well in the head'' show boss Simon had no option but to let her go.
Speaking in the new UK issue of ELLE magazine, she said: ''I can talk about this because I'm OK about it now. But I went through hell and it literally drove me mad. I became so desensitised, you could say terrible things to me and I wouldn't even think about it.
'I'd read awful stuff about myself and not blink. I shut down because I didn't know what else to do.
''Now I can say that Simon was right to get rid of me. I wasn't well in the head when that was going on, so it wasn't going to work.''
However, Cheryl felt as though she and Simon had become close friends from their time on the UK show together and still thinks he should have sacked her to her face not over the phone.
She added: ''He wasn't right not to tell me to my face.''
The 'Crazy Stupid Love' singer - who divorced husband Ashley Cole in 2010 following revelations he had cheated on her with several women - has now returned for the UK series of the show.
Cheryl doesn't think she would have been ready to make her comeback in 2013 because she still felt lost as a person.
The brunette beauty explained: ''It only hit me at the beginning of 2013 that I'd completely lost myself. I'd look in a mirror and think, 'Who the hell is she?'
''Something was seriously wrong: it felt like a nervous breakdown. I had to stop, get out of my life I'd been living and find myself again.''