The programme, which was headed by the music mogul from its 2004 debut until 2010, has failed to attract the amount of viewers it did in its heyday, and Cowell - who now appears on the American version of the programme - is keen to boost its appeal by resuming his seat on the panel.
Sinitta tells Britain's Sunday People newspaper, "This has been the worst series ever and Simon wants to come back. He has so much affection for the U.K. show, he is the boss and he wants to be back on U.K. Tv.
"Simon says X Factor is his first baby. He worries about the show and there is a lot for him to worry about at the moment. The wrong people are going. He finds it frustrating. He doesn't want the whole thing to fall apart."
Cowell's spokesperson adds, "He has not ruled out returning to the show as a possibility in the future. It is also no secret that he has missed working on the U.K. show."