The 'Sex on Fire' singer has confessed he got hooked on pills after having surgery on his shoulder two years ago.
At the time, the band had released their album 'Only By The Night', but Caleb, 28, was on taking such strong medication he doesn't remember much about its launch.
The heartthrob musician - who is now clean - said in an interview with The Sun newspaper: "'Only By The Night' was the most messed up. That was the party record. I don't remember a lot of it. I had surgery on my shoulder so they gave me pain pills which I kind of got hooked on."
Now he has overcome his problems, Caleb believes he and his bandmates - his brothers Jared and Nathan and their cousin Matthew - can be as big as they want.
However, they believe their biggest threat is 'Britain's Got Talent' star Susan Boyle.
The group - whose fifth album 'Come Around Sundown' is out this week - are worried the 49-year-old Scottish singing sensation will sell more records than them.
Caleb said: "I am worried about Susan Boyle. What if she sells more records than us? She is a force of nature that woman."
Like Susan - who is now said to be worth £16 million - the talented stars came from a humble background, growing up in Tennessee, and admit at times it was humiliating.
Now worth millions, Caleb explained: "I had people who wanted to come round to my house but I was too embarrassed because all three of us were sleeping in one bed. It was humiliating.
"There's a line in the song 'New Money' about now having so much money that I can't handle it."