The squeaky clean star has said his fellow Disney Channel stars - who are both three years younger than him - pressured him into smoking a joint when they were teenagers.
Joe, 24, told Vulture magazine: ''The first time I smoked weed was with Demi and Miley. I must have been 17 or 18. They kept saying, 'Try it! Try it!' so I gave it a shot, and it was all right. I don't even smoke weed that often anymore.''
The singer also opened up about his rocky relationship with Demi, claiming the 'X Factor' USA judge - who went to rehab for ''physical and emotional issues'' in 2010 - was actually struggling with full-blown ''drug abuse'' and he only stayed with her because he wanted to help her.
Joe recalled: ''I really got to know her and got to see the ins and outs of what she was struggling with, like drug abuse. I felt like I needed to take care of her, but at the same time I was living a lie, because I wasn't happy but felt like I had to stay in it for her, because she needed help.
''I couldn't express any of that, of course, because I had a brand to protect.''
Joe - who also dated Taylor Swift as well as Demi - joked he was glad he waited for the right girl and not his ''bats**t crazy'' ex.
In the revealing Q&A, he said: ''I lost my virginity when I was 20. I did other stuff before then, but I was sexually active at 20. I'm glad I waited for the right person, because you look back and you go, ''That girl was bats**t crazy. I'm glad I didn't go there.'''
Joe's band - also formed of his two brothers Nick and Kevin - split in October.
The novel's author saw a cut of the film and loved each of the changes the movie's director had made.