The 56-year-old actor - who was married to the star for three years before divorcing in 2003 - admitted his ex-wife is a "wonderful woman" and although she now raises six children with partner Brad Pitt, he knows they will always have feelings for each other.
He said: "Any time you're in a high-profile relationship, you know that question is going to come up forever.
"Angelina is a wonderful woman. I'll love her to the end of my life, and she will love me. As friends."
The Oscar-winning director also opened up about his relationship with his violent father - who died when the star was just 18 - explaining the pair never even had one conversation with each other.
Billy Bob said: "[My father was a] violent Irishman. There was abuse, both verbally and physically ... he was a very intense guy who I don't think I ever had a conversation with.
"[It's made me] want the approval and acceptance of older men."