The former Take That star grew close to Adele after she struck up a friendship with his wife Ayda Field when they both became first time mothers last year (12).
Recent reports suggested the singers will join forces on his upcoming big band covers album, but Williams has now clarified rumours he is to record a duet with the Oscar winner.
He tells Ireland's RTE Ten, "She's my mate and I'm absolutely terrified to ask her! I haven't asked her and I won't be asking her because the whole world wants to sing with Adele right now. Her and my wife Ayda are really good friends and I'm really good friends with Simon (Konecki), her boyfriend. We're good pals but I wouldn't ask Adele to sing on my record."
Williams has confirmed Canadian crooner Michael Buble will make an appearance on the album after the pair got to know each other during appearances on the U.K version of The X Factor.