Rumours about the pair, who play husband and wife on the U.S. Tv show, started swirling in December (13) after the two were spotted Christmas shopping in Brooklyn, New York months after she separated from her carpenter husband Shane Deary after six years of marriage.
The actor admits even his mother was convinced he was dating the Austenland star.
He tells People.com, "She (my mum) calls up going, 'What's going on? I just read this about you'. I tell her, 'You have to take off your Google alerts'.
"I've told her a million times that there are a million inaccuracies that she's read, (but she still) calls and goes, 'Is this true?' I tell her, 'No! When are you going to get it?' You have to brush that stuff off."