The couple - who have been in a relationship since 2006 - are ''working on'' starting a family, the actor has admitted.
Asked if they plan to have children, Joshua said: ''Kids are one of those things that kind of happen, aren't they?
''Certainly we're working on it if that's what you're asking.''
Both Joshua and Diane - who was previously married to Guillaume Canet for five years from 2001 - have previously insisted they have no desire to tie the knot, and though it is still not ''important'' for them to make their union official, they won't rule it out completely.
Joshua told E! Online: ''It's important for the people that it's important for, but neither One Of Us are particularly religious so I don't think that there's any particular push. But never say never.''
Earlier this year, Joshua, 33, sparked engagement speculation after he won a $45,000 necklace at auction and was quoted as announcing it to be an ''engagement gift'' for the 35-year-old blonde beauty and the 'Fringe' star joked the purchase was a ''horrible mistake'' because of the rumours it sparked.
He said: ''I made a horrible mistake at an auction by raising my hand when I shouldn't have, and it started a bunch of rumours that were very, very costly. We're not engaged.''