The country stars shocked fans last month (Jul15) by announcing their divorce, which had already been finalised, after four years of marriage.
Following the news, In Touch magazine editors ran a cover story based on an interview with Groves, who claims she got close to Shelton back in 2011.
Shelton's attorneys have now issued a letter to magazine bosses demanding they retract the story or face legal action.
In the letter, they write, "Mr. Shelton did not have a sexual relationship or affair of any nature whatsoever with Ms. Groves."
However, according to TMZ.com, In Touch editors "stand by" the story and have decided not to retract it.