Elvis Costello, The Avett Brothers, Conor Oberst and The Decemberists' Colin Meloy were also among the stars who performed at the special event, organised by the movie's directors Joel and Ethan Cohen and producer T Bone Burnett.
The show also included a collaboration between Mumford and folk legend Joan Baez, who teamed up to sing John Fahey's Give Me Cornbread When I'm Hungry.
The British star couldn't contain his excitement as Baez left the stage, exclaiming to the audience, "That's f**king Joan Baez, man! Everyone's been too cool to have one of those Omg (oh my god) moments."
The Mumford & Sons frontman went on to invite the movie's star, Oscar Isaac, into the spotlight to duet on Bob Dylan tracks I Was Young When I Left Home and Fare Thee Well, which feature on the film's score.
Mumford's actress wife, Carey Mulligan, also appeared on stage during the gig to perform Didn't Leave Nobody But The Baby with Gillian Welch and Rhiannon Giddens from another Coen Brothers movie soundtrack, O Brother, Where Art Thou?
The Town Hall concert, titled Another Day, Another Time: Celebrating the Music of Inside Llewyn Davis, raised funds for the National Recording Preservation Foundation.