Evans, who notoriously struggles on clay, was knocked out of the Monte-Carlo Masters by Davis Cup team-mate Kyle Edmund.
However he is now through to the last 16 after the win against Zverev, the man who knocked Andy Murray out of the Australian Open.
Both players served out their service games in the first set before Evans break in the tenth to take the set 6-4.
Zverev broke first in the second to lead 3-2 but Evans broke straight back and then broke again at 5-4 to win the match.
He will face Dominic Thiem in the last 16 with the Austrian beating Edmund in straight sets.