Despite recalling Erling Haaland, Lucien Favre's side were lacking in tempo and intensity, similar to the performance against Fortuna Dusseldorf last time out.
And the visitors punished them by scoring with their first attempt after Bote Baku's pinpoint cross was headed in by Jonathan Burkhardt for his first ever Bundesliga goal.
Dortmund did not make any changes at the break despite their below-par display and found themselves 2-0 when Łukasz Piszczek conceded a penalty with a clumsy challenge and Jean-Philippe Mateta converted the spot-kick.
Mainz, who have been in good form on the road, continued to create the better chances after that with the home side failing to improve much despite bringing on Marcel Schmelzer and Mateu Morey.
The result means Favre's side are three points clear of third-placed Leipzig, who they face on Saturday. Mainz are now five points clear of the relegation play-off place and next face Werder Bremen, who are six points behind.
Dortmund showed what they are missing compared to Bayern Munich.
The Bavarians have the ability to win ugly often, as they have done in their last three games against Werder Bremen, Borussia Monchengladbach and Bayer Leverkusen to clinch the title on Tuesday night. Whereas Dortmund have the flair players who make good highlight reels but are unable to win as regularly when they are not at their best.
They managed it against Fortuna last Saturday with a large dose of fortune but their luck ran out tonight. Jaden Sancho, Thorgan Hazard and Julian Brandt were again largely anonymous only this time they could not rely on Haaland to get them out of jail as he was also off his game.
MAN OF THE MATCH
Bote Baku (Mainz): The winger was brilliant with his final delivery giving the Dortmund attackers a lesson in how to be clinical. He set up Burkhardt's opener along with a number of other chances.
BORUSSIA DORTMUND: Bürki 5, Piszczek 4, Hummels 5, Hakimi 5, Can 5, Witsel 4, Guerreiro 5, Hazard 3, Brandt 5, Sancho 4, Haaland 5. Subs: Schmelzer 5, Morey n/a, Schultz n/a.
MAINZ: Muller 6, Aaron 6, St. Juste 7, Niakhate 8, Boetius 5, Latza 7, Kunde 7, Baku 8, Mateta 8, Onisiwo 6, Burkhardt 8. Subs: Oztunali 5, Szalai n/a, Barreiro n/a, Won Ji n/a, Brosinski n/a.
33' - GOAL FOR MAINZ! Baku's pinpoint cross is powered in by Burkhardt with his head after a lovely move.
35' - WHAT A CHANCE! Can with a weak header from three yards which the keeper keeps out.
49' - MAINZ SCORE THE PENALTY! Piszczek is penalised for a foul on Latza after Burki had saved the shot. Mateta coolly sends the keeper the wrong way to make it 2-0!
71' - WHAT A CHANCE! Baku with a lovely ball into the box, its inviting for Boetius whose side-footed effort from close range somehow goes wide.
Since May 2013, Mainz have scored all of their 27 penalties, a Bundesliga record.
As a team, Mainz ran 8km more than their opponents.