The Bavarians, who have clinched the last seven league titles, have now won just one of their last four Bundesliga matches.
Saturday's heavy defeat on Thomas Mueller's 500th game in all competitions for Bayern leaves the future of embattled coach Niko Kovac in doubt.
"I cannot say. You have to ask other people," Kovac said when asked whether he still enjoyed the backing of the Bayern bosses.
I cannot blame the team. With the red card all our plans went out the window. We were then 2-0 behind and had phases where we dominated possession. But after we went down 3-1, we deserved to lose.
"You could see again today that we make simple mistakes that are instantly punished," said Kovac, a former Frankfurt coach, who had infuriated Bayern fans this week by praising his former club's fans as the best in the league.
First-half goals from Filip Kostic and Djibril Sow put the hosts in the driving seat with Bayern down to 10 after the ninth minute dismissal of Jerome Boateng with a red card for a foul.
The Bavarians briefly cut the deficit in the 37th minute through Bundesliga top scorer Robert Lewandowski, who has now netted at least once in each of their 10 league games.
The Poland striker took his league tally to 14 goals but David Abraham restored Frankfurt's two-goal cushion in the 49th.
Martin Hinteregger's header on the hour made sure of the three points and Goncalo Paciencia completed a horrible afternoon for Bayern, piling more pressure on the champions ahead of next week's big clash with Borussia Dortmund.
Gladbach, on 22 points, increased their lead at the top to three over Dortmund, who beat Vfl Wolfsburg 3-0 to snap their unbeaten run in the league and move into second place.
Marcus Thuram got the winner for clinical Gladbach after Kevin Volland cancelled out Oscar Wendt's lead for the Foals with all the goals coming in the first half.
RB Leipzig demolished Mainz 05 8-0 to go third on 18, ahead of Bayern on goal difference.