The governing body announced a review last week but has decided to "remove any doubt of the review's integrity or independence" by moving aside.
Several gymnasts, including Olympians Becky and Ellie Downie and Amy Tinkler, have come forward to highlight the abuse they have suffered in training, saying the mistreatment of athletes had become "ingrained".
- 'Heartbroken' Amy Tinkler says negative experiences led to gymnastics retirement
- 'Beaten, starved and screamed at' – Shocking abuse claims from British gymnasts
Gymnastics whistleblower Downie dropped from Olympic team
The Downies said they had been afraid to speak out previously, while Rio 2016 bronze medallist Tinkler said she made a formal complaint in December 2019 and is still waiting for a response.
A confidential helpline is being set up for British gymnasts who say they have suffered bullying or abuse.
"It is vital the Review is unequivocally independent with full resources to effectively deal with concerns raised by gymnasts," said British Gymnastics CEO Jane Allen.
"In the past week, the complexities have increased, and it is clear to retain the trust of the gymnastics community we have decided to recuse ourselves from any management of the Review.
"Our priority is to learn the lessons and ensure the welfare of all those within gymnastics. By stepping aside, we hope the Review can now proceed unimpeded."
Tweddle and Liukin inspiring hotshot Kinsella to Olympic glory
Hall: Olympic debut caps remarkable turnaround after 'directionless' days