Fans who had already bought tickets and booked accommodation in Tokyo objected to the plan, with both events due to attract big crowds due to the sports' popularity in Japan.
"Of course, the athletes-first approach is extremely important," Tokyo governor Yuriko Koike said.
"To choose Sapporo so suddenly … why Sapporo, who decided this? There was no discussion, and I have a lot of doubts about having this thrust on us. We have made many preparations and there’s no change in my thinking that it should be held in Tokyo."
Koike is set for meetings with the IOC's coordination commission John Coates.
The last summer Olympics which were held in Tokyo, back in 1964, were held in October.