"Maybe I would have been at four Olympics," the superstar Jamaican sprinter told reporters on Friday, before being crowned IAAF male Athlete of the Year for the sixth time after a season in which he completed his amazing Olympic sprint triple triple.
"The athlete of the year award is definitely a big deal - it proves that all the hard work has paid off," he said.
With nine Olympic golds in the drawer, Bolt plans to hang up his spikes next year after a farewell tour of his favourite venues, running the 100m only, and culminating in the world championships in London where he will be gunning for his 12th and 13th world titles if he goes in the relay.
"When I look back the only real thing I think I would change was to have got more serious more quickly," he said.
"I was relying more on my talent. If I'd got more serious I think I would have won done a lot more in my career. It's shocking but it's true. Maybe if I'd started younger I'd have had four Olympics."
Despite his multiple titles and world records, Bolt said the race that would always have a particular place in his heart was the world junior championships in his home country when he won the 200 metres as a 15 year-old-old.
"That was special," he said. "That was the beginning. Jamaica 2002, where it all began. That was the biggest step."
However, it was a defeat five years later that made the biggest impact.
Bolt took silver behind American Tyson Gay over 200 metres in the 2007 Osaka world championships and it was a real wake-up call.
"After I lost I remember going to my coach saying 'I really tried my best but what can I do to win because I really need to win?'," Bolt said.
"He said: 'you're slacking off in the gym. If you want to win you have to get stronger.' So I was like 'alright'. I knew what I needed to do and from then I took the step and just worked and worked."
Within nine months Bolt was the 100m world record holder and weeks later had blasted himself into worldwide consciousness by winning the 100 and 200 metres and 4x100m relay at the Beijing Olympics in spectacular style.
Image credit: From Official Website
Having found a way to win he was in no mood to stop, repeating the feat at London 2012 and, despite early season injuries, doing it again in Rio.
Now 30 and with his speediest days behind him, Bolt is trimming his workrate to enjoy one last season, running a series of 100 metres races only.
"Next year is mainly for the fans," said Bolt, who always gives up a huge amount of time to sign autographs and pose for photos every time he races.
"It's a last opportunity for some people to see me run and a chance for me to say goodbye. I'd like to go back to some of my favourite places to run - Ostrava, Lausanne, maybe Paris."
And then that will be it. The man who has carried his troubled sport sometimes seemingly single-handedly for a decade will step aside for the next generation.
There will be no Michael Phelps-style comeback at the 2020 Tokyo Olympics either.
"Oh no," he said. "My coach and said 'do not retire and come back, don't ever do that'.
"It's a bit different for swimming but for most track and field athletes it never goes well.
"I'm still talking to people about what I will do afterwards and I definitely want to stay involved in the sport but at the moment I am just looking forward to doing nothing."