The English boss had failed to win either of two his previous duels with Ladbrokes Premiership sides but Clark's goal in the second minute of stoppage time made it third time lucky and sealed the Light Blues' place in the last eight.
Martyn Waghorn netted his 28th goal of season with an early spot-kick to put Rangers ahead but had to limp off soon after.
Killie quickly got back on level terms through Rory McKenzie but Gers continued to dominate, eventually getting the goal they so badly craved late on to set up a quarter-final clash with the winner of next week's tie involving Dundee and Dumbarton.
Gers had lost to top-flight opposition St Johnstone in a League Cup tie back in October and were held by the Rugby Park side at Ibrox two weeks ago.
Since then, the Ayrshiremen had appointed Lee Clark as Gary Locke's successor as boss and he was presented to the home support before kick-off.
But the former Huddersfield, Birmingham and Blackpool boss decided to let former Gers skipper Lee McCulloch take charge of the side for the final time while Clark assessed his new squad from the stand.
The fifth-round replay was part two of Rangers' three-match run on plastic but it was the hosts who slipped up inside the first two minutes as Stevie Smith got on the wrong side of Waghorn before tripping the Gers top scorer's heel.
The Englishman showed no sign of nerves after five games without a goal, though, as he sent Jamie MacDonald the wrong way from the spot.
But it was his final involvement after suffering an injury in the initial incident and he was substituted by Clark.
As if that was not bad enough a blow for Warburton's team, McKenzie then cancelled out their opener after eight minutes as he cut in from the right before slotting a perfect left-footed strike past Wes Foderingham.
It was an impressive response by Killie as they harried after Rangers high up the pitch.
Their hunger almost paid off again when Josh Magennis robbed the daydreaming Rob Kiernan, who came close to conceding a penalty himself when he scrambled back to barge the Killie striker over before he could shoot.
Rangers were awarded a free-kick inside the hosts' box after MacDonald bizarrely picked up Conrad Balatoni's back-pass but it went unpunished as the Championship leaders failed to read James Tavernier's ball across goal.
Barrie McKay's swerving volley was another near miss 10 minutes after the interval, while MacDonald looked solid as he pushed away Tavernier's shot from range.
Another save from MacDonald denied Jason Holt's long-range effort but Rangers were gradually beginning to dominate. A goal-saving block from Balatoni on substitute Harry Forrester frustrated them again, however.
But with extra-time looming large, they got the breakthrough as Clark darted to the front post to fire the 90th-minute winner from McKay's corner.