It was left to his replacement Nicky Clark to slot home the 92nd-minute winner after Rory McKenzie had produced a quickfire reply for the hosts.
Rangers will now take on either Dundee or Dumbarton in the quarter-finals at Ibrox.
But Warburton admits his main concern for now is his 28-goal frontman. He said: "With Martyn it was one of those ones where he locked legs as he fell. I don't know if he caught the surface or not but he has an abrasion on his knee.
"He took the penalty, credit to him, but we will see how he is. He is on crutches and we will see how he is in the morning."
Rangers were quick out of the traps and got their noses in front when Smith was caught the wrong side of Waghorn before clipping his heels. Waghorn sent Jamie MacDonald the wrong way as he showed no sign of nerves after five games without a goal.
Killie responded with fierce determination and got a deserved equaliser after eight minutes when McKenzie cut in from the right before flashing a perfectly-placed shot beyond Wes Foderingham's reach.
Rangers, though, eventually grew to dominate proceedings and, with the hosts tiring, Clark broke free from his marker and volleyed home Barrie McKay's corner.
It was Warburton's first win over a top-flight side after having earlier been beaten by St Johnstone in an October League Cup tie ahead of the drawing the original Ibrox tie with Kilmarnock.
He said: "Whether you beat a Northern Premier League team or a Premier League side you still go through or go out, it doesn't matter who you play.
"I felt we should have got a result at Ibrox but w e had 9,000 fans here and had a responsibility to come and deliver the result."
Killie appointed Lee Clark as their new boss on Monday but the former Blackpool manager left it to Lee McCulloch to take the team against his old side while he assessed the squad from the stands.
But despite losing so late on, the caretaker - who stays on as Clark's assistant - said: "There's a lot that's encouraging there for the new manager with the young boys.
"He came in at half-time and full-time and said stuff before the game. I think he'll be pleased with what he saw.
"It was obviously hard to take losing the goal in the last minute.
"Maybe it was tiredness but it shouldn't have been. We had a lot of youngsters who did themselves proud but maybe we just didn't have enough to get us over the line."
To make matters worse, Killie now drop into the Premiership play-off zone after Motherwell's win at Dundee United and McCulloch said: "The league is now the bigger battle."