Caley got off to a dream start in the seventh minute when striker Adam Rooney had his flick deflected past Fraser Forster by Hoops' defender Charlie Mulgrew, before Kris Commons levelled less than two minutes later.
It looked like the visitors would reassert themselves to win their game in hand over the Light Blues and leapfrog them at the top of the table.
However, goals from Grant Munro and Shane Sutherland put the Highlanders on course for an unlikely victory.
And, despite Commons scoring his second of the evening from the penalty spot in injury time, the hosts achieved just that.
The result kept Rangers one point clear of Celtic with three games remaining.
Source: PA
Source: PA