Strachan was livid with the decision to award the Jambos a late penalty that allowed Michal Pospisil to score the visitors' third goal.
After a low-key first half which saw Celtic press forward without any great threat, the game exploded into life after the interval.
Shunsuke Nakamura and Paul Hartley both went close for the home team before Hearts took the lead in the 57th minute.
Laryea Kingston, who performed well in Hearts' midfield, found Kestutis Ivaskevicius with a clever pass. The striker forced his way between Steven Pressley and Stephen McManus before sliding the ball under Artur Boruc.
This was not what the home fans had come to see and they were even more stunned when Hearts went 2-0 up four minutes later.
Kingston was brought down just outside the penalty area and Lee Naylor was booked for his crude challenge on the Hearts star.
Celtic were further punished when Andrew Driver curled his free-kick around the wall and into the top corner of then net.
Celtic hit back almost immediately when former Hearts defender Pressley headed home Nakamura's cross from close range.
However, just as the Bhoys were gathering themselves for an onslaught on the Hearts goal, they fell further behind to a controversial penalty award.
The impressive Kingston was again involved as he burst into the box and Pressley was booked for fouling the midfielder.
It looked a soft award as Kingston appeared to have lost control of the ball but Pospisil needed no second invitation as he made no mistake with the penalty.
It was all too much for Strachan who, after viewing the penalty on a TV monitor, obviously said something which led to the referee ordering him to the stands.
There was still time for Kingston to clear a McManus shot off the line before Celtic began their celebrations - although they were slightly subdued after the defeat.