The Polish goalkeeper let an early shot from Steven Fletcher slip from his grasp as Hibs took the lead and, remarkably, he boobed again three minutes from time as substitute Dean Shiels hit the winner.
Celtic came out of the blocks quickly with Scott McDonald denied twice in the opening three minutes by Yves Makalamby.
But it was Hibs who snatched the lead in the fifth minute through an uncharacteristic slip from Boruc.
There looked to be little threat in Fletcher's deflected shot, but the goalkeeper was slow to react and the ball eluded his grasp to gift the home side the lead.
It was a frantic opening spell with Celtic pressing for an immediate equaliser and the Hibs goal lived dangerously at times.
There was a setback for Hibs after 19 minutes when goalscorer Fletcher was stretchered off with an ankle injury after a collision with Stephen McManus.
Celtic equalised after 26 minutes with a superbly worked goal. McDonald and Jan Vennegoor of Hesselink combined to set up Aiden McGeady and he cracked a great right-foot shot into the bottom corner of the net.
The champions took heart from the goal and Hibs manager John Collins made a tactical switch after half an hour when he brought on Filipe Morais for Lewis Stevenson in a bid to swing things back his way.
But Celtic continued to dominate and Makalamby had to leave his line swiftly to smother the ball at the feet of Lee Naylor after good work from McDonald.
In a rare break, Mickael Antoine-Curier was just inches away from connecting with a cross from Brian Kerr with Celtic exposed at the back. It was a warning Celtic did not heed.
Four minutes from the break, Thierry Gathuessi hit a stunning left-foot shot to beat Boruc again and give the home side a half-time lead.
Hibs opened the second half well and Morais, Antoine-Curier and Shiels all came close with Celtic stretched at the back.
But Celtic hit back and it took a great blocking challenge from Rob Jones to deny Paul Hartley after an hour.
Makalamby then did well to slap away a shot from McGeady but, from the resultant corner, Gary Caldwell rose unchallenged to head an equaliser against his former club.
Amazingly, Hibs snatched the lead again after 87 minutes when Boruc let a Merouane Zemmama shot slip through his fingers and Shiels had an easy tap-in for the winner.