The Hoops were on top for the majority of the match at Hampden with Kilmarnock keeper Cammy Bell starring for his side as he produced a string of fine saves to deny Celtic.
The match appeared to be heading for extra time before substitute Dieter Van Tornhout popped up in the 84th minute to head Lee Johnson's cross past Fraser Forster to clinch Killie's first piece of silverware since their Scottish Cup win in 1997.
But it was referee Willie Collum's decision to deny Celtic a stoppage time penalty after Anthony Stokes went down under a challenge from Michael Nelson that really enraged Lennon.
"It is a stonewall penalty and it was a shocking decision. He is clean through and he is holding the lad off and the guy lunges at him and takes his leg away," the Celtic manager said.
"He doesn't get anywhere near the ball so it's a penalty and a red card in the final minute of a game where we have banged away at the door throughout.
"For me it is a criminal decision and it has cost us the treble. I've seen the incident on television from every angle and I don't know how he can make that decision from where he was.
"To rub salt in the wounds he books Stokes for diving which is awful refereeing. All you want is for the officials to do their jobs properly. I thought overall he had a good game but that is a big call in a cup final at a huge moment when we haven't got much time left.
"For me he has got it horribly wrong."
Celtic could all but clinch the Scottish Premier League title for the first time in four years with victory in Sunday's Old Firm derby and have a Scottish Cup semi-final against Hearts or St Mirren to look forward to.
But Lennon admits he and his players were disappointed to let the chance of a clean sweep of trophies slip through their fingers after losing for the first time since October.
"This has been a fantastic season. The standards the players have set now means there is huge disappointment that we're not going to win the treble," the Celtic manager said.
"We've got a couple of games to go to win the championship. At the start of the season I'd have snapped your hand off for that.
"But such as the way the season has gone today is a massive disappointment when really it is one game. It's one game out of 27 we have lost.Some days your name isn't on the trophy; that's football.
"We came very close to clinching a treble last year and we've come maybe closer this year and we will try to go one better next year."
Source: AFP
Source: AFP