Leigh Griffiths, preferred to Nadir Ciftci in the main striker's role for the first time in Europe this season, gave the Scottish champions the lead in the third minute with a typical poacher's goal before midfielder Nir Biton headed in a second from a Stefan Johansen corner.
However, Berget, who had a short and unsuccessful loan spell at Parkhead last season, pulled a goal back in the 52nd minute with a wonderful volley.
Griffiths headed in his second and sixth of the season in the 61st minute to make it 3-1 but in the fifth of five added minutes added on Berget scored from close range to make the second leg in Sweden next week more fraught than it ought to have been.
It had started so well for the home side.
Celtic looked fired up from the first whistle and got off to the perfect start when midfielder Johansen's pass was stopped by Griffiths around 12 yards out and he quickly steered the ball with his left foot past Johan Wiland.
Celtic Park erupted again in the 10th minute when Biton rose a couple of yards out to head in a Johansen corner, the Norway international coming close to making it three moments later, only to be foiled by the foot of the Malmo goalkeeper.
As the Swedish side stumbled around in something of a daze, Griffiths hit the side netting with a powerful left-footed drive from the edge of the box.
There were occasional slack moments in the Hoops' play but it took 20 minutes for keeper Craig Gordon to be tested and he made a fine save Vladimir Rodic's 25-yard drive.
Celtic's grip on the game loosened slightly, but only slightly, as the first half unfolded further and in the 36th minute Stuart Armstrong flashed an angled-drive past the far post to remind Malmo of further potential damage.
However, it was the Scottish champions who suffered a blow seven minutes into the second half following an Armstrong corner which came to nothing.
Malmo broke and with the Hoops defence still getting organised, Magnus Elkrem fired in a cross from wide on the right which ended up at the unmarked Berget, who volleyed past Gordon and into the far corner.
For the first time the Swedish side looked confident and moments later Gordon parried a Nikola Djurdic drive to safety.
However, just after the hour mark, Griffiths capitalised on slack defending by Anton Tinnerholm who headed a Mikael Lustig cross straight up into the air, by sending a looping header over Wiland and in off the post.
The closing stages were nervy at times as Malmo went looking for the goal which would make the secon leg less daunting and with almost the last kick of the ball, as the home side failed to clear a corner, Berget fired high into the net from close range to change the complexion of the tie.
Source : PA
Source: PA