New Zealand were in complete control at the interval after tries from Cameron Roigard and Will Jordan.
Scotland's furustration was all the greater as the visitors' second score came after they were reduced to 14 as a result of a cynical trip from Leroy Carter.
Beauden Barrett also knocked over a penalty to give the All Blacks a commanding 17-0 lead at the interval.
However, just ten minutes into the second period, Scotland were just three points behind thanks to scores from Ewan Ashman and Kyle Steyn that had the Murrayfield crowd on their feet.
New Zealand were reduced to 14 men again, with skipper Ardie Savea in the bin, and Scotland went close through Ben White, Pierre Schoeman and Darcy Graham.
On the hour mark, Finn Russell's boot levelled a breathless encounter and Wallace Sititi had picked up New Zealand's third yellow card.
However, replacement full-back Damian McKenzie proved to be the matchwinner, touching down under pressure late on then adding a penalty to ensure Scotland's long wait for a win against the All Blacks goes on.
Scotland skipper Sione Tuipulotu described the loss as "bittersweet" and stressed the importance of a win against Argentina, having opened the autumn series with a record score against the United States.
"I told the boys we needed a reaction at half time," he said. "Either we lie down or choose to go for it.
"We had nothing to lose and everything to gain.
"We have to bounce back for next week because Argentina aren't going to make it any easier."