Gregor Townsend's side did just enough to pull off another big result after their Six Nations victories over England and France earlier this year.
Australia, ranked number three in the world and buoyed by a run of five consecutive wins, looked set to get the first points on the board in the fourth minute when they won a penalty just to the right of the posts, but James O'Connor sliced his kick wide from 30 metres out.
Scotland were forced into a change after just 11 minutes when George Turner went off with a head injury to be replaced by debutant Ewan Ashman.
The Murrayfield crowd was in raptures after 22 minutes when Hamish Watson touched down to break the deadlock after the Scots pushed their way over following a lineout just a few metres from the try-line.
After a delay while the TMO checked for an obstruction, Finn Russell made no mistake in kicking the conversion between the posts.
Australia thought they had got themselves off and running in the 35th minute when Michael Hooper forced the ball down under the posts.
However, the try was ruled out after the TMO spotted some foul play by Allan Alaalatoa on Matt Fagerson. The Wallabies prop was duly sent to the sin-bin.
Australia got their first points on the board right on half-time when O'Connor kicked a penalty after Fagerson was caught going off his feet at a ruck.
Four minutes after the restart, the Wallabies got themselves in front when Leota received a lay-off from Nic White, found a gap in the Scots' defence and darted over the line. O'Connor was successful with the conversion.
The match swung back in the hosts' favour in the 59th minute when Ashman was fed by Pierre Schoeman wide on the left and somehow managed to touch the ball down on the line despite the close attention of two Wallabies. Russell sent his kick wide of the post.
O'Connor edged Australia back in front with a penalty in the 64th minute after Watson went in off his feet at a ruck.
However, Scotland regained the initiative in the 68th minute when Russell kicked a penalty from 40 metres after the Australian scrum collapsed. It proved to be decisive in a close contest as Townsend's team maintained their recent impressive momentum.