Winger Darcy Graham's try put Scotland in the lead with just 17 minutes left to play, with Adam Hastings converting for a 15-10 victory for the Six Nations side in the Pool B encounter.
Australia were forced to play 35 minutes with 14 men after Campbell Magnay, sent to the sin-bin for lifting flanker Matt Smith above horizontal after just five minutes, picked up a second yellow for a neck-roll early in the second half.
It proved crucial to an Australian side 7-3 up at the break through Izack Rodda's try after Hastings' 20th minute penalty, although Mack Mason's missed penalty kept the lead to just four.
Tighthead prop Zander Fagerson wasted no time in benefitting from the numerical advantage, driving over in the 46th minute for a score before Mason found his range to put Australia into a 10-8 lead.
Graham got through the Australian defence to put the Scots ahead once more, a lead they never relinquished despite immense pressure with Simon Kennewell coming inches from crossing when putting his foot in touch.
Head coach John Dalziel said: "The result is really special and I'm really pleased for the boys as they've put in so much work in ahead of the game.
"We went out there expecting to do well against this team. A lot of people may have thought we were the underdogs, but there's real belief here.
"We're extremely happy that we got the result - what a start to the tournament - but we've got a huge four days turnaround and a lot of work to do.
"Sometimes when you have the man advantage, you start to think the job is done, but we showed a lot of character in that position. Australia threw a lot at us in the last ten minutes, but the character was there and I felt comfortable with our defence."
Scotland face Six Nations rivals England in their next pool match on Saturday, who comfortably eased past Italy 48-10 at the Manchester City Academy Stadium.