After knocking out Brighton and Stoke City en route to round three for the first time in 16 years, the League One side brushed aside Burnley's challenge to further enhance a home record that has seen them lose just once at the County Ground since August, 2011.
First-half goals from the impressive front pair of Paul Benson and Andy Williams gave Town a half-time lead as they looked to repeat their semi-final win of 1968 in the season they went onto win the cup as a Third Division team.
And even though Austin went into double figures for the season with his sixth goal in three games in the 74th-minute there was never any real threat of Burnley clipping the Robins' wings as Troy Archibald-Henville finally put the Clarets out of their misery with a third goal seven minutes from time.
With injury keeping out Jason Shackell and Junior Stanislas, Martin Paterson and Dean Marney given a breather and Brian Jensen returning in goal after his penalty shoot-out heroics in the previous round, Burnley boss Eddie Howe made five changes to the side that secured a first away Championship win at Derby last Saturday.
On-loan winger Cameron Stewart was also unable to play after appearing in the opening two rounds for Hull City but there was no shortage of experience coming in to replace the absentees in the shape of Michael Duff, Brian Stock and Marvin Bartley.
Swindon-born Joseph Mills also earned a re-call in a left-wing back role as Burnley started with three central defenders but the early focus surrounded Austin on his first return to the County Ground since his move to Turf Moor in January, 2011.
Despite his prolific goal-scoring record for the Robins, Austin was given a rough ride by the Town supporters who were delighted to see him drive a chance just wide after only three minutes after David Edgar had dummied Ross Wallace's corner into his path.
Austin then met another Wallace corner with a header which was blocked on the line by ex-Accrington Stanley winger Gary Roberts and the Clarets' top scorer was involved again when his scuffed effort from a Bartley pull-back was turned into the net from an off-side position by Chris McCann.
Howe must have been happy with the way his side started but with 19 minutes gone the tie swung Swindon's way when they went ahead with their first serious effort at goal.
And again the Clarets' defence was culpable as they stood motionless to allow a low cross from former Burnley loanee Jay McEveley from a short corner to reach Benson, who swivelled to fire past a helpless Jensen and into the bottom corner.
Burnley's reply came from an Austin cross headed at goalkeeper Wes Foderingham by McCann but with Austin getting increasingly frustrated by the close attention of defender Troy Archibald-Henville the Clarets' goal threat gradually diminished.
It might have been different had Austin got the decisive touch ahead of Archibald-Henville to a teasing Stock cross in first-half stoppage time but by then the Clarets were 2-0 down.
Swindon had done little to suggest they might add to their lead but when David Edgar's intended pass to Wallace three minutes before the break was intercepted by Tommy Miller they pounced in clinical style, Miller playing in Williams to slide the ball past Jensen and give the underdogs a sight of the fourth round.
Howe reacted at the interval by replacing Mills with Sam Vokes in a switch back to a 4-4-2 formation but it was Swindon who went closest to scoring again when Benson flicked on for Williams to curl a shot inches wide.
McCann then volleyed over at the other end following a long throw from Mee but the Clarets struggled to create anything clear-cut until Vokes was unlucky to see a header blocked on the line from Kieran Trippier's free-kick with half an hour to go.
A goal then would have given the Clarets hope but Swindon weren't about to let their lead be eroded without a fight as they never gave Burnley a moment's peace, snapping into tackles and forcing mistakes.
Wallace was guilty of one such error as he presented the ball for Roberts to cross to Williams, whose header was beaten away by Jensen, who continued to be the busier of the two goalkeepers.
But just when it looked as though the Clarets would bow out with not much more than a whimper Austin got the goal he craved.
Vokes deserved much of the credit for a superb ball in from the right wing but once again Austin's touch in the penalty area was spot-on as he rounded Foderingham before rolling home his tenth goal of the season.
That might have been the catalyst for a late onslaught from Howe's men but instead it was Swindon who struck again to make sure of their place in Wednesday night's draw.
Again the Clarets were unable to deal with a set-piece as McEveley's delivery was allowed to travel right across the penalty area for Archibald-Henville to stoop and head home at the far post.
And even then the Robins didn't rest on their laurels as Jensen was forced to save well from Williams and Trippier had to clear off the line from Roberts as Burnley were sent packing by Paolo Di Canio's vibrant side.
Burnley: Jensen, Trippier, Duff, Edgar, Mee, Mills (Vokes ht), McCann, Wallace, Bartley, Stock (Marney 56), Austin. Subs: Lafferty, O'Neill, Porter, Long, Grant.
Booked: Austin 51, Mee 68
Swindon: Foderingham, Archibald-Henville (Flint 85), McEveley, Thompson, McCormack, Navarro (Ferry 81), Ritchie, Roberts, Benson (Collins 87), Williams, Miller. Subs: Collins, Bessone, De Vita, Bostock, Bedwell.
Booked: McEveley 30, Miller 49, Archibald-Henville 60, McCormack 90
Referee: James Adcock.
Attendance: 7,353 (Burnley 455)