The North and Midlands were one of four pro districts created when rugby went professional in 1995 but the team was merged with Glasgow in 1998.
Two years ago, the SRU announced Borders would be brought back and pledged to resurrect North and Midlands as Caledonia by 2006.
But the economic downturn and drop in television revenues means the Scotland A team has been scrapped next season and the three pro teams have had their £2m budgets slashed.
Townsend, who returned from France to play for the Borders this season, says another pro team will be one of the casualties.
He said: "Having an extra district in the Borders this season has definitely helped Scottish rugby.
"And I would definitely say a fourth team would be beneficial for the Scottish game but financially I don't think it is going to happen.
"It is going to be put on the backburner and unless money gets injected into the sport it won't happen.
"Not just Scotland, right through the British Isles, the English clubs as well, money is going out of the game and budgets are tight. So I think that will be one thing that slips off the agenda."
Townsend believes that a Caledonia team would have great support from players and fans.
He said: "I have chatted with Kenny Logan and he is desperate for it to happen in Stirling and get a franchise up and running after he finishes playing.
"But I think there are so many places you could have it here, like Inverness, Aberdeen or Dundee.
"Everyone in Scottish rugby knows there is a lot of talent here. There are so many players in the Scottish side who have come through from here - Tom Smith, Stuart Grimes, Jason White."