The Frenchmen scored 16 of their 19 points in the 10 minute periods when Finn Russell and Leone Nakawara were in the sin-bin.
Gregor said: "The two yellow cards put us under a lot of pressure and gave a team like Toulouse, who were off-loading all over the place, a chance to find their gap. But their defence was excellent so we'll have to look at ways to break that down better.
"The penalty count was 14 to five and when you come away from home playing against one of the best teams in Europe with two yellow cards it is very tough to win. We should have put more points on the board in the first 30 minutes when we were dominating the game."
Glasgow will now have to win the reverse fixture at Scotstoun to keep alive their hopes of qualifying for the Champions Cup quarter-finals for the first time.
Gregor added: "It's certainly unusual that you play the same team next week so it is important to know what we need to do better straight away before we go into the deep analysis over the next couple of days.
"But I was very proud of the effort today, I felt we were in control in the first-half but weren't able to put points on the board. And to come here against a team who were on fire at times and be down to 14 men for 20 minutes, and still be in touch of getting a losing bonus point was a great effort."