Disconnected when entering steam games

[Catahya]

Hi there
I've had the O2 premium package for a few months but moved to a different country, and now I'm back again. I have just signed up for the premium package again as I never experienced any problems with the O2 broadband.
The internet has been up for 5 days now with the O2 wireless router II and my boyfriend and I are experiencing problems when launching steam games such as Team Fortress 2 and Left 4 Dead. His computer will always time out the first three or so times whenever he tries to connect to any Team Fortress 2 server and my computer will always fail to connect to any Left 4 Dead server the first time. Please note we nearly always enter the same game so one computer has no problems connecting while the other one does.
Today my boyfriend launched a game of Team Fortress 2 while I was browsing on the net. My boyfriend timed out immediately and the internet died. The internet light on the router was glowing red so we reset the router by unplugging it. It now seems fine.
I just think it is really weird that the connection is so erratic. It doesn't feel to me that it is the connection that is the problem, but the router. I'm not sure I like the O2 router that was provided because it's so limited. When I was on the O2 broadband before I used a Netgear router.
So basically, my question is: Would it be worth it to buy a Netgear router and use that instead? Will it help? Is this a common problem with the O2 router? Is there any way to solve this?

And thanks for reading this wall of text