Tuesday, January 3, 2012


Blocking socket limits

After a long time trying to fix errors / improve performance I have arrived to a sad point: My server with blocking sockets wont support more than 1.018 users, I arrived that limit making an stress test trying to arrive to the 2.000 users barrier, no more clients were accepted after client 1.018.

The problem comes from the design I choose to make the server, I was using blocking sockets, that means for every client we need a thread to wait for data, 1.018 users = 1.018 threads + threads used by the server reached the default limit in Linux (1.024), I have been reading about that , and I know that limit can be modified but it's not a real solution for the problem because my test computer used 90% processor time to handle "only" 1.000 users, so I have to switch to another technology..

That technology is non-blocking sockets, because basically with a single thread you could theoretically handle 1.000 maybe 2.000 connected users, I have been speaking with people that has worked in the game industry and for linux servers the best / fastest way is to use epoll  , epoll is basically used to handle I/O from many file descriptors (connections) in real time, but using epoll to me would have meant throwing away all what I have been working on in the last 3 months.... (Thanks Neils for showing me the right way!)

Finally I decided to use socket sets  , that is the SDL_net non-blocking approach  for that problem , so basically I have to re-write all the networking code for my server...

I have decided to take a rest in the development and  I have started a collaboration with a friend to make a multiplayer platformer, it is very good because I have time to think in other things and learn, I'm just finding that I was making many errors and many things could be improved.

See you soon


Amit said...

How many connections do you really need right now? My suggestion is to leave it the way it is, and redo it later, when you have lots of users.

Also you may find this page to be useful: http://www.kegel.com/c10k.html

JB said...

Thanks for the idea and the link too, I'm still reading it Amit, but I need to make the change, I'm wasting too much processor time, probably with that design I could make the server to work with 500 users and all the features reasonably fast with a quad core processor, but I know that even with a single core I should be able to handle that amount of users, I don't expect to arrive to 8k-10k users, but if I can reach 3-4k it would be ok to me.. making changes to the network later would be a pain in the a%$..

Amit said...

How do you know 500 users is too low? The last game I worked with, Realm of the Mad God, has 85 users per physical server. It's NOT because of networking. It's because they have to simulate the monsters, the message processing, the loot, the bullets, the chat, the dungeons, the combat, and everything else. It wouldn't matter at all if their networking could handle 500 users or 5000 users, because that's not where the time is going.

You need your proxy servers to handle lots of connections, but those will all get multiplexed onto a very small number of game server connections. So your game server doesn't actually have to use async sockets.

I guess all I'm saying is that I'm worried you're optimizing the wrong thing, optimizing for a single server without proxies, when later you're planning to add proxy servers anyway. I think you are going to have to rewrite some of the networking anyway when you add proxy servers later, and you could wait until then to rewrite the game server networking. That way you only have one rewrite instead of two. :)