Handle user accounts with MySql

Hello

    Now we have a solid way to store and request data the next step is going to be the creation of user accounts.

First we are going to prepare the database, to do so we will connect to the mysql server:

mysql -h localhost -u root -p

Create a database

crete database rpgdata;

Use the newly created database

use rpgdata;

Then create a table, that will store the user accounts

 CREATE TABLE accounts (     id INT AUTO_INCREMENT,     fullname VARCHAR(60),     email VARCHAR(120),     password VARCHAR(30), 

    lastlogin TIMESTAMP,

 PRIMARY KEY (id));

We want to login into the game with an existent account or to create a new one, after that step we will get into game character creation (the next step), to simplify things I'm gonna make a text-only client , and a simplified server too.

The logic will be:

1. Client connect to the server socket
2. Client displays two options:2a-create a new character, 2b-login with an existing character
3. If we choose to create a new character the client will show list of text fields we need to fill to create the account
4. If we choose to login with an existing account we will fill our username (email) and password. 

In the first case, it asks for:

• Full user name
• email(we will use it like our user account)
• password
• password (another time), to prevent mistakes when typing

        cout << "NEW ACCOUNT CREATION" << endl;
        cout << "Full name?: ";
        cin  >> fullname;
        cout << "e-mail?: ";
        cin  >> username;
        cout << "Password?: ";
        cin  >> password;
        cout << "Repeat Password?: ";
        cin  >> password2;

        if(password.compare(password2)==0)
            {
                done=true;
            }else{
                cout << "Passwords don't match" << endl;
            }

After filling all the data, it will pack it and sent it to the server

std::stringstream Msg;

Msg << fullname << "#"<< username << "#" << password << "#";

NET.SockSend(CREATE_NEW_ACCOUNT,Msg.str().c_str());

After we receive the message in the server we are going to unpack the data and get the email.

switch (command){
            case CREATE_NEW_ACCOUNT:
                cout << "New account request" << endl;
                data2.clear();
                split(data[3],'#',data2);//I split it into strings

                fullname = data2[0];
                username = data2[1];
                password = data2[2];

                cout << "Fullname " << fullname << endl;
                cout << "Username " << username << endl;
                cout << "Password " << password << endl;

 We will make a query to the MySql server with that email account

                aquery.clear();
                aquery = "select * from accounts where email = '";
                aquery+= username;
                aquery += "'";
                cout << aquery << endl;
                aSQL.executequery(aquery.c_str());//executing select query

If there is already an account with the same email we will reject the data and send a "duplicate account" message to the client, if we don't find it, then we save a new account to the server and send a "account created properly" message to the client

 if (aSQL.getrow())
                    {
                        //Username duplicate..
                        cout << "Duplicate username.." << endl;
                        aNET.Clients[atoi(data[0].c_str())]->SockSend(DUPLICATE_ACCOUNT,"");
                    }else{
                        //We have not found username, ok to insert..
                        cout << "We have not found username, ok to insert.." << endl;
                        aquery.clear();
                        aquery = "insert into accounts values (NULL,'";
                        aquery+= fullname;
                        aquery+="' , '";
                        aquery+= username;
                        aquery+="' , '";
                        aquery+= password;
                        aquery+="', NULL)";
                        cout << aquery << endl;
                        aSQL.executequery(aquery.c_str());//executing select query
                        cout << "New user inserted" << endl;
                        aNET.Clients[atoi(data[0].c_str())]->SockSend(NEW_ACCOUNT_ACCEPTED,"");
                        }
                    break;

After we create a new user, it will return to the main menu.

In the second case, it asks for:

• email
• password

After filling the data, it will pack and sent it to the server

    cout << "ACCOUNT LOGIN" << endl;
    cout << "e-mail?: ";
    cin >> username;
    cout << "Password?: ";
    cin >> password;
    std::stringstream Msg;
    Msg << username << "#" << password << "#";
    NET.SockSend(LOGIN_WITH_ACCOUNT,Msg.str().c_str());

After we receive the message in the server we are going to unpack the data and get the email.

            case LOGIN_WITH_ACCOUNT:
                cout << "Login request" << endl;
                data2.clear();
                split(data[3],'#',data2);//I split it into strings

                username = data2[0];
                password = data2[1];
                cout << "Username " << username << endl;
                cout << "Password " << password << endl;

With the email we are going to make a query to the server asking for accounts with that email.

                aquery.clear();
                aquery = "select * from accounts where email = '";
                aquery+= username;
                aquery += "'";
                cout << aquery << endl;
                aSQL.executequery(aquery.c_str());//executing select query

If we receive no accounts, obviously, there is no such account.

 if (aSQL.getrow())
                    {

        /... 

                    }else{
                    //We have not found username..
                    cout << "Username not found.." << endl;
                    aNET.Clients[atoi(data[0].c_str())]->SockSend(UNKNOW_USERNAME,"");
                    }

If we have found that account we check the password match the one the user has typed.

//Username found
                    cout << aSQL.row[0] << " " << aSQL.row[1] << " " << aSQL.row[2] << " " << aSQL.row[3] << " " << aSQL.row[4] << endl;
                    std::string pass;
                    pass = aSQL.row[3];
                    if (pass.compare(password)==0)
                        {
                        cout << "Username/password match, login ok" << endl;
                        aNET.Clients[atoi(data[0].c_str())]->SockSend(ACCOUNT_ACCEPTED,"");
                        aNET.Clients[atoi(data[0].c_str())]->Login  = true;
                        }else{
                        //wrong password
                        cout << "Password mismatch.. " << endl;
                        aNET.Clients[atoi(data[0].c_str())]->SockSend(BAD_PASSWORD,"");
                        }

If the password is right, it will send an ACCOUNT_ACCEPTED message.

In the client side, after sending data to creating a new account or login with an existing account we will wait to the server to send us a reply

switch(command)
            {
            case ACCOUNT_ACCEPTED:
            cout << "ACCOUNT_ACCEPTED" << endl;
            reply = true;
            logedin = true;
            break;

            case NEW_ACCOUNT_ACCEPTED:
            cout << "NEW_ACCOUNT_ACCEPTED" << endl;
            reply = true;
            break;

            case DUPLICATE_ACCOUNT :
            cout << "DUPLICATE_ACCOUNT " << endl;
            reply = true;
            break;

            case UNKNOW_USERNAME:
            cout << "UNKNOW_USERNAME" << endl;
            reply=true;
            break;

            case BAD_PASSWORD:
            cout << "BAD_PASSWORD" << endl;
            reply=true;
            break;
            }//switch

And basically that's all, We have now authenticated users, ready for the next step: Player creation process!

Notes:

1. Don't look for encryption: it is not yet implemented so the server / client are not secure, the reason for that is 1-Simplicity 2-Easier to understand incoming / outgoing packets and 3-It is very easy to add encryption later.
2. Messages are not encoded: no binary numbers, just plain text, the reason the same in point 1.1 and 1.2 : making things simpler to understand.

The full code can be downloaded at https://code.google.com/p/jbfprj/downloads/list or at the svn

See you!

A server socket class

Hello

I'm going to describe a new class, (last one before I start with the server)

We already have a class to store persistent data, now we need a way to communicate with the world.

I need a class to start a TCP socket server then accept incoming TCP connections , I need to be able to send data to any of the clients too, and finally enqueue the incoming data so I can process it when I need to do.

What the class is going to do is:

1. Open a server socket in a port X
2. Start a thread that will be listening in port X
3. When we receive connections it will save the client info (ip/port) to a vector so we can communicate with the client, every client starts a new thread to get the incoming data from each client (it is a blocking-socket server)
4. The data is stored into two different queues depending if the client has logged in  or not, I made that because I will apply different priority for the queues , the main (client updates) must be processed with maximum priority other way clients will be lagging.. in the other hand client login request can wait a pair of seconds without big problems.

To achieve that I use two classes, first one is cNetwork, it is the main server, second one os cClient, used to store information from every connected client, the header from both classes are stored in the same file:


<network.h>

#ifndef NETWORK_H
#define NETWORK_H

#define MAXLEN 1024
#include "SDL/SDL.h"
#include "SDL/SDL_net.h"
#include "SDL/SDL_thread.h"

#include <vector>
#include <queue>
#include <iostream>
#include <sstream>

using namespace std;

class cNetwork;

class cClient //the connected clients

{
public:
//functions
cClient();//creator
virtual ~cClient();//destructor
void StartThreads();//Start listener thread
static int listener( void *data );//Listener Thread
void Send(const char* Msg);//Send data to client

//variables
cNetwork *_server;//To comunicate with main server instance
TCPsocket socket;//client socket
SDL_Thread *ListenerThread;//The network client thread
int         threadReturnValueclient;
Uint16 UID;//Unique identifier
bool Connected;//socket connected?
bool Login; //loged in?

};

class cNetwork//main network server class
{
      friend class cClient;

    public:
        //functions
        cNetwork();
        virtual ~cNetwork();
        void Start(Uint16 MAXUsers, Uint16 Port);
        void CleanClient(unsigned int Client);//clean a client so we can use it
        static int Master( void *data );//Master listener function/thread
        void ClearClients();//Clear disconnected clients
        void Finish();//close all client connections and finish server

        //variables
        queue<std::string> InputQueue;//Here we store received data
        queue<std::string> LoginQueue;//Here we store login requests
        SDL_sem *ClientsLock; //Semaphore to prevent server from accesing a variable from different threads
        cNetwork *myserver;//server pointer
        vector<cClient*> Clients;//Here I store clients
        SDL_Thread *MasterThread;//The Master thread >> accepts new clients
        int         threadReturnMaster;//return value for master thread
        bool ShutDown;//closing the server?

    private:
        //private variables
        IPaddress ip;//Server ip address
        TCPsocket ServerSock; //Server socket
};

#endif // NETWORK_H

Then I have the client implementation:

<client.cpp>

#include "network.h"

cClient::cClient()
{
    //ctor
    ListenerThread = NULL;
    Connected = false;
}

cClient::~cClient()
{
    //dtor
    //Stop the thread
    SDL_KillThread( ListenerThread );
    //close the socket
    SDLNet_TCP_Close(socket);
}

void cClient::StartThreads()
{
    //Create and run the thread +
    ListenerThread = SDL_CreateThread( &listener, this );
}

int cClient::listener( void *data )
{ //While the program is not over
    cout << "Starting client listener Thread..." << endl;
    int result;
    char in_msg[MAXLEN];
    bool quit=false;

    while( quit == false )
        {
        memset(in_msg, 0, MAXLEN); // Clear in buffer
        if ( ((cClient*)data)->Connected)//If I'm connected...
            {
                result=SDLNet_TCP_Recv(((cClient*)data)->socket,in_msg,MAXLEN);//Check for incoming data
                    if(result<=0)
                        {
                            //NO DATA
                            // TCP Connection is broken. (because of error or closure)
                            SDLNet_TCP_Close(((cClient*)data)->socket);
                            cout << "Socket closed..." << endl;
                            ((cClient*)data)->Connected = false;
                            quit=true;
                        }

                stringstream idmsg;
                idmsg << ((cClient*)data)->UID << "#" << in_msg;// I will add the UID from the client to the incoming message
                try{
                    SDL_SemWait(((cClient*)data)->_server->ClientsLock);//lock smaphore to prevent client to freeze
                    cout << idmsg.str();//only for debuging

                    if  (((cClient*)data)-> Login==true)//Am I loged in?
                        {
                            ((cClient*)data)->_server->InputQueue.push(idmsg.str());  //I'm logged in
                        }else{
                            ((cClient*)data)->_server->LoginQueue.push(idmsg.str());  //New player requests
                        }
                    SDL_SemPost(((cClient*)data)->_server->ClientsLock);//unlock smaphore
                    }
                catch(exception& e)
                    {
                        cout << "QUEUE IN ERROR: " << e.what() << endl;
                    }
            }//if connected
        }//while
    cout << "Closing client listener thread..." <<endl;
    return 0;
}

void cClient::Send(const char* Msg)
{
    if (Connected==true)
        {
            int len;
            int result;
            len=strlen(Msg+1); // add one for the terminating NULL
            result=SDLNet_TCP_Send(socket,Msg,len);
            if(result<len) //If I can't send data probably I've been disconnected so....
                {
                    cout << "SDLNet_TCP_Send: " << endl << SDLNet_GetError();
                    Connected = false;
                }
        }else{
            cout << "Not connected!!" << endl;
            }
}

Notice the client stores the data in two different queues depending if the client has loged in or not, finally we have the main socket server class:

<network.cpp>

#include "network.h"

cNetwork::cNetwork()
{
    //ctor
if(SDLNet_Init()==-1) {//Start SDL_NET
cout << "SDLNet_TCP_INIT: \n " << SDLNet_GetError();
exit(2);
}

//Clean thread variables
MasterThread = NULL;
ClientsLock = NULL;
ClientsLock = SDL_CreateSemaphore( 1 );//semafor protector, previene que mas de un thread vuelque información a la vez a la cola
}

cNetwork::~cNetwork()
{
    //dtor
SDLNet_TCP_Close(ServerSock);//Close socket
SDLNet_Quit();//Close SDL_NET
}


void cNetwork::CleanClient(unsigned int Client)//clean a client so we can use it
{
//cout << "cleaning client: " << Client << endl;
Clients[Client]->UID =Client;
Clients[Client]->socket = NULL;
Clients[Client]->_server = myserver;
Clients[Client]->Connected = false;
Clients[Client]->Login = false;
}

void cNetwork::Start(Uint16 MAXUsers, Uint16 Port)
{

//1-initialize MAXUsers clients
//2-Start sock server

unsigned int x;
for (x=1;x<=MAXUsers;x++)
{
Clients.push_back(new cClient());//insert a new client into the clients vector
CleanClient(Clients.size() -1);
}

cout << "OPENING SERVER SOCKET... " << endl;
if(SDLNet_ResolveHost(&ip,NULL,Port)==-1) {
cout << "SDLNet_TCP_ResolveHost:" << endl  << SDLNet_GetError();
exit(1);
}

ServerSock=SDLNet_TCP_Open(&ip);
if(!ServerSock) {
cout << "SDLNet_TCP_open: " << endl << SDLNet_GetError();
exit(2);
}

ShutDown = false;
//Create and run the threads
MasterThread = SDL_CreateThread( &Master, this );

}



int cNetwork::Master( void *data )//Master listener function/thread
{
TCPsocket new_tcpsock; //Temporary socket to store incoming connections
new_tcpsock=NULL;

cout << "Waiting for incoming connections... " << endl;

bool doneMain=false;
while(!doneMain)//MAIN LOOP
    {
    if(!new_tcpsock)//We have a new client incoming
    {
        new_tcpsock=SDLNet_TCP_Accept(((cNetwork*)data)->ServerSock); // accept a connection coming in on server_tcpsock
        SDL_Delay(5);//No new clients, wait a little
    }else{
        cout << "New client incoming..." << endl;
        unsigned int x;
        for(x=0;x<((cNetwork*)data)->Clients.size();x++)
        {
        if (((cNetwork*)data)->Clients[x]->Connected==false)
            {
            ((cNetwork*)data)->CleanClient(x);
            ((cNetwork*)data)->Clients[x]->socket=new_tcpsock;//asign the socket
            ((cNetwork*)data)->Clients[x]->Connected = true;
            ((cNetwork*)data)->Clients[x]->Login = false;
            ((cNetwork*)data)->Clients[x]->StartThreads();//start client listener thread
            break;
            }
    }//for
        new_tcpsock=NULL;//release temporary socket var

        }//if new data

if (((cNetwork*)data)->ShutDown==true)doneMain =true;

}//while
cout << "Exiting Main thread..." << endl;
return 0;
}

void cNetwork::ClearClients()//Clear disconnected clients
{
unsigned int x;
bool done = false;
//SDL_SemWait(ClientsLock);
if (Clients.size()>0)
{
x=0;
while (!done)
{
 if (!Clients[x]->Connected)
            {
            Clients.erase(Clients.begin()+x);
           // cout << "Number of clients:" << Clients.size() << endl;
            if (x>0)x--;
            }//if !connected
x++;
if(x>=Clients.size())done=true;
}
}//clients size

}

void cNetwork::Finish()
{
    ShutDown =true;
    SDL_WaitThread(MasterThread,&threadReturnMaster);
    unsigned int x;
    for(x=0;x<Clients.size();x++)
        {
            Clients[x]->Connected=false;//force disconnection
        }//for

    while (Clients.size()>0)
        {
            ClearClients();
        }
}

There are many things here, but I will resume it to the maximum, to do that, below there is a simple example to use the class:

#include "network.h"//socket server class 

cNetwork NET;
NET.myserver = &NET;
NET.Start(100,55555); //start networking, up to 100 clients on port 55555

std::sting text;

SDL_SemWait( NET.ClientsLock );
text = NET.InputQueue.front();//take a message out from the queue
NET.InputQueue.pop();
SDL_SemPost( NET.ClientsLock ); 

NET.Clients[X]->Send("data");//will send "data" to client X

NET.Finish();//finish socket server 

• We include the class
• Then instantiate it
• Start the listener thread (at this moment it is fully functional)
• Then I extract a message from the input queue (let's suppose we have a connected client that send it..)
• I send a message to the server ("data")
• Finally, I close the server

Want to test the class? basically start an empty project, add the three files I have described above and then add the example, after you compile the project you will be able to connect just using telnet (telnet localhost 55555)

That's all for now, in the next post I will mix everything to make the client login a reality, I will make a text based client too and hope everything is clear that way.

Blocking-sockets Windows binaries released!

        I have compiled the projects under MS Windows, surprisingly almost everything worked as expected and the binaries are available in  Downloads(-RPGServer / Client / Bot Tutorial ->Windows Binaries), feel free to download and test it.

      There were only two problems:

1. Server include paths were wrong, I had <SDL/SDL.h> so I replaced them with <SDL.h> and compiled fine
2. The bot was giving me errors in the line srand(time(NULL)), solved with a #include <time.h>, curious it worked fine under linux.

When running there is only an issue: no cout output under windows so server starts, and actually works! but there is no output at all, anyway it's ok to test.

        The server has certain limits like not being able to accept more than 1018 clients and to use too much resources, but if you plan to work with a limited amount of users (200-300) it will make the job.

        I would be glad to hear (constructive) comments about the code, the bugs you may found and well, basically your opinion.

Thanks to everybody!

Blocking-Sockets server code released

Hello again!

         After I decided to stop the development for a while I tought it would be nice to share the code with everybody interested, so after a huge cleanup (probably not enought) I have uploaded the code for the server, the client and a silly bot I did to test multiple client connections, they are now in Downloads (-RPGServer / Client / Bot Tutorial -> Source code), feel free to download and test it.

        The server has certain limits like not being able to accept more than 1018 clients and to use too much resources, but if you plan to work with a limited amount of users (200-300) it will make the job.

     Server is untested under windows (it works!), client and bot wont care too much for the OS (will work in linux / windows), I don't have a machine with Mac OS X to test so I couldn't test it (any volunteer to try with an apple computer?)

        I would be glad to hear (constructive) comments about the code, the bugs you may found and well, basically your opinion.

Thanks to everybody!

So I want make an MMORPG, Where do I start from? PART 4

Messaging and optimizations:

<Messaging:>

Networking is the core of an mmorpg, and just in the top of the network we have the messaging system. What we need is to create a way to communicate between server and clients.

Clients will send commands to the server for example: move forward, turn right, atack!... In the server we could have comands to send the map, the clients data like player / npc position,etc...

Most of the people (should) use serialization to send data, for example an item could be packed in an object / serialize it and send over the net, in the destination it would be deserialized and here we have our object/item again, but I have used another way to transfer messages that is hard to code but easier to understand: Plain-text messages.

Probably now you are wondering about how does it works, suppose we want our client to send to the server we are moving forward, then we will just send a command telling the server we have pressed "forward", suppose we decide the text command is going to be "/acce", when the server receives a string "/acce" from our client it will understand the client is moving forward so it will move the client forward in the map and return the new updated position to the client; The server will send "/ping" to the clients every x seconds ,when the clients receive "/ping" it will return "/pong" to measure delay in the communication between server/clients.

What are the advantages when using this method? basically it simplifies debugging , it's easier to understand an incoming packet when you can actually see the command and not a binary-format hex value, basically it is good for didactic purposes.

The disadvantages? performance and scalability , "/acce" uses 5 bytes to send "forward", too much for a simple commands in the other side for every new command you have to modify both client and server, that can be overwhelming!

I resume here all the commands I used for the demo:

Client side:

• /nick > Set players nickname
• /imag > Set players image
• /acce  > Move forward
• /dece > Move backward
• /righ > Turn right
• /left > Turn left
• /idle > Stay idle
• /pong > Answer a /ping request
• /quit > Exit server

Server side:

• /SUID > Send Unique identifier to the player
• /upda > Send non-player updates (X,Y,Nicknames...)
• /updb > Send player updates (X,Y...)
• /ping > Send a /ping request to calculate client latency
• /kill > Sent when a player logoffs

So we have our server running, in our client we create a player and then connect to the server, then the client will wait until our server sends "/SUID" with the UID for our client, the UID wont change until we logoff, after receiving the UID our client will send our nick and image, that is how our "enemies" will see us, then we are ready to explore.

We will start moving around, when we move we will be sending "/acce", "/dece", "/righ" and "/left" to the server that will update our position in the map, if we don't move a "/idle" message will be sent, when the server updates our position it will mark if the client has changed state so it will send the command "/upda" with our updated x and y to the clients in a range (500px), so server wont send updates to clients that can't see our moves, that will limit too network traffic.

When a clients exits the game it sends a "/quit" to the server, server will clean the players data and then it will send a "/kill" command to the clients so they will clean the exiting client...

<9 messages for the client + 5 messages for the server = total 14 diferent messages! , Probably the simplest messaging system you will have ever seen>


<Optimizations:>

After I made the server I saw two simple ways to improve performance: first one in the client second one in the server:

1. In the client I was sending 25 messages/ second to the server: I was sending data no mather if it was moving or if it was idle, but I thought that instead sending messages all the time I could only send data when a change was detected, for example: the client is idle, so the first time, it will send "/idle", but if it keeps standing still it wont keep sending "/idle", server will save the command until a new one arrives, so after a while I start to move forward, I send a "/acce" to the server first time, but again if I keep on moving forward it wont send again "/acce".. that divided outgoing messages from 25 to 2 or 3 messages when moving and 0! standing still.
2. In the server I checked the messages and found that "/upda" >> (client updates) was about 99% of the outgoing traffic, that generated a huge problem when a client was in a crowded area, suppose there are 50 clients near, all them are moving so every client is generating 50 updates / frame, with a frame rate of 20 /s that was 50 * 50 * 20 = 50.000 messages in a second!!! that was making the server collapse, so I had an idea, why not nest packets? when a "/upda" is generated I made the server to don't send that message instantly , instead of that it "packed"  into a bigger message, when more "/upda" messages arraived they were packed up to 10 updates in a bigger one, so in the same "crowded"  are messages sent will be reduced to 5.000, that was much more reasonable.

 With that pair of improvements server was able to handle 1000 clients at the same time with almost no delays , traffic had spikes of 18.000 incoming messages and 200.000 messages sent in a second, but then I reached blocking socket's limits...


<Blocking versus non blocking sockets in the next post>

So I want make an MMORPG, Where do I start from? PART 3

<It had been a long time since I made my last post , part of the problem was because I've been on a trip to Paris (A city everybody should visit at least once in their life), another part of the problem is my fight with non - blocking sockets, but now i'm back and I feel refreshed , with energy to continue with the project.>

Today I'm going to speak about a few things: the server structure ,about messaging  and finally about blocking sockets vs non blocking sockets:

Server Structure:

I plan to make my server architecture based in something like that:

Clients (a.k.a. players) will connect to a login/ authentication server , when validated they will connect to the proxy server, proxy will redirect players to the less loaded server, (balancing server load), server instances will be deployed across the servers on demand ( if a server is more powerful it could handle more instances), all server instances + login server + proxy srv will access the SQL server to write /read all persistent data to be stored, such as:

• Usernames / passwords for the login server
• Maps , players and objects for the server instances
• Number of instances, server instance sockets and health status of the instances (load and failures) for the proxy server.

That is a draft of the design I plan to do, at the moment I'm just working to make a server instance + sql server working, the remaining features will be done later (LOGIN / PROXY).

<Messaging in the next post>

So I want make an MMORPG, Where do I start from? PART 2

Welcome back  to the MMORPG series, part 2!

<Incoming connections....>

After expending one year to learn c++ and SDL I felt ready for the next step: Networking.

I did a chat server just to learn how SDL_net works, It is very interesting and I recommend to try to code one to everybody who dares to create an RPG/MMORPG server because it is basically the same with lower processing needs..

In a chat server (basically ) what happens is:

1. Clients connect to the server
2. They Login with user name / password
3. Every time they type something and press <ENTER> message is sent to all the other connected clients.

(I'm not including here channels and other things... it is just for demonstration purposes only)

If you want you can Download the sources for the server and client and test it, (Unfortunately it only works in Linux because if you run it under windows, it will redirect all the output to a file, so basically is useless..).

But for the MMORPG server I had to analyze it before starting....

What happens (basically )in a MMORPG server is:

(Client side:)

1. Clients connect to the server
2. They Login with user name / password (Or they can create a new user...)
3. Wait for the server to send UID (See below), and world data
4. Transmit actions to the server (update position, use skills, pic objects, attack, etc)
5. Update world state from data received from the server.
6. Goto point 4...

 (Server side:)

1. Allocate resources to accept and process data from incoming clients  (That includes queues and threads to process data)
2. For every client that logs into the server, assign an Unique identifier that is not going to change until client disconnects, the UID is going to be the identifier for all the process in the server (and the client too), the client will receive too the basic world data to start playing (like map data, players and enemies...)
3. Server processes incoming data from connected clients (that includes moving around / and all the skills the player have), for example a player moves forward, server receives a message from client(x) that wants to move forward, so it's going to increase client(x)->Speed
4. Update world state , here it comes were timing is important, the amount of times a world update is applied will change the speed of the game for ALL PLAYERS, for example, players wont send position changes to the server, but  move forward, backward, turn right/left commands, what happens when a world update is executed is that clients will move according to their speed and direction and update X/Y position, so no matter if a client tries to hack the client to run at 100 FPS and not the 25FPS set as default, it wont run faster...
5. Make a list of the players that have changed state (position, skills used)..
6. Send updated data from the clients with changed state to all the clients connected and in a X range...
7. Go to point 3....

I know it is a very raw way to describe what the server does, and you may be missing many features such as maps, skills, enemies / NPCS, AI, etc... but my aim is to make it a simple yet stable network server, then I will add the persistence to do world with  a database (Mysql).

To describe how my server works I'm going to explain how it evolved so it's easier to understand why I changed things...

So let's start to speak with the 3 versions / milestones my server has reached:

<Version Number 0.1:>

Classes:

• cSockServer: Stores global data , a vector to store all the clients connected, server socket, etc...
• cSockClient:Stores data for every client such as socket to communicate

Threads:

• 1x Master thread: Master thread listen for the clients and assigns a new socket to everyone.
• 1xProcessor thread:It processes the data into the input queues and updates wold data
• 1xUpdater thread: Insert updated data to the output queues
• 1xListener Thread/Client:listen for incoming data and inserts it into the Input queue.
• 1xSender thread/Client:Sends data from the output queue to the client.

Queues / vectors:

• Clients vector:Here I store clients data (sockets, UIDs, etc)
• Input queue:Every client has one, used to store incoming data
• Output Queue:Every client has one, used to store outgoing data

Good things of the design:

• Very simple.
• Functional.

Bad Things of the design:

• Very bad performance, when there was more than 3 clients connected you started to feel it lagging even working with the loop-back interface...
• There is no time control, so the server will have to be confident in the clients <Horrid mistake>
• Wasted data: Two threads per client, two queues per client.. too much

What I learned from this version:
Threads are a must when speaking of a socket server, so in this very first version what I saw is I needed to use them in a more efficient way, I was using many threads to just send and receive data to the client but only three to process data.. other big mistake was to use so many queues in the clients, it was a pain in the a** to check them and probably not efficient at all.., with all this things in my head I decided to make an updated version:


<Version Number 0.1.7:>


Classes:

• cSockServer: Stores global data , a vector to store all the clients connected, server socket, etc...
  
• cSockClient:Stores data for every client such as socket to communicate

Threads:

• 1x Master thread: Master thread listen for the clients and assigns a new socket to everyone.
• 2xProcessor thread:It processes the data into the input queue and updates wold data
• 1xUpdater Thread, inserts into output queue the data to send
• XxSender threads: Send data from the output queue
• 1xListener Thread/Client:listen for incoming data and inserts it into the Input queue.

Queues / vectors:

• Clients vector:Here I store clients data (sockets, UIDs, etc)
  
• Input queue:One in the server, used to store incoming data
• Output Queue:One in the server, used to store outgoing data

Good things of the design:

• Centralizing queues make it to waste less data.
• Having more threads to process data makes it to be able to handle more clients (watch the screen-shot in the top of the post), 101 clients connected...
• Timing centralized in the server, more secure now
  

Bad Things of the design:

• Still bad performance,after adding 20 clients it started to degrade performance no matter how many threads you added to the server

What I learned from this version:
Threads collide, if you use semaphores / mutexes you will prevent it from happening but, after adding 5 threads to process data from a queue, there is no performance improvement because threads have to wait each-other. It was a design fault, to continue increasing performance I had to add more queues with different semaphores so the threads wont collide so much, so that brings me to the last version:



<Version Number 0.1.12:>

<867 Clients connected at the same time, everyone constantly moving sending data to the server, 25 incoming packets /s per client, total, +2.000 incoming messages processed/s,  peak:18.000 messages SENT in a second, that is like 18 packets /millisecond, latency/lag for the clients from 49ms to 113ms average 70-80ms>



Classes:

• cSockServer: Stores global data , a vector to store all the clients connected, server socket, etc...
  
• cSockClient:Stores data for every client such as socket to communicate

Threads:

• 1x Master thread: Master thread listen for the clients and assigns a new socket to everyone.
• 2xProcessor threads:It processes the data into the input queue and updates wold data
• 1xUpdater Thread: inserts clients with updated data to a queue to update
• 4xUpdaterQueues:it gets the client UID from the UpdaterQueue and check which clients are near and need to get an update state, so it inserts the outgoing data to the Pool of Output queues
• Pool of Sender threads:The thread gets data from the Pool of Queues and Send data to the clients
• 1xListener Thread/Client:listen for incoming data and inserts it into the Input queue.

Queues / vectors:

• Clients vector:Here I store clients data (sockets, UIDs, etc)
  
• Input queue:One in the server, used to store incoming data
• <vector>Pool Output Queue:It can handle many queues so if the client number rise, you can add more queues to prevent thread collisions.
• <vector>Pool Sender thread Queue: Used to store / add Sender threads dynamically.

Good things of the design:

• Finally threads start to unleash their power, as many threads / output queues you add, as many users can handle, 867 clients (BOTs) are starting to be something good for a server....

Bad Things of the design:

• No optimized at all, there are many parts in the server that could change to make it work smoother, but for didactic purposes, I wont change them at the moment.

What I learned from this version:
I have learned to be very careful with the threads, many threads can access to a variable and read from it and processing power will go up, but when writing data you are forced to use semaphores, so speed stops increasing after a certain number of threads.

I added a command for the server that was "AddPower" it adds one output queue and 5 sender threads, so you can adjust performance, but speaking with a friend he told me that I could automate it so automatically it added more queues /threads, so I did! checking if the queues were filled and needed to be processed faster was easy, and the result is the server you have in the picture absorving 867 clients data....

Thanks Jhonny D!

See you soon in the MMORPG series....

So I want make an MMORPG, Where do I start from? PART 1

That is indeed a good question, asked so many times and answered many times too, in many sites the answer is like:

<If you have a team>

1. Get a team: coders + artists + world designers +well like 10 persons.
2. Get ready to expend 2 years (at least) on it
3. Pay big amount of money to the ISP
4. Have a design document

<Lone developer>

1. Forget about it :(

 What can I add to that? I basically decided to made it no matter how hard it was; I'm not somebody that has never programmed and think that their "RPG maker experience is enough", I'm not asking how to make it because If you are asking that, you probably are so far from the objective you desire that you wont get it never.

I tried to divide the BIG puzzle into small pieces, c++, SDL, SDL_image, SDL_ttf, SDL_NET, SDL_thread, Gimp, MYSQL....

With those pieces I made some bigger ones: A tileset, a simple game engine, game states, player creation process, a simple chat server....

And now it comes the good thing, I got a Simple (RPG DEMO) server with a client, I didn't made it on time (1 year was my objective), but now , after millions of doubts I know I will finish the game...

My objectives now are:

1. Start a series of posts explaining how I did it.
2. Clean Server / client code and upload them (Yes, I'm going to release it soon)
3. Set up MYSQL server

So first thing first:

Q:What do I need to learn to send / receive date across the network?
A:Network sockets

Q: How does network sockets work?
A: Network sockets handles data send/received to an IP network, they are both used by TCP (Connection oriented ) and UDP (Connectionless oriented), in the server you "open" a socket to listen for incoming data, open a socket means bind the program to a particular IP port of the computer, so we could open a socket at port 80 to create a web server, for example, when we create a server socket it waits until a Client machine (program) connects to it, then we create a client sockets that is going to actually communicate with the client. The clients just open a client socket and establish a connection with the server socket.

Q:Why do I need threads?
A: Threads are used to do many things at the same time, for example, a Client for the MMORPG, could have a thread to draw the images on the screen while another one sends/receives network data , the server could have one thread (maybe more?) to handle network data, another one for the AI, another one for Environment events... In the modern multi-core computers, it's a way to use 100% of the processor power

Q:Why are you using SDL_NET? isn't it an ancient library?
A:It is old, but network sockets are older and are still used today (and probably are going to be used for many, many years), just thing the new IP protocol (IPv6) uses them too... SDL_NET has other advantages too, it can be used in a multi-platform environment (I use it in my client and I can compile it under windows / Linux without touching a single line from the code to make it work :) ), finally it is a low-level library so you feel you have the control over what you are working, there are libraries made upon SDL_NET that add things (NET2 uses multi-threading for example), but you have less control about what is happening, another problem is find documentation , SDL_NET is so small that when you read the manual , you realize how easy is to use it.

<More Q/A soon...>

In the next post: Server code and explanations..

JB

RPG Client Alpha!!!

<Client V0.11>

<Client V0.1>

<Doesn't it looks nice??>


So after two weeks fighting against the elements I have been able to connect my Rpg Demo with the chat server (and basically modifying all the code), but now you can move around with your own player...


You can't do much things, in fact you can just MOVE, but that's exactly the base where I wanted to start from.


I have uploaded the Windows version binaries so basically everybody (windows /linux + wine) can test it, just go to the downloads page and get it!

For alpha testers: To test it, you are going to need an IP number, PM please so I will tell you...


See you

A Simple chat server, code uploaded...

<A chat server / client>


I've just uploaded the full code for the simple chat server and client projects, it is a very simple project, but shows the core needed to create a server including multi-threading and connections /disconnections from clients.

At the moment the clients can do:

• Send messages to other clients.
• Send private messages to another client (/priv nick message)
• Ask for the list of connected users (/List)
• Change nick (/nick newnick)
• And of course, exit the server (/quit)

I don't have plans to upgrade the chat server, instead I'm going to adapt it so many players can connect with a client (rpg demo 4) to the server and anybody can walk around and chat.... That is my next milestone, when I achieve it I'll enable the fights to spice it a little ;)

JB

SDL_net / SDL_threads(1): The Threading beast, is it so dangerous?

A multi-threaded echo server....

<What is the most important thing beyond every mmorpg? is it the history?, maybe the fancy 3d graphics? nope... it is the network play!>

<When I started the project , one year ago my objective was to bring the game on-line, I just didn't thought how many new things I needed to learn, the C++ language , the libSDL  library, all the other extension libraries (SDL_image, SDL_ttf, SDL_net, SDL_thread, ...), how to use blogger... , the Gimp, etc, etc....>

<It has been a long trip, but now I have arrived where I wanted to be, to the network>

<On the top of the page there is an screen-shot showing my first networking demo, a multi-threaded socket server using SDL_net and SDL_thread>

<I start the server and then I can connect with telnet to the server, every thing I type on the telnet window goes to the server and it returns the message to the sender, maybe you can think it is something too easy, but let me show why it's important...>

<Here what you can't see is more important than what you see, that's the project structure:>

<I encapsulate all the network code into two classes: csockserver and csockclient, fist one opens an SDL_net server socket who is going to sit and wait for incoming connections, when it happens (new client connection), it creates a new csockclient and adds it to the vector, then the csockclient starts a thread and the thread wait for data incoming to the server for a client, when it happens it just echoes the data back to the client>

<it is a very simple way to handle data, a perfect base to start adding new features, now if we change just a few lines we will have a chat server, but threaded>

<I have remarked the thread thing because when I started to look for demos / tutorials to use SDL_net I basically found nothing related to using it with threads, the question is a server needs threads, we can't stop the server logic just to see if a client is sending data, and for the server I'm going to use more threads: (for example , for the A.I.) so many things can happen at the same time, making threaded code is very hard, because any small mistake is going to crash the server, make your computer leak memory, or even worse open a security hole....>

<I'm going to finish the next step (the chat server) and I will post the c++ code>

<Ideas / suggestions are always welcome>

<see you, in the chat>

JB

<B.T.W. Ogres are ugly, SDL_net + threads too, what could happen if you mix them?? ;)>