A scalable client/server using select() socket function

• Download binaries - 36.2 KB
• Download source files - 6.2 KB

Introduction

This article is a continuation of my article on IOCP. In that article I had demonstrated the use of I/O Completion ports to create scalable client/server applications. In this article I will demonstrate how to use the select() function to create scalable client/server applications. In this implementation client and server send and display simple string messages.

The select() function works with synchronous sockets and doesn't require threads to be created.

The client I have provided here clientselect.exe is the same as clientiocp.exe in my earlier article. I have just renamed it. The code for this client can be found in my earlier article.

Using the code

The select() function will allow a developer to allocate the sockets in three different sets and it will monitor the sockets for state changes. We can process the socket based on its status. The three sets that are created for sockets are:

• Read Set – Check the sockets belonging to this group for readability. A socket will be considered readable when:
• A connection is pending on the listening socket
• Data is received on the socket
• Connection is closed or terminated
• Write Set – Check the sockets belonging to this group for writability. A socket will be considered writable when data can be sent on the socket
• Exception Set – Check the sockets belonging to this group for errors.

The sets are implemented using an fd_set structure. The definition of fd_set can be found in winsock2.h.

typedef struct fd_set {
        u_int fd_count;               /* how many are SET? */
        SOCKET  fd_array[FD_SETSIZE];   /* an array of SOCKETs */
} fd_set;

The following are macros that can be used to manipulate the sets. The source code of these macros can be found in winsock2.h.

• FD_CLR – Removes a socket from the set
• FD_ISSET – Helps in identifying if a socket belongs to a specified set
• FD_SET – Assigns a socket to a specified set
• FD_ZERO – Resets the set

For this implementation the client information will be stored in a singly linked list of CClientContext structure.

class CClientContext  //To store and manage client related information
{
private:
     int               m_nTotalBytes;
     int               m_nSentBytes;
     SOCKET            m_Socket;  //accepted socket
     char              m_szBuffer[MAX_BUFFER_LEN];
     CClientContext   *m_pNext; //this will be a singly linked list

public:
     //Get/Set calls
     void SetTotalBytes(int n)
     {
          m_nTotalBytes = n;
     }

     int GetTotalBytes()
     {
          return m_nTotalBytes;
     }

     void SetSentBytes(int n)
     {
          m_nSentBytes = n;
     }

     void IncrSentBytes(int n)
     {
          m_nSentBytes += n;
     }

     int GetSentBytes()
     {
          return m_nSentBytes;
     }

     void SetSocket(SOCKET s)
     {
          m_Socket = s;
     }

     SOCKET GetSocket()
     {
          return m_Socket;
     }

     void SetBuffer(char *szBuffer)
     {
          strcpy(m_szBuffer, szBuffer);
     }

     void GetBuffer(char *szBuffer)
     {
          strcpy(szBuffer, m_szBuffer);
     }

     char* GetBuffer()
     {
          return m_szBuffer;
     }

     void ZeroBuffer()
     {
          ZeroMemory(m_szBuffer, MAX_BUFFER_LEN);
     }

     CClientContext* GetNext()
     {
          return m_pNext;
     }

     void SetNext(CClientContext *pNext)
     {
          m_pNext = pNext;
     }

     //Constructor
     CClientContext()
     {
          m_Socket =  SOCKET_ERROR;
          ZeroMemory(m_szBuffer, MAX_BUFFER_LEN);
          m_nTotalBytes = 0;
          m_nSentBytes = 0;
          m_pNext = NULL;
     }

     //destructor
     ~CClientContext()
     {
          closesocket(m_Socket);
     }
};

Following is InitSets() function. This function will initialize the sets. Assign the sockets to appropriate sets. This function will be called before calling the select() function.

//Initialize the Sets
void InitSets(SOCKET ListenSocket) 
{
     //Initialize
     FD_ZERO(&g_ReadSet);
     FD_ZERO(&g_WriteSet);
     FD_ZERO(&g_ExceptSet);

     //Assign the ListenSocket to Sets
     FD_SET(ListenSocket, &g_ReadSet);
     FD_SET(ListenSocket, &g_ExceptSet);

     //Iterate the client context list and assign the sockets to Sets
     CClientContext   *pClientContext  = GetClientContextHead();
     while(pClientContext)
     {
          if(pClientContext->GetSentBytes() < pClientContext->GetTotalBytes())
          {
               //We have data to send
               FD_SET(pClientContext->GetSocket(), &g_WriteSet);
          }
          else
          {
               //We can read on this socket
               FD_SET(pClientContext->GetSocket(), &g_ReadSet);
          }

          //Add it to Exception Set
          FD_SET(pClientContext->GetSocket(), &g_ExceptSet); 

          //Move to next node on the list
          pClientContext = pClientContext->GetNext();
     }
}

Following is AcceptConnections() function. This function will monitor the sockets using select(). It will also process the sockets according to their status.

//This function will loop on while it will manage multiple clients 
//using select()
void AcceptConnections(SOCKET ListenSocket)
{
     while (true)
     {
          InitSets(ListenSocket);
          if (select(0, &g_ReadSet, &g_WriteSet, &g_ExceptSet, 0) > 0) 
          {
               //One of the socket changed state, let's process it.

               //ListenSocket?  Accept the new connection
               if (FD_ISSET(ListenSocket, &g_ReadSet)) 
               {
                    sockaddr_in ClientAddress;
                    int nClientLength = sizeof(ClientAddress);

                    //Accept remote connection attempt from the client
                    SOCKET Socket = accept(ListenSocket, 
                    (sockaddr*)&ClientAddress, &nClientLength);
                    if (INVALID_SOCKET == Socket)
                    {
                         printf("\nError occurred while accepting socket: 
                        %ld.", GetSocketSpecificError(ListenSocket));
                    }

                    //Display Client's IP
                    printf("\nClient connected from: %s", inet_ntoa
                        (ClientAddress.sin_addr)); 

                    //Making it a non blocking socket
                    u_long nNoBlock = 1;
                    ioctlsocket(Socket, FIONBIO, &nNoBlock);
                    CClientContext   *pClientContext  = new CClientContext;
                    pClientContext->SetSocket(Socket);

                    //Add the client context to the list
                    AddClientContextToList(pClientContext);
               }

               //Error occurred for ListenSocket?
               if (FD_ISSET(ListenSocket, &g_ExceptSet)) 
               {
                    printf("\nError occurred while accepting socket: %ld.", 
                    GetSocketSpecificError(ListenSocket));
                    continue;
               }

               //Iterate the client context list to see if 
               //any of the socket there has changed its state
               CClientContext   *pClientContext  = GetClientContextHead();
               while (pClientContext)
               {
                    //Check in Read Set
                    if (FD_ISSET(pClientContext->GetSocket(), &g_ReadSet))
                    {
                         int nBytes = recv(pClientContext->GetSocket(), 
                           pClientContext->GetBuffer(), MAX_BUFFER_LEN, 0);
                         if ((0 == nBytes) || (SOCKET_ERROR == nBytes))
                         {
                              if (0 != nBytes) //Some error occurred, 
                              //client didn't close the connection
                              {
                                   printf("\nError occurred while 
                                receiving on the socket: %d.", 
                                GetSocketSpecificError
                                (pClientContext->GetSocket()));
                              }

                              //In either case remove the client from list
                              pClientContext = DeleteClientContext
                                (pClientContext);
                              continue;
                         }

                         //Set the buffer
                         pClientContext->SetTotalBytes(nBytes);
                         pClientContext->SetSentBytes(0);

                         printf("\nThe following message was received: %s", 
                            pClientContext->GetBuffer());
                    }

                    //Check in Write Set
                    if (FD_ISSET(pClientContext->GetSocket(), &g_WriteSet))
                    {
                         int nBytes = 0;
                         if (0 < (pClientContext->GetTotalBytes() - 
                            pClientContext->GetSentBytes()))
                         {
                              nBytes = send(pClientContext->GetSocket(), 
                            (pClientContext->GetBuffer() + 
                            pClientContext->GetSentBytes()), 
                            (pClientContext->GetTotalBytes() - 
                            pClientContext->GetSentBytes()), 0);
                              if (SOCKET_ERROR == nBytes)
                              {
                                   printf("\nError occurred while 
                                sending on the socket: %d.", 
                                GetSocketSpecificError
                                (pClientContext->GetSocket()));
                                   pClientContext = DeleteClientContext
                                    (pClientContext);
                                   continue;
                              }

                              if (nBytes == 
                                (pClientContext->GetTotalBytes() - 
                                pClientContext->GetSentBytes()))
                              {
                                   //We are done sending the data, 
                              //reset Buffer Size
                                   pClientContext->SetTotalBytes(0);
                                   pClientContext->SetSentBytes(0);
                              }
                              else
                              {
                                   pClientContext->IncrSentBytes(nBytes);
                              }
                         }
                    }

                    //Check in Exception Set
                    if (FD_ISSET(pClientContext->GetSocket(), &g_ExceptSet))
                    {
                         printf("\nError occurred on the socket: %d.", 
                           GetSocketSpecificError(pClientContext->GetSocket()));
                         pClientContext = DeleteClientContext(pClientContext);
                         continue;
                    }

                    //Move to next node on the list
                    pClientContext = pClientContext->GetNext();
               }//while
          }
          else //select
          {
               printf("\nError occurred while executing select(): %ld.", 
                            WSAGetLastError());
               return; //Get out of this function
          }
     }
}

I have created a function GetSocketSpecificError() to get the error on a specific socket. When working with select() we can't rely on WSAGetLastError() because multiple sockets may have errors and we require socket specific errors.

//When using select() multiple sockets may have errors
//This function will give us the socket specific error
//WSAGetLastError() can't be relied upon
int GetSocketSpecificError(SOCKET Socket)
{
     int nOptionValue;
     int nOptionValueLength = sizeof(nOptionValue);

     //Get error code specific to this socket
     getsockopt(Socket, SOL_SOCKET, SO_ERROR, (char*)&nOptionValue, 
                                            &nOptionValueLength);

     return nOptionValue;
}

The following are linked list manipulation functions. These will be used to work with CClientContext singly linked list.

//Get the head node pointer
CClientContext* GetClientContextHead()
{
     return g_pClientContextHead;
}

//Add a new client context to the list
void AddClientContextToList(CClientContext *pClientContext)
{
     //Add the new client context right at the head
     pClientContext->SetNext(g_pClientContextHead);
     g_pClientContextHead = pClientContext;
}

//This function will delete the node and will return the next node of the list
CClientContext * DeleteClientContext(CClientContext *pClientContext)
{
     //See if we have to delete the head node
     if (pClientContext == g_pClientContextHead) 
     {
          CClientContext *pTemp = g_pClientContextHead;
          g_pClientContextHead = g_pClientContextHead->GetNext();
          delete pTemp;
          return g_pClientContextHead;
     }

     //Iterate the list and delete the appropriate node
     CClientContext *pPrev = g_pClientContextHead;
     CClientContext *pCurr = g_pClientContextHead->GetNext();

     while (pCurr)
     {
          if (pCurr == pClientContext)
          {
               CClientContext *pTemp = pCurr->GetNext();
               pPrev->SetNext(pTemp);
               delete pCurr;
               return pTemp;
          }

          pPrev = pCurr;
          pCurr = pCurr->GetNext();
     }

     return NULL;
}

Screen Shots

History

• August 7, 2007: Initial Release