Pipe template classes

• Download demo project - 58.1 KB
• Download source - 441 KB

1. Introduction

This pipe template library was written to minimize the programmer's effort in creating pipe servers and clients. With this library, the only things you should need do to get a working pipe client and server is to create the data classes and implement the server side function for the corresponding data classes. A second priority task in this project was to gather all the information on pipe errors and critical situations. And of course, you can use the pipe library code as technical documentation on pipes in Windows.

This library was written for MSVC 5.0, but it also works with later versions of MSVC. If you run this library for later versions of MSVC, it is best to use the ATL atlsecurity.h file instead of the one that comes with this library, because the security.h from this package was rewritten to work with the MSVC 5.0 compiler. This suggestion only applies to server code where DACL is used to set pipe security. Any implementation of DACL wrappers may be used for that purpose.

2. Background

This library uses type lists as introduced by Alexandrescu (see his book "Modern C++ Design" for more information, or just look at the file sources\utils\typelist.h). Type list typedefs are used in the declaration of the server base class.

3. Using the Library

3.1. Quick Start

3.1.1. Functionality

The library supports transmitting the following messages:

On the client side

1. Create the pipe client;
2. Open it;
3. Test it, determine if it is connected to a server side;
4. Create the data objects and send them to a server;
5. Test the connection to a server and, if needed, reconnect;
6. Close the client pipe.

On the server side

1. Create the server object;
2. Start server objects (when creating a worker thread, wait for the registration of a pipe client that in turn creates a worker thread per client communication channel);
3. When a communication channel is created, then begin transmitting data from the client side;
4. Data from every client is composed into the messages;
5. Server decides what message came from the client and calls the corresponding function of the server;
6. Communications continue till the server stops.

3.1.2. Messages

The unit of data transmission is a message.

A message is:

• an object of a class that was derived from the CPipeDataBaseImpl class template;
• with a declared GUID that identifies this class from all other messages processed by a server;
• in which virtual functions are implemented to save/load data.

Let's look at the class declaration of a message. Take, for example, the declaration from the test server (test_server\Messages.h file)

//first of all declare GUID 
//{E67A8D9E-EEB7-42d5-A1F9-25BDEE214A08}
static const GUID PipeMsgGUID_State = 
{0xe67a8d9e, 0xeeb7, 0x42d5, { 0xa1, 0xf9, 0x25, 0xbd, 0xee, 0x21, 0x4a, 0x8 }};
/*
    state information to registry/unregistry clients on server and verify there activity
*/
struct CPipeMsg_State : public CPipeDataBaseImpl<CPipeMsg_State>
{
    enum StateEn {S_Null=0,S_Started,S_Running,S_Stoped};

    CPipeMsg_State(const CString& _sName = _T(""),StateEn _state = S_Null)
        :m_sName(_sName),m_state(_state)
    {
    }

    DECLARE_PIPEDATAGUID(PipeMsgGUID_State);

    virtual bool save(IPipeCommunicator* _ppipe) const
    {
        VERIFY_EXIT1(NOT_NULL(_ppipe),false);
        if(!_ppipe->write(m_sName)) return false;
        if(!_ppipe->write((long)m_state)) return false;
        return true;
    }

    virtual bool load(const IPipeCommunicator* _ppipe)
    {
        VERIFY_EXIT1(NOT_NULL(_ppipe),false);
        long nstate = 0;
        if(!_ppipe->read(m_sName)) return false;
        if(!_ppipe->read(nstate)) return false;
        m_state = (StateEn)nstate;
        return true;
    }
    const CString& get_ClientName() const {return m_sName;}
    StateEn get_state() const {return m_state;}

protected:
    CString m_sName;        // name of the client
    StateEn m_state;        // state inforamtion
};//struct CPipeMsg_State

Every message type is declared in its own class. In this case, you can see a class for transmitting state information (CPipeMsg_State). As you can see in the code fragment, the data class is derived from the CPipeDataBaseImpl template, that receives the created class as a template parameter. Also, you should use the macro DECLARE_PIPEDATAGUID to set a GUID for the message class. In addition, the code fragment shows how to implement the virtual functions save() and load(), to save and load elements of the declared class into or from the pipe.

It is important to implement exactly the same sequence of calls in the save() and load() functions as shown here to write and read data to and from a pipe. It is also important that the order be the same in both functions.

The interface IPipeCommunicator gives the implementation of some base data types which are used for writing and reading class elements to and from the pipe. You should use the functions of this interface (base implementations) to save and load your message class data.

From an architect point of view, the message class describes which data is saved and loaded to a pipe and how the data is saved and loaded while working with the pipe. The message classes are used in both the client and server sides of the pipe channel in this library. For that reason, the message declaration files should be included in both parts of the pipe implementation.

3.1.3. Client side

Let's take a look at the client side of this library (file test_client\test_client.cpp of the project test_client).

The client side simply creates an object of class CClientPipeComm and uses it to transmit data to a server. If we ignore the details of implementing the communications interface, the essence of a pipe can be expressed in a few simple lines:

CPipeMsg_Text txt(sName,str);
pipe.save(&txt);
bFail = pipe.nead_toReconnect();

and:

CPipeMsg_State si(_sName,_state);
_pipe.save(&si);
//...
bFail = pipe.nead_toReconnect();

As you can see, to send some data to the server side, you just need to create a message class object and then call the save() method of the CClientPipeComm class object. This function will return true or false to indicate the success of the operation. In addition, if the state of the pipe changes (i.e., disconnects from the server), the function nead_toReconnect() will return true to say that you need to reconnect to send data.

Class CClientPipeComm description

bool open(const CString& _sServerName
    ,const CString& _sPipeName
    ,DWORD _dwDesiredAccess = GENERIC_WRITE
    ,DWORD _nWaitTime = NMPWAIT_USE_DEFAULT_WAIT
    ,bool _bWriteThroughMode = false
    ,DWORD _dwPipeMode = PIPE_READMODE_BYTE
    ,LPSECURITY_ATTRIBUTES _psa = NULL
);

This function opens a pipe on the client side.

Parameters:

• _sServerName - computer name where a server is started which the client is connected to (if it is the local machine, you should set this parameter to a "." string value).
• _sPipeName -- the pipe channel name to open.
• _dwDesiredAccess - for this release of the library, this should always be set to GENERIC_WRITE.
• _nWaitTime - sets the max pipe open time (if server exists, but can't respond right now).
• _bWriteThroughMode - sets the flag to use "write through" mode to process the write operation and wait until the server returns the read status to guarantee the transmitting of data (if set to true), or to use data buffering where the write function returns without waiting for data to be transmitted).
• _dwPipeMode - sets the mode of pipe (best to use PIPE_READMODE_BYTE).
• _psa - sets the security attributes for this pipe.

See the file pipe\namepipebase.h for a detailed description and error information. There, the CNamedPipeWrap class is defined, which implements the low level pipe operations (see the function open() for detailed information).

void close()

This function closes the pipe on the client side.

operator bool() const
bool operator ! () const

Above are operators to determine if the pipe is valid or not. A pipe is valid if a connection was created successfully.

bool save(IPipeDataBase* _data)

Sends the data to the server. If the operation fails, the client will try to reconnect and then try to resend data one time. If reconnection fails, it returns false. If data was successfully sent, then it returns true.

bool nead_toReconnect() const

This function returns true if, after a save operation, it needs to reconnect to send or resend data.

bool reconnect_client(DWORD _nWaitTime = NMPWAIT_USE_DEFAULT_WAIT)

Reconnects the client side of the pipe. If after calling this function, the connection is restored, then the pipe object will became valid; otherwise, it will be invalid. (You can test the pipe with operator bool() or bool operator!().)

Parameter:

• _nWaitTime - sets the max time for reconnecting.

void async_reconnect_client(DWORD _nWaitTime = NMPWAIT_USE_DEFAULT_WAIT)

Reconnects the client side of the pipe asynchronously. Creates the worker thread which tries to reconnect.

bool reconnecting() const

Returns true if asynchronously reconnecting now. If the function returns false before the function void async_reconnect_client() was called, you can test if the pipe is now connected.

void flush()

Flushes buffers of the client pipe.

An example of client code can be found in the test_client project.

3.1.4. Server Side

Server class creation

An example of a server can be found in the test_server project.

// declare type list of messages to receive by pipe server
typedef TYPELIST_2(CPipeMsg_State,CPipeMsg_Text) PipeServerMsgsList;

// declare pipe server class
struct CTestPipeServer : public CServerPipeCommImpl<CTestPipeServer,PipeServerMsgsList>
{
     CTestPipeServer(const CString& _sPipeName,CStatistics& _stat)
        :CServerPipeCommImpl<CTestPipeServer,PipeServerMsgsList>(_sPipeName)
        ,m_stat(_stat)
    {
        m_psa = &m_sa;
    }

    bool process(CPipeMsg_State* _pstate)
    {
        VERIFY_EXIT1(NOT_NULL(_pstate),false);

        m_stat.on_state(_pstate->get_ClientName(),_pstate->get_state());

        CAutoLock __al(g_output);
        g_output << _T("\"") 
                 << _pstate->get_ClientName() 
                 << _T("\" state:") 
                 <<  _pstate->get_state();
        g_output.endline();
        return true;
    }

    bool process(CPipeMsg_Text* _ptext)
    {
        VERIFY_EXIT1(NOT_NULL(_ptext),false);

        m_stat.on_text(_ptext->get_ClientName());

        CAutoLock __al(g_output);
        g_output << _T("\"") 
                 << _ptext->get_ClientName() 
                 << _T("\" :") 
                 << _ptext->get_Text();
        g_output.endline();
        return true;
    }

    CSecurityAttributes& get_security() {return m_sa;}
protected:
    CSecurityAttributes m_sa;
    CStatistics& m_stat;
};//struct CTestPipeServer

The example shows the server class for a pipe communicator that receives messages (CPipeMsg_State and CPipeMsg_Text). First of all, as you see, you need to create the typelist class as a list of all receive messages, derive your server class from CServerPipeCommImpl, and pass to it the typelist class created before. Then you just need to implement the functionality for each of the message types; in other words, you need to implement the function process() with the corresponding message type class as a parameter. (In the case of this example, these are the bool process(CPipeMsg_State* _pstate) and bool process(CPipeMsg_Text* _ptext) functions.) That is all that's needed to create a pipe server, because the base class, CServerPipeCommImpl, implements everything else needed for the server: read data from a pipe channel, create the message classes from chunks of data, and make calls to a corresponding function for processing messages received from the client.

This project also illustrates the initialization and creation of the server class. Let's see the corresponding code fragment:

CStatistics g_stat;        // statistics    
//...
void run_server()
{
    CTestPipeServer* pserver = new CTestPipeServer(_T("TestPipeLib"),g_stat);

    //activate security
    CSid everyone = security::Sids::World();
    CSid owner = security::Sids::Self();

    CDacl dacl;
    dacl.AddDeniedAce(everyone,security::AccessRights_WriteDac);
    dacl.AddAllowedAce(everyone,security::AccessRights_GenericWrite
                               |security::AccessRights_GenericRead);
    dacl.AddAllowedAce(owner,security::AccessRights_GenericAll);

    CSecurityDesc secrdesc;
    secrdesc.SetDacl(dacl);

    pserver->get_security().Set(secrdesc);

    // run pipe server
    pserver->start();

    cout << _T("Type 'q' -- to exit and 'i' -- for inforamtion") << endl;

    char ch = 0;
    while((ch=toupper(getch()))!='Q')
    {
        if(ch=='I') g_stat.print_all();
    }

    pserver->stop();
    delete pserver;

    cout << _T("pipe server was stoped");
}

This function shows the server class creation, initialization of the access rights to the created pipe, and starting the server. From the worker thread, the process() functions will be called for the corresponding messages. This function also demonstrates closing a pipe channel in the line pserver->stop();.

4. Architecture and Internal Description of the Library

4.1. Brief Description of Classes

class CAutoLock 

Synchronization of code fragments.

struct CClientPipeComm

Pipe client (sending messages to the server).

struct CNamedPipeWrap

wrapper class for WinAPI of the pipe in Windows. This class gives the functions (that work with the WinAPI) for the client and server pipe classes.

template<typename _FromType,typename _ToType,long _bufsize = 1024*4>
struct convert_string

Template class for string conversion optimized for memory use. (This class uses either the memory block on the stack, or heap memory if memory for string conversion is bigger than that pre-allocated on the stack.)

struct COSVersion

Class that encapsulates the information about the system, in particular the Windows version.

struct CPipeBufferChunk

Data buffer for one chunk of pipe data.

template<typename _Type>
struct CPipeCommunicatorImpl

Template class that implements the IPipeCommunicator interface for read and write of base data types to a pipe channel.

template<typename _PipeDataClass>
struct CPipeDataBaseImpl

Template class that implements the base operations with messages over a pipe channel.

struct CPipeReadedBuffer

The data buffer for the pipe channel (to receive data into).

template<typename _Server,typename _TypeList>
struct CServerPipeCommImpl

The class that implements the base of a pipe server.

template<typename _Server,typename _TypeList>
struct CServerPipeReadThread

The class that implements communications over a pipe channel with a concrete client. The process() functions are called from the thread of this class.

struct DateInfo

The auxiliary class to transmit CTime info over a pipe channel.

template<typename _TypeList>
struct FindDataTypeAndProcess
template<>
struct FindDataTypeAndProcess<NullType>

The auxiliary template classes to determine the concrete message class by reading its GUID from the pipe channel. It is used in the server class before loading concrete message information.

interface IPipeCommunicator

The interface that implements operations with base data types (to read and write from and to a pipe channel).

However, if the library should use more base types, I would use template class(es) instead of a class with virtual functions.

interface IPipeDataBase

The interface of a message data.

template<typename _FromType,typename _ToType>
struct string_converter

The auxiliary template class. It is used by the convert_string class and implements string conversion (Unicode <=> ASCII).

4.2. Library Functionality

4.2.1. Client side

(See paragraph 3.1.1. "Functionality" for information.)

The open, close, and get information functions of the client class just call the corresponding function of the CNamedPipeWrap class that wraps the pipe WinAPI.

The save() function will first save the message GUID into the pipe stream, and then save data for the message class object (by calling the function save() of the message class). Also, it tests whether a reconnect is needed, and it will try once to reconnect if write data failed because of disconnect trouble.

4.2.2. Server side

The DWORD thread_main() function implements the parallel thread function, which creates a new communication object and creates a new pipe object (preadthread->create(m_sPipeName,m_psa)), then waits for the client connection (preadthread->connect(bstoped)) and starts the created thread. If the server class function stop() is called (directly or indirectly from the destructor), then the server stops waiting for the client to connect and stops the parallel thread. (The destructor also waits for read threads to stop, sending them an event to stop transmitting data or processing.)

The server thread that communicates with the concrete pipe client (class CServerPipeReadThread) reads the parallel thread data from the pipe channel. All pipe data is transmitted in small chunks (independent of the pipe channel type (PIPE_TYPE_MESSAGE)).

Reading data is done as follows:

1. Form a data block (there is a limit to the maximum data block size; see the CNamedPipeWrap::read_pipe() constant sec_criticalbuffersize).
2. The client should transmit at least one data block. This is the meta-block (GUID of message class that follows), and it should precede any data blocks. When the server receives the meta-block, it prepares for receiving the message corresponding to the GUID. The server uses the template class FindDataTypeAndProcess to find the corresponding message class for the GUID.
3. If a message class is found, then:
1. The server calls the load() function of the message class to load data for that message.
2. The server calls the process() function for the loaded message object.
3. Jump to step 1.
4. If a GUID or message is not found, then jump to step 1 (and ignore the invalid block data).

4.2.4. Data Buffer

It's possible that some people might think that it is too expensive to create classes for data buffering of pipes, but there are reasons why it should be done. A system buffer of every pipe channel is allocated by the system, and their size should be small (512 bytes is almost an optimum size). Also, consider that message size can be almost anything, so it is obvious that we need to create a data buffer.

5. Extras About the Library

5.1. Compiler Information

This is a template library, so you don't need to link any additional libraries to use this code. The included DLL module also contains unit tests. See the MonitorTestSuite project for more information about running the unit tests. This project uses a simple unit test interface, and test_pipe.dll imports functions for executing the units test in the pipe library. (See sources\pipe\pipe_tests.cpp for the file with the unit tests.)

extern "C"
void _declspec(dllexport) test_suite(ITestSuiteHelper* _ts)
// pipe\pipe_tests.cpp

The above export function is used by MonitorTestSuite to execute units test, so test_pipe.dll is needed to execute unit tests, but you don't need to link this library with code that uses the pipe library.

5.2. Possible Redesign

Duplex mode. At present, a client just sends data and a server just receives the data, but there are some "features" of the pipe API on a Windows system. As an alternative, it would be better to create a server at both sides to be able to send and receive data from any sender, whether client or server.