The problem
IP multicasting is the process of sending a single message to a multicast group and this message is duplicated by the switch according to the number of users registered in that group. This technique allows a user to send a single message once to multiple receivers. The most widely used multicast transport is UDP.
UDP multicasting provides great performance and network utilization; however, it suffers from two disadvantages:
- Maximum message size is limited to 64 KB.
- UDP packets may be discarded by network elements (routers, switches, etc...).
Although the first disadvantage may be partially overcome by different compression algorithms, the second disadvantage may become a real problem.
PGM to the rescue
Over the years, many reliable multicast protocols were developed and tested in academic and commercial scenarios; however, PGM (Pragmatic General Multicast) was the most appropriate, and therefore adopted by Microsoft’s implementation as part of MSMQ 3.0, which must be installed on a Windows XP and later OS in order to run this sample.
PGM is applicable for:
- Stock and news updates
- Data conferencing
- Real-time video transfer
- Bulk data transfer
PGM is a transport layer protocol which interfaces directly with the network layer using raw sockets. That means that unlike most widely used protocols, PGM doesn’t build on top of TCP or UDP, but implements its own transport mechanism.
PGM protocol specification defines five packet types:
- ODATA – original data content
- NAK – negative acknowledgment
- NCF – NAK confirmation
- RDATA – data retransmission (repair)
- SPM – source path message
PGM moves the responsibility for packet reception to the receiver side, and by that, eliminates the need for ACKs which would flood the sender’s link. Instead, NAK (Negative Acknowledgment) are sent by the receiver when it detects some packet is missing from the sequence.
In addition, PGM uses FEC (Forward error correction) for ODATA packets, RDATA packets, or both. FEC allows the receiver to assemble the packet even if some parts of it are lost in the way or scrambled.
PGM is an asynchronous protocol, which means that when a socket is created as a sender, it may only send data, and similarly, a receiver may only receive it.
PGM achieves its efficiency through a hierarchical structure as shown below. SPM messages are sent by PGM senders to build the hierarchy tree.
Using these techniques, PGM guarantees that a receiver in the group either receives all data packets from transmissions and repairs, or is able to detect (rare) unrecoverable data packet loss. PGM has above 91% percent network utilization, and scales well on bandwidth limited networks as well as on high-speed networks (>100 Mbps).
WCF implementation
Custom transport
In order to take advantage of PGM features inside the WCF framework, there is a need for a custom transport. Writing a custom transport binding element is not an easy job, but thanks to the WCF SDK samples and a great avant-garde work by Roman Kiss, it is an achievable task.
MEP
Since PGM is inherently a one way protocol, we will use OneWay MEP, i.e., implement IInputChannel/IOutputChannel interfaces, and the same for Session interfaces.
Architecture
PgmTransportElement is the entry point. It is created by the WCF configuration engine. By calling the overloaded CreateBindingElement method, the PgmTransportBindingElement is created.
protected override BindingElement CreateBindingElement()
{
PgmTransportBindingElement bindingElement =
new PgmTransportBindingElement();
this.ApplyConfiguration(bindingElement);
DiagnosticsManager.Instance.Configure(Diagnostics);
return bindingElement;
}
This class is responsible for the creation of channel factories on the sender side:
public override IChannelFactory<TChannel>
BuildChannelFactory<TChannel>(BindingContext context)
{
if (typeof(TChannel) == typeof(IOutputChannel))
{
return (IChannelFactory<TChannel>)
(object)new PgmChannelFactory(this, context);
}
else if (typeof(TChannel) == typeof(IOutputSessionChannel))
{
if (DataMode == DataMode.Stream)
{
throw new InvalidOperationException("Sessionful contracts" +
" cannot use Stream mode. Either configure " +
"dataMode to Message or change the contract to non sessionful");
}
return (IChannelFactory<TChannel>)(object)
new PgmSessionChannelFactory(this, context);
}
else
{
throw new ArgumentException("Unsupported channel type " +
typeof(TChannel).Name);
}
}
and channel listeners on the receiving side:
public override IChannelListener<TChannel>
BuildChannelListener<TChannel>(BindingContext context)
{
if (typeof(TChannel) == typeof(IInputChannel))
{
return (IChannelListener<TChannel>)
(object)new PgmChannelListener(this, context);
}
else if (typeof(TChannel) == typeof(IInputSessionChannel))
{
if (DataMode == DataMode.Stream)
{
throw new InvalidOperationException("Sessionful contracts" +
" cannot use Stream mode. Either configure dataMode " +
"to Message or change the contract to non sessionful");
}
return (IChannelListener<TChannel>)
(object)new PgmSessionChannelListener(this, context);
}
else
{
throw new ArgumentException("Unsupported channel type " +
typeof(TChannel).Name);
}
}
This implementation contains two channel types:
PgmInputDatagramChannel/PgmOutputDatagramChannel – one way message exchange with no session.PgmInputSessionChannel/PgmOutputSessionChannel – one way message exchange with session.
Each input channel encapsulates the PgmReceiver object, and each output channel encapsulates the PgmSender object. Those objects, in turn, interface directly with PGM sockets, and are responsible for the actual data delivery.
InputChannel and its descendants’ implementation are based on an InputQueue generic container which is available from the UDP sample. This is an advanced implementation of a Producer/Consumer collection which decouples between processing of incoming data from the network and WCF channel stack processing of Message objects. In other words, data may be read faster than it can be processed; therefore, it is stored in a queue and processed sequentially.
Configuration options
Mutual options
maxMessageSize – sets the maximum size of the message.dataMode – PGM socket implementation contains two data modes: Message and Stream.
In Message mode, the socket is created with the SocketType.Rdm value and hence preserves message boundaries. A stream socket is created with SocketType.Stream, and is similar to other streaming protocols like TCP in which a single send may not result in a single receive. Therefore, a framing implementation is required – I decided to implement a length framing which means that each message is prefixed with a fixed sized array which contains the message length. This way, the receiver knows how much data to read from the socket and assemble a valid message.
Sender options
sendRate – By default, the send rate is configured to 56Kbps which is relatively slow, so I decided to set it to the default value of 5Mbps, and it can be extended to larger values.lateJoin – A receiver which joins the session in the middle of a transmission may request data which it lost. This value indicates the percentage of time in which a late joiner may request for retransmission of data.fecMode – As previously mentioned, FEC helps restore partially lost or incorrectly received packets. There are four options:
Disabled – FEC is disabled.OnDemand – FEC is applied on RDATA packets.ProActive – FEC is applied on ODATA packets.Both – Both OnDemand and ProActive are applied.
senderWindowAdvance – If the receiver finds a missing packet from a sequence, it sends NAK to the sender and the sender sends RDATA to recover the lost packet. This value indicates a sending window advance percentage.setMessageBoundary – It is not desired to send a large buffer in a single send command. Therefore, you can inform the socket of a large packet size, and then the socket will send it in a more network friendly manner.multicstTTL – Sets the time to live for each packet, i.e., how many network elements the message may route through before discarded.senderInterface – On multi-home machines, it is possible to set the desired network interface for PGM transmission.
Diagnostics
Both PGM senders and receivers provide a rich set of real time statistics. Those values are logged in dedicated files by setting the diagnostics option of PgmTransport to Enabled. Also, the time interval and the directory can be set using configuration.
Known issues
When importing WSDL data from a WCF service running on top of PGM transport, SvcUtil.exe fails to import the PgmTransport element and all its attributes. I tried to implement the WSDL and Policy export interfaces taken from the UDP sample, but with no success. Therefore, after running svcutil.exe, you need to copy the missing PgmTransport element manually.
PGM supports streaming mode; however, it is not fully supported yet by my implementation.
