Steganography 17 - FTP Through a Proxy

• Download source - 27.7 KB

Introduction

This article explains how to write a proxy server for the File Transfer Protocol. It shows how to enhance the proxy server to insert additional data into the transmissions and extract secrets received from a client. The actual FTP server is not touched by the piggyback conversation between stego-client and proxy.

The scenario is: You want to send text to a remote machine or download a certain message from it, but you suspect eavesdroppers to observe your internet connection. The proxy experiment is an approach to make the secret information dissolve in unsuspicious everyday file transfers. (It will take only a few changes to let the proxy filter out forbidden content or send silent alerts on specific user actions.) There are four steps to take:

1. Capture the command channel
2. Capture the data channel
3. Filter the data channel for incoming hidden messages, insert outgoing messages
4. Write a stego-enabled FTP client

The Basics of File Transfer Protocol

FTP is a very old protocol and a bit like SMTP/POP. The client contacts the server on well known port 21 to establish a command channel. The commands and responses sent on this channel are plain readable ASCII text. But as SMTP and POP send long content like mail attachments along with the commands, FTP establishes a data channel for each multiline or binary transfer.

Usually the client asks for a data channel with the PASV (passive mode) command. That makes the server open a random port, write the IP address and port number on the command channel and wait. The client has to parse the information and connect to the port, then the transmission can start.

The older variation would be "active mode": The client opens a data channel port (turns into a server itself), sends the PORT command and waits for the server (which turns into a client) to connect. But as modern firewalls don't allow any application to act as a server anyway, I'll support passive mode only.

Step 1: Command Channel Write-Through

A proxy for the command channel seems to be the easiest part: We need two sockets, the first one accepts a connection from a client, the other one connects to the FTP server. Whenever socket A receives anything, it must be sent through socket B immediately. But ... what if the server responds to a PASV command?! If our proxy would delegate it to the client unchanged, the data channel would be established behind our back!

That means, we have to intercept passive mode connection data. With that information, we can open another pair of sockets: A client socket connecting to the actual data channel, and a server socket waiting for the actual client to connect. Proxy sockets always work in teams of two, so I wrote the class SocketSet. It encapsulates two sockets that perform write-through to/from each other.

The class CommandChannelSocketSet overwrites the ReceivedFromServer method to intercept and process specific responses. First step, we want to catch data channels, so we have to decode everything the FTP server sends and check the status code. If the status is 227 (entering passive mode), we just place another proxy in the middle of the new data channel.

protected override void ReceivedFromServer(IAsyncResult result)
{
  AsyncReceiveInfo receiveInfo = result.AsyncState as AsyncReceiveInfo;
  int bufferContentLength = socketToServer.EndReceive(result);
  byte[] buffer = receiveInfo.Buffer;

  if (bufferContentLength > 0)
  {
    String utf8Text = Encoding.UTF8.GetString(buffer, 0, bufferContentLength);

    if (utf8Text.StartsWith(FTP_ENTERING_PASSIVE_MODE))
    {  // The server has opened a data channel.
      // parse IP information

      String address;
      int remotePort, localPort;

      String connectionInfo = GetTextBetweenBrackets(utf8Text);
      String[] connectionInfoParts = connectionInfo.Split(',');

      // get the IP address of the waiting socket
      address = String.Join(".", connectionInfoParts, 0, 4);

      // get the port of the waiting socket
      int portHighByte = int.Parse(connectionInfoParts[4]);
      int portLowByte = int.Parse(connectionInfoParts[5]);
      remotePort = (portHighByte << 8) + portLowByte;

      // create local data channel socket
      IPAddress localAddress = 
        Dns.GetHostEntry(Dns.GetHostName()).AddressList[0];
      Socket localDataSocketToClient = new Socket(
        AddressFamily.InterNetwork,
        SocketType.Stream,
        ProtocolType.Tcp);
      AsyncAcceptInfo acceptInfo = new AsyncAcceptInfo(
        localDataSocketToClient,
        address,
        remotePort);

      // open local data channel on [port+1], because when debugging offline,
      // [port] is already in use by the local FTP server
      localPort = remotePort + 1;
      localDataSocketToClient.Bind(new IPEndPoint(localAddress, localPort));
      localDataSocketToClient.Listen(1);
      localDataSocketToClient.BeginAccept
        (new AsyncCallback(DataChannelConnected), acceptInfo);

      // Here our own data channel socket is ready.
      // The callback [DataChannelConnected] will connect us to
      // the server's data channel as soon as someone asks for it.

      // format PASV response for client
      String addressText = localAddress.ToString().Replace('.', ',');
      int localPortHighByte = localPort >> 8;
      int localPortLowByte = localPort - (localPortHighByte << 8);
      String responseConnectionInfo = String.Format(
        "{0},{1},{2}",
        addressText,
        localPortHighByte,
        localPortLowByte);
      utf8Text = ReplaceTextBetweenBrackets(utf8Text, responseConnectionInfo);
      buffer = Encoding.UTF8.GetBytes(utf8Text);
      bufferContentLength = buffer.Length;
    }

    // send buffer to client
    socketToClient.Send(buffer, 0, bufferContentLength, SocketFlags.None);
  }

  // wait for next command
  BeginServerToClient();
}

A second later, the client will connect to the waiting proxy socket. What to do then? That's easy: Create another SocketSet and let the data transfer flow right through our proxy:

private void DataChannelConnected(IAsyncResult result)
{
  // welcome, client!
  AsyncAcceptInfo acceptInfo = result.AsyncState as AsyncAcceptInfo;
  Socket remoteDataSocketToClient = acceptInfo.WaitingSocket.EndAccept(result);

  // start another proxy
  this.dataSockets = new SocketSet(remoteDataSocketToClient,
                                   acceptInfo.ServerAddress,
                                   acceptInfo.ServerPort, true);
}

Alright, all transfer goes through our proxy server. But ... how do we know when to stop? There are two scenarios:

1. LIST command or file download: The server sends a 150 (begin file transfer) status message on the command channel. It contains the transmission size in bytes.
2. File upload: The size is not known in advance, so we have to wait for a 226 (passive transfer complete) status message on the command channel.

In the first case, we can parse the size from the status message and tell the SocketSet to close automatically after reading/sending that amount of data. The second case is even easier: Received status code? Shutdown data sockets!

At this point, my application worked fine in stupid write-through mode, but it crashed when I ran it in "stego mode". Why? Well, the proxy's job is to merge an additional message into the streams that flow through the data channel. That means, the size of the transmission grows. The proxy has to guess the final size of stream + secret and then manipulate the status message. ReceivedFromServer grows longer:

protected override void ReceivedFromServer(IAsyncResult result)
{
  AsyncReceiveInfo receiveInfo = result.AsyncState as AsyncReceiveInfo;
  int bufferContentLength = socketToServer.EndReceive(result);
  byte[] buffer = receiveInfo.Buffer;

  if (bufferContentLength > 0)
  {
    String utf8Text = Encoding.UTF8.GetString(buffer, 0, bufferContentLength);

    if (utf8Text.StartsWith(FTP_ENTERING_PASSIVE_MODE))
    {  // The server has opened a data channel.
      [...]
    }
    else if (utf8Text.StartsWith(FTP_PASSIVE_TRANSFER_COMPLETE))
    {  // The transfer is complete.
      if (this.dataSockets != null)
      {  // complete transfer to client, then close data channel
        this.dataSockets.Close();
      }
    }
    else if (utf8Text.StartsWith(FTP_BEGIN_FILE_TRANSFER))
    {  // The server is beginning to send something on the data channel.
       // parse expected size from a status message like
       // "150 Opening binary mode data connection for /file.name 
       // (12345 bytes)."
      String sizeInfo = GetTextBetweenBrackets(utf8Text);
      if (sizeInfo.Length > 0)
      {
        String sizeInfoBytes = sizeInfo.Split(' ')[0];
        long size = long.Parse(sizeInfoBytes);
        if (this.dataSockets != null)
        {  // tell the data channel proxy how much the server will send
          this.dataSockets.ExpectedDownloadSize = size;

          // fake the status message to make the client 
          // download everything we send
          long sizeAfterHiding = Stego.CalculateSizeAfterHiding(size);
          utf8Text = ReplaceTextBetweenBrackets
            (utf8Text, sizeAfterHiding.ToString() + " bytes");
          buffer = Encoding.UTF8.GetBytes(utf8Text);
          bufferContentLength = buffer.Length;
        }
      }
    }
    // send buffer to client
    socketToClient.Send(buffer, 0, bufferContentLength, SocketFlags.None);
  }
  // wait for next command
  BeginServerToClient();
}

The rest of the FTP talk may pass the proxy without interference. Let's take a closer look at the data channel!

Step 2: Data Channel Write-Through

There is no special class for the data channel sockets. SocketSet acts as a transceiver in default mode: It receives buffers from socket A, sends them through socket B and vice versa. Whenever a thread hosting a listening socket accepts a connection (which means a new socket representing the connection is created) it delegates the connection and the proxy target to a new SocketSet:

// somewhere in class SessionManager
this.mainServerSocket.Listen(10);
Socket socketToClient = this.mainServerSocket.Accept();
CommandChannelSocketSet sockets = 
    new CommandChannelSocketSet(socketToClient, "localhost", 21);

// somewhere in class CommandChannelSocketSet
AsyncAcceptInfo acceptInfo = result.AsyncState as AsyncAcceptInfo;
Socket remoteDataSocketToClient = acceptInfo.WaitingSocket.EndAccept(result);
SocketSet dataSockets = new SocketSet(remoteDataSocketToClient,
                                      acceptInfo.ServerAddress,
                                      acceptInfo.ServerPort, true);

// Constructor
public SocketSet(Socket socketToClient, String serverAddress, 
    int serverPort, bool isStegoMode)
{
  // connect second socket
  this.socketToClient = socketToClient;
  this.socketToServer = new Socket
        (AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
  this.socketToServer.Connect(serverAddress, serverPort);

  // initialize fields, read configuration
  [...]

  // prepare for hiding/reading: read secret file
  [...]

  //initialize stego-utility
  [...]

  // start listening
  BeginClientToServer();
  BeginServerToClient();
}

The Begin* methods initialize buffers and call BeginReceive on their sockets. BeginServerToClient also resets the ManualResetEvent socketToServerCanClose to signal that the sockets may not be closed before the received buffer has been transferred to the FTP client. If the socket to the FTP server gets closed - in case of failures or a successfully completed transfer - the SocketSet waits for the event to be set again before it disconnects the other socket. This makes sure that a slow FTP client can read the whole stream from our proxy, though the proxy read it faster and the FTP server already thinks the transfer is done.

The interesting part is the ReceivedBy* methods. Before we send the streams on, we can add or remove certain bytes from the buffers. The correction of transfer size messages has already been done by the command channel proxy. When the FTP client sends something, our personal stego-stream gets read and removed from the buffer. Only the clean file goes to the FTP server.

protected virtual void ReceivedFromClient(IAsyncResult result)
{
  AsyncReceiveInfo receiveInfo = result.AsyncState as AsyncReceiveInfo;
  int bufferContentLength = socketToClient.EndReceive(result);

  if (bufferContentLength > 0)
  {
    SendBufferToServer(receiveInfo.Buffer, 
        bufferContentLength, storeToFileName.Length > 0);
    BeginClientToServer();
  }
  else
  {
    ForceClose();
  }
}

private void SendBufferToServer
    (byte[] buffer, int bufferContentLength, bool useStegoMode)
{
  if (useStegoMode && isStegoMode && stegoUtility != null)
  {  // remove message from buffer
    using (FileStream messageFile = new FileStream(
        storeToFileName, FileMode.Append, FileAccess.Write, FileShare.Read))
    {
      buffer = stegoUtility.Extract(buffer, bufferContentLength, messageFile);
      if (buffer != null) bufferContentLength = buffer.Length;
    }
  }

  if (buffer != null)
  { // send the clean buffer to server
    socketToServer.Send(buffer, 0, bufferContentLength, SocketFlags.None);
  }
}

When the FTP server sends something, a part of the proxy's local secret-messages-file gets inserted among the buffered bytes and sent along with the download.

protected virtual void ReceivedFromServer(IAsyncResult result)
{
  AsyncReceiveInfo receiveInfo = result.AsyncState as AsyncReceiveInfo;
  int bufferContentLength = socketToServer.EndReceive(result);

  if (bufferContentLength > 0)
  {
    byte[] buffer = receiveInfo.Buffer;
    completedDownloadSize += bufferContentLength;
    SendBufferToClient(buffer, bufferContentLength, this.isStegoMode);
  }

  if (expectedDownloadSize > completedDownloadSize)
  {  // waiting for more data or no limit defined (expectedDownloadSize < 0)
    BeginServerToClient();
  }
  else
  {  // job done, allow closing
    socketToServerCanClose.Set();
  }
}

Step 3a: Insertion Stego

How do we merge two binary buffers? If you have read my articles about wave or bitmap steganography, you will start looking for a key, now. But let's keep things simple this time. We know the carrier stream's size from the status message we intercepted on the command channel. The secret data we want to send to the FTP client is in a file only accessible to the user account of the proxy service (if you like, that can be an encrypted file). If the secret file is longer than the FTP transfer, we cut off a smaller piece, send it, and wait for the next FTP transaction to send the whole text.

So, we can be sure that the message stream is not longer than the carrier stream. That means we can split the carrier stream into equal blocks, one block per message byte. Insert one message byte at the beginning of each block and we're done! But ... how does the other side know the block size without knowing the length of the message? Of course we need a little header, so we insert the length of the message at the beginning of the carrier stream. Then the client can read the length first, calculate the block size and extract the inserted bytes. Message and carrier stream will both be reconstructed.

Another but ... when the client sends a file, it does not tell us the size in advance. The proxy would have to buffer the whole stream before it could extract/remove the client's message and send the cleaned carrier stream to the FTP server. That would be not cool at all - and it could lead to timeouts during large uploads, because the FTP server would be waiting for buffers that couldn't be cleaned and sent on before the slow client has uploaded everything to the proxy. To upload and extract write-through, we need the block size at the stream's beginning. Knowing the block size of n, we can move every nth byte from the received buffers to a secret file and re-send the buffer right away.

That means, we need two header fields:

1. Block size: The number of FTP transmission bytes between two secret message bytes
2. Message size: The length of the hidden message

The header will be sent through the outgoing socket first, followed by the first message byte with the first carrier block, the second byte with the second carrier block and so on.

That's the harmless theory. In reality the incoming socket reads a series of buffers. The length of each buffer depends on the configuration and speed of the FTP server. See what would happen if we'd manipulate and send each buffer as soon as it arrived:

Whenever bufferSize mod blockSize is greater than 0 (which means: as good as everytime) disturbing tails of carrier bytes would be sent between the blocks. From the client's point of view, both hidden message and plain file would be garbage.

The solution is a cache that buffers parts of the buffers: We cut off a number of blocks from each received buffer, process and send those, and cache the rest of it until the next buffer arrives. Then we concatenate cache and buffer to a longer stream, again cut off a number of blocks, cache the remaining bytes, and so on until the message is hidden or the file transfer is finished.

The transfer on the data channel can start seconds before the status message with the expected stream length arrives on the command channel, because we're working with two independent sockets. That means, we cannot calculate how much of the secret message fits into the carrier stream or how large the block between two hidden bytes must be. But we have to put the incoming data stream somewhere until the status message arrives.

Is there a parking lot for buffers waiting to be processed ... yes, there's the cache! It has to collect incoming data until (A) we know exactly what to do with it, and (B) there's enough data to call it a block and attach a secret message byte to it. If the method Hide is called before ExpectedStreamSize gets set, it merely collects the buffers. As soon as the SocketSet lets it know the stream size, it starts hiding.

public long ExpectedStreamSize
{
  get { return hide_ExpectedStreamSize; }
  set
  {
    this.hide_ExpectedStreamSize = value;

    if (hide_ExpectedStreamSize > -1)
    {
      // break the transmission into one block per message byte

      if (hide_ExpectedStreamSize < hide_Message.Length)
      { // Message is longer than transmission.
        // send only <buffer.Length> message bytes
        // and wait for the next transmission
        hide_MessagePart = new byte[hide_ExpectedStreamSize];
      }
      else
      { // Message is shorter than transmission.
        // send the whole text
        hide_MessagePart = new byte[hide_Message.Length];
      }

      // calculate size of one block
      this.blockSize = (int)Math.Floor
        ((float)hide_ExpectedStreamSize / hide_MessagePart.Length);

      // copy a part of the full message that can be hidden 
      // in the expected stream
      Array.Copy(hide_Message, 0, hide_MessagePart, 0, hide_MessagePart.Length);
    }
  }
}

public byte[] Hide(byte[] buffer, int size, long remainingStreamSize)
{
  if (hide_ExpectedStreamSize < 0)
  { // Up-/Download size is not known yet.
    // queue the buffer and wait
    foreach (byte b in buffer) hide_waitingBuffer.Add(b);
    return null;
  }

  byte[] returnArray = null;

  // copy buffers to collection
  Collection<byte> transmissionBuffer = new Collection<byte>();
  foreach (byte b in hide_waitingBuffer) transmissionBuffer.Add(b);
  for (int n = 0; n < size; n++) transmissionBuffer.Add(buffer[n]);

  if (transmissionBuffer.Count < hide_blockSize)
  { // The buffer is too small for a block.
    // wait until the socket received more data
    hide_waitingBuffer = transmissionBuffer;
  }
  else
  {
    hide_waitingBuffer.Clear();

    // transmission must be an integer number of blocks
    int countBlocksInCurrentBuffer = 
        (int)Math.Floor((float)transmissionBuffer.Count / hide_blockSize);
    int maxTransmissionSize = countBlocksInCurrentBuffer * hide_blockSize;

    // move the unused rest of the current buffer to the waiting collection
    MoveEndToCollection
        (transmissionBuffer, hide_waitingBuffer, maxTransmissionSize);

    int insertAtIndex = 0;

    if (hide_CurrentMessageIndex == 0)
    { // sending first buffer
      // append <blockSize> and [messageLength] to buffer
      PrependInt32(hide_blockSize, transmissionBuffer);
      PrependInt32(hide_MessagePart.Length, transmissionBuffer);
      insertAtIndex = 8;
    }

    // insert message into carrier
    for(int blockIndex=0; blockIndex<countBlocksInCurrentBuffer; blockIndex++)
    {
      if (hide_CurrentMessageIndex == hide_MessagePart.Length)
      { // end of message
        break;
      }

      transmissionBuffer.Insert
        (insertAtIndex, hide_Message[hide_CurrentMessageIndex]);
      // proceed with next part
      hide_CurrentMessageIndex++;
      // each insertion adds 1 byte, so the next insert position is '+n'
      insertAtIndex += hide_blockSize + 1;
    }

    // return new transmission buffer
    byte[] newBuffer = new byte[transmissionBuffer.Count];
    transmissionBuffer.CopyTo(newBuffer, 0);
    returnArray = newBuffer;
  }
  return returnArray;
}

Step 3b: Divide Two Streams

The message has been hidden - invisible in a binary file, or visible as little typos in a text file. Soon the transmission is going to arrive - buffer by buffer - at the client's socket. We have to loop over the stream in steps of blockSize and, in each iteration, move one byte to the local destination file. Again, we have a problem: When a user uploads a file, it arrives as a sequence of independent buffers which we want to write-through as fast as possible. No more store-and-forward than necessary, please!

We can solve that problem with the same trick we used before: The binary parking lot, also known as waitingBuffer. The Extract method stores incoming buffers until the header is there, then goes on just as the Hide method did.

public byte[] Extract(byte[] buffer, int size, Stream outMessage)
{
  byte[] returnArray = null;

  // copy buffers to collection
  Collection<byte> transmissionBuffer = new Collection<byte>();
  foreach (byte b in extract_waitingBuffer) transmissionBuffer.Add(b);
  for (int n = 0; n < size; n++) transmissionBuffer.Add(buffer[n]);

  if (extract_blockSize < 0 && transmissionBuffer.Count >= 8)
  {  // The first 8 octets arrived.
    // read/remove block and message size
    this.extract_blockSize = RemoveInt32(transmissionBuffer, 7);
    this.extract_RemainingMessageSize = RemoveInt32(transmissionBuffer, 3);
  }

  if (transmissionBuffer.Count < extract_blockSize || extract_blockSize < 0)
  {  // The buffer is too small for a block.
    // wait until the socket received more data
    extract_waitingBuffer = transmissionBuffer;
  }
  else
  {
    extract_waitingBuffer.Clear();

    // transmission must be an integer number of blocks
    int countBlocksInCurrentBuffer = (int)Math.Floor
        ((float)(transmissionBuffer.Count / (extract_blockSize+1)));
    int maxTransmissionSize = 
        countBlocksInCurrentBuffer * (extract_blockSize+1);
    // move the unused rest of the current buffer to the waiting collection
    MoveEndToCollection
        (transmissionBuffer, extract_waitingBuffer, maxTransmissionSize);

    // read and remove message from carrier
    int transmissionBufferIndex = 0;
    while (extract_RemainingMessageSize > 0 && 
        transmissionBufferIndex < transmissionBuffer.Count)
    {
      outMessage.WriteByte(transmissionBuffer[transmissionBufferIndex]);
      transmissionBuffer.RemoveAt(transmissionBufferIndex);
      transmissionBufferIndex += extract_blockSize;
      extract_RemainingMessageSize--;
    }

    // return new transmission buffer
    byte[] newBuffer = new byte[transmissionBuffer.Count];
    transmissionBuffer.CopyTo(newBuffer, 0);
    returnArray = newBuffer;
  }
  return returnArray;
}

Step 4: The Client Application

The .NET Framework contains a very simple FTP client: FtpWebRequest. It supports neither session control nor command/data channels to different IP addresses, but for our demo client it should be okay. Here you can see a little GUI for FtpWebRequest with steganographic extra-features.

The picture shows a SteganoFTP client that has just connected to the proxy specified in the upper group box. The remote directory listing was sent via data channel, so the proxy embedded a text from its message file into it. The application displays the message below the cleaned information (which is the listing).

The user can now select a file and enter a secret message in the boxes on the left. The upload button embeds the text message in the selected file and sends both to the proxy. That triggers (A) an upload to the FTP server the proxy is connected to, and (B) an upload into the proxy server's secret file.