How Proxy Server serves FTP clients?

• Download source files - 55.4 Kb
• Download demo project - 159 Kb

• Introduction
• Background
  • User connect/authentication
  • PASV and PORT commands handling
• Proxy Server Role
• Points of Interest
• References
• Thanks to...

Introduction

In this article I will talk about connecting FTP clients through Proxy Server. So, reader of this article should have a background about the following topics:

1. TCP/IP protocol and Socket programming.
2. File Transfer Protocol (FTP) - RFC 959.
3. Proxy Server functionality - RFC 1919.

Actually FTP clients can connect to FTP servers directly and can transmit and receive files or data directly through direct sockets connections, but in some cases security to FTP clients is needed and most of the internet clients access the internet from LAN network that is connected to the internet through Firewalls that support NAT (Network Address Translation) or full LAN isolation through proxy server.

Background

FTP servers is listening for clients requests on port 21, so any client wants to deal with any server will just connect to the server and send it identification data to authenticate itself then ask for its request, all that in a TCP connection as in figure 1.

The connection is kept opened between the server and client to exchange FTP commands between them. If the client wants to get data, it can use two ways to request it:

1. The Client specifies a non-default client side data port with the PORT command.

   client:  PORT h1,h2,h3,h4,p1,p2
   server:  200 Command okay.        
   // where h1 is the high order 8 bits of the internet host address.

   Then the server sends any data through connecting to this host address and send data, then close the connection.
2. The Client requests the server side to identify a non-default server side data port with the PASV command.

   client:  PASV
   server:  227 Entering Passive Mode (h1,h2,h3,h4,p1,p2).        
   // where h1 is the high order 8 bits of the internet host address.

   Then the client connects this host address and receives its reply for data request at this channel, then the server closes the connection.

Proxy Server role between FTP client and FTP Server

User connect/authentication

First, I will describe user connect/authentication process and how various proxies implement requests differently. When FTP client connects to the FTP servers, it just resolves the FTP server address and connects to port 21 (default FTP port) and the FTP server responses with a welcome message, telling the FTP client that it is ready to receive user authentication information, but what is the case with proxy server? How can the FTP client tell the proxy server, the address of the FTP server that it wants to connect? The answer is, it can be done by various ways, as you can see from the following figure, that is taken from the firewall settings of the CuteFTP client:

We will take each one alone:

1. General: will not use the proxy and connect directly.
2. SITE site:

   client: // connect to proxy server
   proxy:  220 FTP Virtual Server
   client: USER
   proxy:  331 send password.
   client: PASS
   proxy:  230 Login OK. Proceed.
   client: site ftp.cuteftp.com
   proxy:  // connect to ftp.cuteftp.com
   server: 220 GlobalSCAPE Secure FTP Server (v. 2.0)
   proxy:  USER anonymous
   server: 331 Password required for anonymous.
   proxy:  PASS ********
   server: 230 Login OK. Proceed.

   As you can see, the client sends authentication information first to the proxy which keeps it tell, it receives the FTP server name at the SITE command (e.g. site ftp.cuteftp.com), then the proxy connects the FTP server and resends the login information to the FTP server.

3. USER user@site:

   client: // connect to proxy server
   proxy:  220 FTP Virtual Server
   client: USER anonymous@ftp.cuteftp.com
   proxy:  // connect to ftp.cuteftp.com
   server: 220 GlobalSCAPE Secure FTP Server (v. 2.0)
   proxy:  USER anonymous
   server: 331 Password required for anonymous.
   client: PASS ********
   server: 230 Login OK. Proceed.

   As you can see, the client sends user name and FTP server at the USER command (e.g. USER anonymous@ftp.cuteftp.com), then the proxy connects the FTP server and sends the user name to the FTP server and dialog continues between the client and the server.

4. USER with login:

   client: // connect to proxy server
   proxy:  220 FTP Virtual Server
   client: USER
   proxy:  331 send password.
   client: PASS ********
   proxy:  230 Login OK. Proceed.
   client: USER anonymous@ftp.cuteftp.com
   proxy:  // connect to ftp.cuteftp.com
   server: 220 GlobalSCAPE Secure FTP Server (v. 2.0)
   proxy:  USER anonymous
   server: 331 Password required for anonymous.
   proxy:  PASS ********
   server: 230 Login OK. Proceed.

   As you can see, the client sends authentication information first to the proxy which keeps it tell, it receives the FTP server name at the USER command (e.g. USER anonymous@ftp.cuteftp.com), then the proxy connects the FTP server and resends the login information to the FTP server.

5. USER/PASS/ACCT:

   client: // connect to proxy server
   proxy:  220 FTP Virtual Server
   client: USER
   proxy:  331 send password.
   client: PASS
   proxy:  230 Login OK. Proceed.
   client: open ftp.cuteftp.com
   proxy:  // connect to ftp.cuteftp.com
   server: 220 GlobalSCAPE Secure FTP Server (v. 2.0)
   proxy:  USER anonymous
   server: 331 Password required for anonymous.
   proxy:  PASS ********
   server: 230 Login OK. Proceed.

   As you can see, the client sends authentication information first to the proxy which keeps it tell, it receives the FTP server name at the open command (e.g. open ftp.cuteftp.com), then the proxy connects the FTP server and resends the login information to the FTP server.

6. OPEN site:

   client: // connect to proxy server
   proxy:  220 FTP Virtual Server
   client: USER anonymous@ftp.cuteftp.com
   proxy:  // connect to ftp.cuteftp.com
   server: 220 GlobalSCAPE Secure FTP Server (v. 2.0)
   proxy:  USER anonymous
   server: 331 Password required for anonymous.
   proxy:  PASS ********
   server: 230 Login OK. Proceed.

   As you can see, the client sends authentication information first to the proxy which keeps it tell, it receives the FTP server name at the open command (e.g. open ftp.cuteftp.com) or the USER command (e.g. USER anonymous@ftp.cuteftp.com), then the proxy connects the FTP server and resends the login information to the FTP server.

You can check each case with your FTP client alone, and may be your FTP client has different settings, and you can have a look to the code that handles all of these cases:

send(SocketClient, "220 FTP Virtual Server\r\n", 24, 0);
pRequest->nPort = 21;
CString strPass, strWelcome;
bool bSendWelcome = true;
while(true)
{
    pRequest->nLength = recv(SocketClient, str.GetBuffer(1024), 1024, 0);
    str.ReleaseBuffer(pRequest->nLength);
    if(memicmp(str, "USER", 4) == 0)
    {
        if((nIndex = str.Find("@")) != -1)
        {
            bSendWelcome = false;
            pRequest->strHost = str.Mid(nIndex+1);
            pRequest->strHost.TrimRight();
            HostPort(pRequest->strHost, pRequest->nPort);
            pRequest->strRequest = str.Left(nIndex)+"\r\n";
            break;
        }
        else
        {
            bSendWelcome = true;
            pRequest->strRequest = str;
            str.TrimRight();
            if(str.GetLength() == 4)
                pRequest->strRequest.Insert(5, "anonymous");
            // if host still not found then ask for login password
            send(SocketClient, "331 send password.\r\n", 20, 0);
        }
    }
    else    if(memicmp(str, "SITE", 4) == 0 || memicmp(str, "OPEN", 4) == 0)
    {
        if((nIndex = str.Find(" ")) == -1)
            goto END_REQUEST;
        pRequest->strHost = str.Mid(nIndex+1);
        pRequest->strHost.TrimRight();
        HostPort(pRequest->strHost, pRequest->nPort);
        break;
    }
    else    if(memicmp(str, "PASS", 4) == 0)
    {
        strPass = str;
        send(SocketClient, "230 Login OK. Proceed.\r\n", 24, 0);
    }
    else    if(pRequest->strHost != "")
        break;
}

// connect to ftp server
if(ConnectHost(SocketHost, pRequest) == false)
    goto END_REQUEST;
if(pRequest->strRequest.IsEmpty() == false)
{
    // Receive ready command
    nIndex = RequestRecv(pRequest, SocketHost, 
                         strWelcome.GetBuffer(1024), 1024);
    strWelcome.ReleaseBuffer(nIndex);
    // send USER command
    RequestSend(pRequest, SocketHost,
                 pRequest->strRequest.GetBuffer(0), 
                 pRequest->strRequest.GetLength());
    if(strPass.IsEmpty() == false)
    {
        // Receive reply of USER command
        nIndex = RequestRecv(pRequest, SocketHost,
                          str.GetBuffer(1024), 1024);
        str.ReleaseBuffer(nIndex);
        // send password
        RequestSend(pRequest, SocketHost, 
                 strPass.GetBuffer(0), strPass.GetLength());
        if(bSendWelcome)
        {
            // Receive reply of PASS command
            RequestRecv(pRequest, SocketHost, 
                                str.GetBuffer(1024), 1024);
            // send welcome message to client
            send(SocketClient, strWelcome.GetBuffer(0),
                                  strWelcome.GetLength(), 0);
        }
    }
}

PASV and PORT commands handling

Any proxy server should have at least one external address (WAN address) and one internal address (LAN address) as in figure 2. If the client is connected through Proxy Server, then it has a LAN address (internal address), that means it can't connect directly to the internet. If the FTP client sends a PORT command, it will specify its LAN address or another LAN address to receive server connection, but the server can't by-path the proxy server and access the LAN addresses, the same problem with the PASV case as the LAN client needs to connect to external address at the server side to open the data channel, but it can't by-path the proxy server too. Here is the proxy server role of replacing these addresses like the NAT way, which I will describe in the two cases:

1. In the PORT command case:
   • The client sends PORT command with the LAN address and a port.
   • The proxy server detects the PORT command and changes the LAN address by its WAN address and directs the command to the FTP server.
   • The proxy server creates a listen socket at the received port and creates two threads:
     • one to receive from the FTP server and send to the FTP client.
     • and the second to receive from the FTP client and send to the FTP server.
   • The FTP server receives the PORT command and connects the command address (proxy server WAN address) and sends its data packets.
   • The proxy server receives data and transmits it to the FTP client.

   This scenario is repeated each time the client fires a PORT command as in figure 2.

   

2. In the PASV command case:
   • The client sends PASV command.
   • The proxy server sends the PASV command to the FTP server.
   • The proxy server detects the reply and changes the FTP server WAN address with its LAN address and directs the reply to the FTP client.
   • The proxy server creates a listen socket at the received port and creates two threads:
     • one to receive from the FTP server and send to the FTP client.
     • and the second to receive from the FTP client and send to the FTP server.
   • The FTP client receives the reply and connects reply address (proxy LAN address) (channel 1 between FTP client and proxy).
   • The proxy server connects to the listen socket created at the FTP server (channel 2 between proxy and FTP server).
   • The FTP server sends data through channel 2 with the proxy server which sends it to the FTP client through channel 1.

   This scenario is repeated each time the client fires a PASV command as in figure 3.

   

Points of Interest

1. This demo is a part of a large project of a proxy server that supports:
   • Internet access and offers Access Control for all LAN users.
   • Automatic Site filtering depending on pages contents, and user defined rules.
   • Protocols: HTTP, FTP, SOCKS4, SOCKS5, SMTP, POP3, DNS.
   • Full requests log files and users access visual log.
   I just included the section of code that handles the FTP process.
2. I didn't describe the attached code with this article as you don't have to reuse it as I just wanted to describe the idea only, if any one needs more details he can contact me though mail, or send the part of code that needs description.
3. If you want to try this demo you should adjust your FTP client to use proxy server, and adjust proxy address and port (21) and don't forget to check the checkbox of "PASV mode".
4. Don't try to test this demo with FTP client with PORT command in the same machine, as the FTP client will create a listen socket at the command address (address sent with the command), and the proxy will replace the command address with its machine address which is the same address, and fails to listen to the same port. So, you can check the PASV command only.
5. If you want to read about Proxy Servers you can check RFC 1919.

References

1. RFC 959 File Transfer Protocol. J. Postel, J.K. Reynolds. Oct-01-1985.
2. RFC 1579 Firewall-Friendly FTP. S. Bellovin. February 1994.

Thanks to...

1. I owe a lot to my colleagues for helping me in implementing and testing this work. (JAK)