Introduction
WARNING: This code is designed to manipulate files on a remote system. Be sure you know what you do before you use it!
---
Out of pure curiosity I was playing a bit with my AVM Fritz!Box at home looking around, searching what you could do with it - besides letting it be a router of course. You can activate a Telnet server by calling #96*7* from a phone connected to the Fritz!Box (#96*8* to deactivate). So, I started looking around on the file system and found a lot LUA scripts which I wanted to have a closer look at. But simply using cat and having to scroll through the lines in a PuTTY console wasn't quite comfortable so I was looking for a way to copy those files to my computer because Notepad++ supports syntax highlighting for LUA scripts and it's much easier to handle different files simultaneously. Also navigation through the directory tree with cd and ls didn't really satisfy me as a Windows user. ;b
I started a Google search for a Telnet File Explorer but couldn't find one - ok, to be honest I didn't search really deeply because I like this kind of challenge. So, I began to write such an explorer which in this project is specialized to be used with a Fritz!Box but should work on any Telnet server with minor changes.
---
Background
The remote side: Basic Linux Commands
The AVM Fritz!Box uses a *nix like BusyBox system where many standard commands are available (See this German page for a complete list). I'm not used to use *nix like systems as much as I would like to, so let's have a look at the commands I'm using on the remote Telnet machine - if you're familiar with this, be welcome to skip it:
Command 'LS'
ls [-1AacCdeFilnpLRrSsTtuvwxXhk] [{filenames}]
While in Windows you use dir in *nix systems you always use ls. There are a lot command line switches but I'll only explain those I will be using:
| Argument | Description |
| |
-a | Show hidden entries (starting with a dot '.') |
| |
-l | Show a detailed list with a one entry per line |
| |
-e | Show a complete TimeStamp with weekday, seconds and year |
| |
-R | Recursively list all sub-directories |
| |
{filenames} | A start directory and/or a filter for the listing |
| |
To get all information I want, I use all of these arguments like this:
ls -a -l -e -R /
This would result in a output like this:
./:
drwxr-x--- 0 root root 271 Fri Feb 7 17:17:59 2014 mount
drwxr-x--- 3 root root 154 Fri Feb 7 17:17:59 2014 home
-rwxrwx--- 0 root root 198271 Fri Feb 7 17:17:59 2014 somefile.txt
./mount:
./home:
drwxr-x--- 1 root root 154 Fri Feb 7 17:17:59 2014 mysubfolder
-rwxr-x--- 0 root root 78651 Fri Feb 7 17:17:59 2014 readme.txt
lrwxr-x--- 0 root root 0 Fri Feb 7 17:17:59 2014 link -> ./somefile.txt
./home/mysubfolder:
-r-xr-x--- 2 root root 2328156 Fri Feb 7 17:17:59 2014 anotherfile.dat
Note that I only found these dots '.' before the path descriptions (those stopping with a colon ':') when I used the root path as a start directory - otherwise they started with a slash '/'. The information listed here for each file or folder is this:
| Part | Description |
| |
1<sup>st</sup> char of 1<sup>st</sup> block | It's ether a 'd' for directory, 'l' for link or '-' for a normal file. |
| |
rest of 1<sup>st</sup> block | Permissions: 3 times 'rwx' for 'r'=read, 'w'=write and 'x'=execute, if not allowed, there is a dash '-' at that position. The first 'rwx' is for root users, the second for the current group and the third for all users. |
| |
2<sup>nd</sup> block | Number of sub-entries |
| |
3<sup>rd</sup> block | Owner |
| |
4<sup>th</sup> block | Group |
| |
5<sup>th</sup> block | Size in bytes |
| |
6<sup>th</sup> to 10<sup>th</sup> block | Modification date and time |
| |
11<sup>th</sup> block | File or directory name |
| |
12<sup>th</sup> and 13<sup>th</sup> block | If exists, the link target |
| |
On the client side I'll simply grab a snapshot of the whole tree at start-up. This takes some time but is much less prone to errors than dynamically loading single parts afterwards.
Command 'CAT'
cat {filename}
Simply outputs the content of a file to the console. Although this method if prone to changing the output a bit, it is the only way to get a files content without an addition connection on a Fritz!Box because there is no hexdump command or an equivalent to convert non-printable chars. Also a line-break will always be converted to the current new-line standard which in my case is <CR><LF> even if the original file only uses <LF>. For text files this doesn't matter that much but it is impossible to load a binary file that way.
Command 'NC'
nc [{addr} {port}|-l -p {port}] [<|>] {filename}
'NC' means netcat, so it's cat over the net. This is not the complete functionality on netcat but what I intend to use. It uses a separate connection to send or receive data while you can set files as input or output. Here's what the command line switches mean:
| Argument | Description |
| |
{addr} {port} | Defines the target address and port to connect to |
| |
-l -p {port} | Starts NC as a listener waiting for a connection on the port |
| |
< {filename} | Defines a input file which content is to be sent as soon as a connection is established |
| |
> {filename} | Defines a output file to write received data to as soon as a connection is established |
| |
In general there are two ways of exchanging files:
- client->server: The server opens a port and the client connects to it
- client<-server: The client opens a port and the server connects to it
It doesn't matter if there is a file to be sent or received - what matters is the environment of the two systems. Depending on which side can open a reachable port easier that one should be the host opening the port. In either way files can be transferred in both directions.
Here two examples for copying files from the server to the client (first the server, then the client listening):
nc -l -p 4322 < /home/afile.txt
nc 192.168.178.20 4321 < /home/afile.txt
and two analogical examples for copying files from the client to the server:
nc -l -p 4322 > /home/newfile.txt
nc 192.168.178.20 4321 > /home/newfile.txt
Command 'ECHO'
This command is usually used to output text lines on the screen within scripts but if redirected it can be used to create text files. The syntax with redirection is:
echo -n $'{some text}' [>|>>] {filename}
The parameter -n prevents the echo command to append a tailing newline.
The file data has to be escaped though in some cases. I'll be using strict quoting - the text will be enclosed like $'{sometext}' and I will escape most chars for compatibility:
- A zero character (0x00) cannot be written because it terminates the string.
- All chars with a value below 0x20 are control char and will be escaped with octal escaping format:
0xnnn where nnn is the three digit octal value of the char. - All chars with a value above 0x7F are extended ASCII chars and will also be escaped with octal escaping format as above.
- All 'single quote' chars (0x27) obviously have to be escaped because they would otherwise end the quotation, so I'll use
\' as direct escaping. - The backslash char (0x5C) will be escaped directly with
\\. - All other chars will be transferred as they are.
This way, I do not have to send a file line by line but can split it in parts. Although most *nix machines do not have a quickly reached command-line length limitation of 8191 chars (resp. 2047 on older versions) like Windows, it seems a good idea to me though not to send everything in one command but to chunk it in about 1000 char segments. This is an example:
echo -n $'<html>\012<head><title>\\\'</title></head>\012<body></body>\012</html>' > n.txt
It will create a file n.txt containing four lines:
<html>
<head><title>\'</title></head>
<body></body>
</html>
This method should (like cat) only be used for text files of course because there might be some other chars besides 0x00 that cannot be written as intended.
Command 'MKDIR'
This command creates a new directory. If used with the command-line option -p all non-existing parent directories will be created too. Here is an example:
mkdir -p /var/tmp/newfolder/andsubfolder
Command 'RM'
This command can remove (delete) a file or directory tree. If used with the the command-line option -f there will be no confirmation prompt; if -r is used a directory and all its content is deleted recursively. Here are two examples - first deleting a file, then deleting a directory tree:
rm -f /usr/file.tmp
rm -f -r /var/tmp/my_tmp_folder
---
General code structure
I am a bit more comfortable with the client side because C#.NET is something I am familiar with. First I will explain the construct of my code. Here is how the classes are connected:
- The class
TelnetFileInfo is simple a data class which will store Information about a file. - The class
TelnetDirectoryInfo extends the TelnetFileInfo class, adds a List of TelnetDirectoryInfo as SubDirectories and a List of TelnetFileInfo as Files and a method to load a complete file tree with a specified start directory. This class will store a directory/file tree. To load a complete tree you will have to pass a TelnetConnection instance - that is, why it's marked 'uses TelnetConnection'. - The class
TelnetConnection is the heart of this project as it handles most of the logic concerning the telnet communication. For Sending and receiving files using netcat it uses the class TcpConnectionHelper to handle a connecting or listening second connection. - The static class
TcpConnectionHelper is a helper to create connecting or listening connections and do automated data transfer as soon as connected.
General work-flow
I designed a simple Form with a TreeView on the left and a ListView an the right side. Below the ListView I added some controls to initiate uploads and downloads. In Addition I designed a simple Window to show status information because the download takes some time so it is running in a Thread and the user gets to see that something is done.
The TreeView will show the Directory tree after it is loaded. There are three different types of entries which have different icons and colors representing them:
- Directories are displayed in red, enclosed in brackets and have a folder icon.
- Files are displayed black with a common document icon.
- Links are displayed blue with an empty sheets icon having a green arrow at it. Note that links can be targeted at files as well as at directories which is not differenced.
The ListView will simply display the information of the currently selected entry.
The 'Download Method' DropDownMenu (which I will rename to 'Transfer Method') sets one of the following three methods to upload or download files (or directories):
- NETCAT client >> server: A second connection is used to connect to the server on the specified port using NETCAT on the server to receive or send a file.
- NETCAT client << server: A second connection is used listening on the specified port until the server connects to it and sends or receives a file.
- CAT/ECHO (text files only!!!): The existing connection is used to send or receive a file via console input or output. This is (like explained above) NOT suitable for binary files but in case the environment does neither allow the client nor the server to open another listening port this method can be used as a fall-back to at least get text files transferred.
Using the application you should do the following steps:
- Enter your password in the Password TextBox.
- Press [Enter] or click the connect button and wait until the directory tree is loaded.
- To Download: Select a file or directory, enter a suitable port in the '* port:' TextBox and click the 'Download' button, select a target, click the 'OK' button and wait until the operation is complete.
- To Upload: Select a directory, enter a suitable port in the '* port:' TextBox and click the 'Upload' button, select a file to upload, click the 'OK' button and wait until the operation is complete. For now it is only possible to upload files into existing directories.
Remember, that there is little or no error handling or input check so be careful with what you do!
---
Using the code
I will explain the use of the code from the bottom up.
TcpConnectionHelper
The methods ConnectToClient and WaitForConnection are internally used. I only made them public to have a possibility to use customized actions. There are description on how to use inside the code.
All SendData* and ReceiveData* methods are using a similar pattern as they are only different constellations and wrappers for each other. They are also well documented in the code. These methods are designed to work as an end-point for netcat started on the remote system. Because netcat does not automatically terminate the connection (unless you set a timeout which I did not try) I designed these methods to end the connection after a constant timeout without data transferred. All Methods ending with *Connecting use the ConnectToClient method internally and are connecting actively to the specified address and port from the arguments. All methods ending with *Listening use the WaitForConnection method internally and are waiting for the remote to connect to the specified local port from the arguments. In each case you can use either a byte array, a file path or a Stream to transfer data. All methods using byte array or file path as arguments (or return value) are simply wrappers for the corresponded method using a Stream. The following example shows how to transfer an image file from the first computer computer to a second which displays it directly in a PictureBox:
void FirstComputer()
{
string fileToSend = @"C:\someImage.jpg";
string remoteAddress = "192.168.178.21";
int remotePort = 5432;
TcpConnectionHelper.SendFileConnecting(remoteAddress, remotePort, fileToSend);
}
void SecondComputer()
{
int localPort = 5432;
byte[] fileData = TcpConnectionHelper.ReceiveDataListening(localPort);
Bitmap image = new Bitmap(new MemoryStream(fileData));
pictureBox1.Image = image;
}
---
TelnetConnection
This class represents a connection to a telnet server. You can use several connection simultaneously. You have to set the server address, port, username and password in the constructor. Constructors which do not have this parameters are using standard values: The username is left empty and the port is set to the standard telnet port 23.
A connection is started when the Connect() method is called. This starts a Thread which connects to the remote server. If a 'password:' or 'login:' prompt is received, is enteres the stored password from the constructor, if a 'username:' prompt is received (does not apply to Fritz!Box telnet login), it enters the stored username from the constructor. The Connect() method is also called internally by public methods to check if a connection is established or to connect if not. The connection handling uses a state machine architecture internally. The thread periodically looks for data to receive and then for data to be send. The class has three private Actions which are used to handle received data:
Action<string> CurrentLineHandler: The variable _conn_PushAllData must be false for this Action to be called. Whenever the connection thread encounters a line-break, this Action is called with the captured line.Action<IEnumerable<byte>> CurrentRawDataHandler: The variable _conn_PushAllData must be true for this Action to be called. Whenever any data is received this Action is called with the data received.Action CurrentTimeoutHandler: If _conn_PushAllData is false this is called when there is data in the buffer but no line-break and no data is received within the timeout, if true it gets called periodically when no data has been received within the timeout.
You can set dummies and reset _conn_PushAllData to false by calling the ResetConnHandlers() method. Everything received while the dummies are set is ignored.
To send data the AddSendData(...) method should be called either passing a byte array or a string. A string will be converted to a byte array using ASCII encoding. The data is store in a thread-safe way into a buffer and sent to the server by the connection thread.
The SendCommand(string cmd, int timeout) method queues the passed cmd data to be sent and then waits until no new line is received within the passed timeout in milliseconds to return a string containing all received lines. The terminal usually sends the cmd data back to the client before sending the result - this data is filtered out.
The GetRecursiveFileList(string rootPath, Action<string> onNewLine) method sends a LS command to the server and then calls the onNewLine Action every time a new line was received. The LoadCompleteTree(...) method in the TelnetDirectoryInfo class uses this method and parses the lines.
There are three methods each for downloading (DownloadFile*) and uploading (UplaodFile*) files. Path definitions are always meant to be full paths not relative ones.
- The ending
*Active is referring to the client connecting actively to the server using a second connection (the TcpConnectionHelper.[Send|Receive]DataConnecting methods are used) after sending a NC command to the telnet server. The required parameters are localPath, remotePath and remotePort. Note that depending an uploading or downloading the path parameters are in different order. - The ending
*Passive is refering to the client listening for a second connection initiate by the telnet server (the TcpConnectionHelper.[Send|Receive]DataListening methods are used) after sending a NC command to the telnet server. The required parameters are localPath, remotePath, localAddr and localPort. Note that depending an uploading or downloading the path parameters are in different order. If the localAddr parameter is empty the methods queries all local IPs and picks the first to use for the NC command. - The method
DownloadFileCat(...) calls a CAT command on the telnet server and stores the output into a file using only the existing connection. The required parameters are remotePath and localPath. - The method
UplaodFileEcho(...) method splits up a file in about 1000 byte sized chunks and wraps them up in echo commands executed on the telnet server using only the existing connection and writing a file by redirecting the command's output into it. The required parameters are localPath and remotePath.
The method Delete(...) deletes a file or directory tree using a RM command. The required parameters are remotePath and isDirectory.
The method CreateDirectory(...) creates a directory - if necessary with parent folders. The required parameter is remotePath.
Here is an example which uploads a file to the remote server using the echo command then downloads (and deletes) it using netcat listening on the server side. It then checks if both files are identical:
public void CheckUploadIdentical()
{
string baseFile = @"C:\myImage.jpg";
string remotePath = "/tmp/img_temp"
string remoteFile = remotePath + "/myImage.jpg";
string checkFile = @"C:\myImage_check.jpg";
TelnetConnection conn = new TelnetConnection("fritz.box", "MyPassword123");
conn.Connect();
conn.CreateDirectory(remotePath);
conn.UploadFileEcho(baseFile, remoteFile);
conn.DownloadFileConnecting(remoteFile, checkFile, 5432);
conn.Delete(remoteFile, false);
conn.Delete(remotePath, true);
bool result = true;
using (FileStream s1 = new FileStream(baseFile, FileMode.Open, FileAccess.Read))
using (FileStream s2 = new FileStream(checkFile, FileMode.Open, FileAccess.Read))
{
while (true)
{
int b1 = s1.ReadByte();
int b2 = s2.ReadByte();
if (b1 == -1 && b2 == -1)
break;
if (b1 != b2)
{
result = false;
break;
}
}
}
if (result)
MessageBox.Show("The files are identical.");
else
MessageBox.Show("The files are different.");
}
---
TelnetDirectoryInfo
This class is used to reflect the file system. You can call the static LoadCompleteTree(...) method to load a directory tree from the telnet server represented by the passed TelnetConnection conn. The passed parameter rootPath defines which directory is used as a start directory - the standard is /. The result will be always be the root directory of the server but only the passed rootPath will be filled with data. The third parameter is an Action<string> onNewPath that is called every time a new directory description started in the data received from the server. It is intended (and in that way used by the application) to give a feedback to the user that there is still work going on when he has to wait. The following example loads the complete directory tree and counts how many files were found (not counting links):
public TelnetDirectoryInfo LoadFiles()
{
TelnetConnection conn = new TelnetConnection("fritz.box", "MyPassword123");
conn.Connect();
TelnetDirectoryInfo root = TelnetDirectoryInfo.LoadCompleteTree(conn, "/", path =>
{
this.Invoke(new Action(() => label1.Text = path));
});
this.Invoke(new Action(() => label1.Text = CountFiles(root).ToString()));
return root;
}
private int CountFiles(TelnetDirectoryInfo currRoot)
{
int count = 0;
count += currRoot.Files.Count(f => !f.IsLink);
currRoot.SubDirectories.ForEach(s => count += CountFiles(s));
return count;
}
This second example extracts all paths of files which have a given ending from a TelnetDirectoryInfo recursively - again ignoring links:
public List<string> GetAllFilesEndingWith(TelnetDirectoryInfo root, string ending)
{
List<string> result;
result = root.Files.FindAll(f => !f.IsLink && f.Name.EndsWith(ending))
.Select(f => f.Path + "/" + f.Name).ToList();
root.SubDirectories.ForEach(s => result.AddRange(GetAllFilesEndingWith(s, ending)));
return result;
}
History
- 2014-09-10: Included username in GUI, updated source and binary
