Using C and an embedded web-server to provide browser based source editing

• Download source code - 2.63 MB

Introduction

I have always thought it would be neat to be able to navigate and edit source files on a remote system through a web browser. This is not to say that there are no tools in existence today for remotely accessing file systems and files. There may even be some tools that provide similar functionality for web browsers. Nevertheless, this seemed like an interesting challenge.

My main requirement for the solution was to use as little CSS and JavaScript as possible; my skill levels in these areas is limited. I also wanted the application to be in pure C. In the end, I came up with a solution that leveraged the functionality of an already existing embedded web-server library, Snorkel. Specifically, I used the Snorkel SDK to write a light-weight web-server that exposes remote file systems through an HTTP interface.

Snorkel Developer's Guide

• Download Snorkel Developers Guide (PDF) - 1.94 MB

Using the Code

The diagram above illustrates the project's design. The solution uses two components: a server component for accepting incoming requests, and a plug-in for processing requests for viewing and editing source files.

We begin with the server component.

17    void
18    main (int argc, char *argv[])
19    {
20      int i = 1;
21      int http_port = 0;
22      int https_port = 0;
23      char *pszIndex = 0;
.
.
27      snorkel_obj_t http = 0;
.
.
.
114      /*
115       *
116       * always call first to initialize
117       * snorkel API
118       *
119       */
120      if (snorkel_init () != SNORKEL_SUCCESS)
121        {
122          perror ("could not initialize snorkel\n");
123          exit (1);
124        }
125    
126      /*
127       *
128       * create a server object
129       *
130       */
131      http = snorkel_obj_create (snorkel_obj_server,
132                                 2, pszIndex);
133      if (!http)
134        {
135          perror ("could not create server object!\n");
136          exit (1);
137        }
138    
.
.
.
146      if(snorkel_obj_set(http,snorkel_attrib_bubbles,NULL)
147          != SNORKEL_SUCCESS)
148        {
149          fprintf (stderr,
150                  "error encountered setting bubbles!");
151          exit (1);
152        }
153    
154      snorkel_obj_set (http, snorkel_attrib_show_dir, 1);  
156      /*
157       * add listeners
158       *
159       */
160      if (http_port)                /* http port */
161        {
162          if (snorkel_obj_set (http,
163                         snorkel_attrib_listener,
164                         http_port, 0) != SNORKEL_SUCCESS)
165            {
166              fprintf (stderr,
167             "error could not add listener for port %d\n",
168               http_port);
169              exit (1);
170            }
171        }
.
.
.
230      fprintf (stderr,
231             "\n\n[HTTPS] starting embedded server...\n");
232    
233      /*
234       *
235       * start server, no more editing or creation
236       * of objects allowed after this point
237       *
238       */
239      if (snorkel_obj_start (http) != SNORKEL_SUCCESS)
240        {
241          perror ("could not start server\n");
242          snorkel_obj_destroy (http);
243          if (logobj)
244            snorkel_obj_destroy (logobj);
245          exit (1);
246        }
. 
. 
.
256      fprintf (stderr,
257     "\n[HTTP] started.\n\n--hit enter to terminate--\n");
258      fgets (szExit, sizeof (szExit), stdin);
.    
.
.
279    }    

In main, we initialize the Snorkel API by making a call to snorkel_init. Snorkel_init must be called before calling any Snorkel API. Next, we create an HTTP object by invoking the snorkel_obj_create function. The snorkel_obj_create function provides object creation for all Snorkel objects. Unlike C++ and Java, Snorkel objects are not derived from classes. They are more like Windows handles, encapsulations of related data obfuscated by a void pointer. The function takes three parameters: the object type (snorkel_obj_server), the number of threads to create (two), and a pointer to a null terminated string containing the fully qualified path to the directory that we want to export.

On line 146, we identify the plug-in directory. By passing a NULL pointer for the third argument, we instruct the API to use the default plug-in directory: (current directory)/bubbles.

Plug-ins, known as bubbles in the Snorkel API, are self-contained runtime components that export functions with URI mappings for processing HTTP requests. In this example, server logic for processing HTTP-GETs and HTTP-POSTs related to file editing and viewing reside in a plug-in (bubble). Placing the functionality into a bubble allows for reuse by other Snorkel based embedded servers.

As with any web-server, an index file is required to resolve the initial HTTP request sent by a browser. By default, an error occurs if the index directory, the directory provided on line 131, does not contain an index file (index.html). The Snorkel API provides functionality for directory navigation through a browser, if an index file is not present; however, by default, it is disabled. On line 154, we enable browser based directory navigation by calling snorkel_obj_set with the server attribute snorkel_attrib_show_dir. Since we are not providing the server with the location of an index file, it will display the directory listing within the browser instead of producing the default error, page not found.

The directory listing provided by the Snorkel runtime provides file navigation from the root directory (the directory identified on line131) and all of its sub-directories. Directory listings contain links for both files and sub-directories. Names delimited by a beginning and ending bracket denote directory links. The listing headers (Name, Size, and Date Modified) are also links; selecting them sorts the directory listing based on the header type. For example, selecting the Name header sorts the listing based on the file name.

Before we can issue the instruction to start the server, we need to define a listener. Snorkel listeners are objects that process TCP/IP-based protocol requests over user defined ports. A Snorkel server object can support multiple listeners. Listeners can listen for and process requests from HTTP, HTTPS, or proprietary protocol based clients. On lines 160-170, we define a listener assigned to the port http_port, a value obtained from a command line option.

Finally, on line 239, we issue a call to snorkel_obj_start to start the embedded web-server. The snorkel_obj_start API starts the server as a separate thread and returns control back to the calling routine. To prevent the application from exiting, we use the fgets command to wait for input from the end-user to determine when to exit. The entire listing for the program can be found in the "c" directory of the attached bundle in the file file_server.c.

After defining the server component, we next define the plug-in - our Snorkel bubble.

After loading a plug-in into memory, the Snorkel runtime checks for the function bubble_main. If the function exists, the runtime calls the function passing it the associated server object, the object we created in the server's main. In bubble_main, we issue calls to the API snorkel_obj_set with the attribute snorkel_attrib_mime to associate each file type with a content type and a callback function (view_uri). We also associate any URI containing the ending string "update.html" with the function save_file. When the embedded server encounters a request for an associated file type, that is, a file type associated with a mapped function, it calls the function, passing an HTTP request object, a connection object, and a the URL associated with the requested URI. In this solution, the runtime calls the function view_uri for any file containing a matching-mapped file type.

742    byte_t SNORKEL_EXPORT
743    bubble_main (snorkel_obj_t server)
744    {
745      int i = 0;
746    
747      snorkel_obj_set (server, snorkel_attrib_mime, "c",
748                       "text/html", 
749                       encodingtype_text,
750                       view_uri);
751      snorkel_obj_set (server, snorkel_attrib_mime, "cpp",
752                       "text/html", 
753                       encodingtype_text,
754                       view_uri);
755      snorkel_obj_set (server, snorkel_attrib_mime, "h";, 
756                       "text/html", 
757                       encodingtype_text,
758                       view_uri);
.  
.  
.
815      snorkel_obj_set (server,
816                       snorkel_attrib_uri, POST,
817                       "*update.html", encodingtype_text,
818                       save_file);
.
.
.
831      return 1;
832    }

In the function view_uri, we test for the presence of a query-string containing either edit or download. We use the query-string to determine how an end-user wishes to process a mapped file type. A query-string is any string appended to a URI proceeded by a '?'. For example, if the HTTP request contained the URI "http://localhost/c/source.c?edit", the query string is "edit". The Snorkel runtime treats query-strings as a header element, and stores them in the HTTP-header variable "QUERY". On lines 643-645, we use the API function snorkel_obj_get with the attribute snorkel_attrib_header to retrieve the query-string value from the HTTP request header.

593    call_status_t SNORKEL_EXPORT
594    view_uri (snorkel_obj_t http,
595              snorkel_obj_t connection, char *pszurl)
596    {
.
.
.
642    
643      if (snorkel_obj_get
644          (http, snorkel_attrib_header, "QUERY", szquery,
645           (int) sizeof (szquery)) == SNORKEL_SUCCESS)
646        {
647          if (strcmp (szquery, "edit") == 0)
648            return edit_page (http, connection, pszurl);
649          else if (strcmp (szquery, "download") == 0)
650            {
651              if (snorkel_file_stream
652                  (connection, pszurl, 0,
653                   SNORKEL_BINARY) == SNORKEL_ERROR)
654                return HTTP_ERROR;
655              return HTTP_SUCCESS;
656            }
657        }
.    
.
.
738      return SNORKEL_SUCCESS;
739    } 

If the query-string equals "edit", we send the HTTP request object, connection object, and the URL to the function edit_page.

407    call_status_t
408    edit_page (snorkel_obj_t http,
409               snorkel_obj_t connection, char *pszurl)
410    {
. 
. 
.
443      if (stat (pszurl, &stf) != 0)
444        return
445         ERROR_STRING ("resource could not be located\r\n");
446    
447      strftime(sztime,sizeof(sztime),"%m/%d/%y %I:%M:%S %p",
448                localtime (&stf.st_mtime));
449    
450      snorkel_obj_get (http, snorkel_attrib_uri, szuri,
451                       (int) sizeof (szuri));
452      pszfile = strrchr (szuri, '/');
453      pszfile++;
454    
455      if (snorkel_printf(connection,header,szuri, sztime) ==
456          SNORKEL_ERROR)
457        return HTTP_ERROR;
458    
459    
460      if (snorkel_file_stream
461          (connection, pszurl, 0,
462           SNORKEL_UUENCODE) == SNORKEL_ERROR)
463        return HTTP_ERROR;
464    
465      if (snorkel_printf (connection, footer, pszfile) ==
466          SNORKEL_ERROR)
467        return HTTP_ERROR;
468    
469      return HTTP_SUCCESS;
470    }

In the function edit_page, we send back an HTTP reply as an HTML form containing the associated file content in an edit field along with the associated filename in a non-editable field. To write the reply, we use a combination of the functions snorkel_printf and snorkel_file_stream. The function snorkel_printf works like the C fprintf function, using the connection object as an opened stream. We use the function to write the HTML header and footer. The snorkel_file_stream function streams the source file referenced by the URL to the client uuencoded.

We associated the update button, Submit, with the function save_file on lines 815-818 in bubble_main. If the user selects the update button from an edit_page-form, the server calls the exported function save_file to save file modifications to the associated URL.

472    call_status_t SNORKEL_EXPORT
473    save_file (snorkel_obj_t http, snorkel_obj_t con)
474    {
.
.
.
490      if (snorkel_obj_get
491          (http, snorkel_attrib_local_url_path, szurl_file,
492           sizeof (szurl_file)) != SNORKEL_SUCCESS)
493        return HTTP_ERROR;
494    
495    
496      if (snorkel_obj_get
497          (http, snorkel_attrib_uri_path, szuri_file,
498           sizeof (szuri_file)) != SNORKEL_SUCCESS)
499        return HTTP_ERROR;
500    
501      szuri_path[0] = 0;
502      strcat (szuri_path, szuri_file);
503    
504      if (snorkel_obj_get
505          (http, snorkel_attrib_post, "filename", szfile,
506           sizeof (szfile)) == SNORKEL_ERROR)
507        return HTTP_ERROR;
508    
509    
510      if (snorkel_obj_get
511          (http, snorkel_attrib_post_ref, "contents", &psz,
512           &cbpsz) == SNORKEL_ERROR)
513        return HTTP_ERROR;
514    
515    #if defined(WIN32) || defined(WIN64)
516      strcat (szurl_file, "\\");
517    #else
518      strcat (szurl_file, "/");
519    #endif
520      if (szuri_file[strlen (szuri_file) - 1] != '/')
521        strcat (szuri_file, "/");
522    
523      strcat (szurl_file, szfile);
524      strcat (szuri_file, szfile);
525    
526      fd = fopen (szurl_file, "wb");
527      if (!fd)
528        {
529          return
530          ERROR_STRING ("The file could not be saved!\r\n");
531        }
532    
533      ptr = psz;
534      while (ptr)
535        {
536          char *temp = ptr;
537          ptr = strstr (ptr, "\r\n");
538          if (ptr && ptr != temp)
539            {
540              *ptr = 0;
541              if ( fprintf(fd, "%s\n";, temp) < 0)
542                {
543                  fclose (fd);
544                  ERROR_STRING
545                      ("I/O error encountered updating file.\r\n");
546                }
547              ptr += 2;
548            }
549          else if (ptr && ptr == temp) 
550            {
551              if (fprintf (fd, "\n") < 0 )
552                {
553                  fclose (fd);
554                  return
555                    ERROR_STRING
556                          ("I/O error encountered updating file.\r\n");
557                }
558              ptr += 2;
559            }
560          else if (temp && strlen (temp) > 0)
561            {
562              if (fprintf (fd, "%s\n", temp) &lt; 0)
563                {
564                  fclose (fd);
565                  return
566                    ERROR_STRING
567                       ("I/O error encountered updating file.\r\n");
568                }
569            }
570          i++;
571        }
572    
573      fclose (fd);
574      snorkel_printf (con, pszsuccess,
575                      szuri_file, i);
576      return HTTP_SUCCESS;
577    
578    }

To write the file, the save_file function gets the filename stored in the non-editable field of the edit_page-form and the modified file content using the snorkel_obj_get API with the snorkel_attrib_post and snorkel_attrib_post_ref attributes. All data for posted-forms is stored in a table accessible by these attributes. The snorkel_attrib_post attribute obtains HTTP-POST variable values by copying the variable value into a provided buffer, whereas the snorkel_attrib_post_ref returns a pointer to the data stored in an HTTP-POST variable. We use the latter to eliminate the need for the allocation of a buffer large enough to store the modified file's content. To save the file, the save_file function opens the file provided in the variable filename and writes the content contained in the variable contents.

If the query-string value, obtained by view_uri, is "download", view_uri streams the file referenced by the URI back to the requesting client line by line, insuring proper line termination. This might seem inefficient, but thanks to the Snorkel runtime, it is not. The Snorkel runtime automatically buffers small I/O sends to reduce the number of calls to the Socket layer.

Finally, if the URI does not contain a query-string, the view_uri checks the extension and formats the HTML response based on the file type.

The source files for this project (file_server.c and file_server_plugin.c) are located in the "c" directory of the attached bundle.

Running the Server

To run the server, extract the content of the attached bundle.

On Linux

• Append LD_LIBRARY_PATH to include the directory deployment_directory/lib/Linux.
• Change directories to deployment_dir/bin/Linux.
• Enter the command "fsrv -p 8080 -i deployment_directory".

On Windows

• Change directories to deployment_directory\bin\wintel.
• Enter the command "fsrv -p 8080 -i deployment_directory".

Start a browser and point it at http://server_hostname:8080.

Conclusion

I am providing a limited version of the Snorkel runtime library used in this project to CodeProject members free of charge. The provided SDK includes binaries for Linux, SunOS, and Windows platforms. It also includes examples and a developer's guide. Even though Snorkel supports SSL, I have removed SSL versions of the runtime libraries due to US trade laws. The developer's guide is located in the doc folder of the attached bundle, you may want to read the guide prior to playing around with this project. If you have any questions and or additional interests in the provided SDK, feel free to contact me at wcapers64@gmail.com.

History

• March 30, 2010 - Made some grammatical corrections and included a direct link to the Snorkel Developer's Guide.
• April 08, 2010 - Added the Windows 2K version of Snorkel runtime. Note: the Windows 2000 runtime does not leverage thread affinity since the supporting APIs are not present in the OS implementation.
• April 09, 2010 - Made corrections for the Windows 2K version, per request.
  • Download Snorkel for WIn2K - 68.2 KB
• April, 09 2010 - Made some corrections to page 24 of the Developer's Guide.
• April 14, 2010 - Updated Snorkel runtime. The update fixes a CLOSE_WAIT issue, which can occur when a connection is abruptly lost. Also exposed both the linger and timeout attributes for listeners, see modified version of file_server.c and/or the Developer's Guide for how to use. Updated the Developer's Guide in the bundle.
• April 22, 2010 - Due to an error in the SDK build process, in the last update, the Windows 2K (snorkel32_2k.dll) version of the runtime became the de facto version of the runtime. This was because the link libraries for both the Windows 2K and non-Windows 2K versions for the Wintel platform shared the same library name, snorkel32.lib. In this update for the SDK, the Windows 2K version correctly uses the linked library name, snorkel32_2k.lib. Note, this still allows snorkel32_2k.dll to be substituted for snorkel32.dll by altering its name.
• May 18, 2010 - Corrected an issue regarding binary data streaming and MIME-URL callbacks. The defect did not affect callbacks that streamed non-binary content such as HTML, XML, text, etc... Added a network performance enhancement to the Snorkel runtime to improve server response under heavy load conditions.

  Note: the May 18, 2010 update includes an extension of the file server bubble to include GNU Plot files. The modification facilitates data plotting in browsers. Please hold off on using the new functionality and wait for the accompanying article. I will not address questions regarding the new functionality on this page.

• June 16, 2010 - updated Snorkel runtime to 1.0, and added additional command line options to access new functionality.

  Snorkel 1.0 includes:

  • Support for keep-alive.
  • Support for zero-copy (sendfile).
  • Exposed thread governor overload -- now users can create more handlers than there are cores.
  • Minor bug fixes.
  • Added built-in page to display information about embedded server. To access page, append root URI with /about or /snorkel.
• June 21, 2010

  I normally don't post updates to my libraries this fast, but this time, I could not resist the temptation. Build 1.0.1 received a significant performance boost over the weekend, on the Windows side, thanks to changes in how file system information is cached. The changes significantly improved requests per second and average transfer rate by reducing I/O blocking. When bench marked against other web-servers, using Apache's ab test, the performance differences were significant enough to merit this early update.

• June 21, 2010

  Noticed and corrected defects on UNIXes.

• June 25, 2010 -- Snorkel 1.0.2 update
  • Fixed a minor thread-heap allocator defect for allocations pushed outside of thread-heap.
  • Added thread-heap integrity check and auto-repair features.
  • Completed testing and enabled additional performance enhancements that were introduced but disabled for 1.0.1.

  Note to adopters of Snorkel: Snorkel is heavily tested on a daily basis, and bugs are often detected and fixed before they are detected in the field. Until there is an official site for Snorkel updates, I will keep the version bundled with this article and other articles that use the library up to date with the latest version of the API.

• July 8, 2010 -- Snorkel 1.0.4 update
  • Corrected file mismatch between runtime library files in the bin directory and the lib directory for Wintel.
  • I added ability to toggle off/on the size field transmitted by snorkel_printf for non-HTTP streams. To enable or disable the feature, which is on by default for non-HTTP streams, use the following command snorkel_obj_set (snorkel_obj_t non_http_stream, snorkel_attrib_cbprintf, int state (enabled=1,disabled=0)).
  • Added email function. Syntax: snorkel_smtp_message (char *smtp_server, int port, char *from, char *to, char *format_string, arg1, arg2,...argN). Note: the function works just like printf.
  • Exposed a few more attributes.
  • Minor bug fixes.
• July 21, 2010 -- Snorkel 1.0.5 update
• January 6, 2011 – Snorkel 2.0.0.1 update (still free)
  • New improved API
  • Faster performance
  • Added new APIs Aqua (Service SDK) Sailfish (C/C++ application server)
  • Platform support MAC OSX, SunOS, Debian Linux, Windows
  • New download site: http://snorkelembedded.webs.com