Introduction
Axel is a lighter download accelerator for Linux and other Unices. It should compile (and run) on BSD, Solaris, Darwin (Mac OS X) and Win32 (Cygwin) systems.
Porting to NATIVE WIN32 without Cygwin, what I did in the #if WIN32 #endif
, for example:
- Changed Linux file descriptor for the new socket to WIN32
SOCKET
descriptor referencing the new socket - Changed
close
, write
, read
to closesocket
, send
, recv
- Changed
select
blocking thread to WSAEventSelect
Asynchronous I/O - Changed
pthread_t
for the thread to HANDLE
referenced the thread - Changed
pthread_create
, pthread_join
, pthread_cancel
to CreateThread
, WaitForSingleObject
, TerminateThread
- Changed Linux file descripter for the saving file to
HANDLE
referenced the saving state - Changed
open
, read
, write
, close
to CreateFile
, ReadFile
, WriteFile
, CloseHandle
Background
DEBUG Axel to see multi-thread downloading way:
- TCP connect (
socket
, bind
, connect
) to host such as http://localhost provided by IIS - Send http header request:
GET / HTTP/1.0
Host: www.codeproject.com
Range: bytes=1-
User-Agent: Axel 2.4 (WIN32)
- Get the content length from header reply:
HTTP/1.1 206 Partial Content
Content-Type: text/html
Last-Modified: Tue, 20 Sep 2011 03:39:05 GMT
Accept-Ranges: bytes
ETag: "3c2bb1d84677cc1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Fri, 24 Feb 2012 08:41:12 GMT
Connection: close
Content-Length: 688 /* Here is what I need */
Content-Range: bytes 1-688/689
- Divide the content range into threads' number for instance 10:
Downloading 0-67 using conn. 0
Downloading 68-136 using conn. 1
Downloading 137-205 using conn. 2
Downloading 206-274 using conn. 3
Downloading 275-343 using conn. 4
Downloading 344-412 using conn. 5
Downloading 413-481 using conn. 6
Downloading 482-550 using conn. 7
Downloading 551-619 using conn. 8
Downloading 620-688 using conn. 9
- Send http header request with range [from, to] via one of the threads:
GET / HTTP/1.0
Host: localhost
Range: bytes=206-274
User-Agent: Axel 2.4 (WIN32)
recv
the buffer from SOCKET
, WriteFile
the buffer from http header's reply to HANDLE
CreateFile
with temporary file *.st (Axel style), terminate && close the thread when downloaded the content range. Temporary file is helpful to the server host resuming available.
Using the Code
The source file text.c is the main entry considering as a good example and testcase about how to use Axel API.
conf_t conf;
conf_init(&conf);
conf.num_connections = 10;
conf.add_header_count = 0;
axel_t *axel = axel_new(&conf, 0, "http://localhost");
strcpy(axel->filename, "filename.htm");
axel_open(axel);
axel_start(axel);
while (!axel->ready)
{
axel_do(axel);
}
axel_close(axel);
Points of Interest
Cross platform development - played with Axel console under Gentoo Linux to see the Multi-thread download accelerator working way, then ported with NATIVE WIN32 API to Windows. I complied the source code with VS2005 (Choose mbstring
project setting instead of Unicode which brings ^M CTRL-V CTRL-M, unrecognized Chinese character, and binary file trash issue) under Windows 7 64-bit, so it needs more contributors to debug under other Windows distribution.
Improvement
There is WSAPoll
supported only by Windows Vista, 7 && Server 2008 to determine the status of one or more sockets. It might be more efficient than WSAEventSelect
.
History
- 2012-02-27 xzhai BUG FREE :)
- 2012-02-26 xzhai
- Fixed Re-connection bug
- Fixed BIGSIZE file freezing bug
- 2012-02-24 xzhai
- Ported Axel v2.4 to WIN32 without Cygwin