ASProxy: Surf in the web invisibly using ASP.NET power
Surf the web with ASProxy
Current version: 5.5 Beta4
Note: All downloads are from external site (
SourceForge).
Contents
Introduction
With this tool which is written in ASP.NET 2 you can easily pass the filter and see the your desired web pages or you can even download files.
Features:
- Fell free to surf the web pages
- Full support of AJAX enabled and Web 2.0 sites.
- Supporting most login forms
- Download tool to let the user download files regardless of their type
- Resume-supported downloading
- Supporting submission forms for sending registration data and uploading
files
- Supporting cookies
- The capability of displaying the pages of unknown encoding and forcing
them to be displayed in UTF-8 encoding
- Supporting JavaScript and Cascade Style Sheet (CSS) files
- Supporting
JavaScripts
and script tags
- Supporting inline frames and frame-set pages
- The capability of page compression for fast receiving
- The capability of displaying the original URLs and pictures on the
status bar
- Displaying the page title on the browser caption
- Supporting dynamic content created with JavaScript.
- Automatic update to make proxy current
- Password protection against strangers; good for personal use
Installation
Installation guide for Somee (For beginner users)
The first thing you need is to have a free web hosting that supports ASP.NET.
I recommend www.somee.com. To get the free package go to the address below:
http://www.somee.com/FreePackage.aspx
After you created your website, you'll need an FTP tool like FlashFXP or CuteFTP to transfer files to the server.
Open your FTP software and click the "connect" button and a new window will open. Enter your site name in the "Host" field. For example: Name.somee.com
Then you must enter User name and Password. Accept the dialog window. After
you logged in, if there is any folder there, navigate. If not, continue reading
this guide.
Somee's settings to support ASP.NET 2
1- Go to www.somee.com
2- Log in your account.
3- Go to Control Panel and click "Web" button.
4- In the following page titled "General Information", your account information
is displayed. Above the "General Information" header, there are four buttons.
Click the last one named "IIS Setup".
5- At the bottom of the newly opened page, you will see a drop-down list with
the title of ASP.NET Version. Choose the "2.0.50727". Then click "Update"
button!
Uploading Files to Your Own Website
Please notice that you must copy all the files in "PrecompiledWeb" folder to
the server not the folder itself. Now your proxy is ready for use! To see the
sample website created according to this guide, please visit:
http://asproxy3.somee.com or http://asproxy35.somee.com
ASProxy Configuration
Most of the configurations are stored in Web.Config file in appSettings section.
Below are the default configurations:
<appSettings>
<add key="WebProxyMode" value="0"/>
<add key="WebProxyHost" value="0.0.0.0"/>
<add key="WebProxyPort" value="8080"/>
<add key="ASProxyLoginNeeded" value="false"/>
<add key="ASProxyLoginUser" value="admin"/>
<add key="ASProxyLoginPassword" value="admin"/>
<add key="ASProxyServerPort" value="8080"/>
<add key="ASProxyServerAdminPassword" value=""/>
<add key="ErrorLogEnabled" value="false"/>
<add key="ASProxyAutoUpdateEnabled" value="false"/>
<add key="ASProxyAutoUpdateInfoUrl" value="http:</appSettings>
Automatic update:
One of the new features in ASProxy is automatic update which updates the
proxy automatically according to your preference.
This feature is disabled by default. To enable it open the "Web.Config" file and locate "ASProxyAutoUpdateEnabled"
key in "appSettings"
section. Then change its value to "true". That's it!
After enabling this feature, ASProxy will check the URL entered in "ASProxyAutoUpdateInfoUrl"
key
for update every 3 weeks.
If a new version is available, it will download the update and install the
update on itself.
You may have noticed that you can change the update information location in "ASProxyAutoUpdateInfoUrl"
key.
But remember the entered URL should exist and the information file format
should be correct, otherwise the update will fail.
Private proxy:
With this new feature you can disable other people's access to your own proxy
site. This is a valuable option for personal sites to get rid of abusers and
save the bandwidth. To enable this feature follow these instructions:
Open the "Web.Config" file, and go to "appSettings" section.
Locate "ASProxyLoginNeeded" key, then change its value to "true".
To specify username; locate the "ASProxyLoginUser" key and change its
"value" to your desired username.
To specify password; locate the "ASProxyLoginPassword" key and change its
"value" to your desired password.
After this setting is enabled any unauthorized user will be redirected to
login.aspx page. If the username and the password match they will have access to ASProxy.
Access the Web through network proxy:
In some network structures the web server is behind a proxy server and the web
server cannot access to the internet directly. The web application needs to be
configured to access the web.
ASProxy needs some configurations to pass the network proxy:
Open the "Web.Config" file and locate "WebProxyMode" key in "appSettings"
section. The value may be set to one of these:
0 : means that this configuration is disabled and the
application can connect to internet directly. For security reasons this is default option.
1 : means that ASProxy can detect the configurations automatically.
3 : means that the custom values will be used. To enter the custom
values enter the network proxy IP in "WebProxyHost" key and enter its port in "WebProxyPort"
key.
Error log:
This feature is only for developers. If any unhandled error occurs in
application execution the error details will be logged in a new file named
"exceptions.xml" in "bin" folder.
To enable this feature, open the "Web.Config" file and locate "ErrorLogEnabled"
key in "appSettings"
section and change its value to "true".
ASProxyServer Configuration
ASProxyServer is another type of proxies introduced with ASProxy. A version
of ASProxy presented as IP Proxy. Here are the setup and configuration
instructions:
In ASProxyServer package the ServerConfig.aspx is included. Open this page on
your browser (for example: proxy.mysite.com/ServerConfig.aspx). Enter the
administration password (specified in "web.config" file in "ASProxyServerAdminPassword"
key) in "Admin password" field. Specify your desired port
address and then click on "Start proxy" button. The proxy will start to
work and the state will change to "Listening".
To access the created proxy follow these configurations in your browser but
remember the IP address is the address that hosts ASProxy. (For example
mysite.com which its IP is something like 156.25.16.69)
For "Internet Explorer": Open the "Internet Options" in
"Control Panel". Go to the "Connections" tab and click the "LAN Settings" button.
In "Proxy Server" group, check the "User a proxy server for your LAN" check box
and enter the IP address in "Address" field and the port number in
"Port" field. Then OK everything!
For "FireFox": Select "Options" from "Tools" menu and go to
"Advanced" section. Go to "Network" tab and click on the
"Settings" button on the "Connections" group. Check the "Manual proxy
configuration" radio button. Enter the IP address in "HTTP Proxy" field and
the port number
in "Port" field. And check the "Use this proxy for all protocols" check box.
Then OK everything!
Note: Do these configurations when you are online. These
options may cause some problems when you are offline. To disable them in
"IE" uncheck the "User a proxy server for your LAN" and in "FireFox" select
"Direct connection to internet" radio button.
Important note: "Application Pool" of ASProxy should have
"System" identity to work correctly. The default value is "Network". You can access
this option in IIS 6. The ASProxyServer installer will do this for you if
installation is done successfully.
If you do not have enough information about IIS 6, do not change the configurations.
How It Works
Data is usually filtered in three ways:
The first and the simplest way is through controlling the incoming and outgoing
addresses. These addresses are stored in a black list. If one of these addresses
is detected when requested or responded the filtering software will block the
data transfer.
The second way is almost like the first one, but there is a list containing
the unallowed words. The filtering software will automatically detect these
words in the address and block the data.
In the third way, in addition to the first and second ways, a search is done
into the received data. The search engine will search for the certain links,
addresses and words and if the number of these items reaches the certain limit,
the page will be filtered. This way of filtering is more reliable but lessens
the network speed, thus it is usually used in non-public centers where there is
a need to have more control over the users' activities.
All these three ways work based on one a list of certain words and addresses.
And all the filter crossing tools let you cross the filters that work with the
fist and second ways. Notice that the third way is defeated in a few cases.
How ASProxy Works
The users must enter the site on which ASProxy is installed. The URL is
entered in a text box provided and the request is sent correctly depending on
how much the site has been popular and recognized by the filtering system. The
ASProxy engine receives the URL and processes the request. After several-stage
processes, the whole addresses of the requested page are converted and sent to
the ASProxy-installed site. The result is magnificent, because the site will not
be detected as an anti-filtering one and it will not be blocked. Below is the
diagram of the process:
Background
ASProxy tool consists of the following processing classes taking the
responsibilities as explained for each:
-
"DataManagement" section:
"DataManagement" section includes classes which process raw data from
web.
- "ASProxyEngine" class: This class receives the requests
and sends them for processing and returns the result to the requester. This class encapsulates all necessary
activities with its properties and methods.
- "WebDataCore" class: Sends and receives the requests
from web. This class does all necessary configurations to get actual data
from web. This class is ASProxys core.
- "CookieManager" class: Saves and restores cookies
among several requests. "WebDataCore" class always use this class.
- "HttpCompressor" class: Compresses the pages to
increases the transfer speed. Compression can be enabled from the checkbox in
"default"
page. (Disabled by default for compatibility reasons)
- "ResumableDownload" class: Sends any data to the user
(which we call "Download"). This class makes the resume-support downloads
possible.
"DataProcessor" section includes classes to process
common MIME content types that are used in HTML pages.
- "Processors" class: This class especially focuses on
encoding type conversions.
This section includes some subsections for better classification. The class names in this section follow these
rules for these suffixes:
Parser: Classes defined with this suffix, find
positions of tags in html or changes their properties.
Processor: Classes defined with this suffix, perform all
their required operations. The "ASProxyEngine" class uses these classes directly.
Replacer: Classes defined with this suffix, change or replace a
part of text or data (usually Html). To do this, these classes use their
implemented methods.
The "DataProcessor" section operations are implemented in three subsections.
- HTML Classes: The classes of this part perform all necessary
operations to remove or replace HTML codes in a requested page. This is the
most important place in ASProxy to get desired data in correct format.
- JavaScript Classes: The classes of this part perform
operations on JavaScript codes. This part is the hardest in implementation in ASProxy.
- CSS Classes: Perform necessary operations to replace
addresses in CSS files. The main goal of this part is to replace "background"
attributes.
-
"Configurations" section:
This section includes classes which save or restore saved data.
- "ASProxyConfig" class: Saves and restore ASProxy
settings among requests. The default settings are retrieved from "Web.config"
and user changes are retrieved from the cookie.
- "SiteExceptions" class: Stores unhandled errors list
for future debugging. Error logging is stopped by default and can be changed
in "Web.config" file.
Contains common and useful classes.
- "Performance" class: contains some
replacement functions for simple dotNET Framework string functions that
considerably increases ASProxy engine speed.
- "UrlProvider" and "UrlBuilder" classes:
contains some functions for working with URLs.
- "HtmlTags" class: contains some functions for working
with HTML.
- "ASProxyFunctions" classes: contains some generally
useful functions used in ASProxy.
Tracing Background Working (Developers)
Here we look over the ASProxy working steps.
Below we request to visit a site (for example
www.softprojects.org ), so expect that
the response data type should be HTML.
Deep1
2
3
3
3.1
3.2
3.3
3
-
4
4
4.1
4.2
4.2
4.2.1
4.2.2
4.2.3
4.2.4
4.2
-
4.3
4.3
4.3.1
4.3.1
4.3.1.1
4.3.1.2
4.3.1.3
4.3.1
4.3
4.4
4
5
| Execution traceRequest a site:
Initialize an ASProxyEngine
Test content type by PreExecute
{
IMAGE: Link to images.aspx page and display it in page as img tag. END.
Unknown: Redirect to download page. END.
HTML: Continue!
}
in ASProxyEngine
{
Initialize a WebDataCore
Execute the request by WebDataCore instance
{
Initialize "PostBack" or "Get" methods information.
Add cookies.
Get the response.
Restore returned cookies and save them.
}
if html process requested
{
Test content mime type
{
HTML: Initialize HtmlProcessor and process data.
JavaScript: Initialize JSProcessor and process data.
CSS: Initialize CSSProcessor and process data.
}
}
return processed data.
}
Display response in a label!
|
A Review on the Main Functions
For the user to request the URL, first of all, an instance of the main class
needs to be created for processing the request (gethtml.aspx):
engine = new ASProxyEngine(ProcessTypeForData.HTML,true);
engine.RequestInfo.ContentType = MimeContentType.text_html;
engine.Initialize(Request);
engine.Execute(Response);
In this code, we specify that our data type is HTML. Then we call
"initialize" method of ASProxyEngine to specify the request
information such as request URL.
At the end we call "Execute" method to confirm the operations. This method
will apply the result to the response directly.
How to implement the operation in a new page:
To do this add a new page to the ASProxy project and add a textbox, a button
and a label. To save some resources set the "EnableViewState"
property of Label to false; we do not need its state. Then write these codes in
button click event:
protected void Button1_Click(object sender, EventArgs e)
{
SalarSoft.ASProxy.ASProxyEngine engine =
new SalarSoft.ASProxy.ASProxyEngine(SalarSoft.ASProxy.ProcessTypeForData.HTML);
engine.RequestInfo.ContentType = SalarSoft.ASProxy.MimeContentType.text_html;
engine.Initialize(TextBox1.Text);
string myResponse;
engine.Execute(out myResponse);
Label1.Text = myResponse;
}
Run the new page and enter an URL and click the button; the URL will be shown.
Nothing will work on that page. It's just HTML
code and it is not useful by itself. So to get better results we change the code.
protected void Button1_Click(object sender, EventArgs e)
{
SalarSoft.ASProxy.ASProxyEngine engine =
new SalarSoft.ASProxy.ASProxyEngine(SalarSoft.ASProxy.ProcessTypeForData.HTML, true);
engine.RequestInfo.ContentType = SalarSoft.ASProxy.MimeContentType.text_html;
engine.Initialize(TextBox1.Text);
string myResponse;
engine.Execute(out myResponse);
Label1.Text = myResponse;
}
A "true" at the constructor of ASProxyEngine is added. This "true"
specifies that the options should be restored automatically from the cookie or default
options.
If you want more control over the settings you can set them manually. To do
this the OptionsType class is needed. And if you want to access to the stored
settings you should use ASProxyConfig class.
Look at this example:
protected void Button1_Click(object sender, EventArgs e)
{
SalarSoft.ASProxy.ASProxyEngine engine =
new SalarSoft.ASProxy.ASProxyEngine(SalarSoft.ASProxy.ProcessTypeForData.HTML, false);
engine.RequestInfo.ContentType = SalarSoft.ASProxy.MimeContentType.text_html;
SalarSoft.ASProxy.OptionsType opt = SalarSoft.ASProxy.ASProxyConfig.GetCookieOptions();
opt.Images = false;
opt.AcceptCookies = false;
engine.Options = opt;
engine.Initialize(TextBox1.Text);
string myResponse;
engine.Execute(out myResponse);
Label1.Text = myResponse;
}
In this example, the options is restored by "GetCookieOptions"
method and then Images and AcceptCookies are disabled.
Now everything seems OK, but the links do not work. The current page should
detect the URLs in query and display them. To do this we have to write the same
codes in Page_Load event:
protected void Page_Load(object sender, EventArgs e)
{
SalarSoft.ASProxy.FilesConsts.DefaultPage =
System.IO.Path.GetFileName(Request.Url.AbsolutePath).ToLower();
bool isClicked=(Request.Form<button1.id>!=null);
if(isClicked==false && SalarSoft.ASProxy.UrlProvider.IsASProxyAddressUrlIncluded(Request.QueryString))
{
SalarSoft.ASProxy.ASProxyEngine engine =
new SalarSoft.ASProxy.ASProxyEngine(SalarSoft.ASProxy.ProcessTypeForData.HTML,false);
engine.RequestInfo.ContentType = SalarSoft.ASProxy.MimeContentType.text_html;
SalarSoft.ASProxy.OptionsType opt = SalarSoft.ASProxy.ASProxyConfig.GetCookieOptions();
opt.Images = false;
opt.AcceptCookies = false;
engine.Options = opt;
engine.Initialize(TextBox1.Text);
string myResponse;
engine.Execute(out myResponse);
Label1.Text = myResponse;
}
}</button1.id>
In this code, we set the "DefaultPage" value that specifies the
default page for proxy and if it changes all link destinations will change. So
be careful in using this property; it may cause ambiguity.
The other important method used in this code is "IsASProxyAddressUrlIncluded".
This method checks the current query for URL existence.
The important point of this code is detecting the type of post back.
The code has to detect if the post back is done through pressing button1 or not.
The result of this detection is set to
"isClicked" variable. In "Page_Load" event the operations will
not
run when the button1 is not clicked and the query has URL address or the post
back has occurs from a submit form.
Implementation
Below you see some important method implementation.
Here is implementation of "Initialize" method that has 2 overloads. And you
will see important related methods such as "InitializeRequestQueriesByHttpRequest"
and "DoAutoDetection".
public void Initialize(HttpRequest httpRequest)
{
InitializeRequestQueriesByHttpRequest(httpRequest);
DoAutoDetection(httpRequest);
}
public void Initialize(string requestUrl)
{
RequestInfo.RequestUrl = requestUrl;
RequestInfo.RequestedQueries = requestUrl;
DoAutoDetection(null);
}
private void DoAutoDetection(HttpRequest httpRequest)
{
if (httpRequest != null)
{
RequestInfo.RequestedQueries = GetQueryCollectionString(httpRequest.QueryString);
if(!string.IsNullOrEmpty(httpRequest.ContentType))
RequestInfo.ContentTypeString = httpRequest.ContentType; RequestInfo.InputStream = httpRequest.InputStream;
}
if (RequestInfo.HttpRequestMethod == WebRequestMethod.POST)
{
if (httpRequest != null)
{
RequestInfo.PostedFormData = httpRequest.Form.ToString();
RequestInfo.PostedFormData = HttpUtility.HtmlDecode(RequestInfo.PostedFormData);
}
}
else
RequestInfo.PostedFormData = "";
if(RequestInfo.RequestInQueryMethod== WebRequestMethod.GET)
{
RequestInfo.PostedFormData = RequestInfo.PostedFormData;
if (fAutoDetection)
{
RequestInfo.RequestUrl =
UrlBuilder.AppendAntoherQueries(RequestInfo.RequestUrl,
RequestInfo.PostedFormData);
}
}
}
private void InitializeRequestQueriesByHttpRequest(HttpRequest httpRequest)
{
if (fAutoDetection)
{
RequestInfo.HttpRequestMethod = ASProxyFunctions.StringToRequestMethod(httpRequest.HttpMethod);
bool tmpBool = false;
string url = httpRequest.QueryString[Consts.qUrlAddress];
if (!string.IsNullOrEmpty(url))
{
string decode = httpRequest.QueryString[Consts.qDecode];
if (decode != null)
{
try
{
tmpBool = Convert.ToBoolean(Convert.ToInt32(decode));
}
catch
{
tmpBool = false;
}
}
if (tmpBool)
{
url = UrlProvider.DecodeUrl(url);
}
RequestInfo.RequestUrl = url;
}
tmpBool = false;
string postForm = httpRequest.QueryString[Consts.qIsPostForm];
if (postForm != null && postForm != "")
{
try
{
tmpBool = Convert.ToBoolean(Convert.ToInt32(postForm));
}
catch
{
tmpBool = false;
}
}
if (tmpBool)
RequestInfo.RequestInQueryMethod = WebRequestMethod.POST;
else
RequestInfo.RequestInQueryMethod = WebRequestMethod.GET;
}
}
Here is the implementation of "PreExecution" method. This method will
perform the
request but doesn't return any results. This is useful to check the response
headers and determine to continue the operations with calling "Execute" method
or not.
Note that after calling "PreExecution" method we can call "Execute" method or
we can call "Execute" method directly without calling "PreExecute" method.
public void PreExecution()
{
if (fAutoDetection)
{
if (RequestInfo.ContentType == MimeContentType.image_gif
|| RequestInfo.ContentType == MimeContentType.image_jpeg)
this.fUseRequestUrlAsReferer = true;
}
if (fWebData == null)
{
do
{
if (fWebData != null && fWebData.ResponseInfo.AutoRedirect)
{
Uri autoRedirectLocation = fWebData.ResponseInfo.AutoRedirectLocation;
CookieCollection cookies = fWebData.ResponseInfo.Cookies;
bool redirectIsInternal = fWebData.ResponseInfo.AutoRedirectIsInternal;
RequestInfo.RequestUrl = autoRedirectLocation.ToString();
fWebData.Dispose();
fWebData = null;
fWebData = new WebDataCore(RequestInfo.RequestUrl);
if (redirectIsInternal)
{
fWebData.RequestInfo.Cookies = cookies;
}
}
else
fWebData = new WebDataCore(RequestInfo.RequestUrl);
fWebData.UseRequestUrlAsReferer = fUseRequestUrlAsReferer;
fWebData.RequestInfo.RequestMethod = RequestInfo.RequestInQueryMethod;
fWebData.RequestInfo.PostDataString = RequestInfo.PostedFormData;
fWebData.RequestInfo.ContentType = RequestInfo.ContentTypeString;
fWebData.RequestInfo.InputStream = RequestInfo.InputStream;
fWebData.AcceptCookies = fOptions.AcceptCookies;
fWebData.DisplayErrorPageAsResult = false;
fWebData.Execute();
if (fWebData.Status == LastActivityStatus.Error)
{
this.fLastErrorMessage = fWebData.ErrorMessage;
this.fLastStatus = LastActivityStatus.Error;
this.fLastException = fWebData.LastException;
return;
}
} while (fWebData.ResponseInfo.AutoRedirect);
SetResponseInformation();
}
}
Note that the loop is used to execute the requests to implement Redirect
behavior.
Version History
Current version: 5.5 beta4
Version 4.5 2008/08/09:
* Cookie management improved. Cookies will save by specified domain name by site.
This feature ables to pass most sites login process.
Version 4.4 2008/08/08:
* Some response headers added to support some feature.
such as Cache-Control allows to cahce images and static contents.
This feature increases page loading speed and decreases server pressure.
Version 4.3 2008/06/13:
* Fix: Problem with queries that only contains parameters fixed. (e.g. <a href="?pass=yes">Testing</a>)
* ASProxyEncoder (Dynamic content coder, "asproxyencoder.js" file) URLs encoding functions improved.
* Fix: Problem with POST cookies solved. This problem didn't let to pass login pages.
* Fix: Now ASProxy can work properly in sub directories.
* Fix: BASE tag behavior improved.
* Fix: Some mistakes in "CorrectLocalUrlToOrginal" fucntion in "asproxyencoder.js" are fixed.
* Fix: Space character problem with downloading filename solved by replacing spaces with dash character.
* Download tool displays error messages and error url correctly.
Version 4.2 2008/03/27:
* ASProxy for mono released.
* Support embeded data with "data:" prefix.
* Enhanced displaying links and images original address.
Version 4.1 2008/03/08:
* Supports Basic, Digest and Integrated Windows Authentications.
* Enhanced displaying links and images original address.
* Some options renamed. "Display orginal url" renamed to "Original URLs" and "Always use UTF-8" renamed to "Force UTF-8".
* UI Improved.
Version 4.0 2007/12/29:
* Automatic update available!
This feature is disable by default, it can be enabled by the configure in Web.Config
file in ASProxyAutoUpdateEnabled key.
* Username and password protection for private proxies now is available! Just
change the settings in Web.Config file (key names are: ASProxyLoginNeeded ,
ASProxyLoginUser and ASProxyLoginPassword).
* Most problems in login forms are fixed. This problem was because of self post back
handling problem in "default.aspx" page.
* Some ASProxyServer bugs are fixed.
* Cookie handling mechanism improved. Now cookies are stored correctly.
* Supports inline background style in style tag. For example: <style>backgound-image:url(image.jpg);</style>
* Some bugs with user agent are fixed.
Special thanks to Ali Mohammadzadeh for English translation assistance.
