Accessing Google Search Algorithm via XML

Introduction

XML has become the hottest new kid on the block for data transmittal. For the first time ever in the history of computing something so new has been adopted by all the major players and standardized! Starting out as a de-facto standard for data exchange, XML is expected to soon become a dejured standard.

The article shows how to access the Google search facility using XML and ASP.

Prerequisites and Code

First of all, you guys must understand that XML is not a language. It is a mechanism to facilitate seamless and transparent data exchange between tightly and lightly-coupled systems. The support for XML has been included in most of the major programming languages. For our purposes, the programming environment used would be ASP with VBScript

You must also have MSXML shipped by Microsoft in IE to run this code. If you are using IE 5 or higher, you already have MSXML installed on your computer. If you do not have MSXML parser installed, you would get a PROG ID error when trying to create XMLHTTP object below. To download MSXML, visit Microsoft

To access Google Search Data, you would need to create a XMLHTTP object:

set xml = Server.CreateObject("Microsoft.XMLHTTP")

If you are using version 3.0 of XMLHTTP, you would use the following code:

Set xml = Server.CreateObject("MSXML2.ServerXMLHTTP")

This code creates an instance of XMLHTTP. XMLHTTP would be used to send requests and receive responses over HTTP. All we have done till this point is to create an object that would hold the responses sent by Google to our application.

Once the object has been instantiated, you would need to establish connection with Google like this:

xml.Open "GET", "http://www.google.com/search?q=features+of+XML", False
xml.Send 

This would establish the connection between Google and your application and send back the data using GET.

Notice the URL that we are accessing. It includes the full URL with querystring parameters that would be sent to Google. In a real-world application, these hard-coded values should ideally be user submitted. For that, you would have to create a form, submit values, receive them and then pass them in the XMLHTTP open method.

Once you have established a connection and sent a request, all that needs to be done is to receive the response, display it and destroy the object to release resources. This is done by:

Response.Write (xml.responseText)
xml = Nothing 

Understanding the XMLHTTP Open Method

XMLHTTP supports various methods one of which is the Open method. Open method takes the following form:

open(Method, URL, Async, User, Password)

• Method: A string value that specifies the method used to open the connection. In this case, we are using GET. Other possible values include POST, PUT or PROPFIND
• RL: Either a relative or an absolute URL
• Async: A boolean value to indicate whether the calls would be asynchronous. The default value is TRUE.
• User: An optional string value for sites requiring authentication
• Password: An optional string value for sites requiring authentication.

Recap

Lets have a look at the full source code:

  
  ' Create an xmlhttp object:

    set xml = Server.CreateObject("Microsoft.XMLHTTP")

  ' Version 3.0 of XMLHTTP, use:

  ' Set xml = Server.CreateObject("MSXML2.ServerXMLHTTP")


  ' Opens the connection to the remote server.

    xml.Open "GET", "http://www.google.com/search?q=XMLHTTP+FAQ", False

  ' Actually Sends the request and returns the data:

    xml.Send

    Response.Write (xml.responseText)
    xml = Nothing

Possible Extensions to the Code

The XMLHTTP API can be used to access both binary and simple HTML data from any web site. Thus, you can easily use XMLHTTP API's to request remote files or request remote web pages. The above example is a classic means to include Google search facility on your own web site. The point to remember here is that XMLHTTP can be used to fetch remote data from any web site. Therefore, you can just as well use Yahoo or even Altavista to receive data.

XMLHTTP can also be used to access secure web sites or data that requires authentication. The Open method can be used to specify the username and password (both optional) as the third and fourth parameters. However, that would be a topic for another day. Enjoy and good luck!