Retrieve Information from Yahoo! APIs

Introduction

This tip aims at getting numerous amount of data using Yahoo! APIs using C#. Yahoo! APIs provide various information like weather, geographical information like country, city, ocean details, local US complete city information etc.

What is API?

API is Application Programming Interface. APIs can either be Desktop or Web APIs. Desktop APIs are usually DLL file which have functions defined in them. Microsoft itself provides numerous APIs in various DLLs like user32.dll, advapi32.dll, etc. Numerous functions are defined in them. Similarly, Web APIs are requests triggered by a client usually with a secret key for authentication. The response is usually a JSON or XML which needs to be parsed as required for our requirement.

Who Provides APIs?

Companies like Google, Yahoo!, Starbucks, Netflix and multiple others provide lot of APIs which can be utilized to get necessary information. Mostly APIs are required to sign up and secret keys are to be included in the request so that the server-side can understand it's a legitimate request.

What is YQL?

YQL is Yahoo! Query Language. It's similar to SQL and is designed to retrieve and manipulate data from APIs through a single Web interface.

https://developer.yahoo.com/yql/console/ is the YQL console which has complete details on tables, sample queries. Results of the query are either retrieved in JSON or XML as chosen. This utilizes Yahoo! APIs eliminating the need to write complex API programming. No need of any secret keys as well.

Yahoo! also provides the corresponding REST query so that Webclient can utilize the URL and formulate the JSON/XML response as needed.

What Information Does YQL Contain?

Weather, World geography, local US yellow pages, Google and its associated websites like news, search, Microsoft Bing, etc. Currently YQL holds 1064 tables.

Parse XML Output

Next comes the big work of parsing the XML from the REST API response. API response XMLs are complex, and can't easily be converted as such.

Normally, parsing XML is done by XMLSerialize.Deserialize() function. In order to store the deserialized object, we need to write class files which correspond properly stores node values for XML.

But writing C# wrapper classes for large XML with numerous nodes is a tedious task. They sometimes end up in wrong class / variable assignment as well. But Visual Studio provides very easy way to accomplish this task.

C# Wrapper Classes for XML Deserialization

• Save a sample output from APIs to a XML file
• Open the XML in Visual Studio (2008, 2010, 2013, 2015)
• Once done, you will see a XML menu in the menu bar
• Click it and hit "Create Schema"

• Once you hit it, XSD files will be created.
• Based on the XML schemas, single to 'N' number of XSD files can be created.

• Save all XSDs to a location say D:\Temp
• Once done, the next step is to generate class files as needed
• Open Visual Studio command prompt from Start Menu
• As per below screenshot, please generate class file:

• xsd.exe is a XML schemas / Datatype support utility which is used to generate XSD, Class files
• /c - denotes generate class file || /l - denotes language CS, VB || /n - denotes Namespace || List of XSD files separated by spaces
• So wrapper class is generated for any complex XML file.

How to Code in C#?

Once you generate the class file, the top most class which holds every other classes is 'query'. Itès the base node in XML as well.

static void Main(string[] args)

        {
            // Initialize XML Serializer Class which hosts the Deserialize() function
            // typeof(query) - query is root node of XML
            XmlSerializer XMLdeserialize = new XmlSerializer(typeof(query));          

            // Initialize Web Client and set its encoding to UTF8
            WebClient wc = new WebClient();
            wc.Encoding = Encoding.UTF8;

            // Form Actual URL - REST API call
            StringBuilder sbURL = new StringBuilder();
            sbURL.Append(@"https://query.yahooapis.com/v1/public/yql?q=");
            // YQL is select * from geo.places where text='sfo'
            sbURL.Append(AntiXssEncoder.HtmlFormUrlEncode
		(@"select * from geo.places where text='sfo'")); // Anti XSS encoder - 
							// Prevent cross site scripting
            sbURL.Append(@"&diagnostics=true");           

            // Download string (XML data) from REST API response
            string XMLresult = wc.DownloadString(sbURL.ToString());

            // Form Stream from the string we got
            byte[] byteArray = Encoding.UTF8.GetBytes(XMLresult);
            MemoryStream ms = new MemoryStream(byteArray);
            StreamReader reader = new StreamReader(ms);

            // Deserialize XML
            object obj = XMLdeserialize.Deserialize(reader);
            query XMLData = (query)obj;
            reader.Close();

            // Display Values
            Console.WriteLine("Place Type : " + XMLData.results.place.placeTypeName.Value);
            Console.WriteLine("Name : " + XMLData.results.place.name);
            Console.WriteLine("Country : " + XMLData.results.place.country.Value);
            Console.WriteLine("Postal : " + XMLData.results.place.postal.Value);
            Console.WriteLine("Time Zone : " + XMLData.results.place.timezone.Value);

            Console.ReadLine();

        }

Code has detailed comments which will help to understand the flow detail. Below is sample output parsed from REST API XML.

History

• 31-Jan-2016: First revision

Conclusion

Any further comments to improve the tip are welcome.

References

1. https://en.wikipedia.org/wiki/Web_API - Web API
2. https://developer.yahoo.com/yql/guide/ - YQL Guide
3. https://developer.yahoo.com/yql/console/ - YQL console to write and test queries
4. https://msdn.microsoft.com/en-in/library/he66c7f1(v=vs.110).aspx - MSDN Deserializer
5. https://satishlalam.wordpress.com/2009/06/05/auto-generating-c-wrapper-for-parsing-xml-files/ - Create class from XML file
6. http://blogs.msdn.com/b/mapo/archive/2008/02/20/xsd-exe-error-generating-classes-for-schema-myschema-xsd-the-element-http-schemas-domainname-com-types-mytype-is-missing.aspx - Combine XSD to generate class file