Introduction
The article aims to present a generic approach of fetching and visualizing live and historical stock data in Silverlight. The example is implemented using Google Finance as data source for the live and historical stock data and Visiblox Charts for Silverlight in presenting the fetched data. The example is, however composed, so that changing either the data source or the data visualization library should be fairly straightforward to implement. The article's structure is the following:
Obtaining Live and Historical Stock Data
The first problem to solve is finding a data source providing live and historical stock data and understand how its API works. There are two data sources that provide freely accessible stock information with sensible limitations: Google Finance and Yahoo Finance. Both data sources provide delayed live stock data information as well as historical data. In this article, I choose to go with Google Finance. Let's take a look at how its API works.
Google Finance API Overview
In our application, what we want to do is be able to query the live price of the stock and also fetch historical prices (as noted before Google Finance, like all free data sources offer time delayed "live" data). The only documented public Google Finance API is the portfolio API, however this API does not allow fetching stock prices in an easy way. Fetching live and historic stock data unfortunately is not documented.
The Google Finance pages, however, query against an undocumented REST API. This API can easily be understood by backtracking requests made to the site. The public API calls that our application will be needing are as follows.
Live Stock Data API
To obtain live stock data, the following call needs to be made: http://www.google.com/finance/info?q={stockCode}. The q parameter specifies the stock code of the instrument, as specified on Google Finance.
So for example, querying the latest Microsoft (MSFT) stock can be done via this call: http://www.google.com/finance/info?q=MSFT. The returned data is in JSON format, the above call would return something similar:
Historical Stock Data API
The public historical stock data API is also undocumented, thus I've had to backtrack the calls made on the Google Finance pages in order to discover it. According to this, historical stock data can be requested with the following call: http://finance.google.co.uk/finance/historical?q={stockCode}&startdate={startDate}&enddate={endDate}&output=csv.
The parameters of the call are the following:
stockCode - Stock code of the instrument, as specified on Google Finance startDate - Start date of the period in MMM+d,+yyyy format (e.g. Jan+01,+2010) endDate - End date of the period in the same format
An example query, fetching Microsoft (MSFT) stock prices between 28 Jan 2010 and 1 Feb 2010 would be the following: finance.google.co.uk/finance/historical?q=MSFT&startdate=Jan+28,+2010&enddate=Feb+01,+2010&output=csv. The output of this call is a simple comma separates values (CSV) file with a somewhat similar structure:
Date, Open, High, Low, Close, Volume
1-Feb-10,28.39,28.48,27.92,28.41,85931099
29-Jan-10,29.90,29.92,27.66,28.18,193888424
28-Jan-10,29.84,29.87,28.89,29.16,117513692
Working Around the Silverlight Cross-Domain Issue
Having mapped the REST API of Google Finance, it's obvious that in our solution we'll be querying against these URLs. However, in order for Silverlight to access cross-domain resources, the target domain needs to explicitly allow access by defining a crossdomain.xml or clientaccesspolicy.xml and specifying required settings. Since the domain we'd like to query doesn't have such permissions, Silverlight clients can't directly query this data source.
There are some solutions to this issue, all of which involve a level of proxying the request. A possible approach would be to use Yahoo pipes, another solution is to set up a proxy, e.g., using Google App Engine that would forward requests to and back from the target domain. In my solution, I chose to implement the latter solution as a proxy provides a universal solution to cross-domain issues, whereas Yahoo Pipes have to be re-configured for every scenario.
I won't be going in depth on how to set up the proxy using Google App Engine, for details on this, see the blog article on using Google App Engine as proxy for Silverlight and Flash cross-domain requests. The benefits of using Google App Engine as a proxy is that resources are not charged until a reasonable daily limit has been reached and all one needs to get started with it is a Google account.
For the example to work, I've uploaded the proxy implementation to a custom app engine domain (you'll find it in the source). To submit a proxied request to access live stock prices for Microsoft (MSFT), the following request needs to be made: http://{customdomain}.appspot.com/proxy?url=http://www.google.com/finance/info?q=MSFT.
Note on Quotas
Using Google App Engine as a proxy has been chosen purely for demo purposes. Note that if the proxy receives too many requests and the free quota is used up, it will stop functioning.
Fetching and Parsing the Stock Data
Having discovered the data source API, the next step is to fetch and parse some stock data. To do so, let's design the data structure of the app and implement querying and parsing of the data.
Designing the Data Structure
We need to represent two different data structures in our application: live data and historic data.
Live Data
The Google Finance data source returns about 10 parameters of the live data, let's just model the 4 most important of these: the stock code, timestamp, price and change since last update creating the LiveStockData class:
public class LiveStockData
{
public string StockCode { get; set; }
public DateTime TimeStamp { get; set; }
public double Price { get; set; }
public double Change { get; set; }
}
Historic Data
The Google Finance data source returns the date, high, low, open, close and volume of the data. Let's add the stock code to this information and create our class - HLOCHistoricStockData - to represent the historic HLOC data:
public class HLOCHistoricStockData
{
public string StockCode { get; set; }
public DateTime Date { get; set; }
public double Price { get; set; }
public double Open { get; set; }
public double Close { get; set; }
public double High { get; set; }
public double Low { get; set; }
public double Volume { get; set; }
}
Generic Change Events
As the application will be fetching URLs, which is an asynchronous process, the easiest way of getting notified of these changes is via subscribing to events. It is logical in our case to either return the fetched LiveStockData or HLOCHistoricStockData data structures. To do this in a type safe way, let's design a generic change handler event with generic arguments:
public delegate void StockDataEventHandler<T>(object sender, EventWithDataArgs<T> e);
public class EventWithDataArgs<T>:EventArgs
{
public T Data { get; internal set; }
public EventWithDataArgs(T data)
{
this.Data = data;
}
}
We'll use StockDataEventHandler later on as the generic event handler for change notifications.
Fetching and Parsing Live Stock Data
Having designed the data structure, it's time to go ahead and fetch the live data. The idea would be to create a class that once created, could be subscribed to. The class would constantly query the data source and fire an event whenever the price of the subscribed stock has changed.
Let's sketch out what the interface for such a class would look like and call this interface ILiveStockDataQuerier:
public interface ILiveStockDataQuerier
{
event StockDataEventHandler<livestockdata> OnDataUpdated;
void Subscribe(string stockCode);
}
Moving forward, let's implement this interface. Since we'll be wanting to constantly query, this will need to be done on the background thread, using a BackgroundWorker to avoid the application from hanging. When someone subscribes to a stock code, we also want to store that information to know which stock to keep querying.
private string _subscribedStockCode;
private BackgroundWorker _queryService;
public void Subscribe(string stockCode)
{
_subscribedStockCode = stockCode;
_queryService = new BackgroundWorker();
_queryService.DoWork += new DoWorkEventHandler((s, e) =>
{ this.QueryData(_subscribedStockCode); });
_queryService.RunWorkerAsync();
}
private void QueryData(string stockCode)
{
WebClient queryWebClient = new WebClient();
queryWebClient.OpenReadCompleted +=
new OpenReadCompletedEventHandler(liveWebClient_OpenReadCompleted);
var url = String.Format(GoogleLiveStockDataUrl
, stockCode);
var proxiedUrl = Helpers.GetProxiedUrl(url);
queryWebClient.BaseAddress = proxiedUrl;
queryWebClient.OpenReadAsync(new Uri(proxiedUrl, UriKind.Absolute));
}
Once the URL is returned, we need to parse it and fire a change event with the live stock information. I've used the popular JSON.NET library to help parse the JSON response. The code for this is as follows:
void liveWebClient_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
if (e.Error != null)
return;
var webClient = sender as WebClient;
if (webClient == null)
return;
using (StreamReader reader = new StreamReader(e.Result))
{
string contents = reader.ReadToEnd();
contents = contents.Replace("//", "");
var originalCulture = Thread.CurrentThread.CurrentCulture;
Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US");
try
{
var array = JArray.Parse(contents);
if (array.Count > 0)
{
var o = array[0];
double value = o["l"].Value<double>();
double change = o["c"].Value<double>();
DateTime date = DateTime.ParseExact(o["lt"].Value<string>(),
"MMM d, h:mmtt EST", CultureInfo.CurrentUICulture);
var data = new LiveStockData()
{
StockCode = _subscribedStockCode,
TimeStamp = date,
Price = value,
Change = change
};
if (OnDataUpdated != null)
{
OnDataUpdated(this,
new EventWithDataArgs<livestockdata>(data));
}
Thread.Sleep(1000);
this.QueryData(_subscribedStockCode);
}
}
catch (JsonReaderException ex)
{
}
}
}
Fetching and Parsing Historical Stock Data
Having implemented the class dealing with querying and parsing the live updates, let's implement the class that fetches and parses historical stock prices. We'd be expecting this class to perform the following: can be instructed to retrieve data on a given stock for a given time range, parses the response into a HLOCHistoricStockData class and raises an event passing this class on. Based on these requirements, let's sketch up the interface - IHistoricalStockDataQuerier for this class:
public interface IHistoricalStockDataQuerier
{
event StockDataEventHandler<IList<hlochistoricstockdata> >OnQueryCompleted;
void QueryData(string stockCode, DateTime startDate, DateTime endDate);
}
Implementing querying the stock data is pretty straightforward:
public void QueryData(string stockCode, DateTime startDate, DateTime endDate)
{
WebClient historicWebClient = new WebClient();
historicWebClient.OpenReadCompleted +=
new OpenReadCompletedEventHandler(historicWebClient_OpenReadCompleted);
Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US");
var url = String.Format(GoogleHistoricalStockDataUrl,
stockCode,
startDate.ToString("MMM+dd\\%2C+yyyy"),
endDate.ToString("MMM+dd\\%2C+yyyy")
);
var proxiedUrl = Helpers.GetProxiedUrl(url);
historicWebClient.BaseAddress = proxiedUrl;
historicWebClient.OpenReadAsync(new Uri(proxiedUrl, UriKind.Absolute));
}
Parsing the result isn't that tricky either as we're receiving simple csv:
void historicWebClient_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
if (e.Error != null)
return;
var webClient = sender as WebClient;
if (webClient == null)
return;
try
{
using (StreamReader reader = new StreamReader(e.Result))
{
var stockDataList = new List<hlochistoricstockdata>();
string contents = reader.ReadToEnd();
var lines = contents.Split('\n');
bool firstLine = true;
foreach (var line in lines)
{
if (firstLine)
{
firstLine = false;
continue;
}
if (line.Length == 0)
continue;
var parts = line.Split(',');
var stockData = new HLOCHistoricStockData()
{
Date = Convert.ToDateTime(parts[0]),
Open = Convert.ToDouble(parts[1]),
High = Convert.ToDouble(parts[2]),
Low = Convert.ToDouble(parts[3]),
Close = Convert.ToDouble(parts[4]),
Price = Convert.ToDouble(parts[4]),
Volume = Convert.ToDouble(parts[5]),
StockCode = GetStockCodeFromUrl(webClient.BaseAddress)
};
stockDataList.Add(stockData);
}
if (OnQueryCompleted != null)
{
OnQueryCompleted(this, new EventWithDataArgs
<IList<hlochistoricstockdata>>(stockDataList));
}
}
}
catch (Exception ex)
{
}
}
</hlochistoricstockdata>
One part of this code does need some explanation. In the live data querying example, we had the code of the queried stock in the _subscribedStockCode private variable. In the historic version, we're not doing this, instead at the result of every request, we can access it by parsing it from the original request URL, calling GetStockCodeFromUrl. This is a minor detail, however this way not only do we not have to store what stock we're querying, but also can allow multiple queries to run simultaneously for multiple stocks as when a query returns it contains the code of the stock it belongs to. The implementation of the GetStockCodeFromUrl method is the following:
private string GetStockCodeFromUrl(string url)
{
string pattern = ".*?q=([^&]+)*";
var matches = Regex.Matches(url, pattern, RegexOptions.None);
string stockName = matches[0].Groups[1].Value;
return stockName;
}
Visualizing the Fetched Stock Data
Having created the two classes - GoogleLiveStockDataQuerier and GoogleHistoricalStockDataQuerier - that query and parse the data and raise events when they have done so, we have everything needed to visualize the stocks.
Visiblox Charts for Silverlight Overview
Visualization will be done with the free version of Visiblox Charts for Silverlight. There are several Silverlight components available - the most popular being the open source Silverlight Toolkit charts. The reason I've chosen this tool in this scenario is because it comes with built-in zooming and panning and it has decent performance when rendering hundreds of points on screen.
In this project, the live stock data will be constantly updated as the chart's title, while the historical stocks will be rendered as points on a line series.
Visualizing Historical and Live Data
To visualize the data, we need to create a chart, add a line series, perform querying the data source and when the query is complete, update the line series for historical data and the chart title for the live data. I'll also be adding panning and zoom to help navigating the data.
Creating a Chart
To create a chart, we need to add a Chart object to the visual tree. We'll also be needing to create LineSeries to visualize the retrieved data. Also, in this example, I'd like to use the IList list returned by GoogleHistoricalStockDataQuerier to visualize the historic stock data without doing too much work. Thus I'll bind this business object collection to the line series by setting a BindableDataSeries as the DataSeries property of the LineSeries and setting up the correct XValueBinding and YValueBinding properties on it. This way, all I'll have to do later is set the ItemsSource on the BindableDataSeries to have the historic stock data update.
Creating the Chart with a LineSeries whose data series is binded could be done both in XAML or code behind. In this example, I'm choosing to go with XAML:
<UserControl x:Class="StockVisualization.MainPage"
(...)
xmlns:charts="clr-namespace:Visiblox.Charts;assembly=Visiblox.Charts"
>
<StackPanel x:Name="LayoutRoot" Orientation="Vertical">
<charts:Chart x:Name="StockChart" Width="600" Height="300"
LegendVisibility="Collapsed">
<charts:Chart.Series>
-->
<charts:LineSeries.DataSeries>
-->
<charts:BindableDataSeries XValueBinding="{Binding Date}"
YValueBinding="{Binding Price}"/>
</charts:LineSeries.DataSeries>
</charts:Chart.Series>
</StackPanel>
</UserControl>
Creating Input Controls
In order for the example to work, we'll need to have a way of allowing the user to specify the kind of stock to query. For simplicity, I'll be using a ComboBox with pre-defined values in this example.
Querying live and historical stock data does not happen instantly and it would be desirable to let the user know that something is happening in the background. For this reason, I've created a ProgressBar to show whenever querying is in process.
The XAML for these controls for the example is as follows:
<StackPanel x:Name="LayoutRoot" Orientation="Vertical">
<StackPanel Orientation="Horizontal" HorizontalAlignment="Center">
<TextBlock>Stock code:</TextBlock>
<ComboBox x:Name="ComboStockCode">
<ComboBoxItem Content="MSFT" IsSelected="True"/>
<ComboBoxItem Content="GOOG"/>
<ComboBoxItem Content="VOD"/>
<ComboBoxItem Content="BCS"/>
</ComboBox>
<Button x:Name="BtnUpdate" Content="Update" Click="BtnUpdate_Click"/>
</StackPanel>
<StackPanel Orientation="Horizontal" x:Name="PanelFetchingData"
Visibility="Collapsed" Margin="0,5,0,5" HorizontalAlignment="Center">
<ProgressBar IsIndeterminate="True" Width="150" Height="15"/>
</StackPanel>
<charts:Chart x:Name="StockChart" Width="600" Height="300"
LegendVisibility="Collapsed">
-->
</charts:Chart>
<TextBlock HorizontalAlignment="Center" Width="400"
TextWrapping="Wrap" Text="Drag mouse to pan, scroll with mousewheel to zoom."/>
</StackPanel>
Visualizing the Live Stock Data
Visualizing the live stock data is pretty obvious: all we need to do is create an instance of GoogleLiveStockDataQuerier and subscribe to its OnDataUpdated event. When this fires, we'll have to update the Title of the chart. The code for this is as follows:
private GoogleLiveStockDataQuerier _liveDataQuerier;
public MainPage()
{
_liveDataQuerier = new GoogleLiveStockDataQuerier();
_liveDataQuerier.OnDataUpdated +=new StockDataEventHandler
<livestockdata>(_liveDataQuerier_OnChanged);
}
void _liveDataQuerier_OnChanged(object sender, EventWithDataArgs<livestockdata> e)
{
if (e.Data != null)
{
Dispatcher.BeginInvoke(() =>
{
StockChart.Title = String.Format("{0}: {1} ({2})",
e.Data.StockCode, e.Data.Price, e.Data.Change);
}
);
}
}
Visualizing the Historical Stock Data
Visualizing the historical stock data is also pretty straightforward. When the user clicks the Update button, the QueryData method of the GoogleHistoricalStockDataQuerier instance is called. When this method is called, it will raise an OnQueryCompleted event that will return an IList in the Data member of the event. This IList can then just be set as the ItemsSource of the BindableDataSeries we have defined on the chart.
The code for all of this is much simpler than explained, the relevant parts are as follows:
private GoogleHistoricalStockDataQuerier _historicDataQuerier;
public MainPage()
{
_historicDataQuerier = new GoogleHistoricalStockDataQuerier();
_historicDataQuerier.OnQueryCompleted += new StockDataEventHandler
<ilist<hlochistoricstockdata>>(_historicDataQuerier_OnQueryCompleted);
UpdateChartData();
}
public void UpdateChartData()
{
StockChart.Title = "";
PanelFetchingData.Visibility = Visibility.Visible;
var stockCode = (ComboStockCode.SelectedItem as ComboBoxItem).Content.ToString();
_liveDataQuerier.Subscribe(stockCode);
_historicDataQuerier.QueryData(stockCode, DateTime.Now.AddMonths(-6), DateTime.Now);
}
void _historicDataQuerier_OnQueryCompleted(object sender, EventWithDataArgs
<IList<hlochistoricstockdata>> e)
{
PanelFetchingData.Visibility = Visibility.Collapsed;
(StockChart.Series[0].DataSeries as BindableDataSeries).ItemsSource = e.Data;
}
Adding Interactivity to the Chart
Until now, the example fetches the last half year's data and shows all data points on the chart. This means that we're already showing about 200 data points which might be a bit overcrowded at some points:
It would make sense to show less points on the screen and allow zooming and panning, allowing the user to navigate the charts in a more interactive way.
Zooming and panning are behaviours on Visiblox charts. In order to use them, one just needs to set them as the
Behaviour property of the chart. Since we'll be wanting to use both zooming and panning at the same time, we'll need to add a
BehaviourManager that will allow multiple behaviours being used on the chart. This can be done in XAML:
<charts:Chart x:Name="StockChart" Width="600" Height="300"
LegendVisibility="Collapsed">
<charts:Chart.Behaviour>
<charts:BehaviourManager AllowMultipleEnabled="True">
<charts:PanBehaviour YPanEnabled="False" IsEnabled="True"/>
<charts:ZoomBehaviour IsEnabled="True"/>
</charts:BehaviourManager>
</charts:Chart.Behaviour>
</charts:Chart>
In the above code sample, we've set YPanEnabled to be false on the PanBehaviour meaning that we'll only be able to pan among the X axis - it wouldn't make much sense to allow the user to pan up and down the Y axis in this case.
The only problem we're facing after having added zooming and panning is that there's not much to pan as all points are displayed on the screen by default. We can change this by setting the Range of the X axis to only show points from the last month whenever the chart is updated in the UpdateChartData method. To do so, we'll simply re-define the Range of the X axis:
if (StockChart.XAxis == null)
{
StockChart.XAxis = new DateTimeAxis();
}
StockChart.XAxis.Range = new DateTimeRange()
{ Minimum = DateTime.Now.AddMonths(-1), Maximum = DateTime.Now };
Having added these few lines, the chart can now be panned by dragging the mouse and zoomed by using the mousewheel.
Here is a screenshot of the compiled version of the example:
Thoughts on Reusing the Code
In the article, I've broken down the problem of visualizing live and historical stock data into 3 main steps:
By defining the data structures for stock data and declaring interfaces for fetching them, my goal was to allow the data provider to be easily swapped to any other (e.g. Yahoo Finance or another service with a public API).
The charting component used for visualizing the data can also easily be substituted with any other Silverlight charting library. Implementation of the visualization in this case will be specific to the component chosen. However, displaying a chart with either a line or candlestick series should not be difficult using any charting library.
Conclusion
In this article, I've implemented quite a generic way of fetching and visualizing live and historic stock data. I've used a specific data provider and specific charting component during the implementation. The solution was designed to easily enable changing both the data source or the data visualization library. You can download the source code here.
I hope you have found this article useful. Feel free to leave comments on it below.
History
- 19th January, 2011: Initial post
