Introduction
Google has a number of services available for which no APIs exist currently. I aim to make APIs for such services, the first of those being Google Bookmarks. After lots of hours spent intercepting HTTP requests, I have finally created a class capable of retrieving, adding, and modifying bookmarks for a given account. To date, I have not found a single piece of code outside of FireFox extensions for utilizing the Google Bookmarks service. I hope you find this of use. I have some rather interesting ideas for this class so watch out for more articles.
Using the Class
There are only 5 steps to implementing this control, 4 if you only intend to read items.
- Add the
GoogleBookmarks
class to your project - Declare an instance of the class
- Define a function to handle the
BookmarksRefreshed
event - Initialize the class with the username and password
- Interact with the bookmarks
Declare an Instance
Because we will assign an event handler to this, you will want to declare it globally:
GBDemoClass.GoogleBookmarks GBookmarks = new GoogleBookmarks();
Define an event handler in the Form Load
event:
private void Form1_Load(object sender, EventArgs e)
{
GBookmarks.BookmarksRefreshed += new EventHandler<bookmarksrefreshedeventargs />(GBookmarks_BookmarksRefreshed);}
}
Handle the Event
In the event handler, the only var passed is e.Username
. This would be of use if you had multiple class instances for using multiple accounts, you know which one is being refreshed if they share a handler. I suppose it would be easier and more efficient if the bookmarks were provided as an argument somehow, but for now this works. Here we reference the global GBookmarks
object declared above. We iterate through using a foreach
and add the entries to our ListView
.
void GBookmarks_BookmarksRefreshed(object sender, BookmarksRefreshedEventArgs e)
{
listView1.Items.Clear();
foreach (GBDemoClass.BookmarkProperties bookmark in GBookmarks)
{
ListViewItem iBookmark = new ListViewItem();
iBookmark.Text = bookmark.Title;
iBookmark.SubItems.Add(bookmark.URL);
iBookmark.SubItems.Add(bookmark.Category);
iBookmark.Subitems.Add(bookmark.Id);
listView1.Items.Add(iBookmark);
}
}
Now we just need to initialize the class, call Refresh
, sit back and let it all happen.
GBookmarks.Init("UserName", "Password");
GBookmarks.Refresh();
When the bookmarks have been retrieved, the BookmarksRefreshed
event is fired and your list is updated. This can easily be applied to a ToolStrip or other similar control to build a menu.
Add / Edit / Remove
GBookmarks.Add(string Category, string Title, string URL);
GBookmarks.Remove(string ID);
The Add
function handles both adding and editing. The key seems to be the URL. Adding any combination of Category / Title that has a URL that already exists in another bookmark will modify it rather than add a duplicate. This is how Google has designed the service and is not done by my code. I have a separate function for removing, however the same method applies, "if you were to call Add
with blank Category and Title but URL filled in, you would remove the entry". I swear this was the case yesterday, but while re-writing this article, I have found the previous sentence to be false. Today I found that to remove an entry a different set of POST vars are used, and the ID
is now needed.
Additionally, Google has integrity checks on the server side and you are unable to add a URL that would not be valid.
How It All Works
For those who want to know more about how this works behind the scenes, read on.
Initializing the Class
During initialization, the username
and password
are passed to the Init
function. Note that if a login cookie exists, calling Init
more than once with alternate credentials will not produce the desired results. I've made the Login
method public
to make the object reusable for multiple accounts. The use of ArrayLists
from the previous version has been removed. As well, Refresh
is no longer called at the end of Init
.
public void Init(string username, string password)
{
if (string.IsNullOrEmpty(username))
{
throw new ArgumentNullException("username");
}
if (string.IsNullOrEmpty(password))
{
throw new ArgumentNullException("password");
}
_Username = username;
_Password = password;
}
Before calling Refresh
, you can use the UseTimer
property (Boolean
) to determine whether or not to utilize the auto-refresh timer. This is used in conjunction with the UpdateInterval
property (Int
) that sets the update time in milliseconds. The default interval is 10 minutes. Note that both UseTimer
and UpdateInterval
can be set at any time prior to or after initialization but the change will not be reflected until the next time Refresh
is called.
Let's move on to the next function, Refresh
.
public bool Refresh()
{
if (string.IsNullOrEmpty(_Username) || string.IsNullOrEmpty(_Password))
{
throw new InvalidOperationException("Username or Password not set");
}
_Bookmarks.Clear();
if (!UpdateBookmarks())
return false;
OnBookmarksRefreshed(_Username);
UpdateTimer();
return true;
}
Prior to triggering the BookmarksRefreshed
event, the above code first calls the UpdateBookmarks
function, and later processes any changes to the timer, or returns false
if there was an error. You will notice this class tends to be quite layered. That is, there is a fairly complex call stack once the process is begun, so try to follow along. Next we will look at UpdateBookmarks
.
private bool UpdateBookmarks()
{
CheckLogin();
DownloadBookmarksRSS();
return ParseRSS();
}
private void CheckLogin()
{
if (!ValidateCookies())
Login();
}
In this function, we first determine if we are logged in already or not. This is done with the use of cookies and the CheckLogin
function. The process of tracking the cookie took many trials and many different approaches to the task. In the end, we check for a cookie from the correct URL with a name of SID
and a value not equal to EXPIRED
. I will not paste the function (ValidateCookies
) here so examine the source if you are interested.
Our next step is to perform the login. We do this in Login
:
private void Login()
{
string PostData = EncodePostData(
new string[] { "service", "nui", "hl", "Email", "Passwd",
"PersistentCookie", "rmShown", "continue" },
new string[] { "bookmarks", "1", "en", _Username, _Password, "yes", "1",
"http://www.google.com/bookmarks/lookup%3Foutput%3Dxml%26num%3D10000&" }
);
POST(_BookmarksLoginURL, PostData);
}
This function URLEncodes the post data and calls POST
, which associates the Cookie Container, sets various headers and submits the POST
data. After successfully calling this function, ValidateCookies
would return true
indicating we are logged in. If you want to see how the POST
or EncodePostData
functions work, again, examine the code.
Now it is time to actually retrieve the bookmarks. This is done in the DownloadBookmarksRSS
function. Let's have a look at it, shall we?
private void DownloadBookmarksRSS()
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_BookmarksRssUri);
request.CookieContainer = _CookieContainer;
using (WebResponse response = request.GetResponse())
using (StreamReader reader =
new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
_Data = reader.ReadToEnd();
}
}
catch
{
_Data = string.Empty;
}
}
_BookmarksRssUri
and _Data
are defined globally if you're wondering where they came from.
Pretty simple huh? Lucky for us, Google now includes the GUID
and ID
tags in the RSS feed. Previously it was necessary to fetch and parse both the RSS feed and XML feed to obtain all required information. If anything goes wrong in the DownloadBookmarksRSS
function, we set _Data
to an empty string
causing ParseRSS
to return false
below. If everything goes OK, we now call ParseRSS
and add populate our bookmarks.
private bool ParseRSS()
{
if (!string.IsNullOrEmpty(_Data))
{
string title = string.Empty;
string url = string.Empty;
string category = string.Empty;
string guid = string.Empty;
string id = string.Empty;
using (XmlReader reader = XmlReader.Create(new StringReader(_Data)))
{
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Element)
{
switch (reader.Name)
{
case "item":
if (title != string.Empty)
if (id != string.Empty)
_Bookmarks.Add(new BookmarkProperties(id, guid, title, category, url));
title = string.Empty;
url = string.Empty;
category = string.Empty;
guid = string.Empty;
id = string.Empty;
break;
case "title":
title = reader.ReadString();
break;
case "link":
url = reader.ReadString();
break;
case "smh:signature":
_Signature = reader.ReadString();
break;
case "guid":
guid = reader.ReadString();
break;
case "smh:bkmk_id":
id = reader.ReadString();
break;
case "smh:bkmk_label":
category = reader.ReadString();
break;
}
}
}
if (title != string.Empty)
if (id != string.Empty)
_Bookmarks.Add(new BookmarkProperties(id, guid, title, category, url));
}
return true;
}
return false;
}
We must check for the existence of an ID
with the RSS feed because there will always be an entry by Google with a link to http://google.com/bookmarks. This link will not have an ID
.
The RSS data is received as XML, so here we use an XmlReader
to iterate through all of the nodes. In the last version, it was known that each entry will begin with an item
tag, and end with a smh:bkmk_label
tag. While adding the ability to add and modify bookmarks, I discovered that the smh:bkmk_label
tag is not present if the bookmark is not in a category. Previously, we would consider the entry read when the label tag was found. This effectively hid all entries that lacked a category. I am surprised nobody else caught this by now. To work around this, it now performs a check each time a new item is read. If title is not empty then add what we've got. As well, we have to do the same check after all items have been read or we will miss the last one. After capturing all of the fields, we then call _Bookmarks.Add
to add it to the list. Eventually I will clean up this function to remove the case
statements and read the fields directly from the XML feed rather than the RSS feed.
After all of these functions have executed, the BookmarksRefreshed
event is fired and you may act on the available data. The only value provided to the event handler is the Username
, in case multiple objects are being used simultaneously on different accounts.
Adding and Editing
As mentioned before, the Add
method is used for both adding and updating an entry. It appears that each bookmark must have a unique URL. Also, URLs are case sensitive, so you could have an entry for google.com as well as Google.com. An attempt to add an entry with a URL that already exists will result in its category and title changed to whatever is being added.
public string Add(string Category, string Name, string URL)
{
CheckLogin();
string PostString = EncodePostData(
new string[] { "q", "title", "labels", "zx" },
new string[]
{ URL, Name, Category, Convert.ToString(DateTime.Now.Second * 3175) }
);
WebResponse response = POST("http://www.google.com/bookmarks/mark", PostString);
string respData;
if (response == null)
respData = "";
else
using (StreamReader reader =
new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
respData = reader.ReadToEnd();
}
return respData;
}
You'll notice the var zx
when adding and editing an entry. This var is some form of time stamp but I have not managed to determine how it is calculated. I do know that it loops multiple times a day. As such, we just set it to the current second * 3175. Thanks to Marschills for providing some information on this.
Previously (less than 24 hours ago!) I found that adding an existing URL with a blank category and title caused it to be removed; this does not seem to be the case anymore and if done now will only cause the category to be cleared. Through some more sniffing, I determined that removing an entry is identical to adding except the POST
vars are different. To remove an entry we must set dlq
to the entry’s ID
, op
to remove
, and zx
to our random value.
Thanks to Steven Hansen
Ok, so you saw the _Bookmarks
object up there. Steve Hansen’s post in the forums below turned the main class into an enumerable one and modified the BookmarkProperties
class, holding the items in a List
and removing the use of ArrayLists
.
_Bookmarks
is defined globally as such:
private List<bookmarkproperties /> _Bookmarks = new List<bookmarkproperties />();
Additionally, the GoogleBookmarks
class is made enumerable by declaring as:
public class GoogleBookmarks : IEnumerable<bookmarkproperties />
Have a look at the code for any details I have not mentioned in here.
In Closing
This class took a lot of trial and error and a lot of research to complete. As stated previously, I know of no other control, documentation, or so much as an example of accomplishing this task in any .NET language. I hope you find this article useful and if you do, please vote! I also welcome any and all feedback or comments. Find a bug? Know of a way to improve it? Post it here and I will most likely incorporate it into the next release. Enjoy!
Current Limitations
- Google allows multiple Categories (Labels) to be assigned to a single bookmark. As it is, this class does not handle multiple categories for a single item and the last read will be used. This will be handled in the next version.
History
- September 4, 2007 – Initial posting
- September 5, 2007 - Corrected all the evil things the post template did to the article!
- March 23, 2008 – Second version. Implemented
Add
/ Edit
/ Remove