Introduction
Google has a number of services available for which no APIs exist currently. I aim to make APIs for such services, the first of those being Google Bookmarks. After lots of hours spent intercepting HTTP requests, I have finally created a class capable of retrieving, adding, and modifying bookmarks for a given account. To date, I have not found a single piece of code outside of FireFox extensions for utilizing the Google Bookmarks service. I hope you find this of use. I have some rather interesting ideas for this class so watch out for more articles.
Using the Class
There are only 5 steps to implementing this control, 4 if you only intend to read items.
- Add the
GoogleBookmarks class to your project - Declare an instance of the class
- Define a function to handle the
BookmarksRefreshed event - Initialize the class with the username and password
- Interact with the bookmarks
Declare an Instance
Because we will assign an event handler to this, you will want to declare it globally:
GBDemoClass.GoogleBookmarks GBookmarks = new GoogleBookmarks();
Define an event handler in the Form Load event:
private void Form1_Load(object sender, EventArgs e)
{
GBookmarks.BookmarksRefreshed += new EventHandler<bookmarksrefreshedeventargs />(GBookmarks_BookmarksRefreshed);}
}
Handle the Event
In the event handler, the only var passed is e.Username. This would be of use if you had multiple class instances for using multiple accounts, you know which one is being refreshed if they share a handler. I suppose it would be easier and more efficient if the bookmarks were provided as an argument somehow, but for now this works. Here we reference the global GBookmarks object declared above. We iterate through using a foreach and add the entries to our ListView.
void GBookmarks_BookmarksRefreshed(object sender, BookmarksRefreshedEventArgs e)
{
listView1.Items.Clear();
foreach (GBDemoClass.BookmarkProperties bookmark in GBookmarks)
{
ListViewItem iBookmark = new ListViewItem();
iBookmark.Text = bookmark.Title;
iBookmark.SubItems.Add(bookmark.URL);
iBookmark.SubItems.Add(bookmark.Category);
iBookmark.Subitems.Add(bookmark.Id);
listView1.Items.Add(iBookmark);
}
}
Now we just need to initialize the class, call Refresh, sit back and let it all happen.
GBookmarks.Init("UserName", "Password");
GBookmarks.Refresh();
When the bookmarks have been retrieved, the BookmarksRefreshed event is fired and your list is updated. This can easily be applied to a ToolStrip or other similar control to build a menu.
Add / Edit / Remove
GBookmarks.Add(string Category, string Title, string URL);
GBookmarks.Remove(string ID);
The Add function handles both adding and editing. The key seems to be the URL. Adding any combination of Category / Title that has a URL that already exists in another bookmark will modify it rather than add a duplicate. This is how Google has designed the service and is not done by my code. I have a separate function for removing, however the same method applies, "if you were to call Add with blank Category and Title but URL filled in, you would remove the entry". I swear this was the case yesterday, but while re-writing this article, I have found the previous sentence to be false. Today I found that to remove an entry a different set of POST vars are used, and the ID is now needed.
Additionally, Google has integrity checks on the server side and you are unable to add a URL that would not be valid.
How It All Works
For those who want to know more about how this works behind the scenes, read on.
Initializing the Class
During initialization, the username and password are passed to the Init function. Note that if a login cookie exists, calling Init more than once with alternate credentials will not produce the desired results. I've made the Login method public to make the object reusable for multiple accounts. The use of ArrayLists from the previous version has been removed. As well, Refresh is no longer called at the end of Init.
public void Init(string username, string password)
{
if (string.IsNullOrEmpty(username))
{
throw new ArgumentNullException("username");
}
if (string.IsNullOrEmpty(password))
{
throw new ArgumentNullException("password");
}
_Username = username;
_Password = password;
}
Before calling Refresh, you can use the UseTimer property (Boolean) to determine whether or not to utilize the auto-refresh timer. This is used in conjunction with the UpdateInterval property (Int) that sets the update time in milliseconds. The default interval is 10 minutes. Note that both UseTimer and UpdateInterval can be set at any time prior to or after initialization but the change will not be reflected until the next time Refresh is called.
Let's move on to the next function, Refresh.
public bool Refresh()
{
if (string.IsNullOrEmpty(_Username) || string.IsNullOrEmpty(_Password))
{
throw new InvalidOperationException("Username or Password not set");
}
_Bookmarks.Clear();
if (!UpdateBookmarks())
return false;
OnBookmarksRefreshed(_Username);
UpdateTimer();
return true;
}
Prior to triggering the BookmarksRefreshed event, the above code first calls the UpdateBookmarks function, and later processes any changes to the timer, or returns false if there was an error. You will notice this class tends to be quite layered. That is, there is a fairly complex call stack once the process is begun, so try to follow along. Next we will look at UpdateBookmarks.
private bool UpdateBookmarks()
{
CheckLogin();
DownloadBookmarksRSS();
return ParseRSS();
}
private void CheckLogin()
{
if (!ValidateCookies())
Login();
}
In this function, we first determine if we are logged in already or not. This is done with the use of cookies and the CheckLogin function. The process of tracking the cookie took many trials and many different approaches to the task. In the end, we check for a cookie from the correct URL with a name of SID and a value not equal to EXPIRED. I will not paste the function (ValidateCookies) here so examine the source if you are interested.
Our next step is to perform the login. We do this in Login:
private void Login()
{
string PostData = EncodePostData(
new string[] { "service", "nui", "hl", "Email", "Passwd",
"PersistentCookie", "rmShown", "continue" },
new string[] { "bookmarks", "1", "en", _Username, _Password, "yes", "1",
"http://www.google.com/bookmarks/lookup%3Foutput%3Dxml%26num%3D10000&" }
);
POST(_BookmarksLoginURL, PostData);
}
This function URLEncodes the post data and calls POST, which associates the Cookie Container, sets various headers and submits the POST data. After successfully calling this function, ValidateCookies would return true indicating we are logged in. If you want to see how the POST or EncodePostData functions work, again, examine the code.
Now it is time to actually retrieve the bookmarks. This is done in the DownloadBookmarksRSS function. Let's have a look at it, shall we?
private void DownloadBookmarksRSS()
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_BookmarksRssUri);
request.CookieContainer = _CookieContainer;
using (WebResponse response = request.GetResponse())
using (StreamReader reader =
new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
_Data = reader.ReadToEnd();
}
}
catch
{
_Data = string.Empty;
}
}
_BookmarksRssUri and _Data are defined globally if you're wondering where they came from.
Pretty simple huh? Lucky for us, Google now includes the GUID and ID tags in the RSS feed. Previously it was necessary to fetch and parse both the RSS feed and XML feed to obtain all required information. If anything goes wrong in the DownloadBookmarksRSS function, we set _Data to an empty string causing ParseRSS to return false below. If everything goes OK, we now call ParseRSS and add populate our bookmarks.
private bool ParseRSS()
{
if (!string.IsNullOrEmpty(_Data))
{
string title = string.Empty;
string url = string.Empty;
string category = string.Empty;
string guid = string.Empty;
string id = string.Empty;
using (XmlReader reader = XmlReader.Create(new StringReader(_Data)))
{
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Element)
{
switch (reader.Name)
{
case "item":
if (title != string.Empty)
if (id != string.Empty)
_Bookmarks.Add(new BookmarkProperties(id, guid, title, category, url));
title = string.Empty;
url = string.Empty;
category = string.Empty;
guid = string.Empty;
id = string.Empty;
break;
case "title":
title = reader.ReadString();
break;
case "link":
url = reader.ReadString();
break;
case "smh:signature":
_Signature = reader.ReadString();
break;
case "guid":
guid = reader.ReadString();
break;
case "smh:bkmk_id":
id = reader.ReadString();
break;
case "smh:bkmk_label":
category = reader.ReadString();
break;
}
}
}
if (title != string.Empty)
if (id != string.Empty)
_Bookmarks.Add(new BookmarkProperties(id, guid, title, category, url));
}
return true;
}
return false;
}
We must check for the existence of an ID with the RSS feed because there will always be an entry by Google with a link to http://google.com/bookmarks. This link will not have an ID.
The RSS data is received as XML, so here we use an XmlReader to iterate through all of the nodes. In the last version, it was known that each entry will begin with an item tag, and end with a smh:bkmk_label tag. While adding the ability to add and modify bookmarks, I discovered that the smh:bkmk_label tag is not present if the bookmark is not in a category. Previously, we would consider the entry read when the label tag was found. This effectively hid all entries that lacked a category. I am surprised nobody else caught this by now. To work around this, it now performs a check each time a new item is read. If title is not empty then add what we've got. As well, we have to do the same check after all items have been read or we will miss the last one. After capturing all of the fields, we then call _Bookmarks.Add to add it to the list. Eventually I will clean up this function to remove the case statements and read the fields directly from the XML feed rather than the RSS feed.
After all of these functions have executed, the BookmarksRefreshed event is fired and you may act on the available data. The only value provided to the event handler is the Username, in case multiple objects are being used simultaneously on different accounts.
Adding and Editing
As mentioned before, the Add method is used for both adding and updating an entry. It appears that each bookmark must have a unique URL. Also, URLs are case sensitive, so you could have an entry for google.com as well as Google.com. An attempt to add an entry with a URL that already exists will result in its category and title changed to whatever is being added.
public string Add(string Category, string Name, string URL)
{
CheckLogin();
string PostString = EncodePostData(
new string[] { "q", "title", "labels", "zx" },
new string[]
{ URL, Name, Category, Convert.ToString(DateTime.Now.Second * 3175) }
);
WebResponse response = POST("http://www.google.com/bookmarks/mark", PostString);
string respData;
if (response == null)
respData = "";
else
using (StreamReader reader =
new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
respData = reader.ReadToEnd();
}
return respData;
}
You'll notice the var zx when adding and editing an entry. This var is some form of time stamp but I have not managed to determine how it is calculated. I do know that it loops multiple times a day. As such, we just set it to the current second * 3175. Thanks to Marschills for providing some information on this.
Previously (less than 24 hours ago!) I found that adding an existing URL with a blank category and title caused it to be removed; this does not seem to be the case anymore and if done now will only cause the category to be cleared. Through some more sniffing, I determined that removing an entry is identical to adding except the POST vars are different. To remove an entry we must set dlq to the entry’s ID, op to remove, and zx to our random value.
Thanks to Steven Hansen
Ok, so you saw the _Bookmarks object up there. Steve Hansen’s post in the forums below turned the main class into an enumerable one and modified the BookmarkProperties class, holding the items in a List and removing the use of ArrayLists.
_Bookmarks is defined globally as such:
private List<bookmarkproperties /> _Bookmarks = new List<bookmarkproperties />();
Additionally, the GoogleBookmarks class is made enumerable by declaring as:
public class GoogleBookmarks : IEnumerable<bookmarkproperties />
Have a look at the code for any details I have not mentioned in here.
In Closing
This class took a lot of trial and error and a lot of research to complete. As stated previously, I know of no other control, documentation, or so much as an example of accomplishing this task in any .NET language. I hope you find this article useful and if you do, please vote! I also welcome any and all feedback or comments. Find a bug? Know of a way to improve it? Post it here and I will most likely incorporate it into the next release. Enjoy!
Current Limitations
- Google allows multiple Categories (Labels) to be assigned to a single bookmark. As it is, this class does not handle multiple categories for a single item and the last read will be used. This will be handled in the next version.
History
- September 4, 2007 – Initial posting
- September 5, 2007 - Corrected all the evil things the post template did to the article!
- March 23, 2008 – Second version. Implemented
Add / Edit / Remove
