SharePoint 2010 Client Object Model, Part 1

• Download ConsoleDemo - 11.72 KB
• Download SPDemo - 511.3 KB

SharePoint 2010 Client Object Model

In previous versions of SharePoint when it was necessary to access ListItems or other objects from within a SharePoint environment the only choice available was to use the server object model, perhaps from the code behind in a webpart or application page, or in a Service running on the SharePoint machine. Outside of a SharePoint environment, the only option was to use Web Services with all of the inherent limitations and inefficiencies.

Although the Web Services are still available, and can be useful for some situations, there is a better way of working with SharePoint 2010; the Client Object Model. The Client Object Model allows developers to utilize SharePoint directly from .NET managed code, in a Windows WPF application for instance, Silverlight, in browser and out of browser, or from JavaScript without the need to directly utilize Web Services. This article will discuss the inner workings of the Client OM and how to use it effectively, not just to work Lists and ListItems, but other SharePoint objects not available via Web Services. This article will focus on the details while Part 2 will focus practical usage in all supported environments.

Supported Environments

The SharePoint Client Object Model is supported in three environments; .NET managed code, JavaScript and SilverLight.

.NET Managed Code Assemblies

To use the Client Object Model in .NET Managed code you must add references to two assemblies in your Visual Studio project. Both of these assemblies can be found in [SharePoint Root]\ISAPI folder, which is typically C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI\.

Microsoft.SharePoint.Client.dll (282kb)
Microsoft.SharePoint.Client.Runtime.dll (146kb)

As you can see from the indicated sizes these are not large assemblies and won't bloat an application very much. Despite their relatively diminutive size however they pack a great deal of punch, as you will soon see.

Silverlight Assemblies

Just like .NET managed code, to use the SharePoint Client OM from within a Silverlight application you must add two assemblie references. These assemblies are a little trickier to find and are located in the [SharePoint Root]\Templates\Layouts\ClientBin folder.

Microsoft.SharePoint.Client.Silverlight.dll (266kb)
Microsoft.SharePoint.Client.Silverlight.Runtime.dll (142kb)

The location is understandable considering the the Layout folder is mapped to each SharePoint Web Application and the heavy use of Silverlight within SharePoint 2010. You will notice the size of these assemblies is just slightly smaller than their .NET managed code counterparts. Despite this, there is no difference in functionality.

JavaScript code

To make use of the Client OM in JavaScript you must include the SP.js file on the page. The corresponding SP.Runtime.js will be added automatically by SharePoint.

FormDigest and Security

SharePoint uses the FormDigest control to generate a security token based on the user, site and time period and uses it to validate any action posting data and updating the content database. This control is required to be present on all pages where the Client OM is being used. However, since it is included in the SharePoint master pages, v4.master and default.master, it should not be an issue. Just keep it in mind when constructing your own master pages.

Client OM Architecture

Before getting to actual usage we'll take a brief look at the architecture of the Client Object Model to understand how it functions.

As can be seen in the above diagram all versions of the Client OM go through the WCF Web Service named Client.svc. This Web Service is located, along with all other SharePoint Web Services, in the folder [SharePoint Root]\ISAPI. The Client OM is responsible for packaging any requests into XML and calling the Web Server, however, when this call takes place is controlled by the developer as you will see. The response from the Client.svc Web Service is sent as JSON and the Client OM is responsible for transforming this into appropriate objects for use in the environment the call was made from.

The Client.svc Web Service has one method, ProcessQuery which takes a Stream object, an XML formatted stream as indicated above, and returns a Stream, JSON formatted. The implementation for this method can be found in the assembly Microsoft.SharePoint.Client.ServerRuntime.dll and the private class ClientRquestServiceImpl. Delving into this method, the first thing you find is an authentication check.

if (Utility.ShouldForceAuthentication(HttpContext.Current))
{
    WebOperationContext.Current.OutgoingResponse.StatusCode = HttpStatusCode.Unauthorized;
    WebOperationContext.Current.OutgoingResponse.SuppressEntityBody = true;
    return new MemoryStream();
}

internal static bool ShouldForceAuthentication(HttpContext httpContext)
{
    return (((httpContext != null) 
      && (string.Compare(httpContext.Request.Headers["X-RequestForceAuthentication"], 
           "true", StringComparison.OrdinalIgnoreCase) == 0)) 
      && (((httpContext.User == null) || (httpContext.User.Identity == null)) 
            || !httpContext.User.Identity.IsAuthenticated));
}

The Client OM uses Windows Authentication by default, however, later I'll show how to utilize Claims Based Authentication. As shown in this code, authentication is not performed by this Web Service, it simply returns and Unauthorized status if authentication has not been completed previously. Authentication is handled by one of the legacy Web Services, Sites.asmx. When authentication has been confirmed the response content type is set the JSON as stated previously...

WebOperationContext.Current.OutgoingResponse.ContentType = "application/json";
WebOperationContext.Current.OutgoingResponse.Headers.Add("X-Content-Type-Options", "nosniff");

...then the XML from the input stream is processed.

using (ClientMethodsProcessor processor = new ClientMethodsProcessor(inputStream, host))
{
    processor.Process();
    stream = processor.CreateResultUTF8Stream();
}

The XML that comes to this Web Service method is a collection ObjectPath elements with attributes and child elements that detail the request being made. The ClientMethodsProcessor constructor method creates a Dictionary of these elements which is used in the Process method and eventually in the ProcessOne method based on the type of action the element represents.

private void ProcessOne(XmlElement xe)
{
    this.ThrowIfTimeout();
    if (xe.Name == "Query")
    {
        this.ProcessQuery(xe);
    }
    else if (xe.Name == "Method")
    {
        this.ProcessMethod(xe);
    }
    else if (xe.Name == "StaticMethod")
    {
        this.ProcessStaticMethod(xe);
    }
    else if (xe.Name == "SetProperty")
    {
        this.ProcessSetProperty(xe);
    }
    else if (xe.Name == "SetStaticProperty")
    {
        this.ProcessSetStaticProperty(xe);
    }
    else if (xe.Name == "ObjectPath")
    {
        this.ProcessInstantiateObjectPath(xe);
    }
    else if (xe.Name == "ObjectIdentityQuery")
    {
        this.ProcessObjectIdentityQuery(xe);
    }
    else if ((xe.Name == "ExceptionHandlingScope") || (xe.Name == "ExceptionHandlingScopeSimple"))
    {
        this.ProcessExceptionHandlingScope(xe);
    }
    else if (xe.Name == "ConditionalScope")
    {
        this.ProcessConditionalScope(xe);
    }
}

You'll see later during actual implementation and usage how this all fits together and functions.

Using the Client Object Model

The Client OM is very similar to the Server Object Model most SharePoint developers should be familiar with. As the table below shows the class names are essentially the same, except for ClientContext, just without the SP prefix. There are some differences and some classes specific to the Client OM, which I'll highlight as we go. All of the objects you will work with are derived from ClientObject which contains properties used for creating the ObjectPath used to form a query as described in the previous section.

Server OM	Client OM
SPContext	ClientContext
SPSite	Site
SPWeb	Web
SPList	List
SPListItem	ListItem

A very simple example of using the Client OM in Managed code is as follows

using(ClientContext ctx = new ClientContext("http://mysite"))
{
    ctx.Load(ctx.Site);
    ctx.ExecuteQuery();
}

or with JavaScript

var ctx = SP.ClientContext.get_current();
var site = ctx.get_site();
ctx.load(site);
ctx.executeQueryAsync(Function.createDelegate(this, OnSuccess),
            Function.createDelegate(this, OnFail));

Both examples are almost the same with the prominent exception being the asynchronous method for JavaScript. The request is, of course, out of band in the browser so requires asynchronous processing. This is the same for Silverlight. The one thing to note in either of these examples is that the Web Service is not actually called until the Execute method is called. As I'll show in the examples this is very useful when forming complex queries and optimizing the amount of data transmitted rather than making several calls to the Web Services.

Now I'll take a look at the individual components of this process. I'll use a Managed Code Console application for these examples but in Part 2 of this article I'll demonstrate usage with JavaScript and Silverlight.

Creating a ClientContext

Just as with the Server OM, when using the Client OM the first thing that must be done is to create a context to operate within, a ClientContext in this case.

using(ClientContext ctx = new ClientContext("http://mysite"))
{
}

The input to the constructor must be a full URL. The ClientObject is derived from ClientRunTimeContext which implements IDisposable so it should be properly disposed after use. This isn't as critical as when using the Server OM but still important for proper usage. The ClientContext object contains properties for the Site and Web for the context being used, along with an overridden ExecuteQuery method, which I'll cover shortly. The ClientRunTimeContext contains methods such as Load and LoadQuery, which, again, I'll cover shortly. It should be noted that the default authentication method for the Client OM is Windows Authentication. If the site you are working with uses Forms Based Authentication there are some additional steps that must be taken. I'll get to that also.

Once you have a ClientContext it can be used to get references to the Site or to a Web within the site. As mentioned above Site and Web are properties of the ClientContext object. Just as with the Server OM you can also get Web objects by using the RootWeb property or OpenWeb method of the Site. One method not available in the Server OM is the OpenWebById method which takes the Guid ID for the Web that will be opened.

public Web OpenWebById(Guid gWebId)
{
    return new Web(base.Context, new ObjectPathMethod(base.Context, base.Path, "OpenWebById", new object[] { gWebId }));
}

After the ClientContext has been instantiated there are few properties available, the most important of which is PendingRequest. This property contains a reference to a ClientRequest object, which, as the name implies, is the object that will make the actual query to SharePoint and process the results. It does this via a WebRequestExecutor object exposed by the RequestExecutor property. As described above all requests are processed via the Client.svc Web Service. You can see this in the WebRequest property when debugging an application.

ctx.PendingRequest.RequestExecutor.WebRequest.RequestUri
// http://mysite/_vti_bin/client.svc/ProcessQuery

Creating a request: Load

Having a ClientContext is only the beginning of using the Client OM. As described above requests are made and processed based on a collection of ObjectPath elements. These are constructed by using the Load or LoadQuery methods.

public void Load<T>(T clientObject, params Expression<Func<T, object>>[] retrievals) where T: ClientObject;
public IEnumerable<T> LoadQuery<T>(ClientObjectCollection<T> clientObjects) where T: ClientObject;
public IEnumerable<T> LoadQuery<T>(IQueryable<T> clientObjects) where T: ClientObject;

using(ClientContext ctx = new ClientContext(URL))
{
    ctx.Load(ctx.Site);
    ctx.ExecuteQuery();
}

Both of of these methods (LoadQuery has two overloads) take as parameters objects that are derived from ClientObject. Remember from above this object contains the information necessary to form the ObjectPath for the object that will be used in the query. One thing to keep in mind when working with the Client OM is that no matter how many Load or LoadQuery statements you have, nothing is actually done until a call to ExecuteQuery is made. This allows you to build complex queries using multiple objects but lets the Client OM code construct the ObjectPaths required without extraneous data. Taking a look at the Load method a ClientObject and params, which, of course, is optional. In the sample above the Site property is used to pass a Site object, which, as noted early, is derived from ClientObject. The retrievals parameter is used to specify what properties or objects should be in the return. I'll get to that in a moment but first I'll look inside the Load method.

public void Load<T>(T clientObject, params Expression<Func<T, object>>[] retrievals) where T: ClientObject
{
    if (clientObject == null)
    {
        throw new ArgumentNullException("clientObject");
    }
    DataRetrieval.Load<T>(clientObject, retrievals);
}


internal static void Load<T>(T clientObject, params Expression<Func<T, object>>[] retrievals) where T: ClientObject
{
    ClientObjectCollection objects = clientObject as ClientObjectCollection;
    if ((retrievals == null) || (retrievals.Length == 0))
    {
        clientObject.Query.SelectAllProperties();
        if (objects != null)
        {
            clientObject.Query.ChildItemQuery.SelectAllProperties();
        }
    }
    else
    {
     ...removed for clarity
    }
}

When the Load method is called the parameters are passed to the Load method of an internal object, DataRetrieval. The first thing it checks for is whether the retrievals parameter exists. If not, or it is empty, the API will assume all properties are to be retrieved. Using Fiddler you can inspect the request and results after ExecuteQuery is called and you should see something similar to this.

POST http://mySite/_vti_bin/client.svc/ProcessQuery HTTP/1.1
X-RequestDigest: 0x8D593CC2539B9[truncated for space]0000
Content-Type: text/xml
X-RequestForceAuthentication: true
Host: mySite
Content-Length: 554
Expect: 100-continue

<Request AddExpandoFieldTypeSuffix="true" SchemaVersion="14.0.0.0" LibraryVersion="14.0.4762.1000" 
        ApplicationName=".NET Library" 
        xmlns="http://schemas.microsoft.com/sharepoint/clientquery/2009">
    <Actions>
        <ObjectPath Id="2" ObjectPathId="1" />
        <ObjectPath Id="4" ObjectPathId="3" />
        <Query Id="5" ObjectPathId="3">
            <Query SelectAllProperties="true">
                <Properties />
            </Query>
        </Query>
    </Actions>
    <ObjectPaths>
        <StaticProperty Id="1" TypeId="{3747adcd-a3c3-41b9-bfab-4a64dd2f1e0a}" Name="Current" />
        <Property Id="3" ParentId="1" Name="Site" />
    </ObjectPaths>
</Request>

As you can see ObjectPath elements have been constructed from the ClientObject. Note the SelectAllProperties="true". The results are as follows:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/json
Server: Microsoft-IIS/7.5
SPRequestGuid: 22e3f795-a5c9-492d-a551-3e950b8f1c10
Set-Cookie: WSS_KeepSessionAuthenticated={4e4cffb3-203e-4857-8f5a-90a4b18789a4}; path=/
X-SharePointHealthScore: 0
X-Content-Type-Options: nosniff
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.6029
Content-Length: 460

[
{
"SchemaVersion":"14.0.0.0","LibraryVersion":"14.0.6106.5001","ErrorInfo":null
},2,{
"IsNull":false
},4,{
"IsNull":false
},5,{
"_ObjectType_":"SP.Site",
"Id":"\/Guid(2aa4b949-05b2-456e-8392-c694d41c95d0)\/",
"ServerRelativeUrl":"\u002f",
"Url":"http:\u002f\u002fmySite",
"UIVersionConfigurationEnabled":false,
"MaxItemsPerThrottledOperation":5000,
"AllowDesigner":true,
"AllowRevertFromTemplate":false,
"AllowMasterPageEditing":false,
"ShowUrlStructure":false
}
]

Because the Site object doesn't have many properties the results are relatively small. You can see that a JSON object is returned with the values for the properties filled in. If you were to try accessing a property prior to calling ExecuteQuery, as below, a PropertyOrFieldNotInitializedException would be thrown.

using(ClientContext ctx = new ClientContext(URL))
{
    ctx.Load(ctx.Site);

    // Will throw PropertyOrFieldNotInitializedException
    Console.WriteLine("Site.URL: {0}", ctx.Site.Url);

    ctx.ExecuteQuery();
}

When attempting to access a collection, such as, Site.Features, the exception will be CollectionNotInitializedException

Note, however, in this example the exception will still be thrown since the only object that was loaded was Site, which does not include all of the collections within it. To include the Features collection the Load method would need to be as such

ctx.Load(ctx.Site.Features);

However, when attempting to access a property on the Site object a PropertyOrFieldNotInitializedException would be thrown because the Load method has not included it, only the Features collection. As a comparision the request and response for the above code would look like this

POST http://mySite/_vti_bin/client.svc/ProcessQuery HTTP/1.1
X-RequestDigest: 0xF1C00A2A5C[truncated for space]0000
Content-Type: text/xml
X-RequestForceAuthentication: true
Host: mySite
Content-Length: 714
Expect: 100-continue

≷Request AddExpandoFieldTypeSuffix="true" SchemaVersion="14.0.0.0" LibraryVersion="14.0.4762.1000" 
        ApplicationName=".NET Library" xmlns="http://schemas.microsoft.com/sharepoint/clientquery/2009">
  ≷Actions>
    ≷ObjectPath Id="2" ObjectPathId="1" />
    ≷ObjectPath Id="4" ObjectPathId="3" />
    ≷ObjectPath Id="6" ObjectPathId="5" />
    ≷Query Id="7" ObjectPathId="5">
      ≷Query SelectAllProperties="true">
        ≷Properties />
      ≷/Query>
      ≷ChildItemQuery SelectAllProperties="true">
        ≷Properties />
      ≷/ChildItemQuery>
    ≷/Query>
  ≷/Actions>
  ≷ObjectPaths>
    ≷StaticProperty Id="1" TypeId="{3747adcd-a3c3-41b9-bfab-4a64dd2f1e0a}" Name="Current" />
    ≷Property Id="3" ParentId="1" Name="Site" />
    ≷Property Id="5" ParentId="3" Name="Features" />
  ≷/ObjectPaths>
≷/Request>


HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/json
Server: Microsoft-IIS/7.5
SPRequestGuid: ad372075-501e-4974-a466-2bd1004e8110
Set-Cookie: WSS_KeepSessionAuthenticated={4e4cffb3-203e-4857-8f5a-90a4b18789a4}; path=/
X-SharePointHealthScore: 0
X-Content-Type-Options: nosniff
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.6029
Content-Length: 2863

[
{
"SchemaVersion":"14.0.0.0","LibraryVersion":"14.0.6106.5001","ErrorInfo":null
},2,{
"IsNull":false
},4,{
"IsNull":false
},6,{
"IsNull":false
},7,{
"_ObjectType_":"SP.FeatureCollection","_Child_Items_":[
{
"_ObjectType_":"SP.Feature",
"_ObjectIdentity_":"740c6a0b-85e2-48a0-a494-e0f1759d4aa7:site:2aa[truncated for space]5d0:feature:2acf[truncated for space]18b",
"DefinitionId":"\/Guid(2acf27a5-f703-4277-9f5d-24d70110b18b)\/"
},{
"_ObjectType_":"SP.Feature",
"_ObjectIdentity_":"740c6a0b-85e2-48a0-a494-e0f1759d4aa7:site:2aa[truncated for space]5d0:feature:695b[truncated for space]162",
"DefinitionId":"\/Guid(695b6570-a48b-4a8e-8ea5-26ea7fc1d162)\/"
},{
"_ObjectType_":"SP.Feature",
"_ObjectIdentity_":"740c6a0b-85e2-48a0-a494-e0f1759d4aa7:site:2aa[truncated for space]5d0:feature:c85[truncated for space]fb5",
"DefinitionId":"\/Guid(c85e5759-f323-4efb-b548-443d2216efb5)\/"
... Removed for brevity ...
}
]
}

As you can see there are additional elements added to the Query and ObjectPaths elements and the returned JSON object doesn't include the properties for the Site as the previous results did. If you wanted both the properties and the collections, two Load methods would be needed.

ctx.Load(ctx.Site);
ctx.Load(ctx.Site.Features);

This, however, has a large drawback, the size of the request and responses are increased.

Method	Request Size (bytes)	Response Size (bytes)
Properties	554	460
Collections	719	2865
Properties and Collections	824	3201

The Site object is relatively small, with few properties and collections, but you can see how this could easily get out of hand. To get some control over this bloating you'll need to use the retrievals parameter to limit the results.

Limiting the result set

As shown above the signature of the Laod method takes a params collection of Expressions, which are derived from LambdaExpression.

public sealed class Expression<TDelegate> : LambdaExpression

I'll switch to the Web object for a larger set of examples and show a simple usage below.

Web web = ctx.Site.RootWeb;
ctx.Load(web, w => w.Title);

The first parameter is still the ClientObject derived object that will be used to form the ObjectPaths, but the second is a normal Lamda expression that you may be accustomed to when using Linq. Comparing the above to this

Web web = ctx.Site.RootWeb;
ctx.Load(web);

you can see that although the request size increases slightly the results are reduced quite dramatically.

Method	Request Size (bytes)	Response Size (bytes)
Full	508	736
Title Only	698	297

Because <cdoe>retrievals is a params collection you can string expressions together, such as

ctx.Load(web,
    w => w.Title,
    w => w.Description);

    or 

ctx.Load(web,
    w => w.Title,
    w => w.Description,
    w => w.Lists);

With the second example you must keep in mind that only the properties of the Lists will be loaded, not the collections, such as Fields. If you want those in the results you would use an Include statement.

Web web = ctx.Site.RootWeb;
ctx.Load(web, w => w.Title,
                w => w.Lists
                    .Include(l => l.Fields));

The Include method also takes Lamda expressions so you could include both properties and collections

Web web = ctx.Site.RootWeb;
ctx.Load(web, w => w.Title,
                w => w.Lists
                    .Include(l => l.Title,
                            l => l.Fields));

and since Fields is also a collection an even deeper Include could be added

Web web = ctx.Site.RootWeb;
ctx.Load(web, w => w.Title,
                w => w.Lists
                    .Include(l => l.Title,
                            l => l.Fields
                                .Include(f => f.InternalName)));

As with all of the uses of the Load method you must be cautious. The below table shows how the size of the results can grow based on the depth of the collections, and also how judicious use of the Lambda expressions can dramatically reduce it. You will rarely need all of the properties of an object or its collections so it should be planned out what is needed and form the Load based on those needs.

Method	Request Size (bytes)	Response Size (bytes)
One Include	984	1,946,393
Two Includes	1057	1,946,671
Three Includes	1193	192,660

Even if you use two Load methods it will still be more efficient than one large, deeply nested Include

Web web = ctx.Site.RootWeb;
ctx.Load(web, w => w.Title,
                w => w.Lists
                    .Include(l => l.Title));

ctx.Load(web, w => w.Title,
                w => w.Lists
                    .Include(l => l.Fields
                                .Include(f => f.InternalName, 
                                f => f.Group)));

Method	Request Size (bytes)	Response Size (bytes)
Two Loads	1201	211,489

Creating a request: LoadQuery

The Load methods shown above are used to "fill in" the object passed to it, for instance the Web object. This is fine when loading properties of the object or its collections but it is not good for access ListItems. The only thing you can get using Load is the ItemCount. To get the ListItems you'll need to use the LoadQuery method.

public IEnumerable<T> LoadQuery<T>(ClientObjectCollection<T> clientObjects) where T: ClientObject;
public IEnumerable<T> LoadQuery<T>(IQueryable<T> clientObjects) where T: ClientObject; 

As you can see from the two signatures of this method it can take either a ClientObjectCollection, such as List, or an IQueryable object, both of which of course must be derived from ClientObject so the appropriate ObjectPaths can be constructed. For the latter override the IQueryable object can be passed using either method syntax or Linq syntax, which I'll explore shortly.

Using the first override the lines below would produce the same results. The difference being that Load will "fill in" the web.Lists collection but LoadQuery will return a collection of Lists and not use the web.Lists parameter.

ctx.Load(web.Lists);

var lists = ctx.LoadQuery(web.Lists);

Another difference between these methods is the construction of the request and return. The request for the Load is shown below. For brevity I won't show the result but it is 15,657 bytes.

POST http://mySite/_vti_bin/client.svc/ProcessQuery HTTP/1.1
X-RequestDigest: 0x798DBB28C[truncated for space]0000
Content-Type: text/xml
X-RequestForceAuthentication: true
Host: mySite
Content-Length: 796
Expect: 100-continue

<Request AddExpandoFieldTypeSuffix="true" SchemaVersion="14.0.0.0" LibraryVersion="14.0.4762.1000" 
        ApplicationName=".NET Library" 
        xmlns="http://schemas.microsoft.com/sharepoint/clientquery/2009">
  <Actions>
    <ObjectPath Id="2" ObjectPathId="1" />
    <ObjectPath Id="4" ObjectPathId="3" />
    <ObjectPath Id="6" ObjectPathId="5" />
    <ObjectPath Id="8" ObjectPathId="7" />
    <Query Id="9" ObjectPathId="7">
      <Query SelectAllProperties="true">
        <Properties />
      </Query>
      <ChildItemQuery SelectAllProperties="true">
        <Properties />
      </ChildItemQuery>
    </Query>
  </Actions>
  <ObjectPaths>
    <StaticProperty Id="1" TypeId="{3747adcd-a3c3-41b9-bfab-4a64dd2f1e0a}" Name="Current" />
    <Property Id="3" ParentId="1" Name="Site" />
    <Property Id="5" ParentId="3" Name="RootWeb" />
    <Property Id="7" ParentId="5" Name="Lists" />
  </ObjectPaths>
</Request>

The LoadQuery method produces the below request and a result that is 15,574 bytes

POST http://mySite/_vti_bin/client.svc/ProcessQuery HTTP/1.1
X-RequestDigest: 0x798DBB28C8[truncated for space]0000
Content-Type: text/xml
X-RequestForceAuthentication: true
Host: mySite
Content-Length: 646
Expect: 100-continue

<Request AddExpandoFieldTypeSuffix="true" SchemaVersion="14.0.0.0" LibraryVersion="14.0.4762.1000" 
        ApplicationName=".NET Library" 
        xmlns="http://schemas.microsoft.com/sharepoint/clientquery/2009">
  <Actions>
    <Query Id="21" ObjectPathId="7">
      <Query SelectAllProperties="false">
        <Properties />
      </Query>
      <ChildItemQuery SelectAllProperties="true">
        <Properties />
      </ChildItemQuery>
    </Query>
  </Actions>
  <ObjectPaths>
    <Property Id="7" ParentId="5" Name="Lists" />
    <Property Id="5" ParentId="3" Name="RootWeb" />
    <Property Id="3" ParentId="1" Name="Site" />
    <StaticProperty Id="1" TypeId="{3747adcd-a3c3-41b9-bfab-4a64dd2f1e0a}" Name="Current" />
  </ObjectPaths>
</Request>

Although the requests are formed differently the results, aside from the size, are the same; a collection of Lists with all properties available. The size differences in this example are very modest but could become more significant with larger, more complex queries. Just keep it mind when choosing the method to use.

Just as with the Load method it may be advantageous to limit the results being returned. This is where the second overload for LoadQuery is used. The below example uses method syntax to form a Lambda expression to only include the Title property in the results. Again, as the table below shows, the sizes of the request and response are significantly different when filtering is applied.

Web web = ctx.Site.RootWeb;
var lists = ctx.LoadQuery(web.Lists.Include(l => l.Title));

Method	Request Size (bytes)	Response Size (bytes)
Full List	797	15,657
Include Title	705	2340

As I mentioned above query syntax can also be used with the LoadQuery method.

Web web = ctx.Site.RootWeb;
var query = from l in web.Lists
            select l;
var lists = ctx.LoadQuery(query);

Here I am creating an IQueryable object from the Linq expression that will be used to return an IEnumerable<List> containing the Lists for the Web. The previous examples could be replicated as this

Web web = ctx.Site.RootWeb;
var query = from l in web.Lists
                .Include(l => l.Title)
            select l;
var lists = ctx.LoadQuery(query);

Working with ListItems

Although getting the titles for the Lists within a Web is useful, most of the time the Client OM will be used to work with ListItems; reading, inserting, updating and deleting. With all the advances SharePoint 2010 has provided, at this point you still must work with CAML via the CamlQuery class.

CamlQuery query = new CamlQuery();
ListItemCollection items = list.GetItems(query);
ctx.Load(items);    

Although no CAML has been supplied this will return all of the ListItems, with all field values. To get only specific fields you would use a standard CAML query, such as

CamlQuery query = new CamlQuery();
ListItemCollection items = list.GetItems(query);
ctx.Load(items);  
query.ViewXml = @"<View Scope='RecursiveAll'>
  <ViewFields>
    <FieldRef Name='Title' />
    <FieldRef Name='FirstName' />
  </ViewFields>
</View>";

If you have a List with many columns, writing the CAML can be very tedious and error prone. Luckily, CamlQuery has a static method to help with this, CreateAllItemsQuery. This method will return the following CAML

<View Scope="RecursiveAll">
  <Query>
  </Query>
</View>

There is also a static method for all folders, CreateAllFoldersQuery

<View Scope="RecursiveAll">
  <Query>
    <Where>
      <Eq>
        <FieldRef Name="FSObjType" />
        <Value Type="Integer">1</Value>
      </Eq>
    </Where>
  </Query>
</View>

As I showed above you can use standard CAML syntax to form the query, including Where and RowLimit elements. You can also use an overload of the CreateAllItemsQuery

public static CamlQuery CreateAllItemsQuery(int rowLimit, params string[] viewFields);

which would be used like this

CamlQuery query = CamlQuery.CreateAllItemsQuery(10, "Title", "FirstName");

ExecuteQuery

Although ExecuteQuery has been used in the previous examples for completeness I will briefly review how it works.

public virtual void ExecuteQuery()
{
    ScriptTypeMap.EnsureInited();
    ClientRequest pendingRequest = this.PendingRequest;
    this.m_request = null;
    pendingRequest.ExecuteQuery();
}

The EnsureInited method uses a lock to ensure the initialization of the object is only performed once. The lock is necessary because the Init method uses reflection to check custom attributes and create an instance of classes implementing the IScriptTypeFactory interface. After initialization the ClientRequest.ExecuteQuery method will call the BuildQuery method.

private ChunkStringBuilder BuildQuery()
{
    SerializationContext serializationContext = this.SerializationContext;
    ChunkStringBuilder builder = new ChunkStringBuilder();
    XmlWriterSettings settings = new XmlWriterSettings {
        OmitXmlDeclaration = true,
        NewLineHandling = NewLineHandling.Entitize
    };
    XmlWriter writer = XmlWriter.Create(builder.CreateTextWriter(CultureInfo.InvariantCulture), settings);
    writer.WriteStartElement("Request", "http://schemas.microsoft.com/sharepoint/clientquery/2009");
    writer.WriteAttributeString("AddExpandoFieldTypeSuffix", "true");
    writer.WriteAttributeString("SchemaVersion", ClientSchemaVersions.CurrentVersion.ToString());
    writer.WriteAttributeString("LibraryVersion", "14.0.4762.1000");
    if (!string.IsNullOrEmpty(this.m_context.ApplicationName))
    {
        writer.WriteAttributeString("ApplicationName", this.m_context.ApplicationName);
    }
    writer.WriteStartElement("Actions");
    
    ...removed for clarity...
    
    writer.WriteEndElement();
    writer.WriteStartElement("ObjectPaths");
    Dictionary<long, ObjectPath> dictionary = new Dictionary<long, ObjectPath>();
    while (true)
    {
        List<long> list = new List<long>();
        foreach (long num in serializationContext.Paths.Keys)
        {
            if (!dictionary.ContainsKey(num))
            {
                list.Add(num);
            }
        }
        if (list.Count == 0)
        {
            break;
        }
        for (int i = 0; i < list.Count; i++)
        {
            ObjectPath path = this.m_context.ObjectPaths[list[i]];
            path.WriteToXml(writer, serializationContext);
            dictionary[list[i]] = path;
        }
    }
    writer.WriteEndElement();
    writer.WriteEndElement();
    writer.Flush();
    return builder;
}

As you can this is where all the ObjectPaths created from the objects and properties used in any Load or LoadQuery methods prior to the ExecuteQuery method being called are constructed into an XML string that will be sent to the server along with setting the properties such as LibraryVersion and ApplicationName. After the query has been built the only thing left to do is send it to the server using the ExecuteQueryToServer method below.

private void ExecuteQueryToServer(ChunkStringBuilder sb)
{
    this.m_context.FireExecutingWebRequestEvent(new WebRequestEventArgs(this.RequestExecutor));
    this.RequestExecutor.RequestContentType = "text/xml";
    if (this.m_context.AuthenticationMode == ClientAuthenticationMode.Default)
    {
        this.RequestExecutor.RequestHeaders["X-RequestForceAuthentication"] = "true";
    }
    Stream requestStream = this.RequestExecutor.GetRequestStream();
    sb.WriteContentAsUTF8(requestStream);
    requestStream.Flush();
    requestStream.Close();
    this.RequestExecutor.Execute();
    this.ProcessResponse();
}

This method will send the query, then, using the ProcessResponse method, translate the results into a JSON object.

Authentication

As I mentioned earlier the Client OM uses Windows Authentication by default when utilizing Managed Code. With JavaScript or Silverlight the code is being executed within the context of an application already so it will use the existing security token. You can see this by using a tool such as Fiddler to monitor the web traffic being generated. When using Managed Code there will calls to _vti_bin/sites.asmx requesting the FormsDigest information, which contains the security token.

POST http://mySite/_vti_bin/sites.asmx HTTP/1.1
Content-Type: text/xml
SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation
X-RequestForceAuthentication: true
Authorization: NTLM TlRMTVNTUAABAAAAt7II4gkACQAsAAAABAAEACgAAAAGAbEdAAAAD01BUktOSVNDSEFMS0U=
Host: mySite
Content-Length: 0

When working with a site using Claims based authentication, such as Forms Authentication, an extra step is needed prior to calling ExecuteQuery to set the type and credentials to use. Everything else in your code remains the same.

ctx.AuthenticationMode = ClientAuthenticationMode.FormsAuthentication;
ctx.FormsAuthenticationLoginInfo = new FormsAuthenticationLoginInfo("loginName", "password");

The FormsAuthenticationLoginInfo class is used by the ClientContext.FormsAuthenticationLogin method by first calling the EnsureLogin method to check for authentication cookies.

internal CookieCollection EnsureLogin(Uri contextUri)
{
    if ((this.m_authCookies == null) || !this.m_cookieValid)
    {
        this.m_authCookies = new Dictionary<Uri, CookieInfo>();
        this.m_cookieValid = true;
    }
    contextUri = new Uri(contextUri, "/");
    CookieInfo info = null;
    if (!this.m_authCookies.TryGetValue(contextUri, out info) || (info.Expires <= DateTime.UtcNow))
    {
        info = this.Login(contextUri);
        this.m_authCookies[contextUri] = info;
    }
    return info.Cookies;
}

If the cookie is not present, or has expired, a call to Login which will call the authentication service to complete the Login.

Uri authenticationServiceUrl;
if (this.AuthenticationServiceUrl == null)
{
    authenticationServiceUrl = new Uri(contextUri, "/_vti_bin/authentication.asmx");
}
else
{
    authenticationServiceUrl = this.AuthenticationServiceUrl;
}
Authentication authentication = new Authentication(authenticationServiceUrl) {
    CookieContainer = this.CookieContainer
};
LoginResult result = authentication.Login(this.LoginName, this.Password);

What's next

Hopefully this article has given you background on what the SharePoint Client Object Model is and how it works. In Part 2 I will cover actual, such as, updating and adding ListItems, creating Lists and more, using .NET Managed code, JavaScript and Silverlight.

History

Initial posting: 10/13/11