0

Background:

I am trying to create a C# command line utility to extract list item information from lists that may exist anywhere within a particular site collection. All lists that I am trying to extract from were created from a particular template with ID 10003 (Custom template).

The powers that be are still deciding how frequently this thing is supposed to be run, but I'm expecting an answer on the order of every few minutes or so, so I need execution time to be no more than a minute, and don't think that's achievable with my current approach.

I have a site collection with 7 immediate children and ~200 total descendant sub-sites, and these lists may appear in any one of them. Most instances will only have a couple of items, but some of them will have a few thousand. I'm expecting ~10-20k results total.

This utility will be run on a remote server, and internally we'd prefer to use the CSOM over rest. I'm familiar with SP web part development but this is the first time I've needed to use the CSOM.

The Farm is a SP 2010 On Prem with 3 WFEs, updated with latest CU.

Immediate Issues:

  • I'm throwing a "Cannot complete this action." on a call to context.ExecuteQuery(), and I'm not sure why.
  • I am aware that I'm probably over-calling context.ExecuteQuery(), and would like to know a better way to load all of these lists.
  • My current code only gets the immediate child webs, when I need all descendants of the root.

Code:

My current attempt looks like this:

using (var ctxt = new ClientContext(url))
        {
            var webs = ctxt.Web.Webs;
            ctxt.Load(webs);
            ctxt.ExecuteQuery();
            var allItems = new List<ListItem>();
            foreach (var web in webs)
            {
                ctxt.Load(web.Lists);
                ctxt.ExecuteQuery();
                foreach (var list in web.Lists)
                {
                    if (list.BaseTemplate == 10003)
                    {
                        ctxt.Load(list);
                        ctxt.ExecuteQuery();
                        var items = list.GetItems(query);
                        ctxt.Load(items);
                        ctxt.ExecuteQuery(); // <- **throws "Cannot complete this action." on first iteration of loop.**
                        allItems.AddRange(items);
                    }
                }
            }                
            results = allItems.Select(ConvertToNeededResultType).ToList();
        }

The Query looks like:

<View Scope='RecursiveAll'>
<Webs Scope='SiteCollection' /> <!--**If I omit this line, I get a CollectionNotInitialized exception on allitems.AddRange(items)**--/>
<Lists ServerTemplate='10003' />
<Query>
    <OrderBy>
        <FieldRef Name='CreatedOn' Ascending='False' />
    </OrderBy>
    <Where>
        <And>
            <Eq>
                <FieldRef Name='FSObjType' />
                <Value Type='Integer'>0</Value>
            </Eq>
            <Geq>
                <FieldRef Name='CreatedOn'/>
                <Value Type='DateTime' IncludeTimeValue='FALSE'>{0}</Value>
            </Geq>
        </And>
    </Where>
<Query>
<ViewFields>
    <FieldRef Name='Title' Nullable='TRUE' />
    <FieldRef Name='URL' Nullable='TRUE' />
    <FieldRef Name='CreatedOn' Nullable='TRUE' />
    <FieldRef Name='Category' Nullable='TRUE' />
    <FieldRef Name='Attachments' Nullable='TRUE' />
    <FieldRef Name='ID' />
    <ProjectProperty Name='Title' />
    <ListProperty Name='Title' />
</ViewFields>
</View>

where the {0} contains a date that is the number of days I want to go back with the list in format: "yyyy-MM-ddTHH:mm:ssZ"

What I am looking for:

I am looking for either advice on how to resolve the specific issues enumerated above with my current code, or a suggestion with examples of how to more efficiently achieve the same result.

J. Patton
  • 27
  • 10

1 Answers1

0

I never did figure out the "Cannot complete this action" bit, but I've solved the requirements another way: I needed two methods, recursive for crawling the webs and retrieving items:

public static IEnumerable<WebLocation> GetWebLocations(string rootWebUrl)
{
   List<WebLocation> results;
   using (var cntxt = new ClientContext(rootWebUrl))
   {
       var web = cntxt.Web;
       cntxt.Load(web, w => w.Webs, w => w.Id, w => w.ServerRelativeUrl, w => w.Lists);
       cntxt.ExecuteQuery();
       results = GetWebLocations(cntxt, web, Guid.Empty);
   }
   return results;
}

private static List<WebLocation> GetWebLocations(ClientContext cntxt, Web currentWeb, Guid parentId)
{
   var results = new List<WebLocation>();
   var currentId = currentWeb.Id;
   var url = currentWeb.ServerRelativeUrl;
   var location = new WebLocation { ParentSiteID = parentId, SiteID = currentId, WebUrl = url, HotLinksItems = new List<HotLinksItem>() };
   foreach (var list in currentWeb.Lists)
   {
       cntxt.Load(list, l => l.BaseTemplate, l => l.RootFolder.ServerRelativeUrl);
       cntxt.ExecuteQuery();
       if (list.BaseTemplate == 10003)
       {
          var itemCollection =
                       list.GetItems(new CamlQuery
                       {
                           ViewXml = "<View Scope='RecursiveAll'><ViewFields><FieldRef Name='Title' Nullable='TRUE' /><FieldRef Name='ID' /><ProjectProperty Name='Title' /><ListProperty Name='Title' /></ViewFields></View>"
                       });
           cntxt.Load(itemCollection);
           cntxt.ExecuteQuery();
           foreach (var item in itemCollection)
           {
               var hotlink = new HotLinksItem
               {
                   Title = item["Title"] != null ? item["Title"].ToString() : null,
                   ID = item["ID"] != null ? item["ID"].ToString() : null,
               };
               location.HotLinksItems.Add(hotlink);
           }
       }
   }

}

It still feels like I'm doing too many calls, but trimming them down with the sub selections has helped performance a lot, All in all, against my farm this thing runs in a little over 30 seconds. (Before trimming down return items, it ran in about 3 minutes.)

J. Patton
  • 27
  • 10