Posts Tagged ‘sharepoint 2010’

When using SharePoint as a platform for document management there is often a requirement to assign unique identifiers to documents. Document ID feature has been introduced in SharePoint 2010 and is also available in SharePoint 2013. Being limited to a site collection introduces a big disadvantage since it means that your document identifiers will be unique within one site collection. This would be a pretty useful feature except document management systems usually spread across multiple site collections or even web applications. There are two ways to ensure Document IDs are unique across site collections: either set a different Document ID prefix on each site collection or develop a custom Document ID provider. In this post I will describe what Document ID is and how it works, as well I’ll provide a solution to a Custom Document ID Provider and explain it’s details.

How the gears are actually turning

Lets start with how Document ID works in SharePoint. The whole thing consists of three main parts:

  1. Activation – Document ID feature;
  2. Activation – Document ID enable/disable timer job;
  3. Document ID Generator event receiver/Document ID assignment job

Document ID Service feature

This is a site collection feature which enables Document ID functionality on a site collection level. This needs to be activated in all site collections where you intend to use Document IDs. Once activated, this feature does a couple things:

  1. Adds a Document ID settings option under Site Collection Administration;
  2. Generates a Document ID prefix automatically;
  3. Checks the size of the site collection and depending on the result will either assign Document IDs straight away or create work items to do that. The function call used for that is DocIdHelpers.IsSiteTooBig(site, 1, 20, 40). Let me disassemble this for you: “Please check this site collection and if it has at least 1 web site or 40 lists or 20 libraries, consider it being TOOBIG“. *chuckle*. Notice ORs instead of ANDs.
    Keep in mind that work items are created with a scheduled date and there is no way to change it. In case you don’t believe you can query ScheduledWorkItems table in the Content database:

    declare @enable uniqueidentifier
    set @enable = cast(‘749FED41-4F86-4277-8ECE-289FBF18884F’ as uniqueidentifier)
    select * from [dbo].[ScheduledWorkItems] where [Type] = @enable

    This will return a work item which will be processed by Document ID enable/disable job and delivery date will be 30 minutes from the moment you activated the feature. This means that no matter how many times you run the timer jobs in the first 30 minutes after activating the feature, nothing will happen. Take a break, have a coffee and navigate to Document Id Settings. If you still see:
    1
    that means the timer job hasn’t done it’s job yet.
    Top Tip #331: In case you can’t wait – change the system time, restart timer service and run the timer jobs.

Timer jobs

There are two of them:

  1. Document ID enable/disable job
    Title: Document ID enable / disable
    Type: Microsoft.Office.DocumentManagement.Internal.DocIdEnableWorkItemJobDefinition
    Work item type: 749FED41-4F86-4277-8ECE-289FBF18884F
    Description: Work item that sets the change in the type of content in all sites to reconfigure Document ID
    The default schedule: Every day 21:30-21:45

    This will process all work items on all site collections in a web application, and make sure that Document ID prefix is pushed to all subsites. Document ID field will be added to all content types which inherit from Document and Document Set. More precisely, three fields will be added: Document ID, Document ID Value and Persist ID. In addition to adding these columns, SharePoint adds an event receiver to each of the content types so that they run every time a document or document set is uploaded to SharePoint. The server uses ItemAdded event to ensure that document ID providers can use item metadata when assigning document IDs.

  2. Document ID assignment job
    Title: Document ID assignment
    Type: Microsoft.Office.DocumentManagement.Internal.DocIdWorkItemJobDefinition
    Work item type: A83644F5-78DB-4F95-9A9D-25238862048C
    Description: Work item that assigns Document ID to all elements in the site collection
    The default schedule: Every day 22:00-22:30

    This will push the settings to all lists and assign Document IDs to the documents.

Document ID Generator Event Receiver

When items are added to a site collection, SharePoint assigns or reassigns a Document ID. When a new item is added, SharePoint first checks to see if the item has a Document ID. If it does, it checks to see if Preserve ID attribute is set to True or False, and then sets it to False if it is currently set to True. If the item does not already have a Document ID, the server gets a Document ID for the item from the specified provider, writes it to metadata, and sets Preserve ID attribute to False. I hope this is not confusing.

This is how out-of-the-box Document ID works in a nutshell. Having a semi-fixed pattern and the fact that the functionality is limited to a site collection you end up with another bright idea ended up almost useless for an enterprise document management system.

Web application Document ID

Out-of-the-box functionality should be sufficient for some of the folk, but it wasn’t in my case. I needed the IDs to be in a specific format and notation as a requirement by the enterprise, so I had to implement a custom Document ID Provider. This had to spray unique identifiers across all site collections within a web application.

The solution consists of the following elements:
3

  1. Code to generate Document ID;
  2. List instance to store last Document ID and a Scheme;
  3. Settings page;
  4. Custom action to add a link to Settings page;
  5. A feature activate custom Document ID provider.

Document ID Provider

This is the core of the solution. To create a custom Document ID Provider we need to derive a class from Microsoft.Office.DocumentManagement.DocumentIdProvider. The class contains 3 abstract methods and 1 abstract property that we need to implement:

  1. public override string GenerateDocumentId(SPListItem listItem). This method is called when a new Document ID needs to be created. Current item is handed over to this method as a parameter. This is the place where you generate a Document ID or get it from a another system.
  2. public override string GetSampleDocumentIdText(SPSite site). Returns an example Document ID value that will be displayed in Document ID search web part. The method is called when Find By Document ID web part is rendered.
  3. public override bool DoCustomSearchBeforeDefaultSearch. This property determines how documents will be retrieved by their Document ID. If it’s set to False, documents will be retrieved using SharePoint Search first. If it’s set to True, GetDocumentUrlsById will be used before SharePoint Search. Note that this only defines the priority of the search method and that second search method will be used only if the first one doesn’t return a result.
  4. public override string[] GetDocumentUrlsById(SPSite site, string documentId). Returns an array of URLs pointing to documents with a specified Document ID. Implementing this has an advantage over using default SharePoint search – most of the time it’s faster: there is no need to wait for search crawls to finish. Otherwise, if you will be letting SharePoint Search do the work, this method should return an empty array of strings.
    If DoCustomSearchBeforeDefaultSearch is True, then returning an empty array of strings will tell DocIdRedir.aspx to try searching again using SharePoint Search.
    If DoCustomSearchBeforeDefaultSearch is False and neither SharePoint Search nor GetDocumentUrlsById returned any results, a message “No documents with the ID were found in this site collection” will be displayed.

Document ID List

I’ve chosen a list to store all information because of the following reasons:

  1. SPListItem.ID works well as a unique identifier.
  2. Use of SPList.Properties to store configuration data.
  3. Getting all information for Document ID from one object.

Let me explain how the whole thing comes together:

docid

  1. A document is added to a library;
  2. Document ID Generator Event Receiver kicks in;
  3. Document ID Provider generates a Document ID and returns it.

This is how Document ID Provider generates a new ID:

docid-2

So simple it didn’t even need a scheme :(

A Document ID is generated by merging an SPListItem.ID and a Scheme. Let’s take an example scheme of “DMS-0000000” and an ID of “5”. What I wanted it to end up with is “DMS-0000005” – not adding number 5 at the end but instead using zeros as a placeholder for numbers.  This magic code here does the thing:

internal static string FormatID(int id, string scheme)
{
    return string.Format(string.Format(“{{0:{0} }}”, scheme), id).TrimEnd(‘ ‘);
}

Since the string is being formatted twice, you might want to add various checks (or replacements like Replace(“E0”, “\\E0”) etc.) if the scheme includes any of the formatting keywords which you can find here.

Settings Page

An Application Page to edit and store Document ID settings in list’s property bag. For now, there is one editable setting on the page which is a Document ID scheme.

Custom action

A custom action was added to List Settings under General Settings to display a link to Document ID Settings page:
4

<CustomAction
    Id=”DocumentIDSettings”
    GroupId=”GeneralSettings”
    Location=”Microsoft.SharePoint.ListEdit”
    Sequence=”1000″
    RegistrationId=”10071″
    Title=”Document ID Settings”>
    <UrlAction Url=”_layouts/DocumentID/Settings.aspx”/>
</CustomAction>

Custom Document ID Provider feature

Feature receiver needs to run some code to set your custom Document ID provider for a site collection:

public class ProvisionCustomDocIdProviderEventReceiver : SPFeatureReceiver
{
    public override void FeatureActivated(SPFeatureReceiverProperties properties)
    {
        DocumentId.SetProvider(properties.Feature.Parent as SPSite, new CustomDocumentIdProvider());
    }    
    public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
    {
        DocumentId.SetDefaultProvider(properties.Feature.Parent as SPSite);
    }
}

This is how a custom Document ID provider works.

You can find the complete VS2013 project here. Modifying the solution in a few easy steps could make it Farm wide. Play around with the source code. There isn’t much of coding and it’s pretty straightforward.

Common issues

Document ID is not generated

The most common one. Aghm,- probably the only one. Things that I would suggest to take a look at in this particular order:

  1. Check if the Document ID Service feature is activated. *DOH*
  2. Check Document ID Settings under Site Collection Administration and see if there is a message “Configuration of the Document ID feature is scheduled to be completed by an automated process”. If this is the case, wait 30 minutes or refer to the beginning of this post how to speed the Document ID provisioning process.
  3. Check if Document ID fields are available in current web.
  4. Check if Document ID Generator Event Receivers are added to the library.
  5. Check if Content Types are not marked as Read Only or Sealed.
  6. If you are using a custom library template, check if Document ID works in standard document library. If that’s the case – there might be issues with your template.

Hope this helps!

Permissions play an important role in SharePoint. Today in my first blog post I’d like to share a solution to a rare exception when SharePoint fails to assign permissions and you are left with an item that you can’t even delete. But first some background on the project which made me break the first rule of SharePoint – never mess with a SharePoint database directly.

The project that I finished working on half a year ago is a business process management system. Lots of lists, lots of libraries, over 30.000 sites (still counting) and of course the biggest pain for users (performance) and support team (issues) – item level permissions. These are assigned using event receivers (ItemAdded and ItemUpdated) based on a Department site column which was used in every site content type. This wouldn’t be an issue if we had kept everything according to Microsoft’s best practices and recommendations. The magic number of 5.000 was left far behind – after half year of usage some lists already had over 25.000 items with item level permissions. Almost every action (opening an item, saving it etc.) took from 3 to 10 seconds.

It was slow, but it worked until we hit about 15.000 items in a single list. Then every once in a while (1 in a 1000) an item was created without any permissions – we called them orphans. Too bad the exception was random and we couldn’t reproduce it. ULS logs would show this error:

System.Data.SqlClient.SqlException: Violation of PRIMARY KEY constraint ‘Perms_PK’. Cannot insert duplicate key in object ‘dbo.Perms’. The duplicate key value is (9f07b6ef-25e1-4a7a-b06e-f60019e20255, 0x, ok/objektai/16855_.000).  The statement has been terminated.   

Users (except for site collection administrators) weren’t able to see these items in lists. They could open view or edit forms but couldn’t do anything that called an SPListItem.Update() or SPListItem.Delete(). Trying to view item permissions would result in an error:

Cannot complete this action

Luckily the URL contained ID of the SPListItem so we were able to check the properties of that item using PowerShell.

http://site/web/_layouts/User.aspx?obj={60782643-80D2-4F96-956F-C74A56263DC6},16855,LISTITEM&List={60782643-80D2-4F96-956F-C74A56263DC6

Usually items with unique permissions look like this:

HasUniqueRoleAssignments True
EffectiveBasePermissions FullMask
FirstUniqueAncestorSecurableObject Microsoft.SharePoint.SPListItem
ReusableAcl Microsoft.SharePoint.SPReusableAcl
RoleAssignments {Microsoft.SharePoint.SPRoleAssignment, Microsoft.SharePoint.SPRoleAssignment…}
AllRolesForCurrentUser {Full Control, Read, Limited Access, View Only…}
FirstUniqueAncestor Microsoft.SharePoint.SPListItem

but instead our items looked like this:

HasUniqueRoleAssignments
EffectiveBasePermissions FullMask
FirstUniqueAncestorSecurableObject
ReusableAcl
RoleAssignments
AllRolesForCurrentUser
FirstUniqueAncestor

Yes, all of these blank values were nulls. There were no possible ways of fixing these orphans either by code or using PowerShell – we couldn’t delete them (UnauthorizedAccessException) and SPListItem.ResetRoleInheritance() was returning an error System.Runtime.InteropServices.COMException (0x80004005): Cannot complete this action. The only way to do ResetRoleInheritance was to use a SharePoint database stored procedure proc_SecResetItemPerm. It takes SiteId, WebId, OldScopeId, Item Url and DocId as parameters so first we need to get them. PowerShell works great for that. Here is an example script with it’s output:

Add-PSSnapin Microsoft.Sharepoint.Powershell
$site = Get-SPSite http://site
$site.ID
#>>9f07b6ef-25e1-4a7a-b06e-f60019e20255

$web = Get-SPWeb http://site/web
$web.ID
#>>c3536be5-a419-4c27-88fd-269316c18757

$list = $web.Lists[“List”]
$item = $list.GetItemById(16855)
$item.Url
#>>list/16855_.000

$item.UniqueId
#>>fab7dc68-50ad-4ea0-9c99-230dfdbc0567

Now the last parameter that we need is OldScopeId. The way to get it is by querying Perms table:

USE [WSS_Content]
GO

SELECT TOP 1 [ScopeId]
FROM [WSS_Content].[dbo].[Perms]
WHERE ScopeUrl = ‘web/list/16855_.000’
GO

Since we have all the parameters we can execute proc_SecResetItemPerm stored procedure. We do this by writing a query against WSS_Content database:

USE [WSS_Content]
GO

GODECLARE @return_value int,
@NewScopeId uniqueidentifier,
@RequestGuid uniqueidentifier

EXEC @return_value = [dbo].[proc_SecResetItemPerm]
@SiteId = ‘9f07b6ef-25e1-4a7a-b06e-f60019e20255’,
@WebId = ‘c3536be5-a419-4c27-88fd-269316c18757’,
@OldScopeId = ‘B2D7B4C0-8E02-4E5F-A611-020BE5770A8F’,
@Url = N’web/list/16855_.000′,
@DocId = ‘fab7dc68-50ad-4ea0-9c99-230dfdbc0567′,
@NewScopeId = @NewScopeId OUTPUT,
@RequestGuid = @RequestGuid OUTPUT

SELECT @NewScopeId as N’@NewScopeId’,
@RequestGuid as N’@RequestGuid’

SELECT ‘Return Value’ = @return_value
GO

The query returns NewScopeId and we end up with broken role inheritance and no new roles assigned. Now we can assign new permissions or delete that item.

Hope it helps!

Paul.