Posts Tagged ‘SharePoint 2013’

When using SharePoint as a platform for document management there is often a requirement to assign unique identifiers to documents. Document ID feature has been introduced in SharePoint 2010 and is also available in SharePoint 2013. Being limited to a site collection introduces a big disadvantage since it means that your document identifiers will be unique within one site collection. This would be a pretty useful feature except document management systems usually spread across multiple site collections or even web applications. There are two ways to ensure Document IDs are unique across site collections: either set a different Document ID prefix on each site collection or develop a custom Document ID provider. In this post I will describe what Document ID is and how it works, as well I’ll provide a solution to a Custom Document ID Provider and explain it’s details.

How the gears are actually turning

Lets start with how Document ID works in SharePoint. The whole thing consists of three main parts:

  1. Activation – Document ID feature;
  2. Activation – Document ID enable/disable timer job;
  3. Document ID Generator event receiver/Document ID assignment job

Document ID Service feature

This is a site collection feature which enables Document ID functionality on a site collection level. This needs to be activated in all site collections where you intend to use Document IDs. Once activated, this feature does a couple things:

  1. Adds a Document ID settings option under Site Collection Administration;
  2. Generates a Document ID prefix automatically;
  3. Checks the size of the site collection and depending on the result will either assign Document IDs straight away or create work items to do that. The function call used for that is DocIdHelpers.IsSiteTooBig(site, 1, 20, 40). Let me disassemble this for you: “Please check this site collection and if it has at least 1 web site or 40 lists or 20 libraries, consider it being TOOBIG“. *chuckle*. Notice ORs instead of ANDs.
    Keep in mind that work items are created with a scheduled date and there is no way to change it. In case you don’t believe you can query ScheduledWorkItems table in the Content database:

    declare @enable uniqueidentifier
    set @enable = cast(‘749FED41-4F86-4277-8ECE-289FBF18884F’ as uniqueidentifier)
    select * from [dbo].[ScheduledWorkItems] where [Type] = @enable

    This will return a work item which will be processed by Document ID enable/disable job and delivery date will be 30 minutes from the moment you activated the feature. This means that no matter how many times you run the timer jobs in the first 30 minutes after activating the feature, nothing will happen. Take a break, have a coffee and navigate to Document Id Settings. If you still see:
    1
    that means the timer job hasn’t done it’s job yet.
    Top Tip #331: In case you can’t wait – change the system time, restart timer service and run the timer jobs.

Timer jobs

There are two of them:

  1. Document ID enable/disable job
    Title: Document ID enable / disable
    Type: Microsoft.Office.DocumentManagement.Internal.DocIdEnableWorkItemJobDefinition
    Work item type: 749FED41-4F86-4277-8ECE-289FBF18884F
    Description: Work item that sets the change in the type of content in all sites to reconfigure Document ID
    The default schedule: Every day 21:30-21:45

    This will process all work items on all site collections in a web application, and make sure that Document ID prefix is pushed to all subsites. Document ID field will be added to all content types which inherit from Document and Document Set. More precisely, three fields will be added: Document ID, Document ID Value and Persist ID. In addition to adding these columns, SharePoint adds an event receiver to each of the content types so that they run every time a document or document set is uploaded to SharePoint. The server uses ItemAdded event to ensure that document ID providers can use item metadata when assigning document IDs.

  2. Document ID assignment job
    Title: Document ID assignment
    Type: Microsoft.Office.DocumentManagement.Internal.DocIdWorkItemJobDefinition
    Work item type: A83644F5-78DB-4F95-9A9D-25238862048C
    Description: Work item that assigns Document ID to all elements in the site collection
    The default schedule: Every day 22:00-22:30

    This will push the settings to all lists and assign Document IDs to the documents.

Document ID Generator Event Receiver

When items are added to a site collection, SharePoint assigns or reassigns a Document ID. When a new item is added, SharePoint first checks to see if the item has a Document ID. If it does, it checks to see if Preserve ID attribute is set to True or False, and then sets it to False if it is currently set to True. If the item does not already have a Document ID, the server gets a Document ID for the item from the specified provider, writes it to metadata, and sets Preserve ID attribute to False. I hope this is not confusing.

This is how out-of-the-box Document ID works in a nutshell. Having a semi-fixed pattern and the fact that the functionality is limited to a site collection you end up with another bright idea ended up almost useless for an enterprise document management system.

Web application Document ID

Out-of-the-box functionality should be sufficient for some of the folk, but it wasn’t in my case. I needed the IDs to be in a specific format and notation as a requirement by the enterprise, so I had to implement a custom Document ID Provider. This had to spray unique identifiers across all site collections within a web application.

The solution consists of the following elements:
3

  1. Code to generate Document ID;
  2. List instance to store last Document ID and a Scheme;
  3. Settings page;
  4. Custom action to add a link to Settings page;
  5. A feature activate custom Document ID provider.

Document ID Provider

This is the core of the solution. To create a custom Document ID Provider we need to derive a class from Microsoft.Office.DocumentManagement.DocumentIdProvider. The class contains 3 abstract methods and 1 abstract property that we need to implement:

  1. public override string GenerateDocumentId(SPListItem listItem). This method is called when a new Document ID needs to be created. Current item is handed over to this method as a parameter. This is the place where you generate a Document ID or get it from a another system.
  2. public override string GetSampleDocumentIdText(SPSite site). Returns an example Document ID value that will be displayed in Document ID search web part. The method is called when Find By Document ID web part is rendered.
  3. public override bool DoCustomSearchBeforeDefaultSearch. This property determines how documents will be retrieved by their Document ID. If it’s set to False, documents will be retrieved using SharePoint Search first. If it’s set to True, GetDocumentUrlsById will be used before SharePoint Search. Note that this only defines the priority of the search method and that second search method will be used only if the first one doesn’t return a result.
  4. public override string[] GetDocumentUrlsById(SPSite site, string documentId). Returns an array of URLs pointing to documents with a specified Document ID. Implementing this has an advantage over using default SharePoint search – most of the time it’s faster: there is no need to wait for search crawls to finish. Otherwise, if you will be letting SharePoint Search do the work, this method should return an empty array of strings.
    If DoCustomSearchBeforeDefaultSearch is True, then returning an empty array of strings will tell DocIdRedir.aspx to try searching again using SharePoint Search.
    If DoCustomSearchBeforeDefaultSearch is False and neither SharePoint Search nor GetDocumentUrlsById returned any results, a message “No documents with the ID were found in this site collection” will be displayed.

Document ID List

I’ve chosen a list to store all information because of the following reasons:

  1. SPListItem.ID works well as a unique identifier.
  2. Use of SPList.Properties to store configuration data.
  3. Getting all information for Document ID from one object.

Let me explain how the whole thing comes together:

docid

  1. A document is added to a library;
  2. Document ID Generator Event Receiver kicks in;
  3. Document ID Provider generates a Document ID and returns it.

This is how Document ID Provider generates a new ID:

docid-2

So simple it didn’t even need a scheme :(

A Document ID is generated by merging an SPListItem.ID and a Scheme. Let’s take an example scheme of “DMS-0000000” and an ID of “5”. What I wanted it to end up with is “DMS-0000005” – not adding number 5 at the end but instead using zeros as a placeholder for numbers.  This magic code here does the thing:

internal static string FormatID(int id, string scheme)
{
    return string.Format(string.Format(“{{0:{0} }}”, scheme), id).TrimEnd(‘ ‘);
}

Since the string is being formatted twice, you might want to add various checks (or replacements like Replace(“E0”, “\\E0”) etc.) if the scheme includes any of the formatting keywords which you can find here.

Settings Page

An Application Page to edit and store Document ID settings in list’s property bag. For now, there is one editable setting on the page which is a Document ID scheme.

Custom action

A custom action was added to List Settings under General Settings to display a link to Document ID Settings page:
4

<CustomAction
    Id=”DocumentIDSettings”
    GroupId=”GeneralSettings”
    Location=”Microsoft.SharePoint.ListEdit”
    Sequence=”1000″
    RegistrationId=”10071″
    Title=”Document ID Settings”>
    <UrlAction Url=”_layouts/DocumentID/Settings.aspx”/>
</CustomAction>

Custom Document ID Provider feature

Feature receiver needs to run some code to set your custom Document ID provider for a site collection:

public class ProvisionCustomDocIdProviderEventReceiver : SPFeatureReceiver
{
    public override void FeatureActivated(SPFeatureReceiverProperties properties)
    {
        DocumentId.SetProvider(properties.Feature.Parent as SPSite, new CustomDocumentIdProvider());
    }    
    public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
    {
        DocumentId.SetDefaultProvider(properties.Feature.Parent as SPSite);
    }
}

This is how a custom Document ID provider works.

You can find the complete VS2013 project here. Modifying the solution in a few easy steps could make it Farm wide. Play around with the source code. There isn’t much of coding and it’s pretty straightforward.

Common issues

Document ID is not generated

The most common one. Aghm,- probably the only one. Things that I would suggest to take a look at in this particular order:

  1. Check if the Document ID Service feature is activated. *DOH*
  2. Check Document ID Settings under Site Collection Administration and see if there is a message “Configuration of the Document ID feature is scheduled to be completed by an automated process”. If this is the case, wait 30 minutes or refer to the beginning of this post how to speed the Document ID provisioning process.
  3. Check if Document ID fields are available in current web.
  4. Check if Document ID Generator Event Receivers are added to the library.
  5. Check if Content Types are not marked as Read Only or Sealed.
  6. If you are using a custom library template, check if Document ID works in standard document library. If that’s the case – there might be issues with your template.

Hope this helps!

First of all I would like to say that quick edit JavaScript validation is not documented at all, so the methods to achieve the validation are debatable and not necessary correct. But hey – it’s still a way how to do it :) The only reads you can find on MSDN are here and here. Not very helpful, eh? There are a couple of blog posts (despite the lack of consistent examples) on this topic, but 95% of them involve custom fields and the rest 5% is far from what i needed. A good place for a head start if you’re overriding a template (creating a custom rendering) for a field is here (this covers basic stuff and has some examples which at least are consistent). If you’re creating a new custom field and want custom rendering for it – this is a good place to start for a series or articles (the examples are inconsistent, though, and might be misleading sometimes, but still a good read). And last but not least – this guy here. I was on my own on this one, but first things first…

The requirements were to override Task list field validation. The fields had to be SP-Built-in-ones, unmodified in any way (e.g. Status, Start Date, Due Date and etc.). This is relatively easily done in the forms (we ended up using custom actions, though we could use rendering template overrides as well). But what about list quick edit? The easiest way would have been to create a bunch of calculated fields but that would render them all as read only in quick edit mode and the users wouldn’t get any error messages. So we had to go with something more interesting and challenging – field template overrides. The first thing you do is “throw the hook” – you either specify the JS Link property of a field, view, or in my case – list view web part.

You can do it through the UI: Edit Page -> Edit Web Part -> Miscellaneous -> JS Link and there you specify a site collection relative path of your freshly-created JavaScript file:

web part properties
Or you can do this using Server-Side Object Model:

using (SPLimitedWebPartManager webPartManager = defaultPage.GetLimitedWebPartManager(PersonalizationScope.Shared))
{
    var tasksListViewWebPart= webPartManager.WebParts["taskWP"];
    tasksListViewWebPart.JSLink = "/_layouts/15/QEValidation.js";
    webPartManager.SaveChanges(tasksListViewWebPart);
}

or PowerShell:

$web = Get-SPWeb http://sp2013
$webPartPage = "/home.aspx"
$webPartManager = $web.GetLimitedWebPartManager($webPartPage, [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)
$webpart = $webPartManager.WebParts["taskWP"]
$webpart.JSLink = "/_layouts/15/QEValidation.js";
$webPartManager.SaveChanges($webpart)

Not all web parts have a JS Link property. You won’t be able to set it to an XsltListViewWebPart or any other web part that is not inherited from IListWebPart. The best approach, in my opinion, is setting the JS Link property of an SPView – this would work everywhere the view is being used.

Now what we need is an entry point – a JavaScript file that is loaded when a list web part is being rendered for the user. All the rendering will depend on the contents of template override methods. Since we are specifying a JS file and not a particular method, we need to execute our code somehow. This is done via an IIFE. Basically we are declaring an anonymous JavaScript function at the top of the file which will be invoked automatically. So first thing we do is handle the “load” event, add field template overrides and execute a function called “onInit” in sp.js context (the latter is needed to load some variables from SharePoint which will be used in field rendering):

(function () {
    if (typeof SPClientTemplates === 'undefined')
        return;

    fieldContext = {};
    fieldContext.Templates = {};

    fieldContext.Templates.Fields = {
        "LinkTitle": {
            "View": TitleFieldReadOnlyViewTemplate },

    fieldContext.Templates.Fields = {
        "LinkTitle": {
            "View": TitleFieldReadOnlyViewTemplate },

    [....]

    SPClientTemplates.TemplateManager.RegisterTemplateOverrides(fieldContext);

    $(document).ready(function () {
        SP.SOD.executeFunc('sp.js', 'SP.ClientContext', onInit);
    });
})();

We need to register template overrides before onInit, because latter will be executed asynchronously. Doing this the opposite way (registering template overrides at the end of onInit) would cause problems as all the fields would have already been rendered using default templates. Since field rendering (read-only field, different errors) is dependent on some SharePoint variables (like current user) we will need to include checks in the rendering methods. You can add New, Edit and Display form rendering overrides as well by doing this:

fieldContext.Templates.Fields = {
    "LinkTitle": {
        "View": TitleFieldReadOnlyViewTemplate,
        "DisplayForm": TitleFieldDisplayFormTemplate,
        "NewForm": TitleFieldNewFormTemplate,
        "EditForm": TitleFieldEditFormTemplate }
     };

What you specify on the right-hand side of the colon (e.g. TitleFieldReadOnlyViewTemplate) is a function which will be responsible for field rendering and which should return field contents as HTML. I will not be covering other than View templates in this article. Let’s continue with an example of the rendering method:

function TitleFieldReadOnlyViewTemplate(ctx) {
    var columnName = "Title";
    var quickEditId = "QEDIT_TITLE";

    ReadOnlyViewTemplateInstance(ctx, columnName, quickEditId);

    return RenderField(ctx, columnName);
}
function ReadOnlyViewTemplateInstance(ctx, columnName, quickEditId) {
    cellContext = ctx;

        SP.SOD.executeOrDelayUntilScriptLoaded(function () {
            SP.GanttControl.WaitForGanttCreation(function (ganttInstance) {

                ChangeColumnGetEditControlName(ganttInstance, columnName, quickEditId);

                    SP.JsGrid.PropertyType.Utils.RegisterEditControl(quickEditId, function (ctx) {
                        var editorInstance = new SP.JsGrid.EditControl.EditBoxEditControl(ctx, null);

                        editorInstance.SetValue = function (value) {
                            if (cellContext.inGridMode === true) {
                                _cellContext = editorInstance.GetCellContext();
                                editorInstance._cellContext = _cellContext;

                                if (IsUserOperationManager == true) {
                                    _cellContext.SetCurrentValue({ localized: value });
                                } else if (value != _cellContext.originalValue.localized) {
                                    ctx.jsGridObj.SetCellError(_cellContext.record.recordKey, columnName, "You cannot change the value of this field");
                                }
                            }
                        };

                        editorInstance.Unbind = function () {
                            if (cellContext.inGridMode === true) {
                                _cellContext = editorInstance._cellContext;

                                if (_cellContext != null) {
                                    ctx.jsGridObj.ClearAllErrorsOnCell(_cellContext.record.recordKey, columnName);
                                }
                            }
                        };

                        return editorInstance;
                    }, []);

            });
        }, "spgantt.js");
}

TitleFieldReadOnlyViewTemplate does two things:

  • Registers a new edit control and overrides it’s methods (ReadOnlyViewTemplateInstance);
  • and

  • Renders the field depending on it’s type and returns field contents as HTML (RenderField).

Let’s get to the most interesting part of this slop – ReadOnlyViewTemplateInstance method. As I mentioned before – there is absolutely no documentation about this, but the files involved in the background logic are: spgantt.js and jsgrid.js. Basically, every time a cell is rendered, an SP.GanttControl is created. Then it’s Init() method is called which therefore creates an instance of SP.JsGrid.JsGridControl. Let’s take a closer look at the spaghetti code from a bit earlier:

SP.SOD.executeOrDelayUntilScriptLoaded(function () {
    SP.GanttControl.WaitForGanttCreation(function (ganttInstance) {
    });
}, "spgantt.js");

This part gets called every time a Gantt control is being created. Great, this is exactly what we want! What we do not want is the Gantt control creating a default SP.JsGrid.JsGridControl – we want it customized. So the next thing we do is change column function fnGetEditControlName so it returns our unique control id (ChangeColumnGetEditControlName) and then we override the creation of the SP.JsGrid.JsGridControl itself:

SP.JsGrid.PropertyType.Utils.RegisterEditControl(quickEditId, function (ctx) {
    var editorInstance = new SP.JsGrid.EditControl.EditBoxEditControl(ctx, null);

    return editorInstance;
}, []);

Great! Now we have a default edit control. We need to override a couple of methods and we’ll be good to go:

editorInstance.SetValue = function (value) {
    if (cellContext.inGridMode === true) {
        _cellContext = editorInstance.GetCellContext();
        editorInstance._cellContext = _cellContext;

        if (IsUserOperationManager == true) {
            _cellContext.SetCurrentValue({ localized: value });
        } else if (value != _cellContext.originalValue.localized) {
            ctx.jsGridObj.SetCellError(_cellContext.record.recordKey, columnName, "You cannot change the value of this field");
        }
    }
};

We want to interact only when users are changing data, so first we check if the cell is in edit mode (cellContext.inGridMode). When a user enters a value in the cell, SP.JsGrid.EditControl.EditBoxEditControl.SetValue is being called. By default this method sets a new value without doing any checks. So first we need to check if the user has permissions to edit this field and then either set a new value or display a cell error.

editorInstance.Unbind = function () {
    if (cellContext.inGridMode === true) {
        _cellContext = editorInstance._cellContext;

        if (_cellContext != null) {
            ctx.jsGridObj.ClearAllErrorsOnCell(_cellContext.record.recordKey, columnName);
        }
    }
};

This method is being called when cell is loosing focus. If a user doesn’t have permissions and a new value is not set, we clear any existing errors and unbind the cell. We could have registered a DisplayControl instead of EditControl, but we need to override templates earlier than get user permissions – so the only way to go is register an EditControl and make it read-only when needed.

That’s about it if you want to change a simple text box and add some custom validation to it. If you want to go deeper, take a look at JsGrid.debug.js. You’ll find that an EditBoxEditControl is a small drop in the ocean. If you want to get a good grasp of how this stuff works get down to bedrock of SP.JsGrid.EditControl.EditBoxEditControl class.

Hope this helps!

Paul.

The main advantage of using Hyper-V with differencing disks is disk size. Building 2 or 3 separate machines (SP, AD and SQL) might take up to 140GB of disk space. And the SSDs are still expensive these days. Using differencing disks and 2 VM build (SP/SQL + AD) will use only up to 80GB. That’s right – with all the updates, SQL server and Visual Studio 2012 installed.

structure

The diagram above explains the structure of the virtual machines. Windows Server 2012 Parent VM acts as a parent virtual machine. SP 2013 and AD machines are created using that parent’s virtual disk with differencing option.

 

Parent VM

1. Create a new Windows Server 2012 Standard Virtual Machine. Select New in Hyper-V.

a1

2. Assign a name and set storage location.

2

3. Assign memory. Don’t use Dynamic Memory as it doesn’t work with SharePoint well.

s1

4. Configure networking to use external connection. Connections are managed in Virtual Switch Manager in Hyper-V.

4

5. Select Create a virtual disk. Set size to 60GB.

5

If you need to expand disk size later follow these steps:

  • Right click on virtual machine (Hyper-V). Select Settings…
  • Select Hard Drive. Click Edit…
  • Select Expand. Click Next…
  • Specify new size. Click Finish.
  • Start the virtual machine. Go to Administrative Tools -> Computer Management. Select Disk Management. Right click on C: drive. Select Extend Volume…. Specify new size and click Finish.

 6. Select Windows Server 2012 image and install the OS

6

7. Get it to a state that has all you need on all machines.

Common settings:

  • Configure and install Windows Updates.
  • Disable Windows Firewall.
  • Other desired configuration.
  • sysprep it. Run a command window and enter “C:\Windows\System32\Sysprep\sysprep.exe”. Select Generalize and Shutdown. Click OK.

7

Note: Do not start this machine. Ever. If you start this one all other VMs created from this one will stop working.

 

Active Directory VM

Skip these steps if you want to set up AD and SharePoint on the same VM. In that case make sure you install SharePoint and all its prerequisites before going with the AD installation and use dcpromo to install AD instead of using server manager.

1. Create a new Hard Disk. Select New in Hyper-V.

1

2. Select VHDX virtual disk type.

a2

3. Select Differencing.

a3

4. Specify Name and Location of the new disk.

a4

5. Specify a parent virtual disk created previously.

a5

6. Create a new virtual machine. Select New in Hyper-V.

7. Assign a name and set storage location.

a6

8. Assign memory. Don’t use Dynamic Memory.

a7

9. Set connection to Internal. Connections are managed in Virtual Switch Manager in Hyper-V.

a8

10. Select Use an existing virtual hard disk, select the child hard disk created earlier.

a9

11. Run the VM.

12. Launch Server Manager.

13. Click on Add roles and features.

14. Select Role-based or feature-based installation.

15. Add Active Directory Domain Services and DNS Server roles.

a12

16. In Active Directory Domain Services Configuration Wizard select Add a new forest. Specify a name for your domain.

a13

17. Select Next. Fill Domain Controller Options.

a14

18. Follow other steps and complete the installation.

 

 SharePoint VM

The steps are practically the same as creating the AD machine. Follow the previous steps and just change the name from AD to SP:

1. Create a new Hard Disk. Select New in Hyper-V.

2. Select VHDX virtual disk type.

3. Select Differencing.

4. Specify Name and Location of the new disk.

5. Specify a parent virtual disk created previously.

6. Create a new virtual machine. Select New in Hyper-V.

7. Assign a name and set storage location.

8. Assign memory. Don’t use Dynamic Memory.

s1

9. Configure networking to use external connection. Connections are managed in Virtual Switch Manager in Hyper-V.

s2

10. Select Use an existing virtual hard disk, select the child hard disk created earlier.

s3

11. Right Click on SharePoint VM and select Settings…

12. Select Add Hardware and add an internal connection switch.

13. Connect to SharePoint VM, set network configuration so the internal network adapter is in the same network as the AD machine’s adapter. Connect to the domain created on Active Directory VM. Proceed with SQL and SharePoint installation.

SharePoint installation will not be covered in this guide.

Hope this helps!

Paul.

Recently I’ve been having a hard time dealing with managed metadata default values. I’ve been trying to set default values with a piece of code from SharePoint 2010. I guess this is one out of few places where things got changed from previous version.

First things first – scenario: Users add an item to the list. Then an event receiver creates a site collection. 2 managed metadata fields in the newly created site need to have default values set.

This was doable with a fewer lines of code in SharePoint 2010:

1. Get the term and format a default value. Notice the “-1;#” in String.Format(). This is what causes problems in SharePoint 2013:

[..Some code omitted for briefness..]
var terms = placesTermSet.GetTerms(“Altis”, true);
var defaultTerm = terms[0];
var defaultValue = String.Format(“-1;#{0}{1}{2}”, defaultValue.Labels[0].Value, TaxonomyField.TaxonomyGuidLabelDelimiter, defaultValue.Id);

2. Assign it to the DefaultValue property of the field and update it:

field.DefaultValue = defaultValue;
field.Update(true);

SharePoint 2013 in this case doesn’t like the “-1;#” anymore. This should be replaced with a WssId – a unique term ID from TaxonomyHiddenList. This is unique for every site collection. We need to get that before assigning a default value:

1. Get the taxonomy field which will have the default value:

var field = targetSite.Fields.GetFieldByInternalName(“Places”) as TaxonomyField;
var fieldValue = new TaxonomyFieldValue(field);

2. Get the term which will be used as a default:

[..Some code omitted for briefness..]
var terms = placesTermSet.GetTerms(“Altis”, true);
var defaultTerm = terms[0];

3. Get the WssId, format a default value string, assign it and update the field:

fieldValue.TermGuid = Convert.ToString(defaultTerm.Id);
var dummy = fieldValue.ValidatedString;
if (dummy != null)
{
var defaultValue = String.Format(“{0};#{1}{2}{3}”, fieldValue.WssId, defaultValue.Labels[0].Value, TaxonomyField.TaxonomyGuidLabelDelimiter, defaultValue.Id);
field.DefaultValue = defaultValue;
field.Update(true);
}

Hope this helps!

Paul.