Tuesday, November 29, 2011

SharePoint Server 2010 User Profile

There's no big change between SharePoint Server 2010 and MOSS 2007 in terms of User Profile management. The MOSS 2007 user profile database attached to SSP (Shared Service Provider) can be mounted directly to SharePoint Server 2010 User Profile Service Application. The mount process actually upgrades the MOSS 2007 user profile database. You will notice some new tables and columns are added in the backend database. Following image shows the scheme change made on the UserProfile_Full table in user profile database.

The User Information List is still there in each top site collection where SharePoint foundation manitains the basic user data. A record will be added to the User Information List when a user or a group is first referenced by the SharePoint environment. The initial User Information data, like title, user login, email address, etc., are imported from the authentication provider such as Active Directory. In SharePoint Foundation 2010 the User Information List is updated when you modify user info through "My Settings". All User Information List data are stored in the UserInfo table in content database:

Once the User Profile Service is enabled in SharePoint Server 2010, the User Information List became read-only for end user except the "Mobile Number" property, and user profile database is updated when the user modifying "My Settings", instead of the User Information List (UserInfo table). A timer job is responsible for replicating the user profile data to User Information data for each top site collection, and User Information List data will be overridden unless the corresponding property is not set to "Replicable" in User Profile Service:

Sometime it would cause some confusion when the User Information data are not synced to the User Profile data. Remember regular SharePoint sites always get user info from the User Information List but the User Profile content are the original data source when User Profile Service is enabled.

There's slight difference in SharePoint 2010 when accessing or updating the user profile data comparing with SharePoint 2007. In MOSS 2007, Microsoft.Office.Server.ServiceContext is available and you can create a UserProfileManager object from a ServiceContext. In SharePoint 2010 ServiceContext is replaced by the new Microsoft.SharePoint.SPServiceContext:

void UpdateUserProfiles(string siteUrl)
{
string userName = null;
using (SPSite spSite = new SPSite(siteUrl))
{
//ServiceContext is obsolete in SharePoint 2010
//UserProfileManager userProfileManager = new UserProfileManager(ServerContext.GetContext(spSite));

UserProfileManager userProfileManager = new UserProfileManager(SPServiceContext.GetContext(spSite));

foreach (UserProfile profile in userProfileManager)
{
string account = Convert.ToString(profile[PropertyConstants.AccountName].Value);
if (string.IsNullOrEmpty(account) || account.IndexOf("\\") <= 0)
continue;
userName = account.Substring(account.IndexOf("
\\"));
profile[PropertyConstants.PictureUrl].Value = GetUserPhotoLocation(userName);
profile.Commit();
}
}
}
The method simply goes through each user profile, and updates the user photo by some custom logic (GetUserPhotoLocation method). Notice Microsoft.Office.Server.UserProfiles has become a separate dll in SharePoint 2010, so in order to get above code compiled you need to add Microsoft.Office.Server and Microsoft.Office.Server.UserProfiles to the references.

Thursday, October 27, 2011

Invoke SharePoint Workflow Programmatically

There's some restricted SharePoint sites and content in your SharePoint environment, and users can only interact with secure SharePoint resources through custom webpart(s). A typical example is performance review system. For simplicity let's say each department has a site or subsite to store the review data. You create a SPList for each year's review like PerformanceReview-2011, PerformanceReview-2012, etc. Only the management team members of a department can have direct access to the department performance review site. Every employee can input the data and respond manager's question/review through a custom webpart. Usually RunWithElevatedPrivileges is used inside the WebPart to get through the permission issue for SPList content update as code below:
    SPSecurity.RunWithElevatedPrivileges(delegate()
    {
        using (SPSite site = new SPSite(siteID))
        {
            using (SPWeb web = site.OpenWeb(webID))
            {
                SPList list = web.Lists.TryGetList(listName);
                //Update List Item ...
            }
        }
    });
A workflow associated with the List is supposed to automatically kick off doing some extra stuff such as assigning task to corresponding managers. The problem is that the execution context of above code is system account, and updating List Item with System account won't trigger the workflow. So you have to start the workflow manually inside the WebPart:
    // Start a workflow manually since system account won't trigger workflow automatically
    private static void StartWorkflow(SPListItem listItem, string workflowName)
    {
        try
        {
            var listWorkflowAssociations = listItem.ParentList.WorkflowAssociations;
            foreach (SPWorkflowAssociation wfAssoication in listWorkflowAssociations)
            {
                if (wfAssoication.Name == workflowName)
                {
                    listItem.Web.Site.WorkflowManager.
                      StartWorkflow(listItem, wfAssoication, wfAssoication.AssociationData, true);
                    break;
                }
            }
        }
        catch (Exception ex)
        {
            // Exception handling
        }
    }
Bear in mind that the Associator, Initiator and Current User inside Workflow context all are system account with such implementation. You have to use People field defined in List Item to reference the original user.

Friday, October 21, 2011

Event Receivers Issue After Migration From SharePoint 2007 to SharePoint 2010

We have a SharePoint 2007 site created by Reservations Template (part of WSS 3.0 Application Templates). The site is used for company-wide meeting room booking. After migrating to SharePoint 2010 (database-attachment approach) the room-booking system was not working properly. Same resource can be doubly booked and sometimes you can't book some resources while they show as available in the UI.

After some digging I found the problem was caused by the Event Receiver associated with the Reservations List. Originally in SharePoint 2007 the Reservations List has registered the Event Receivers with assembly of "ReservationEventHandler, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c". That assembly registration became "ReservationEventHandler, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" after migration for some reason. To fix the problem, we simply correct the List's event registry:
static void FixReservationEventReceivers(SPSite site, string webUrl, string listName)
{
    using (SPWeb web = site.OpenWeb(webUrl))
    {
        bool hasAddingEvent = false, hasDeletingEvent = false, hasUpdatingEvent = false;
        SPList list = web.Lists.TryGetList(listName);
        List<SPEventReceiverDefinition> invalidReceivers = new List<SPEventReceiverDefinition>();
        foreach (SPEventReceiverDefinition item in list.EventReceivers)
        {
            if (item.Assembly.Contains("ReservationEventHandler") && item.Assembly.Contains("Version=14.0.0.0"))
                invalidReceivers.Add(item);
            if (item.Assembly.Contains("ReservationEventHandler") && item.Type == SPEventReceiverType.ItemAdding)
                hasAddingEvent = true;
            if (item.Assembly.Contains("ReservationEventHandler") && item.Type == SPEventReceiverType.ItemDeleting)
                hasDeletingEvent = true;
            if (item.Assembly.Contains("ReservationEventHandler") && item.Type == SPEventReceiverType.ItemUpdating)
                hasUpdatingEvent = true;
        }
        foreach (var item in invalidReceivers)
        {
            item.Delete();
        }

        string assemblyName = "ReservationEventHandler, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c";
        string className = "Microsoft.SharePoint.ApplicationTemplates.ReservationEventHandler";
        if (!hasAddingEvent)
            list.EventReceivers.Add(SPEventReceiverType.ItemAdding, assemblyName, className);
        if (!hasDeletingEvent)
            list.EventReceivers.Add(SPEventReceiverType.ItemDeleting, assemblyName, className);
        if (!hasUpdatingEvent)
            list.EventReceivers.Add(SPEventReceiverType.ItemUpdating, assemblyName, className);
    }
}
You may wonder what those event receivers do with the List item. The explanation is lengthy but not worthy knowing to be honest. The template was not well designed and implemented in my opinion. I spent quite a bit of time to customize and optimize the template. Actually we don't need those event receivers anymore after my modification and the performance of the booking system has been improved significantly since then (hundreds of thousands of records in our Reservations List). I may write some notes about that later.

Monday, October 17, 2011

XsltListViewWebPart Custom View and XSL

SharePoint 2010 introduces a new XsltListViewWebPart that is inherited from DataFormWebPart. One great feature in XsltListViewWebPart is that you can define query and fields easily and clearly. Following XML declaration demonstrates how to set a XsltListViewWebPart to show open tasks assign to current user, and link to external custom XSL to display the items.
<WebPartPages:XsltListViewWebPart runat="server" IsIncluded="True" GhostedXslLink="main.xsl" 
    ListDisplayName="Tasks" NoDefaultStyle="TRUE" ViewFlag="8" Title="Tasks" PageType="PAGE_NORMALVIEW"  
    Default="FALSE" DisplayName="Open Tasks" DetailLink="Lists/Tasks" __markuptype="vsattributemarkup" 
    partorder="3"  viewcontenttypeid="0x" >
    <XmlDefinition>
        <View MobileView="TRUE" Type="HTML" DisplayName="All Tasks" Url="Lists/Tasks/AllItems.aspx" 
            Level="1" BaseViewID="1" ContentTypeID="0x" ImageUrl="/_layouts/images/issues.png">
            <Query>
                <OrderBy>
                    <FieldRef Name="Status" Ascending="TRUE"/>
                </OrderBy>
                <GroupBy></GroupBy>
                <Where>
                    <And>
                        <Eq>
                            <FieldRef Name="AssignedTo"/><Value Type="Integer"><UserID/></Value>
                        </Eq>
                        <Neq>
                            <FieldRef Name="Status"/><Value Type="Text">completed</Value>
                        </Neq>
                    </And>
                </Where>
            </Query>
            <ViewFields>
                <FieldRef Name="LinkTitleNoMenu"/>
                <FieldRef Name="Status"/>
                <FieldRef Name="Priority"/>
                <FieldRef Name="DueDate"/>
                <FieldRef Name="PercentComplete"/>
            </ViewFields>
            <RowLimit Paged="TRUE">30</RowLimit>
            <Toolbar Type="Freeform" />
        </View>
    </XmlDefinition>
    <XslLink>Site Assets/XSL/OpenTask.xsl</XslLink>
    <parameterbindings>
        <ParameterBinding Name="dvt_sortdir" Location="Postback;Connection"/>
        <ParameterBinding Name="dvt_sortfield" Location="Postback;Connection"/>
        <ParameterBinding Name="dvt_startposition" Location="Postback" DefaultValue=""/>
        <ParameterBinding Name="dvt_firstrow" Location="Postback;Connection"/>
        <ParameterBinding Name="OpenMenuKeyAccessible" Location="Resource(wss,OpenMenuKeyAccessible)"/>
        <ParameterBinding Name="open_menu" Location="Resource(wss,open_menu)"/>
        <ParameterBinding Name="select_deselect_all" Location="Resource(wss,select_deselect_all)"/>
        <ParameterBinding Name="idPresEnabled" Location="Resource(wss,idPresEnabled)"/>
        <ParameterBinding Name="NoAnnouncements" Location="Resource(wss,noXinviewofY_LIST)"/>
        <ParameterBinding Name="NoAnnouncementsHowTo" Location="Resource(core,noXinviewofY_DEFAULT)"/>
        <ParameterBinding Name="AddNewAnnouncement" Location="Resource(wss,addnewitem)"/>
        <ParameterBinding Name="MoreAnnouncements" Location="Resource(wss,moreItemsParen)"/>
        <ParameterBinding Name="UserID" Location="CAMLVariable" DefaultValue="CurrentUserName"/>
    </parameterbindings>
</WebPartPages:XsltListViewWebPart>
The custom XSL does three things:

1. Replace priority string from "(2) Normal" by "Normal"
2. Replace default empty message "There are no items to show in this view of the {ListName} list. To add a new item, click 'New'." by "No item"
3. Replace the SP2010 Add New Item icon to old SP2007 icon

The XSL source:
<xsl:stylesheet xmlns:x="http://www.w3.org/2001/XMLSchema" xmlns:d="http://schemas.microsoft.com/sharepoint/dsp" 
                version="1.0" exclude-result-prefixes="xsl msxsl ddwrt" 
                xmlns:ddwrt="http://schemas.microsoft.com/WebParts/v2/DataView/runtime" 
                xmlns:asp="http://schemas.microsoft.com/ASPNET/20" 
                xmlns:__designer="http://schemas.microsoft.com/WebParts/v2/DataView/designer" 
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
                xmlns:msxsl="urn:schemas-microsoft-com:xslt" 
                xmlns:SharePoint="Microsoft.SharePoint.WebControls" 
                xmlns:ddwrt2="urn:frontpage:internal" 
                xmlns:o="urn:schemas-microsoft-com:office:office">
    <xsl:include href="/_layouts/xsl/main.xsl"/>
    <xsl:include href="/_layouts/xsl/internal.xsl"/> 

    <xsl:template name="FieldRef_body.Priority" match="FieldRef[@Name='Priority']" mode="body">
                <xsl:param name="thisNode" select="."/>
                    <xsl:choose>
                      <xsl:when test="starts-with($thisNode/@*[name()=current()/@Name], '(1) ')">
              <xsl:value-of select="substring-after($thisNode/@*[name()=current()/@Name], '(1) ')"/>
                      </xsl:when>
            <xsl:when test="starts-with($thisNode/@*[name()=current()/@Name], '(2) ')">
              <xsl:value-of select="substring-after($thisNode/@*[name()=current()/@Name], '(2) ')"/>
            </xsl:when>
            <xsl:when test="starts-with($thisNode/@*[name()=current()/@Name], '(3) ')">
              <xsl:value-of select="substring-after($thisNode/@*[name()=current()/@Name], '(3) ')"/>
            </xsl:when>
                    </xsl:choose>
    </xsl:template>
  
  <xsl:template name="EmptyTemplate">
    <tr>
      <td class="ms-vb" colspan="99">No item</td>
    </tr>
  </xsl:template>

  <xsl:template name="Freeform" >
    <xsl:param name="AddNewText" />
    <xsl:param name="ID" />
    <xsl:variable name="Url">
      <xsl:choose>
        <xsl:when test="List/@TemplateType='119'">
          <xsl:value-of select="$HttpVDir" />/_layouts/CreateWebPage.aspx?List=
          <xsl:value-of select="$List" />&amp;RootFolder=
          <xsl:value-of select="$XmlDefinition/List/@RootFolder" />
        </xsl:when>
        <xsl:when test="$IsDocLib">
          <xsl:value-of select="$HttpVDir" />/_layouts/Upload.aspx?List=
          <xsl:value-of select="$List" />&amp;RootFolder=
          <xsl:value-of select="$XmlDefinition/List/@RootFolder" />
        </xsl:when>
        <xsl:otherwise>
          <xsl:value-of select="$ENCODED_FORM_NEW" />
        </xsl:otherwise>
      </xsl:choose>
    </xsl:variable>
    <xsl:variable name="HeroStyle">
      <xsl:choose>
        <xsl:when test="Toolbar[@Type='Standard']">display:none</xsl:when>
        <xsl:otherwise></xsl:otherwise>
      </xsl:choose>
    </xsl:variable>
    <xsl:if test="$ListRight_AddListItems = '1' and (not($InlineEdit) or $IsDocLib)">
      <table id="Hero-{$WPQ}" width="100%" cellpadding="0" cellspacing="0" border="0" style="{$HeroStyle}">
        <tr>
          <td colspan="2" class="ms-partline">
            <img src="/_layouts/images/blank.gif" width="1" height="1" alt="" />
          </td>
        </tr>
        <tr>
          <td class="ms-addnew" style="padding-bottom: 3px">
            <img src="/_layouts/images/rect.gif" alt="Add a new item" />
            <xsl:text disable-output-escaping="yes" ddwrt:nbsp-preserve="yes">&amp;nbsp;</xsl:text>
            <xsl:choose>
              <xsl:when test="List/@TemplateType = '115'">
                <a class="ms-addnew" id="{$ID}-{$WPQ}" href="{$Url}" 
                   onclick="javascript:NewItem2(event, &quot;{$Url}&quot;);javascript:return false;" target="_self">
                  <xsl:value-of select="$AddNewText" />
                </a>
              </xsl:when>
              <xsl:otherwise>
                <a class="ms-addnew" id="{$ID}" href="{$Url}" 
                   onclick="javascript:NewItem2(event, &quot;{$Url}&quot;);javascript:return false;" target="_self">
                  <xsl:value-of select="$AddNewText" />
                </a>
              </xsl:otherwise>
            </xsl:choose>
          </td>
        </tr>
        <tr>
          <td>
            <img src="/_layouts/images/blank.gif" width="1" height="5" alt="" />
          </td>
        </tr>
      </table>
      <xsl:choose>
        <xsl:when test="Toolbar[@Type='Standard']">
          <script type='text/javascript'>
            if (typeof(heroButtonWebPart <xsl:value-of select="$WPQ" />) != "undefined") {
            <xsl:value-of select="concat(' var eleHero = document.getElementById(&quot;Hero-', $WPQ, '&quot;);')" />
            if (eleHero != null) eleHero.style.display = ""; }
          </script>
        </xsl:when>
        <xsl:otherwise></xsl:otherwise>
      </xsl:choose>
      <xsl:if test="List/@TemplateType = '115'">
        <script type='text/javascript'>
          if (typeof(DefaultNewButtonWebPart <xsl:value-of select="$WPQ" />) != "undefined") {
          <xsl:value-of select="concat(' var eleLink = document.getElementById(&quot;', $ID, '-', $WPQ, '&quot;);')" />
          if (eleLink != null) { DefaultNewButtonWebPart <xsl:value-of select="$WPQ" /> (eleLink); } }
        </script>
      </xsl:if>
    </xsl:if>
  </xsl:template>
</xsl:stylesheet>
Note that linking external XSL in XsltListViewWebPart would have performance impact since a compilation process would occur each time the WebPart is loaded, and the WebPart content can't be cached anymore. So it would be better to use inline XSL if there're many WebParts in a page.

Thursday, October 13, 2011

Using SharePoint WebProvisioned Event to Update Newly Created Site

I've been working on a SharePoint 2010 Site Definition that includes a bunch of Lists and a default page, and the new site (SPWeb) built by the Site Definition should keep old SharePoint 2007 UI because the users are used to the old looks and feels. A DataFormWebPart on the default page shows recent updates for a few different Lists. The problem is that those ListIDs are hard-coded in side DFWP, but ListIDs will be changed when a new site is created. Some tweaks on DFWP, e.g. changing ListID to ListName, could make it reusable, but that only works for single datasource but won't work for multiple datasources (AggregateDataSource). I tried AllUsersWebPart and View approach to provision the WebPart instance to the default page but without luck.

To resolve the problem I use tokens in the DFWP and replace those tokens with real ListIDs after site is created, as following:
<SharePoint:SPDataSource runat="server" DataSourceMode="List" UseInternalName="true" 
            UseServerDataFormat="true" selectcommand="&lt;View&gt;&lt;/View&gt;">
    <SelectParameters><asp:Parameter Name="ListID" DefaultValue="$LIST_TOKEN:{Project Documents}$"/>
    </SelectParameters>
    <DeleteParameters><asp:Parameter Name="ListID" DefaultValue="$LIST_TOKEN:{Project Documents}$"/>
    </DeleteParameters>
    <UpdateParameters><asp:Parameter Name="ListID" DefaultValue="$LIST_TOKEN:{Project Documents}$"/>
    </UpdateParameters>
    <InsertParameters><asp:Parameter Name="ListID" DefaultValue="$LIST_TOKEN:{Project Documents}$"/>
    </InsertParameters>
</SharePoint:SPDataSource>
In above DFWP configuration the "$LIST_TOKEN:{Project Documents}$" token will be replaced by the List/Library ID of "Project Documents". The other issue is that sometimes some WebParts on the default page are automatically closed after the new site is provisioned for some unknown reason. So there're a few issues I need to fix for the Site Definition:

1. Convert SP2010 site back to SharePoint 2007 UI
2. Change master page and css
3. Change DFWP's list Ids on default page
4. Open closed WebParts if exists

The solution is use SPWebEventReceiver to update site properties and the default page:
using System;
using System.Security.Permissions;
using System.Text.RegularExpressions;
using System.Web.UI.WebControls.WebParts;

using Microsoft.SharePoint;
using Microsoft.SharePoint.WebPartPages;

namespace SiteDefinition
{
    /// <summary>
    /// Web Provision Events
    /// </summary>
    public class INGProjectSiteEventReceiver : SPWebEventReceiver
    {
        /// <summary>
        /// Update a site (SPWeb) when it's created
        /// 1. Convert site back to SharePoint 2007 UI
        /// 2. Update master page and css file
        /// 3. Update DFWP's list Ids on default page
        /// 4. Open closed WebParts if exists
        /// </summary>
        public override void WebProvisioned(SPWebEventProperties properties)
        {
            if (properties.Web.WebTemplateId == 11001) // Only applied to targetted template
            {
                properties.Web.UIVersion = 3;
                properties.Web.MasterUrl = properties.Web.Site.RootWeb.MasterUrl;
                properties.Web.AlternateCssUrl = properties.Web.Site.RootWeb.AlternateCssUrl;
                properties.Web.Update();

                UpdateDefaultPageWebPartListIDs(properties.Web);
                OpenClosedWebParts(properties.Web);
            }

            base.WebProvisioned(properties);   
        }

        /// <summary>
        /// Update default page's DataFormWebParts' ListID property with corresponding List ID (GUID)
        /// </summary>
        private void UpdateDefaultPageWebPartListIDs(SPWeb web)
        {

            SPFile defaultPage = web.RootFolder.Files["Default.aspx"];
            if (defaultPage.Exists)
            {
                System.Text.ASCIIEncoding coding = new System.Text.ASCIIEncoding();
                byte[] byteArrayText = defaultPage.OpenBinary();

                if (byteArrayText.Length > 0)
                {
                    string origHtml = coding.GetString(byteArrayText);
                    string newHtml = ReplaceDataSourceToken(origHtml, web);
                    if (!string.IsNullOrEmpty(newHtml))
                    {
                        byte[] newByteArray = coding.GetBytes(newHtml);
                        defaultPage.SaveBinary(newByteArray);
                    }
                }
            }
        }

        /// <summary>
        /// Pre-defined tokens are used inside default page and we need to replace them here:
        /// e.g. "\$LIST_TOKEN:{Tasks}\$" will be replaced with Tasks' List ID (GUID)
        /// </summary>
        private string ReplaceDataSourceToken(string text, SPWeb web)
        {
            string pattern = @"\$LIST_TOKEN:{.*?}\$";
            Match mc = Regex.Match(text, pattern);
            if (mc.Success)
            {
                while (mc.Success)
                {
                    if (!string.IsNullOrEmpty(mc.Value))
                    {
                        string listName = mc.Value.Substring(13, mc.Length - 15);

                        SPList list = web.Lists.TryGetList(listName);
                        if (list != null)
                            text = string.Format("{0}{1}{2}",  
                                text.Substring(0, mc.Index), list.ID.ToString("B"), text.Substring(mc.Index + mc.Length));
                        else
                            text = string.Format("{0}$LIST_NOT_FOUND:[{1}]{2}",
                                text.Substring(0, mc.Index), listName, text.Substring(mc.Index + mc.Length);
                        
                        mc = Regex.Match(text, pattern);
                    }
                }
            }
            return text;
        }

        /// <summary>
        /// Sometimes WebParts are closed automatically and this method opens all closed webParts in the default page
        /// </summary>
        private void OpenClosedWebParts(SPWeb web)
        {
            SPFile defaultPage = web.RootFolder.Files["Default.aspx"];
            if (defaultPage.Exists)
            {
                using (SPLimitedWebPartManager wpManager = 
                    defaultPage.GetLimitedWebPartManager(PersonalizationScope.Shared))
                {
                    foreach (Microsoft.SharePoint.WebPartPages.WebPart wp in wpManager.WebParts)
                    {
                        if (wp.IsClosed)
                        {
                            wpManager.OpenWebPart(wp);
                            wpManager.SaveChanges(wp);
                        }
                    }                
                }
            }
        }
    }
}

Tuesday, September 20, 2011

SharePoint 2010 SPList Backup and Restore

In SharePoint 2010 it's very easy to backup and restore a Site Collection, a Site, or a List. For example if want to backup a SPList, just go to Central Admin => Backup and Restore => Export a site or list, select the List, type the backup file name then click the "Start Export" button and all done. Alternatively we can use powershell to do the same job:
Export-SPWeb -Identity "https://portal.intranet.com/department/it" 
             -ItemUrl "/department/it/Lists/Projects" 
             -Path c:\backup\ITProjectsList.cmp
             -IncludeUserSecurity
To restore the List run the script:
Import-SPWeb -Identity "https://backup.intranet.com/department/it"
             -Path c:\backup\ITProjectsList.cmp
             -IncludeUserSecurity 
             -Force
Be aware that the backup could result in multiple files depending on the SPList size and the machine's setting. If that's the case and you miss any backup file with .cmp extension, you will get following error when run the Import-SPWeb command:
Import-SPWeb: Could not find file 'C:\Users\{user}\AppData\Local\Temp\{guid}\ExportSettings.xml'...
Simply copying all the backup files to the restore location would resolve the problem.

Thursday, May 19, 2011

ASP.NET MVC Controller Out of Control

MVC pattern and Microsoft ASP.NET MVC implementation are quite popular and successful in recent years for many reasons such as modularity, readability, maintainability, and separation of concerns. However, out-of-box ASP.NET MVC may not fit well in some scenarios. You won't get the goodness of MVC without proper implementation. Recently when I reviewed one of our ASP.NET MVC applications I just found another example of how a pattern could become anti-pattern.

The ASP.NET MVC application is an internal authoring system letting people to manage a bunch of products and their attributes. The products then can be published and exported to a list of XML files that are used by front-end servers to display to public. This application's base domain model was quite simply: a Product has many properties and a Product belongs to one or more Category. The complexity goes to product properties and business logic behind them. That resulted in a huge Product controller in the implementation. The controller class is so big that the developer split it to a couple of partial classes to avoid thousands lines of code in one class file. That doesn't smell good at all.

The Product management UI has a few tabs which group different properties in a logical way. AJAX has been extensively used to populate and update those partial Product properties. Each UI component (partial view) has at least two corresponding controller actions, one for read and one for date. The controller was too complex and was doing too much: input and validation, data manipulation (CRUD), view mapping etc. It looks like going back to the old WebForm model and the controller is just like the code-behind we used to do in traditional ASP.NET.

This sounds like a common problem. Google "ASP.NET MVC big fat controller" there're thousands of results. Many people have been struggling on similar pain. Also many people provided suggestions or solutions to resolve the problem. I found one article with great insight on this topic that posted long time ago right after ASP.NET MVC 1.0 was released.

A handful esteemed people even built a new MVC framework on top of ASP.NET from scratch called FubuMVC when they found ASP.NET MVC had the problem along with other issues. If you want to stick with ASP.NET MVC, there're some good thoughts to slim the controller and make the applicaton more readable, extendable and maintainable:

1. ControllerLess action by one controller one action.
2. Grouping controllers.
3. Use AutoMapper to simplify the property mapping between different models.
4. Move logic and code to service layer as much as possible.
5. Use Command pattern to distribute responsibilities.

The last one Command pattern can be extended to Command Query Responsibility Segregation (CQRS) pattern, referring to this channel-9 video about applying CQRS to ASP.NET MVC 3.

Above ideas can be used together when we are doing ASP.NET MVC application developing or refactoring. It sure takes more effort at the beginning but it would pay off in the long run.

Tuesday, April 26, 2011

Dynamically Invoke Generic Method In .NET


Today I cleaned up a lengthy method in our project which I think it's worthy of a post. Our project has a repository that caches all business objects, we called them entities and the entity list is long. The cache is loaded/refreshed when repository is first accessed or some entities have been changed in the back-end store. The method for loading entities is something like:
void LoadEntitiesCache()
{
    List<EntityType1> entity1List = DataStore.GetAllEntitiesByType<EntityType1>();
    SaveEntitiesToCache(Entity1List);
    List<EntityType2> entity2List = DataStore.GetAllEntitiesByType<EntityType2>();
    SaveEntitiesToCache(Entity2List);
    List<EntityType3> entity3List = DataStore.GetAllEntitiesByType<EntityType3>();
    SaveEntitiesToCache(Entit32List);
    ....
    List<EntityTypeN> entityNList = DataStore.GetAllEntitiesByType<EntityTypeN>();
    SaveEntitiesToCache(EntityNList);
}
The way of my clean-up is something like:
void LoadEntitiesCache()
{
    Type [] allEntityTypes = new Type [] { EntityType1, EntityType2, ..., EntityTypeN };
    foreach(Type entityType in allEntityTypes)
    {
        List<entityType> entityList = DataStore.GetAllEntitiesByType<entityType>();
        SaveEntitiesToCache(entityList);
    }
}
The logic is there but above code won't work because you can't invoke a generic method like that, instead reflection must be used for such usage. Following code example shows how to use reflection to invoke a generic method (entity/cache update logic is not included):
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection;
using System.Collections.ObjectModel;
using System.Collections.Concurrent;

public interface IEntity
{
    string Identifier { get; }
    string Type { get; }
}

public class Book : IEntity
{
    public string ISBN { get; set; }
    public string Category { get; set; }
    public string Identifier { get { return ISBN; } }
    public string Type { get { return Category; } }
}

public class CD : IEntity
{
    public string Name { get; set; }
    public string Category { get; set; }
    public string Identifier { get { return Name; } }
    public string Type { get { return Category; } }
}

public class KeyedEntityCollection<T> : KeyedCollection<string, T> where T : IEntity
{
    protected override string GetKeyForItem(T item)
    {
        return item.Identifier;
    }
}

public class Repository
{
    static ConcurrentDictionary<Type, object> _entitiesCache = new ConcurrentDictionary<Type, object>();

    void LoadEntitiesCache()
    {
        // Load all entities from back-end store and save them in _entitiesCache
        DataSource dataSource = new DataSource();

        Type[] entityTypes = new Type[] { typeof(Book), typeof(CD) };
        MethodInfo methodInfo = typeof(DataSource).GetMethod("GetEntitiesByType");
        foreach (var entityType in entityTypes)
        {
            object _cacheObject = null;
            if (_entitiesCache.TryGetValue(entityType, out _cacheObject) && _cacheObject != null)
                continue;

            try
            {
                // Invoke DataStore.GetEntitiesByType<T>() dynamically
                MethodInfo genericMethod = methodInfo.MakeGenericMethod(entityType);
                IEnumerable dataReturn = genericMethod.Invoke(dataSource, null) as IEnumerable;

                // Create EntityCollection dynamically
                Type[] types = new Type[] { entityType };
                Type entityCollectionType = typeof(KeyedEntityCollection<>);
                Type genericType = entityCollectionType.MakeGenericType(types);
                IList genericCollection = Activator.CreateInstance(genericType) as IList;

                if (dataReturn != null)
                {
                    foreach (var entity in dataReturn)
                    {
                        genericCollection.Add(entity);
                    }
                }
                _entitiesCache.AddOrUpdate(entityType, genericCollection, 
                    (type, existingValue) => { return genericCollection; });
            }
            catch (Exception ex)
            {
                // Log error
                throw;
            }
        }
    }

    public Repository()
    {
        LoadEntitiesCache();
    }

    public KeyedEntityCollection<T> GetEntities<T>() where T : IEntity
    {
        object cachedEntityCollection = null;
        _entitiesCache.TryGetValue(typeof(T), out cachedEntityCollection);
        return cachedEntityCollection as KeyedEntityCollection<T>;
    }

    public T GetEntityById<T>(string identifier) where T : IEntity
    {
        var entityCollection = GetEntities<T>();
        if (entityCollection != null && entityCollection.Contains(identifier))
        {
            return entityCollection[identifier];
        }
        return default(T);
    }
}

internal class DataSource
{
    // Get data from database, or xml files, or whatever data store
    public IEnumerable<T> GetEntitiesByType<T>() where T : IEntity
    {
        // Mock some test data here
        if (typeof(T) == typeof(Book))
        {
            List<Book> bookSource = new List<Book>();
            Enumerable.Range(1, 100).ToList().ForEach(
                i => bookSource.Add(new Book() { ISBN = i.ToString(), Category = "Computer" }));
            return bookSource as IEnumerable<T>;
        }
        else if (typeof(T) == typeof(CD))
        {
            List<CD> cdSource = new List<CD>();
            Enumerable.Range(1, 100).ToList().ForEach(
                i => cdSource.Add(new CD() { Name = i.ToString(), Category = "Music" }));
            return cdSource as IEnumerable<T>;
        }
        else return null;
    }
}

class Program
{
    static void Main(string[] args)
    {
        Repository repo = new Repository(); 
        var books = repo.GetEntities<Book>();           // Get all books
        var secondBook = repo.GetEntityById<Book>("2"); // Get the second book
        var cds = repo.GetEntities<CD>();               // Get all CDs
        var thirdCD = repo.GetEntityById<CD>("3");      // Get the third CD
        Console.Read();
    }
}

Friday, February 04, 2011

ASP.NET Cache, MemoryCache and Velocity

Caching is one of the most effective ways to improve performance for high-traffic/high-volume service applications. Caching is considered as a common practice to deal with data access with heavy I/O. ASP.NET Cache from System.Web dll is the only one memory caching implementation you can use in .NET 1.0/2.0/3.0 framework. It's there for so long that we almost forget the question at the very beginning: why put it into the Web context? You have to add System.Web reference even though you are working on class libraries or WinForm applications. Anyway people are just used to this bad design.

Finally Microsoft has a new ObjectCache/MemoryCache caching system from System.Runtime.Caching.dll in the latest version of the .NET 4.0 Framework. The API and usage is similar but it makes more senses now. We can use the MemoryCache in any application (I found it's not applicable to Console apps) without dependency on System.Web. One noticeable change in the new Cache API is that it has only one Add method, and gets rid of the confusion from two Add and Insert methods in ASP.NET Cache. Here's some note from MSDN describing the difference between MemoryCache and traditional ASP.NET Cache:

The MemoryCache class is similar to the ASP.NET Cache class. The MemoryCache class has many properties and methods for accessing the cache that will be familiar to you if you have used the ASP.NET Cache class. The main differences between the Cache and MemoryCache classes are that the MemoryCache class has been changed to make it usable by .NET Framework applications that are not ASP.NET applications. For example, the MemoryCache class has no dependencies on the System.Web assembly. Another difference is that you can create multiple instances of the MemoryCache class for use in the same application and in the same AppDomain instance.

How about that distributed caching system with code name of "Velocity"? It has been re-branded and becomes part of the Windows Server AppFabric - version 1.0 officially released a few months ago. The origin of "Velocity" project was a .NET version of MemCached. I remember our company was considering using "Velocity" three years ago (CTP version at that time). Our lead said let's wait for the mature final version. Now Microsoft eventually makes it the final 1.0 version after four years of work. What a super velocity! It took a new graduate student Brad Fitzpatrick a few months to create the MemCached solution for his LiveJournal site, and now it's pretty much the standard in the OpenSource world. It takes so long for Microsoft to develop a .NET component that's crucial for big systems and enterprise environments. A deep sigh for Microsoft.

Related to .NET caching there's an interesting post talking about loading different cache systems dynamically, including ASP.NET Cache, MemoryCache and AppFabric Cache. The implementation is simple and it could be helpful in some scenario.

Friday, January 28, 2011

Implementing Simple Custom Rules By Code

In my previous two posts Microsoft Workflow Rules Engine was used to execute the business rules. That maybe a bit overkill if there're only a small set of rules to run and the use cases of them are relatively simple. In that case we could implement the rules and control their execution directly. The simple console app below demos the concept of custom rules implementation:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication1
{
    class Program
    {
        #region demo domain objects
        public class Product
        {
            public string Name { get; set; }
            public decimal Price { get; set; }
        }

        public class ShoppingCart
        {
            public List<Product> SelectedProducts { get; set; }
            public List<string> CouponCodes { get; set; }
            public decimal TotalAmount { get { return SelectedProducts.Sum(prod => prod.Price); } }
            internal decimal Discount { get; set; }

            public ShoppingCart()
            {
                SelectedProducts = new List<Product>();
                CouponCodes = new List<string>();
                Discount = 0;
            }
        }
        #endregion demo domain objects

        #region promotion objects 
        class CouponPromotion  
        {
            public string Name { get; set; }
            public string PromotionCode { get; set; }
            public decimal Discount { get; set; }
            public DateTime ExpiredOn { get; set; }
        }

        class TotalAmountPromotion // discount if total amount is greater than a number
        {
            public string Name { get; set; }
            public decimal TotalAmount { get; set; }
            public decimal Discount { get; set; }
            public DateTime ExpiredOn { get; set; }
        }
        #endregion promotion object

        #region custom rules interface and implementation
        interface ICustomRule
        {
            void Execute(ShoppingCart cart);
        }

        class CouponPromotionRule : ICustomRule
        {
            private CouponPromotion mCouponPromotion;
            public CouponPromotionRule(CouponPromotion promotion)
            {
                this.mCouponPromotion = promotion;
            }

            public void Execute(ShoppingCart cart)
            {
                if (mCouponPromotion.ExpiredOn >= DateTime.Today && cart.CouponCodes.Contains(mCouponPromotion.PromotionCode))
                {
                    Console.WriteLine(mCouponPromotion.Name + " applied");
                    cart.Discount += mCouponPromotion.Discount;
                }
            }
        }

        class TotalAmountGreaterRule : ICustomRule
        {
            private TotalAmountPromotion mTotalAmountPromotion;
            public TotalAmountGreaterRule(TotalAmountPromotion promotion)
            {
                this.mTotalAmountPromotion = promotion;
            }

            public void Execute(ShoppingCart cart)
            {
                if (mTotalAmountPromotion.ExpiredOn >= DateTime.Today && cart.TotalAmount >= mTotalAmountPromotion.TotalAmount)
                {
                    Console.WriteLine(mTotalAmountPromotion.Name + " applied");
                    cart.Discount += mTotalAmountPromotion.Discount;
                }
            }
        }
        #endregion custom rules interface and implementation

        static void RuleTest(ShoppingCart cart)
        {
            // Mock CouponPromotionRule and TotalAmountRules
            CouponPromotionRule rule1 = new CouponPromotionRule(new CouponPromotion()
            {
                Name = "Coupon Promotion Rule 1",
                PromotionCode = "123456789",
                Discount = 10,
                ExpiredOn = new DateTime(2100, 1, 1)
            });

            CouponPromotionRule rule2 = new CouponPromotionRule(new CouponPromotion()
            {
                Name = "Coupon Promotion Rule 2",
                PromotionCode = "111111111",
                Discount = 20,
                ExpiredOn = new DateTime(2000, 1, 1)
            });
            TotalAmountGreaterRule rule3 = new TotalAmountGreaterRule(new TotalAmountPromotion()
            {
                Name = "Total Amount Rule 1",
                TotalAmount = 1000,
                Discount = 100,
                ExpiredOn = new DateTime(2100, 1, 1)
            });

            ICustomRule[] rules = new ICustomRule[] { rule1, rule2, rule3 };
            foreach (var rule in rules)
            {
                rule.Execute(cart);
            }
        }

        static void Main(string[] args)
        {
            ShoppingCart shoppingCart = new ShoppingCart();
            Enumerable.Range(1, 5).ToList().ForEach(i => shoppingCart.SelectedProducts.Add(
                   new Product() { Name = "Product" + i.ToString(), Price = 100 * i }));
            shoppingCart.CouponCodes.Add("123456789");
            shoppingCart.CouponCodes.Add("111111111");
            Console.Write("\nShopping Cart before rules execution -- ");
            Console.WriteLine("Total : ${0}\tDiscount : ${1}", shoppingCart.TotalAmount, shoppingCart.Discount);

            RuleTest(shoppingCart);

            Console.Write("Shopping Cart after rules execution -- ");
            Console.WriteLine("Total : ${0}\tDiscount : ${1}", shoppingCart.TotalAmount, shoppingCart.Discount);

            Console.Read();
        }      
    }
}

The result of the test app: