Intall SharePoint 2010 In Windows 7 Virtual Machine

SharePoint 2010 RTM is out there for about two months. I didn't see any issue during the installation on my desktop with Windows 2008 R2. I also wanted to setup a SP2010 environment in my laptop with Windows 7 64-bit. Direct installation of this mess product on my Windows 7 was not an option for me, so I decided to install SP2010 to a virtual machine.

The problem came up because Windows 7 doesn't support Hyper-V and Windows Virtual PC doesn't support 64-bit system, but SP2010 must run in 64-bit environment. So you can not install SP2010 in a virtual machine run inside Windows 7 with Microsoft products. What a Broken As Designed (B.A.D.)!

The solutions go to non-Microsoft products. My first try, the free VMWare Server version 2.0.1 (http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=751fa0d1-356c-4002-9c60-d539896c66ce), was successful without big issues. Following are steps of my installation:

1. Install VMWare Server 2.0.1 with default settings.
2. Go to VMWare Web Access admin page (http://machineName:8222/ui) and select "Virtual Machine -> Create Virtual Machine" with following settings:

• Guest OS: "Microsoft Windows Server 2008 (64-bit)" (I want to install 2008 R2 initially but it's not included in the dropdown list).
• Memory size: 2G, Processors count: 2.
• "Create a New Virtual Disk" with disk capacity of 40G. "Policies -> Optimize for performance" on the same disk configuration page.
• "Add a Network Adapter" with default "Bridged" network connection.
• "Use a Physical Drive"
• "Don't Add a Floppy Drive"
• "Add a USB Controller"

3. Insert Windows Server 2008 SP2 64-bit DVD and Power on the newly created VM instance, follow the instruction to complete the Windows Server 2008 SP2 installation.
4. Activate Windows Server 2008 SP2 and install import updates.
5. Install SQL Server 2008 SP1 64-bit.
6. Install SharePoint 2010 RTM using "Server Farm" and "Complete" setting.
6.1 Skip this step if you run the SharePoint with domain account.
I want to use a local account to run SP2010 in my VM, but it's not allowed by default. So I need to install SharePoint configuration database using PowerShell. Click SharePoint 2010 management shell, Type "New-SPConfigurationDatabase", it failed and prompted something like SQL Server version is too low. Google and install SQL Server 2008 Cumulative update(http://support.microsoft.com/kb/963036). Rerun "New-SPConfigurationDatabase" but still failed with message of "User does not exist or is not unique...". Google and found it's related to Administrator account. Create a separate "SVC_SharePoint" account to connect to the database, then run "New-SPConfigurationDatabase" with success.
7. Run SharePoint configuration Wizard
8. Install important updates
9 (optional). Install Visual Studio 2010
10 (optional). Install Office 2010

The total disk usage after all above installation is about 29G, and the memory consumption after reboot is about 1G. The whole VM is installed in an external USB 2.0 hard disk. The speed is satisfactory.

SharePoint SPList.LastItemModifiedDate And SPWeb.LastItemModifiedDate Are UTC Time

In SharePoint API, a SPList object has a LastItemModifiedDate property that indicates the last item modified time. The MSDN description for this property is: Gets the date and time that an item, field, or property of the list was last modified.

We use this property to validate the memory cache in one project, but the cache is not properly updated when the list item is changed. Finally we figured out this property returns a UTC time not a local time. To correct the issue, simply use the ToLocalTime method:

DateTime listLastUpdate = list.LastItemModifiedDate.ToLocalTime();

Similarly, SPWeb.LastItemModifiedDate is also UTC time, and you need to convert it to local time manually when you use it in the code.

In fact, all DateTime values stored in SQL Server, such as list item's "Modified" time, are in UTC format. SharePoint will convert it back to local time:

SPListItem item = list.Items[0];
DateTime modifiedTime = Convert.ToDateTime(item["Modified"]);

I don't see any reason why this LastItemModifiedDate property is using UTC format. My guess is that the time is not parsed by database value, instead is created something like this in code behind:

DateTime listModifiedDate = new DateTime(value from database);

Is it a bug only existing in SharePoint 2007? I checked the latest SharePoint 2010 RTM release (build number 14.0.4762). To my surprise, they're still UTC time. Maybe MS should document them or give explanation for any such inconsistent stuff inside API.

SQL Server Vs MongoDB - Performance Perspective

Relational database such as SQL Server would be slower than NoSQL solutions. That's understandable. Relational database has its focus and ACID has its cost. But I didn't expect their performance gap is so big until I did some real tests.

The test machine is the same I tested on MongoDB last time:

• OS: Windows Server 2008 64-bit SP2
• CPU: Intel i5 750
• Memory: 6G
• DB: SQL Server 2008 Enterprise edition SP1.

Similar I did the test on MongoDB last time, I create a simple test database with only three columns:

  id    -- uniqueidentifier (Clustered index)
 key   -- varchar(40) (Unique non-clustered index)
 value -- varchar(40) (no indexing)

MongoDB test code was modified to test SQL Server. I ran the console and was eager to see the results. But the first test case of inserting 1-million records in local database seemed to be never completed.

The insert operations were just too slow. Is there anything going wrong with my machine? I installed all Windows/SQL Server important updates, and disable almost all unnecessary processes, deflagment disk, and redid the test. But the result is still very disappointing, less than 400 insert per second.

Then I tried a few options:

1. Change id to be non-clustered index: no big difference
2. Change id type from uniqueidentifier to varchar(40): no big difference
3. Add an extra identity integer column (primary key): no big difference
4. Change key/value from varchar(40) to char(40): no big difference
5. Change key's index fillfactor to 10, 30, 60, 80, 100: slightly difference among them
6. Do insertion using SQL Script:

    DECLARE @id uniqueidentifier
   DECLARE @Count int, @Max int
   SET @Count = 0
   SET @Max = 1000000
   WHILE @Count < @Max       
   BEGIN             
       SET @id = NEWID()            
       INSERT INTO testTable (id, key, value) VALUES (@id, @id, @id)            
       SET @Count = @Count + 1       
   END

no big difference
8. Use stored procedure: improve a little
9. Change Key's unique non-clustered index to non-clustered index: improve a little.
10. Change database recovery model from full to simple: improve a little

The maximum insertion speed could reach around 900 per second after all kind of tuning in my local machine. That's not even close to MongoDB, and I didn't do any extra work to achieve 200000+ insertions/sec.

Googled and found that actually many people are having the same problem. The article SQL Server Slow Performance on Insert compiles a few resources on the topic of slow insertion in SQL Server.

Anyway I was patient enough to wait the 1-million records being inserted. Then I did the rest of the test with following results (local test):

Insert: 900/Second
Update: 1500/Second
Search on clustered-indexed column: 9000/Second
Search on non-clusterd indexed column: 2000/Second
Search on Non-indexed column: 27/Second

Besides the non-indexed column search (table scan), only the search on clustered-indexed column in SQL Server is comparable to MongoDB's.

Test MongoDB In Windows Environment

NoSQL movement becomes more and more popular. Nowadays most NoSQL product environments are Unix-based. MongoDB is one of few solutions that provide Windows installation and .NET provider. This exercise evalutes MongoDB's performance in Windows platform.

The test machines' configuration:

• OS: Windows Server 2008 64-bit SP2
• CPU: Intel i5 750
• Memory: 6G
• MongoDB: 1.4.2 64-bit (http://dl.mongodb.org/dl/win32/x86_64)
• MongoDB Driver: MongoDB-CSharp 0.82 (http://github.com/samus/mongodb-csharp)

Local tests and remote tests are both examined in this exercise. Local tests are conducted inside one physical box. MongoDB server and client process are running in separated machines in remote tests, and connection between server and client machines are 100 Mbit/Sec.

A simple document with key (unique-indexed) value (non-indexed) pair is used for testing. All fields (including internal _id field) will store GUID value:

{
_id: "12345678-1234-1234-1234-123456789012",  // Internal field
key: "12345678-1234-1234-1234-123456789012",  // Unique-indexed
value: "12345678-1234-1234-1234-123456789012" // Non-indexed
}

Manually handling "_id" field without using Oid format is not recommended. The reason of doing that in our tests is efficiently do update and delete operations without extra search.

Local tests are conducted inside one physical machine. MongoDB server and client process are running in separated machines in remote tests, and connection between server and client machines are 100Mbit/Sec.

Local test results:

(All in document actions per second, 1M=1000000)

Document number	1M	2M	4M	8M	10M
Insert	32210	29540	27160	22100	20530
Update	18560	18160	18010	17900	17540
Delete	12660	12510	11980	11610	11550
Search on _id	13340	13330	13350	12990	12130
Search on key	9190	9280	9300	9110	9010
Search on value	7.2	3.6	1.8	0.9	0.7

Remote test results:

Document number	1M	2M	4M	8M	10M
Insert	31450	30700	22420	18810	17600
Update	18180	18160	17400	18100	17830
Delete	12550	12270	11730	11680	11530
Search on _id	2410	2440	2410	2420	2420
Search on key	1470	1470	1450	1450	1440
Search on value	7.2	3.6	1.8	0.9	0.7

Findings:

1. Fastest operation is data insert.
2. Search on _id is faster than search on other indexed field.
3. Search on indexed field seems in constant time, not proportional to size of the data set.
4. Search on non-indexed field shows very poor performance, and the time spent is proportional to the size the data set. This is similar to a table scan scenario in a relational database.
5. There's no big gap in insert/update/delete between local test and remote test.
6. Remote search is significantly slower than local search.

By digging into source code of MongDB-CSharp driver, I found that insert, update and delete functions in the client side are implemented by one-way IO writing without any acknowledgement, i.e. keep sending data to TCP NetworkStream. Streamed data can be efficiently buffered in both client and server side. From client point of view it's asynchronous process. That explains why there's no big difference between remote and local tests in batch insert/update/delete scenarios. On the other hand, read or search (find related functions) requires two-way communication. The client sends a request to the server then waits the response to be completed. The server process time plus the round-trip network delay are all counted for one request action. That's why remote search are significantly slower the tests conducted in local box.

I thought the relatively low speed of indexed search in our test results could be caused by my single-threaded test application. So I modified the code and did the tests with 2, 4, 6, 8 and 10 threads. But surprisingly the performance did not improve at all.

The test code is listed below. It is simple and may be not accurate enough. But it gave me an idea how it performs in general with Windows environment.

Test code:

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Diagnostics;
using System.Threading;
using MongoDB.Driver;

public class Program
{
    struct TimeUnit
    {
        public int NumberOfRecords;
        public long MilliSeconds;
        public TimeUnit(int num, long ms)
        {
            NumberOfRecords = num;
            MilliSeconds = ms;
        }
    }
    const int MaxDocuments = 1000000;
    const int NonIndexSearchCount = 1000;
    const int ThreadCount = 1;
    const string DBConnectionString = "Server=127.0.0.1";
    const string DBName = "TestMongoDB";
    const string DBCollectionName = "TestCollection";
    static string[] testContent = new string[MaxDocuments];
    static object lockObject = new object();
    static Dictionary<string, List<TimeUnit>> resultTimes = new Dictionary<string, List<TimeUnit>>();

    static void Main(string[] args)
    {
        // Initialize data
        for (int i = 0; i < MaxDocuments; i++)
            testContent[i] = Guid.NewGuid().ToString();

        // Start Threads
        List<Thread> threads = new List<Thread>();
        for (int i = 0; i < ThreadCount; i++)
        {
            int max = MaxDocuments / ThreadCount;
            int insertStart = i * max;
            int searchStart = insertStart + max / 2;
            int searchEnd = searchStart + max / 10;
            Thread thread = new Thread(() => TestMongoDB(insertStart, searchStart, searchEnd, max));
            thread.Start();
            threads.Add(thread);
            //ThreadPool.QueueUserWorkItem(o => TestMongoDB(insertStart, searchStart, searchEnd, max));
        }
        Console.WriteLine();
        // Wait threads to be completed
        foreach (Thread thread in threads)
            thread.Join();

        // Display results
        Console.WriteLine("Results:");
        foreach (var key in resultTimes.Keys)
        {
            List<TimeUnit> times = resultTimes[key];
            Console.Write(key + ":");
            long totalTime = 0, totalNumber = 0;
            foreach (TimeUnit st in times)
            {
                totalTime += st.MilliSeconds;
                totalNumber += st.NumberOfRecords;
                Console.Write(st.MilliSeconds + " ");
            }
            Console.WriteLine(string.Format("{0} speed is : {1}/sec", key, totalNumber * 1000.0 / totalTime));
        }
        Console.WriteLine("Done");
        Console.Read();
    }

    /// <summary>
    /// MongoDB test. Document format:
    /// { 
    ///     _id,    // internal field
    ///     key,    // indexed field
    ///     value   // non-indexed field
    /// }
    /// </summary>
    static void TestMongoDB(int insertStart, int searchStart, int searchEnd, int max)
    {
        // Init MongoDB
        Mongo mongo = new Mongo(DBConnectionString);
        IMongoDatabase db;
        IMongoCollection documents;
        Document doc = new Document();
        Stopwatch watch = new Stopwatch();

        mongo.Connect();
        db = mongo[DBName];
        documents = db.GetCollection(DBCollectionName);

        // Setup Index on key field
        lock (lockObject)
        {
            if (!documents.MetaData.Indexes.ContainsKey("key"))
            {
                Document indexDoc = new Document { { "key", 1 } };
                documents.MetaData.CreateIndex(indexDoc, true);
            }
        }

        // Insert data
        watch.Start();
        for (int i = insertStart; i < insertStart + max; i++)
        {
            Document newdoc = new Document();
            newdoc["_id"] = newdoc["key"] = newdoc["value"] = testContent[i];
            documents.Insert(newdoc);
        }
        watch.Stop();
        AddTimeUnit("insert", new TimeUnit(max, watch.ElapsedMilliseconds));
        Console.WriteLine(string.Format("MongoDB insert {0} records: {1} ms",
           max, watch.ElapsedMilliseconds.ToString()));
        watch.Reset();

        // Search on id field
        watch.Start();
        for (int i = searchStart; i < searchEnd; i++)
        {
            doc["_id"] = testContent[i];
            var doc1 = documents.FindOne(doc);
            string value = doc1["value"].ToString();
        }
        watch.Stop();
        AddTimeUnit("search-id", new TimeUnit(searchEnd - searchStart, watch.ElapsedMilliseconds));
        Console.WriteLine(string.Format("MongoDB {0} search on id field over {1} records: {2} ms",
           searchEnd - searchStart, documents.Count(), watch.ElapsedMilliseconds));
        watch.Reset();
        doc = new Document();

        // Search on indexed "key" field
        watch.Start();
        for (int i = searchStart; i < searchEnd; i++)
        {
            doc["key"] = testContent[i];
            var doc1 = documents.FindOne(doc);
            string value = doc1["value"].ToString();
        }
        watch.Stop();
        AddTimeUnit("search-key", new TimeUnit(searchEnd - searchStart, watch.ElapsedMilliseconds));
        Console.WriteLine(string.Format("MongoDB {0} search on indexed field over {1} records: {2} ms",
           searchEnd - searchStart, documents.Count(), watch.ElapsedMilliseconds));
        watch.Reset();
        doc = new Document();

        // Search on non-indexed "value" field
        watch.Start();
        for (int i = searchStart; i < searchStart + NonIndexSearchCount; i++)
        {
            doc["value"] = testContent[i];
            var doc1 = documents.FindOne(doc);
            string value = doc1["value"].ToString();
        }
        watch.Stop();
        AddTimeUnit("search-value", new TimeUnit(NonIndexSearchCount, watch.ElapsedMilliseconds));
        Console.WriteLine(string.Format("MongoDB {0} search on non-indexed field over {1} records: {2} ms",
          NonIndexSearchCount, documents.Count(), watch.ElapsedMilliseconds));
        watch.Reset();
        doc = new Document();

        // Update test
        watch.Start();
        for (int i = searchStart; i < searchEnd; i++)
        {
            doc["_id"] = testContent[i];
            //var doc1 = documents.FindOne(doc);
            doc["key"] = testContent[i];
            doc["value"] = i.ToString();
            documents.Update(doc);
        }
        watch.Stop();
        AddTimeUnit("update", new TimeUnit(searchEnd - searchStart, watch.ElapsedMilliseconds));
        Console.WriteLine(string.Format("MongoDB {0} update over {1} records: {2} ms",
           searchEnd - searchStart, documents.Count(), watch.ElapsedMilliseconds));
        watch.Reset();
        doc = new Document();

        // Delete test
        watch.Start();
        for (int i = searchStart; i < searchEnd; i++)
        {
            doc["_id"] = testContent[i];
            //var doc1 = documents.FindOne(doc);
            documents.Delete(doc);
        }
        watch.Stop();
        AddTimeUnit("delete", new TimeUnit(searchEnd - searchStart, watch.ElapsedMilliseconds));
        Console.WriteLine(string.Format("MongoDB {0} delete over {1} records: {2} ms",
           searchEnd - searchStart, documents.Count(), watch.ElapsedMilliseconds));
    }

    private static void AddTimeUnit(string key, TimeUnit st)
    {
        lock (lockObject)
        {
            if (resultTimes.ContainsKey(key))
            {
                List<TimeUnit> times = resultTimes[key];
                times.Add(st);
            }
            else
            {
                List<TimeUnit> times = new List<TimeUnit>();
                times.Add(st);
                resultTimes.Add(key, times);
            }
        }
    }
}

A SharePoint Feature To Configurate Content Deployment Timout Setting

The default timeout setting in SharePoint 2007 content deployment is 10 minutes. For content deployment on a big site, this may not be long enough, and you may get a timeout exception during the content deployment. The bad news is that you can't change the content deployment timeout setting with stsadm.exe, and you must run custom code to talk with SharePoint API to update this setting. Stefan Gossner provides a console application for such task. But console application is so flexible to reality. Not many admins like the idea of running a console application to update a setting for SharePoint in production environment.

It would be good if we can make the configuration change by browser, idealy a page linked to content deployment section in central administration web site. In this practice I will wrap all that as a SharePoint Feature.

First we need to define the feature.xml:

<?xml version="1.0" encoding="utf-8"?>
<Feature  Id="02ca3716-5938-4788-ad67-17e6039f93da"
          Title="Content Deployment Timeout Configuration"
          Description="Timeout setting for content deployment"
          Version="1.0.0.0"
          Hidden="FALSE"
          Scope="Farm"
          DefaultResourceFile="core"
          xmlns="http://schemas.microsoft.com/sharepoint/">
  <ElementManifests>
    <ElementManifest Location="elements.xml"/>    
  </ElementManifests>
</Feature>

In order to add a link in Content Deployment section in SharePoint central admin, the Feature needs to include a custom action. The elements.xml is as following:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <CustomAction
        Id="Content.Deployment"
        GroupId="ContentDeployment"
        Location="Microsoft.SharePoint.Administration.Operations"
        Sequence="510"
        Title="Content deployment timeout setting">
    <UrlAction Url="/_admin/CDTimeoutSetting.aspx" />
  </CustomAction>
</Elements>

The UrlAction is pointing to the /_admin/CDTimeoutSetting.aspx page which includes a simple UI for timeout update:

<%@ Page Language="C#" MasterPageFile="~/_admin/admin.master" %>

<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral,PublicKeyToken=71e9bce111e9429c" %>
<%@ Assembly Name="Microsoft.SharePoint.ApplicationPages.Administration, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
<%@ Assembly Name="Microsoft.SharePoint.Publishing, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"%> 
<%@ Assembly Name="System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %>

<%@ Import Namespace="Microsoft.SharePoint" %>
<%@ Import Namespace="System.Web.UI" %>
<%@ Import Namespace="Microsoft.SharePoint.Publishing.Administration" %>

<asp:Content ID="Content2" runat="server" ContentPlaceHolderID="PlaceHolderPageTitleInTitleArea">
    Manage Content Deployment Remote Timeout Setting
</asp:Content>
<asp:Content ID="Content4" runat="server" ContentPlaceHolderID="PlaceHolderMain">
    <div style="margin: 15" runat="server" id="divMain">
        <div>
            <p>
                <asp:Label ID="lblInfo" runat="server"></asp:Label>
                <asp:Label ID="lblCurrentTimeout" runat="server" Font-Bold="true"></asp:Label>
            </p>
            <br />
            <p>
                <asp:Label ID="Label2" runat="server">Set timeout in minutes</asp:Label>
                <asp:TextBox ID="txtTimeout" runat="server"></asp:TextBox>
                <asp:RequiredFieldValidator ID="RequiredFieldValidator1" runat="server" Text="* Required" ErrorMessage="Required" ControlToValidate="txtTimeout" Display="Dynamic" ValidationGroup="update"></asp:RequiredFieldValidator>
                <asp:RangeValidator ID="RangeValidator1" runat="server" ErrorMessage="Invalid (1-240)" ControlToValidate="txtTimeout" Display="Dynamic" Type="Integer" MinimumValue="1" MaximumValue="7200" ValidationGroup="update"></asp:RangeValidator>
                <asp:Button ID="btnUpdate" runat="server" Text="Update" ValidationGroup="update" OnClick="btnUpdate_Click" />
            </p>
        </div>
    </div>
</asp:Content>
<script runat="server">
    ContentDeploymentConfiguration config = null;

    void btnUpdate_Click(object sender, EventArgs e)
    {
        if (config == null)
        { 
            config = ContentDeploymentConfiguration.GetInstance();
        }
        Microsoft.SharePoint.Administration.SPWebApplication wa = SPContext.Current.Site.WebApplication;
        wa.FormDigestSettings.Enabled = false;
        SPContext.Current.Web.AllowUnsafeUpdates = true;
        config.RemoteTimeout = Convert.ToInt32(txtTimeout.Text) * 60;
        config.Update();
        SPContext.Current.Web.AllowUnsafeUpdates = false;
    }

    void Page_PreRender(object sender, System.EventArgs e)
    {
        if (SPContext.Current == null || SPContext.Current.Site == null || SPContext.Current.Web == null)
        {
            lblInfo.Text = "Invalid SPContext. Please sign in and try again.";
            return;
        }
        if (config == null)
        {
            config = ContentDeploymentConfiguration.GetInstance();
        }
        lblCurrentTimeout.Text = string.Format("Current timeout setting is: {0} minutes", Convert.ToString(config.RemoteTimeout / 60));
    }
</script>

The manifext.xml for packaking the Feature to a wsp solution package:

<?xml version="1.0"?>
<Solution SolutionId="04a13d4d-21e2-4e72-a65c-163b16e8bbc8" xmlns="http://schemas.microsoft.com/sharepoint/">
  <FeatureManifests>
    <FeatureManifest Location="ContentDeploymentTimeoutSetter\feature.xml" />
  </FeatureManifests>
  <TemplateFiles>
    <TemplateFile Location="ADMIN\CDTimeoutSetting.aspx" />
  </TemplateFiles>
</Solution>

The screen-shot of the central admin configuration page:


The screen-shot of timeout setting page:

SharePoint Feature List Page

In some cases, once the site is created and properly configured, system admins don't want end users, not even site collection administrators, to touch the SharePoint Features for security and other reasons. We built a custom SharePoint solution with custom site definition. System admin requested to set all our custom Features to be hidden so that those Features won't be able to be activated or deactivated by users. Admins themselves like to use stsadm or some other tools to control those features.

All set and everything is okay, except you can not see those Feature status in site settings' pages. So I created a Feature List application page specific for this:



Code:

<%@ Page Language="C#" Inherits="Microsoft.SharePoint.WebControls.LayoutsPageBase" %>

<%@ Assembly Name="System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" %>
<%@ Assembly Name="System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %>
<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral,PublicKeyToken=71e9bce111e9429c" %>
<%@ Assembly Name="Microsoft.SharePoint.ApplicationPages, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"%>

<%@ Import Namespace="System.Web.UI" %>
<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="System.Collections.Specialized" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<%@ Import Namespace="Microsoft.SharePoint.Utilities" %>
<%@ Import Namespace="Microsoft.SharePoint.Administration" %>

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>SharePoint Feature List</title>
    <meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <meta http-equiv="Content-language" content="en">

    <script runat="server">
        public class FeatureItem
        {
            private string _name, _scope, _id;
            private bool _hidden;
            public string Name { get { return _name; } set { _name = value; } }
            public string Scope { get { return _scope; } set { _scope = value; } }
            public string ID { get { return _id; } set { _id = value; } }
            public bool Hidden { get { return _hidden; } set { _hidden = value; } }
            public FeatureItem(string name, string scope, string id, bool hidden)
            {
                Name = name;
                Scope = scope;
                ID = id;
                Hidden = hidden;
            }
        }

        protected override bool AllowNullWeb { get { return false; } }
        protected override bool RequireSiteAdministrator { get { return false; } }
        
        void Page_Load(object sender, System.EventArgs e)
        {
            if (SPContext.Current == null || SPContext.Current.Site == null || SPContext.Current.Web == null)
            {
                lblInfo.Text = "Invalid SPContext. Please sign in and try again.";
                return;
            }

            PopulateFeatures();
        }

        void PopulateFeatures()
        {
            List<FeatureItem> featureList = new List<FeatureItem>();
            string scope = ddlScope.SelectedValue;

            Dictionary<string, SPFeatureCollection> activeFeatures = new Dictionary<string, SPFeatureCollection>();
            activeFeatures.Add(SPFeatureScope.Farm.ToString(), Microsoft.SharePoint.Administration.SPWebService.ContentService.Features);
            activeFeatures.Add(SPFeatureScope.WebApplication.ToString(), SPContext.Current.Site.WebApplication.Features);
            activeFeatures.Add(SPFeatureScope.Site.ToString(), SPContext.Current.SiteFeatures);
            activeFeatures.Add(SPFeatureScope.Web.ToString(), SPContext.Current.WebFeatures);

            if (ddlActive.SelectedValue == "Active")
            {
                if (scope == "All")
                {
                    foreach (SPFeatureCollection features in activeFeatures.Values)
                    {
                        foreach (SPFeature item in features)
                        {
                            try
                            {
                                string name = string.IsNullOrEmpty(item.Definition.DisplayName) ? item.Definition.Id.ToString() : item.Definition.DisplayName;
                                FeatureItem fi = new FeatureItem(name,
                                    item.Definition.Scope.ToString(), item.Definition.Id.ToString(), item.Definition.Hidden);
                                featureList.Add(fi);
                            }
                            catch (Exception ex)
                            {
                                string name = string.IsNullOrEmpty(item.Definition.DisplayName) ? item.Definition.Id.ToString() : item.Definition.DisplayName;
                                FeatureItem fi = new FeatureItem(ex.Message, string.Empty, string.Empty, false);
                                featureList.Add(fi);
                            }

                        }
                    }
                }
                else
                {
                    SPFeatureCollection features = activeFeatures[scope];
                    foreach (SPFeature item in features)
                    {
                        try
                        {
                            string name = string.IsNullOrEmpty(item.Definition.DisplayName) ? item.Definition.Id.ToString() : item.Definition.DisplayName;
                            FeatureItem fi = new FeatureItem(name,
                                item.Definition.Scope.ToString(), item.Definition.Id.ToString(), item.Definition.Hidden);
                            featureList.Add(fi);
                        }
                        catch (Exception ex)
                        {
                            string name = string.IsNullOrEmpty(item.Definition.DisplayName) ? item.Definition.Id.ToString() : item.Definition.DisplayName;
                            FeatureItem fi = new FeatureItem(ex.Message, string.Empty, string.Empty, false);
                            featureList.Add(fi);
                        }
                    }
                }
            }
            else //Inavtive
            {
                foreach (SPFeatureDefinition definition in SPFarm.Local.FeatureDefinitions)
                {
                    Guid featureID = Guid.NewGuid();
                    string name = string.Empty;
                    try
                    {
                        featureID = definition.Id;
                        name = string.IsNullOrEmpty(definition.DisplayName) ? featureID.ToString() : definition.DisplayName;
                        bool isActive = false;
                        if (activeFeatures[definition.Scope.ToString()] != null)
                            isActive = (activeFeatures[definition.Scope.ToString()][featureID] != null);

                        if (!isActive && (scope == "All" || definition.Scope.ToString() == scope))
                        {
                            FeatureItem fi = new FeatureItem(name, definition.Scope.ToString(), definition.Id.ToString(), definition.Hidden);
                            featureList.Add(fi);
                        }
                    }
                    catch (Exception e)
                    {
                        FeatureItem fi = new FeatureItem(name + " : " + e.Message, "Invalid", featureID.ToString(), true);
                        featureList.Add(fi);
                    }
                }
            }

            featureList.Sort(delegate(FeatureItem item1, FeatureItem item2)
            {
                return item1.Name.CompareTo(item2.Name);
            });
            lblTotal.Text = string.Format("Total count: {0}", featureList.Count);

            gvFeature.DataSource = featureList;
            gvFeature.DataBind();
        }
    </script>

</head>
<body>
    <form id="form1" runat="server">
    <div style="margin: 15">
        <p>
            <asp:Label ID="Label2" runat="server" Text="Feature Status:"></asp:Label>
            <asp:DropDownList ID="ddlActive" runat="server" AutoPostBack="true">
                <asp:ListItem Selected="True">Active</asp:ListItem>
                <asp:ListItem>Inactive</asp:ListItem>
            </asp:DropDownList>&nbsp;
            <asp:Label ID="Label3" runat="server" Text="Feature Scope:"></asp:Label>
            <asp:DropDownList ID="ddlScope" runat="server" AutoPostBack="true">
                <asp:ListItem>All</asp:ListItem>
                <asp:ListItem>Farm</asp:ListItem>
                <asp:ListItem>WebApplication</asp:ListItem>
                <asp:ListItem>Site</asp:ListItem>
                <asp:ListItem Selected="True">Web</asp:ListItem>
            </asp:DropDownList>&nbsp;
            <asp:Label ID="lblTotal" runat="server"></asp:Label>
        </p>
        <p>
            <asp:Label ID="lblInfo" runat="server"></asp:Label>
            <asp:GridView ID="gvFeature" runat="server" AutoGenerateColumns="False" BorderWidth="1px"
                BackColor="White" CellPadding="4" BorderStyle="Solid" BorderColor="#3366CC"
                Font-Size="Small" AlternatingRowStyle-ForeColor="ActiveCaption">
                <HeaderStyle ForeColor="White" BackColor="#003399" HorizontalAlign="Left">
                </HeaderStyle>
                <Columns>
                    <asp:BoundField HeaderText="Feature Name" DataField="Name" ReadOnly="true" ItemStyle-Width="380px">
                    </asp:BoundField>
                    <asp:BoundField HeaderText="Scope" DataField="Scope" ReadOnly="true" ItemStyle-Width="120px">
                    </asp:BoundField>
                     <asp:BoundField HeaderText="Hidden" DataField="Hidden" ReadOnly="true" ItemStyle-Width="120px">
                    </asp:BoundField>
                    <asp:BoundField HeaderText="FeatureID" DataField="ID" ReadOnly="true" ItemStyle-Font-Size="X-Small" ItemStyle-Width="240px">
                    </asp:BoundField>
                </Columns>
                <EmptyDataTemplate>
                    <asp:Label ID="lblEmptyMessage" runat="server" Text="No feature found."></asp:Label>
                </EmptyDataTemplate>
            </asp:GridView>
        </p>
    </div>
    </form>
</body>
</html>

SharePoint Content Deployment And Modified Field

I have talked about SharePoint Modified field in here and here. What about content deployment?

After content deployment (full or incremental), the Modified time of a list item in the target site will be the same as the last published time of the original list item. For example, sourceSite.list1.item1 has two latest versions:

Modified: 2010-01-01 11:00AM (version 5.2)
Modified: 2010-01-01 10:00AM (version 5.0)

Suppose a full content deployment is conducted at 2010-02-02, then the targetSite.list1.item1 will have one version:

Modified: 2010-01-01 10:00AM (version 1.0)

Note that the versions in target site are not the same as original's. It always starts from version 1.0, and increases to 2.0, 3.0...

Telerik Editor Issue When Overriding Page Render Method

Telerik RAD Editor is used in one SharePoint WCM site. To include Flash objects and scripts inside RAD Editor, I did following steps:
1. Add "<tool name='FlashManager' />" to Editor configuration file in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\wpresources\RadEditorSharePoint\5.5.1.0__1f131a624888eeed\ToolsFile.xml. This ensures the flash manager icon is available in edit window. Options for adding this setting:
2. Update all RADEditor field controls in page layout from:
<telerik:RadHtmlField id="RadHtmlField1" FieldName="fieldName" runat="server" CssClass="telerikeditor" ></telerik:RadHtmlField>

To:

<telerik:RadHtmlField id="RadHtmlField1" FieldName="fieldName" runat="server" CssClass="telerikeditor" AllowScripts="true" AllowSpecialTags="true"></telerik:RadHtmlField>

This makes scripts and special tags (such as flash <object> tag) legal inside RADEditor.

But the scripts and flash content were not showing. The source html shows something like:

<pre style="display:none" id=RadEditorEncodedTag>....</pre>

I googled and couldn't see a fix about this issue. So I used reflector to find out how Telerik does under the hood. And I figured out that RAD Editor stores non-standard tags like flash objects and scripts in following format:

<pre style="display: none;" id="RadEditorEncodedTag">Base64codingObjects</pre>

RADEditor field Control will decode it during render cycle:

protected override void RenderFieldForDisplay(HtmlTextWriter output)
{
if (!string.IsNullOrEmpty((string) this.ItemFieldValue))
{
    bool canCacheResults = true;
    string html = HtmlEditorInternal.ConvertStorageFormatToViewFormat(
         (string) this.ItemFieldValue, out canCacheResults);
    output.Write(EditorTools.UnEscapeSpecialTags(html));
}
}

The problem is that Render Method is overridden in the custom page layouts, and this RenderFieldForDisplay is not invoked. I then added such decoding logic to the overridden Render method:

        protected override void Render(HtmlTextWriter writer)
    {
         //...
         StringBuilder sb = new StringBuilder();
         StringWriter sw = new StringWriter(sb);
         HtmlTextWriter ourWriter = new HtmlTextWriter(sw);
         base.Render(ourWriter);
         string writeString = EscapeRADEditorTags(sb.ToString());
         Response.Write(writeString);
         //...
    }

    private static string EscapeRADEditorTags(string html)
    {
        try
        {
            MatchEvaluator evaluator = new MatchEvaluator(DecodeMatchManually);
            return Regex.Replace(html, 
                    "<pre\\s+(style=\"display\\s*:none\"\\s*)?id=\"?RadEditorEncodedTag\"?[^>]*>([^<]+)</pre>", 
                    evaluator, RegexOptions.ECMAScript | RegexOptions.Multiline | RegexOptions.IgnoreCase);
        }
        catch (Exception ex)
        {
            Logger.LogException(ex);
            return html;
        }
    }

Everything works properly after this code change.

HttpModule Blocks SharePoint Designer Issue

A HttpModule is added to a SharePoint publishing site. Every thing works fine except that SharePoint designer is not able to open the SharePoint site. After tracing the html traffic, I noticed some communication patterns between SharePoint designer and the SharePoint site. So the workaround is to skip all HttpModule actions for those communications. Following code is added inside the HttpModule:

    if ( httpContext.Request.UserAgent.Contains("WebDAV") 
        || httpContext.Request.UserAgent.Contains("MS FrontPage")
        || httpContext.Request.Url.AbsoluteUri.Contains("_vti_"))
    {
        return;
    }

Now SharePoint designer can open the site without any problem. But sometimes the check-out function inside SharePoint designer is disable for some reason. I can not figure out why that happens sometimes but not always.

SharePoint Limit Of 11 Direct Dependencies

MOSS 2007 Error

"An error occurred during the process of . The page '/_catalogs/masterpage/Support.master' allows a limit of 11 direct dependencies, and that limit has been exceeded.":




Resolution

Search following default setting in web.config:

    <SafeMode MaxControls="200" CallStack="false" DirectFileDependencies="10" 
     TotalFileDependencies="50" AllowPageLevelTrace="false">
      <PageParserPaths>
      </PageParserPaths>
    </SafeMode>

Change DirectFileDependencies value from 10 to a bigger number.