Tuesday, September 20, 2011

SharePoint 2010 SPList Backup and Restore

In SharePoint 2010 it's very easy to backup and restore a Site Collection, a Site, or a List. For example if want to backup a SPList, just go to Central Admin => Backup and Restore => Export a site or list, select the List, type the backup file name then click the "Start Export" button and all done. Alternatively we can use powershell to do the same job:
Export-SPWeb -Identity "https://portal.intranet.com/department/it" 
             -ItemUrl "/department/it/Lists/Projects" 
             -Path c:\backup\ITProjectsList.cmp
             -IncludeUserSecurity
To restore the List run the script:
Import-SPWeb -Identity "https://backup.intranet.com/department/it"
             -Path c:\backup\ITProjectsList.cmp
             -IncludeUserSecurity 
             -Force
Be aware that the backup could result in multiple files depending on the SPList size and the machine's setting. If that's the case and you miss any backup file with .cmp extension, you will get following error when run the Import-SPWeb command:
Import-SPWeb: Could not find file 'C:\Users\{user}\AppData\Local\Temp\{guid}\ExportSettings.xml'...
Simply copying all the backup files to the restore location would resolve the problem.

Thursday, May 19, 2011

ASP.NET MVC Controller Out of Control

MVC pattern and Microsoft ASP.NET MVC implementation are quite popular and successful in recent years for many reasons such as modularity, readability, maintainability, and separation of concerns. However, out-of-box ASP.NET MVC may not fit well in some scenarios. You won't get the goodness of MVC without proper implementation. Recently when I reviewed one of our ASP.NET MVC applications I just found another example of how a pattern could become anti-pattern.

The ASP.NET MVC application is an internal authoring system letting people to manage a bunch of products and their attributes. The products then can be published and exported to a list of XML files that are used by front-end servers to display to public. This application's base domain model was quite simply: a Product has many properties and a Product belongs to one or more Category. The complexity goes to product properties and business logic behind them. That resulted in a huge Product controller in the implementation. The controller class is so big that the developer split it to a couple of partial classes to avoid thousands lines of code in one class file. That doesn't smell good at all.

The Product management UI has a few tabs which group different properties in a logical way. AJAX has been extensively used to populate and update those partial Product properties. Each UI component (partial view) has at least two corresponding controller actions, one for read and one for date. The controller was too complex and was doing too much: input and validation, data manipulation (CRUD), view mapping etc. It looks like going back to the old WebForm model and the controller is just like the code-behind we used to do in traditional ASP.NET.

This sounds like a common problem. Google "ASP.NET MVC big fat controller" there're thousands of results. Many people have been struggling on similar pain. Also many people provided suggestions or solutions to resolve the problem. I found one article with great insight on this topic that posted long time ago right after ASP.NET MVC 1.0 was released.

A handful esteemed people even built a new MVC framework on top of ASP.NET from scratch called FubuMVC when they found ASP.NET MVC had the problem along with other issues. If you want to stick with ASP.NET MVC, there're some good thoughts to slim the controller and make the applicaton more readable, extendable and maintainable:

1. ControllerLess action by one controller one action.
2. Grouping controllers.
3. Use AutoMapper to simplify the property mapping between different models.
4. Move logic and code to service layer as much as possible.
5. Use Command pattern to distribute responsibilities.

The last one Command pattern can be extended to Command Query Responsibility Segregation (CQRS) pattern, referring to this channel-9 video about applying CQRS to ASP.NET MVC 3.

Above ideas can be used together when we are doing ASP.NET MVC application developing or refactoring. It sure takes more effort at the beginning but it would pay off in the long run.

Tuesday, April 26, 2011

Dynamically Invoke Generic Method In .NET


Today I cleaned up a lengthy method in our project which I think it's worthy of a post. Our project has a repository that caches all business objects, we called them entities and the entity list is long. The cache is loaded/refreshed when repository is first accessed or some entities have been changed in the back-end store. The method for loading entities is something like:
void LoadEntitiesCache()
{
    List<EntityType1> entity1List = DataStore.GetAllEntitiesByType<EntityType1>();
    SaveEntitiesToCache(Entity1List);
    List<EntityType2> entity2List = DataStore.GetAllEntitiesByType<EntityType2>();
    SaveEntitiesToCache(Entity2List);
    List<EntityType3> entity3List = DataStore.GetAllEntitiesByType<EntityType3>();
    SaveEntitiesToCache(Entit32List);
    ....
    List<EntityTypeN> entityNList = DataStore.GetAllEntitiesByType<EntityTypeN>();
    SaveEntitiesToCache(EntityNList);
}
The way of my clean-up is something like:
void LoadEntitiesCache()
{
    Type [] allEntityTypes = new Type [] { EntityType1, EntityType2, ..., EntityTypeN };
    foreach(Type entityType in allEntityTypes)
    {
        List<entityType> entityList = DataStore.GetAllEntitiesByType<entityType>();
        SaveEntitiesToCache(entityList);
    }
}
The logic is there but above code won't work because you can't invoke a generic method like that, instead reflection must be used for such usage. Following code example shows how to use reflection to invoke a generic method (entity/cache update logic is not included):
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection;
using System.Collections.ObjectModel;
using System.Collections.Concurrent;

public interface IEntity
{
    string Identifier { get; }
    string Type { get; }
}

public class Book : IEntity
{
    public string ISBN { get; set; }
    public string Category { get; set; }
    public string Identifier { get { return ISBN; } }
    public string Type { get { return Category; } }
}

public class CD : IEntity
{
    public string Name { get; set; }
    public string Category { get; set; }
    public string Identifier { get { return Name; } }
    public string Type { get { return Category; } }
}

public class KeyedEntityCollection<T> : KeyedCollection<string, T> where T : IEntity
{
    protected override string GetKeyForItem(T item)
    {
        return item.Identifier;
    }
}

public class Repository
{
    static ConcurrentDictionary<Type, object> _entitiesCache = new ConcurrentDictionary<Type, object>();

    void LoadEntitiesCache()
    {
        // Load all entities from back-end store and save them in _entitiesCache
        DataSource dataSource = new DataSource();

        Type[] entityTypes = new Type[] { typeof(Book), typeof(CD) };
        MethodInfo methodInfo = typeof(DataSource).GetMethod("GetEntitiesByType");
        foreach (var entityType in entityTypes)
        {
            object _cacheObject = null;
            if (_entitiesCache.TryGetValue(entityType, out _cacheObject) && _cacheObject != null)
                continue;

            try
            {
                // Invoke DataStore.GetEntitiesByType<T>() dynamically
                MethodInfo genericMethod = methodInfo.MakeGenericMethod(entityType);
                IEnumerable dataReturn = genericMethod.Invoke(dataSource, null) as IEnumerable;

                // Create EntityCollection dynamically
                Type[] types = new Type[] { entityType };
                Type entityCollectionType = typeof(KeyedEntityCollection<>);
                Type genericType = entityCollectionType.MakeGenericType(types);
                IList genericCollection = Activator.CreateInstance(genericType) as IList;

                if (dataReturn != null)
                {
                    foreach (var entity in dataReturn)
                    {
                        genericCollection.Add(entity);
                    }
                }
                _entitiesCache.AddOrUpdate(entityType, genericCollection, 
                    (type, existingValue) => { return genericCollection; });
            }
            catch (Exception ex)
            {
                // Log error
                throw;
            }
        }
    }

    public Repository()
    {
        LoadEntitiesCache();
    }

    public KeyedEntityCollection<T> GetEntities<T>() where T : IEntity
    {
        object cachedEntityCollection = null;
        _entitiesCache.TryGetValue(typeof(T), out cachedEntityCollection);
        return cachedEntityCollection as KeyedEntityCollection<T>;
    }

    public T GetEntityById<T>(string identifier) where T : IEntity
    {
        var entityCollection = GetEntities<T>();
        if (entityCollection != null && entityCollection.Contains(identifier))
        {
            return entityCollection[identifier];
        }
        return default(T);
    }
}

internal class DataSource
{
    // Get data from database, or xml files, or whatever data store
    public IEnumerable<T> GetEntitiesByType<T>() where T : IEntity
    {
        // Mock some test data here
        if (typeof(T) == typeof(Book))
        {
            List<Book> bookSource = new List<Book>();
            Enumerable.Range(1, 100).ToList().ForEach(
                i => bookSource.Add(new Book() { ISBN = i.ToString(), Category = "Computer" }));
            return bookSource as IEnumerable<T>;
        }
        else if (typeof(T) == typeof(CD))
        {
            List<CD> cdSource = new List<CD>();
            Enumerable.Range(1, 100).ToList().ForEach(
                i => cdSource.Add(new CD() { Name = i.ToString(), Category = "Music" }));
            return cdSource as IEnumerable<T>;
        }
        else return null;
    }
}

class Program
{
    static void Main(string[] args)
    {
        Repository repo = new Repository(); 
        var books = repo.GetEntities<Book>();           // Get all books
        var secondBook = repo.GetEntityById<Book>("2"); // Get the second book
        var cds = repo.GetEntities<CD>();               // Get all CDs
        var thirdCD = repo.GetEntityById<CD>("3");      // Get the third CD
        Console.Read();
    }
}

Friday, February 04, 2011

ASP.NET Cache, MemoryCache and Velocity

Caching is one of the most effective ways to improve performance for high-traffic/high-volume service applications. Caching is considered as a common practice to deal with data access with heavy I/O. ASP.NET Cache from System.Web dll is the only one memory caching implementation you can use in .NET 1.0/2.0/3.0 framework. It's there for so long that we almost forget the question at the very beginning: why put it into the Web context? You have to add System.Web reference even though you are working on class libraries or WinForm applications. Anyway people are just used to this bad design.

Finally Microsoft has a new ObjectCache/MemoryCache caching system from System.Runtime.Caching.dll in the latest version of the .NET 4.0 Framework. The API and usage is similar but it makes more senses now. We can use the MemoryCache in any application (I found it's not applicable to Console apps) without dependency on System.Web. One noticeable change in the new Cache API is that it has only one Add method, and gets rid of the confusion from two Add and Insert methods in ASP.NET Cache. Here's some note from MSDN describing the difference between MemoryCache and traditional ASP.NET Cache:

The MemoryCache class is similar to the ASP.NET Cache class. The MemoryCache class has many properties and methods for accessing the cache that will be familiar to you if you have used the ASP.NET Cache class. The main differences between the Cache and MemoryCache classes are that the MemoryCache class has been changed to make it usable by .NET Framework applications that are not ASP.NET applications. For example, the MemoryCache class has no dependencies on the System.Web assembly. Another difference is that you can create multiple instances of the MemoryCache class for use in the same application and in the same AppDomain instance.

How about that distributed caching system with code name of "Velocity"? It has been re-branded and becomes part of the Windows Server AppFabric - version 1.0 officially released a few months ago. The origin of "Velocity" project was a .NET version of MemCached. I remember our company was considering using "Velocity" three years ago (CTP version at that time). Our lead said let's wait for the mature final version. Now Microsoft eventually makes it the final 1.0 version after four years of work. What a super velocity! It took a new graduate student Brad Fitzpatrick a few months to create the MemCached solution for his LiveJournal site, and now it's pretty much the standard in the OpenSource world. It takes so long for Microsoft to develop a .NET component that's crucial for big systems and enterprise environments. A deep sigh for Microsoft.

Related to .NET caching there's an interesting post talking about loading different cache systems dynamically, including ASP.NET Cache, MemoryCache and AppFabric Cache. The implementation is simple and it could be helpful in some scenario.

Friday, January 28, 2011

Implementing Simple Custom Rules By Code

In my previous two posts Microsoft Workflow Rules Engine was used to execute the business rules. That maybe a bit overkill if there're only a small set of rules to run and the use cases of them are relatively simple. In that case we could implement the rules and control their execution directly. The simple console app below demos the concept of custom rules implementation:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication1
{
    class Program
    {
        #region demo domain objects
        public class Product
        {
            public string Name { get; set; }
            public decimal Price { get; set; }
        }

        public class ShoppingCart
        {
            public List<Product> SelectedProducts { get; set; }
            public List<string> CouponCodes { get; set; }
            public decimal TotalAmount { get { return SelectedProducts.Sum(prod => prod.Price); } }
            internal decimal Discount { get; set; }

            public ShoppingCart()
            {
                SelectedProducts = new List<Product>();
                CouponCodes = new List<string>();
                Discount = 0;
            }
        }
        #endregion demo domain objects

        #region promotion objects 
        class CouponPromotion  
        {
            public string Name { get; set; }
            public string PromotionCode { get; set; }
            public decimal Discount { get; set; }
            public DateTime ExpiredOn { get; set; }
        }

        class TotalAmountPromotion // discount if total amount is greater than a number
        {
            public string Name { get; set; }
            public decimal TotalAmount { get; set; }
            public decimal Discount { get; set; }
            public DateTime ExpiredOn { get; set; }
        }
        #endregion promotion object

        #region custom rules interface and implementation
        interface ICustomRule
        {
            void Execute(ShoppingCart cart);
        }

        class CouponPromotionRule : ICustomRule
        {
            private CouponPromotion mCouponPromotion;
            public CouponPromotionRule(CouponPromotion promotion)
            {
                this.mCouponPromotion = promotion;
            }

            public void Execute(ShoppingCart cart)
            {
                if (mCouponPromotion.ExpiredOn >= DateTime.Today && cart.CouponCodes.Contains(mCouponPromotion.PromotionCode))
                {
                    Console.WriteLine(mCouponPromotion.Name + " applied");
                    cart.Discount += mCouponPromotion.Discount;
                }
            }
        }

        class TotalAmountGreaterRule : ICustomRule
        {
            private TotalAmountPromotion mTotalAmountPromotion;
            public TotalAmountGreaterRule(TotalAmountPromotion promotion)
            {
                this.mTotalAmountPromotion = promotion;
            }

            public void Execute(ShoppingCart cart)
            {
                if (mTotalAmountPromotion.ExpiredOn >= DateTime.Today && cart.TotalAmount >= mTotalAmountPromotion.TotalAmount)
                {
                    Console.WriteLine(mTotalAmountPromotion.Name + " applied");
                    cart.Discount += mTotalAmountPromotion.Discount;
                }
            }
        }
        #endregion custom rules interface and implementation

        static void RuleTest(ShoppingCart cart)
        {
            // Mock CouponPromotionRule and TotalAmountRules
            CouponPromotionRule rule1 = new CouponPromotionRule(new CouponPromotion()
            {
                Name = "Coupon Promotion Rule 1",
                PromotionCode = "123456789",
                Discount = 10,
                ExpiredOn = new DateTime(2100, 1, 1)
            });

            CouponPromotionRule rule2 = new CouponPromotionRule(new CouponPromotion()
            {
                Name = "Coupon Promotion Rule 2",
                PromotionCode = "111111111",
                Discount = 20,
                ExpiredOn = new DateTime(2000, 1, 1)
            });
            TotalAmountGreaterRule rule3 = new TotalAmountGreaterRule(new TotalAmountPromotion()
            {
                Name = "Total Amount Rule 1",
                TotalAmount = 1000,
                Discount = 100,
                ExpiredOn = new DateTime(2100, 1, 1)
            });

            ICustomRule[] rules = new ICustomRule[] { rule1, rule2, rule3 };
            foreach (var rule in rules)
            {
                rule.Execute(cart);
            }
        }

        static void Main(string[] args)
        {
            ShoppingCart shoppingCart = new ShoppingCart();
            Enumerable.Range(1, 5).ToList().ForEach(i => shoppingCart.SelectedProducts.Add(
                   new Product() { Name = "Product" + i.ToString(), Price = 100 * i }));
            shoppingCart.CouponCodes.Add("123456789");
            shoppingCart.CouponCodes.Add("111111111");
            Console.Write("\nShopping Cart before rules execution -- ");
            Console.WriteLine("Total : ${0}\tDiscount : ${1}", shoppingCart.TotalAmount, shoppingCart.Discount);

            RuleTest(shoppingCart);

            Console.Write("Shopping Cart after rules execution -- ");
            Console.WriteLine("Total : ${0}\tDiscount : ${1}", shoppingCart.TotalAmount, shoppingCart.Discount);

            Console.Read();
        }      
    }
}

The result of the test app:

Thursday, January 27, 2011

Create Windows Workflow 4.0 Rules By Custom String

In my previous post, I presented a way to create Windows Workflow rules by CodeDom expression. Sometimes it's hard to build a meaningful UI for authoring business rules. In such case we may need to use custom string to construct WF rules. The key part is how to parse the rule condition/action string. We could do similar things in WF RuleSetDialog. Can we re-use the parser used by RuleSetDialog? The answer is yes. Beau Crawford gave a solution for WF 3.5. Basically WF internally has a rule condition and action parser under System.Workflow.Activities.Rules.Parser inside System.Workflow.Activities assembly, but the parser is not exposed to public. In order to re-use the internal parser we need to use reflection.

Following code example demos how to archive this by slight modification of the code used in previous post:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.CodeDom;
using System.Reflection;
using System.Workflow.Activities.Rules;
using System.Workflow.ComponentModel.Compiler;

class Product
{
public string Name { get; set; }
public decimal Price { get; set; }
}

class ShoppingCart
{
public List<Product> SelectedProducts { get; set; }
public List<string> CouponCodes { get; set; }
public decimal TotalAmount { get { return SelectedProducts.Sum(prod => prod.Price); } }
internal decimal Discount { get; set; }

public ShoppingCart()
{
SelectedProducts = new List<Product>();
CouponCodes = new List<string>();
Discount = 0;
}
}

class CouponPromotion
{
public string Name { get; set; }
public string PromotionCode { get; set; }
public decimal Discount { get; set; }
public DateTime ExpiredOn { get; set; }
}

/// <summary>
/// WFRuleParser is helper class to parse Windows Workflow condition and action.
/// Invoke ParseCondition and ParseAction two internal methods
/// defined in System.Workflow.Activities.Rules.Parser by reflection
/// </summary>
class WFRuleParser
{
public static RuleExpressionCondition ParseCondition(Type targetObjectType, string expression)
{
RuleValidation ruleValidation = new RuleValidation(targetObjectType, null);
return ExecuteMethod("ParseCondition",
new object[] { ruleValidation }, new object[] { expression }) as RuleExpressionCondition;
}

public static List<RuleAction> ParseAction(Type targetObjectType, string expression)
{
RuleValidation ruleValidation = new RuleValidation(targetObjectType, null);
return ExecuteMethod("ParseStatementList",
new object[] { ruleValidation }, new object[] { expression }) as List<RuleAction>;
}

// Invoke private/internal method using reflection
private static object ExecuteMethod(string methodName, object[] ctorParameters, object[] methodParameters)
{
string ParserTypeName ="System.Workflow.Activities.Rules.Parser, System.Workflow.Activities,
Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
;
try
{
Type type = Type.GetType(ParserTypeName);
ConstructorInfo constructor = type.GetConstructor(BindingFlags.NonPublic | BindingFlags.Instance,
null, new Type[] { typeof(RuleValidation) }, null);
object instance = constructor.Invoke(ctorParameters);
MethodInfo method = instance.GetType().GetMethod(methodName,
BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);
return method.Invoke(instance, methodParameters);
}
catch (Exception ex)
{
throw ex;
}
}
}

class WFRuleBuilder
{
// Create a Rule for a CouponPromotion
public static Rule GetCouponPromotionRule(CouponPromotion couponPromo)
{
Type targetObjectType = typeof(ShoppingCart); // Shopping Cart is the input object for Rules Engine

// condition => if coupone code is matched and promotion is not expired
var conditionString = string.Format(
"this.CouponCodes.Contains(\"{0}\") && DateTime.Today < DateTime.Parse(\"{1}\")",
couponPromo.PromotionCode, couponPromo.ExpiredOn.ToString("yyyy-MM-dd"));
var condition = WFRuleParser.ParseCondition(targetObjectType, conditionString);

// action => add discount
var actionString = string.Format("this.Discount = this.Discount + {0}", couponPromo.Discount);
var action = WFRuleParser.ParseAction(targetObjectType, actionString);

return new Rule(couponPromo.Name, condition, action, null);
}
}

class Program
{
static void Main(string[] args)
{
// Mock 5 products in shopping cart:
// [{"Product1":{Price:$100, Category:"Category1"}}, {"Product2":{Price: $200, Category:"Category1"}}, ...]
ShoppingCart shoppingCart = new ShoppingCart();
Enumerable.Range(1, 5).ToList().ForEach(i => shoppingCart.SelectedProducts.Add(
new Product() { Name = "Product" + i.ToString(), Price = 100 * i }));
shoppingCart.CouponCodes.Add("123456789");
shoppingCart.CouponCodes.Add("111111111");
Console.Write("\nShopping Cart before rules execution -- ");
Console.WriteLine("Total : ${0}\tDiscount : ${1}", shoppingCart.TotalAmount, shoppingCart.Discount);

// Mock 2 CouponPromotions
CouponPromotion promo1 = new CouponPromotion()
{
Name = "Promotion1", PromotionCode = "123456789", Discount = 100, ExpiredOn = new DateTime(2000, 1, 1)
};
CouponPromotion promo2 = new CouponPromotion()
{
Name = "Promotion2", PromotionCode = "111111111", Discount = 100, ExpiredOn = new DateTime(2100, 1, 1)
};


// Build a RuleSet with above two rules
RuleSet ruleSet = new RuleSet();
ruleSet.Rules.Add(WFRuleBuilder.GetCouponPromotionRule(promo1));
ruleSet.Rules.Add(WFRuleBuilder.GetCouponPromotionRule(promo2));

// Print Rule information
Console.WriteLine("\nRules defined in the RuleSet:");
ruleSet.Rules.ToList().ForEach(rule =>
{
Console.WriteLine("\"{0}\" RuleCondition: \n{1}", rule.Name, rule.Condition);
Console.WriteLine("\"{0}\" RuleAction: \n{1}", rule.Name, rule.ThenActions[0]);
});

// Execute Rules
RuleValidation validation = new RuleValidation(typeof(ShoppingCart), null);
if (ruleSet.Validate(validation))
{
RuleEngine engine = new RuleEngine(ruleSet, validation);
engine.Execute(shoppingCart);
}
else // Validation failed, print errors
{
StringBuilder errors = new StringBuilder();
foreach (ValidationError validationError in validation.Errors)
{
errors.Append(validationError.ErrorText);
}
Console.WriteLine("Validation errors:{0}", errors);
}

Console.Write("\nShopping Cart after rules execution -- ");
Console.WriteLine("Total : ${0}\tDiscount : ${1}", shoppingCart.TotalAmount, shoppingCart.Discount);

Console.Read();
}
}
A CouponPromotion has a coupon code and an expiration date property. At run-time, rules engine will check all coupon codes stored in a ShoppoingCart, then apply the discount if they are valid. The logic is:
if ShoppingCart.Contains(couponPromotion.CouponCode) && couponPromotion is not expired
then add CurrentCouponPromotion.Discount to ShoppingCart
Remember the target object the rule engine runs against on is a ShoppingCart instance. The rule action would be update ShoppingCart's Discount property. The exact strings to be parsed for two promotions defined in above example code are:
Promotion1 condition string:
this.CouponCodes.Contains("123456789") && System.DateTime.Today < System.DateTime.Parse("2000-01-01")
Promotion1 action string:
this.Discount = this.Discount + 100
Promotion2 condition string:
this.CouponCodes.Contains("111111111") && System.DateTime.Today < System.DateTime.Parse("2100-01-01")
Promotion2 action string:
this.Discount = this.Discount + 100
Obviously Promotion1 has expired and only Promotion1 is valid and applied. The console app result:

Tuesday, January 25, 2011

Business Rules Implementation Using Windows Workflow 4.0

Recently I have been working on a project for a back-end service providing product information for a telecom company. The pricing of a product or products is relying on a number of business rules. e.g. a cell-phone's price and its monthly charge can vary based on the data/voice plan as well as those value-added features. A discount or multiple discounts could be applied if certain criteria are matched. In addition, complicated validation rules are applied for those products.

Windows Workflow Rules Engine, a free out-of-box feature from Windows Workflow Foundation, plays a center role in our business rules implementation. We construct our own business entities and relationship. On top of those business objects we build our customized rule authoring UI. Those rule-related entities are converted to CodeDom expression at run-time, and then a set of Workflow RuleSets are created based on those CodeDom expression. For each service call, products will be passed to the Workflow Rule Engine as an input, and rules execution would update their metadata or properties if a rule or some rules are satisfied.

Following code sample demos how all this works:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.CodeDom;
using System.Workflow.Activities.Rules;
using System.Workflow.ComponentModel.Compiler;

#region Business Object Models
class Product
{
public string Name { get; set; }
public string Category { get; set; }
public decimal Price { get; set; }
}

class ShoppingCart
{
public List<Product> SelectedProducts { get; set; }
public string CouponCode { get; set; }
public decimal TotalAmount { get { return SelectedProducts.Sum(prod => prod.Price); } }
internal decimal Discount { get; set; }

public ShoppingCart()
{
SelectedProducts = new List<Product>();
Discount = 0;
}

// Check total amount for a given category
public bool TotalCategoryAmountGreaterThan(decimal price, string category)
{
decimal totalPriceForCategory = 0;
foreach (var prod in SelectedProducts)
{
if (prod.Category == category)
totalPriceForCategory += prod.Price;
}

return totalPriceForCategory >= price;
}
}
#endregion

#region Rule-related Object Models
/// <summary>
/// Apply a discount if having the promotion code
/// </summary>
class CouponPromotion
{
public string Name { get; set; }
public string PromotionCode { get; set; }
public decimal Discount { get; set; }
}

/// <summary>
/// Apply a discount when total amount for a given category greater than a value
/// </summary>
class BundlePromotion
{
public string Name { get; set; }
public string ProductCategory { get; set; }
public decimal TotalAmountThreshold { get; set; }
public decimal Discount { get; set; }
}
#endregion

#region Helper class to create rules based on Rule-related objects
class WFRuleBuilder
{
// Create a Rule for a CouponPromotion
public static Rule GetCouponPromotionRule(CouponPromotion couponPromo)
{
CodePrimitiveExpression couponCode = new CodePrimitiveExpression(couponPromo.PromotionCode);
CodePrimitiveExpression couponDiscount = new CodePrimitiveExpression(couponPromo.Discount);

CodePropertyReferenceExpression couponPropertyRef =
new CodePropertyReferenceExpression(new CodeThisReferenceExpression(), "CouponCode");
CodePropertyReferenceExpression discountPropertyRef =
new CodePropertyReferenceExpression(new CodeThisReferenceExpression(), "Discount");

// Create a binary equality expression to compare the property value
CodeBinaryOperatorExpression couponConditionExpression =
new CodeBinaryOperatorExpression(couponCode, CodeBinaryOperatorType.ValueEquality, couponPropertyRef);

// Create a binary addition expression to increase the discount
CodeBinaryOperatorExpression discountExpression =
new CodeBinaryOperatorExpression(discountPropertyRef, CodeBinaryOperatorType.Add, couponDiscount);
CodeAssignStatement discountActionExpression =
new CodeAssignStatement(discountPropertyRef, discountExpression);

Rule couponRule = new Rule(couponPromo.Name);
couponRule.Condition = new RuleExpressionCondition(couponConditionExpression);
couponRule.ThenActions.Add(new RuleStatementAction(discountActionExpression));

return couponRule;
}

// Create a Rule for a BundlePromotion
public static Rule BuildBundlePromotionRule(BundlePromotion bundlePromo)
{
CodePrimitiveExpression priceThreshold = new CodePrimitiveExpression(bundlePromo.TotalAmountThreshold);
CodePrimitiveExpression productCategory = new CodePrimitiveExpression(bundlePromo.ProductCategory);
CodePrimitiveExpression discount = new CodePrimitiveExpression(bundlePromo.Discount);
CodePrimitiveExpression[] parameters = new CodePrimitiveExpression[] { priceThreshold, productCategory };

CodePropertyReferenceExpression discountPropertyRef =
new CodePropertyReferenceExpression(new CodeThisReferenceExpression(), "Discount");

// Create a method invocation expression to check the totalValue at runtime
CodeMethodInvokeExpression methodEvaluationExpression =
new CodeMethodInvokeExpression(new CodeThisReferenceExpression(), "TotalCategoryAmountGreaterThan", parameters);

// Create a binary addition expression to increase the discount
CodeBinaryOperatorExpression discountExpression =
new CodeBinaryOperatorExpression(discountPropertyRef, CodeBinaryOperatorType.Add, discount);
CodeAssignStatement discountActionExpression = new CodeAssignStatement(discountPropertyRef, discountExpression);

Rule priceThresholdRule = new Rule(bundlePromo.Name);
priceThresholdRule.Condition = new RuleExpressionCondition(methodEvaluationExpression);
priceThresholdRule.ThenActions.Add(new RuleStatementAction(discountActionExpression));

return priceThresholdRule;
}
}
#endregion

class Program
{
static void Main(string[] args)
{
// Mock 5 products in shopping cart:
// [{"Product1":{Price:$100, Category:"Category1"}}, {"Product2":{Price: $200, Category:"Category1"}}, ...]
ShoppingCart mockShoppingCart = new ShoppingCart();
Enumerable.Range(1, 5).ToList().ForEach(i => mockShoppingCart.SelectedProducts.Add(
new Product() { Name = "Product" + i.ToString(), Price = 100 * i, Category = "Category1" }));
mockShoppingCart.CouponCode = "123456789";
Console.Write("\nShopping Cart before rules execution -- ");
Console.WriteLine("Total : ${0}\tDiscount : ${1}", mockShoppingCart.TotalAmount, mockShoppingCart.Discount);

// Mock some rules
RuleSet rules = MockRules();

// Execute Rules
RuleValidation validation = new RuleValidation(typeof(ShoppingCart), null);
if (rules.Validate(validation))
{
RuleEngine engine = new RuleEngine(rules, validation);
engine.Execute(mockShoppingCart);
}
else // Validation failed, print errors
{
StringBuilder errors = new StringBuilder();
foreach (ValidationError validationError in validation.Errors)
{
errors.Append(validationError.ErrorText);
}
Console.WriteLine("Validation errors:{0}", errors);
}

Console.Write("\nShopping Cart after rules execution -- ");
Console.WriteLine("Total : ${0}\tDiscount : ${1}", mockShoppingCart.TotalAmount, mockShoppingCart.Discount);

Console.Read();
}

// Mock one CouponPromotion rule and PackageTotalPricePromotion rule for testing
private static RuleSet MockRules()
{
// Mock coupon rule: $10 discount if the user has a coupon code of "123456789"
CouponPromotion couponPromo = new CouponPromotion()
{
Name = "2011 new year coupon",
PromotionCode = "123456789",
Discount = 10
};

// Mock total price rule: $100 discount if purchased more than $1000 for ProductCategory of "Category1"
BundlePromotion bundlePromo1 = new BundlePromotion()
{
Name = "Bundle Promotion for Category1",
ProductCategory = "Category1",
TotalAmountThreshold = 1000,
Discount = 100
};

// Mock total price rule: $200 discount if purchased more than $1000 for ProductCategory of "Category2"
BundlePromotion bundlePromo2 = new BundlePromotion()
{
Name = "Bundle Promotion for Category2",
ProductCategory = "Category2",
TotalAmountThreshold = 1000,
Discount = 200
};

// Create a RuleSet to contain above two rules
RuleSet promotionRules = new RuleSet("Promotion Rules");
promotionRules.Rules.Add(WFRuleBuilder.GetCouponPromotionRule(couponPromo));
promotionRules.Rules.Add(WFRuleBuilder.BuildBundlePromotionRule(bundlePromo1));
promotionRules.Rules.Add(WFRuleBuilder.BuildBundlePromotionRule(bundlePromo2));

// Print Rule information
Console.WriteLine("\nRules defined in the RuleSet:");
promotionRules.Rules.ToList().ForEach(rule =>
{
Console.WriteLine("\"{0}\" RuleCondition: \n{1}", rule.Name, rule.Condition);
Console.WriteLine("\"{0}\" RuleAction: \n{1}", rule.Name, rule.ThenActions[0]);
});

return promotionRules;
}

}
The console app shows that a $10 coupon discount and a $100 bundle discount has been applied:



We have a simple Product class with Name, Category and Price properties. A ShoppingCart sent to a rule engine contains a list of products and an optional coupon code. Rule engine will update ShoppingCart's discount property if applicable.

Two promotion types have been abstracted in the demo code. CouponPromotion simply applies a discount if the promotion code is matched. BundlePromotion is setup the way that a discount is applied if the total amount for certain category is greater than a number. For such rules we are easily build a nice authoring UI without using the out-of-box RuleSetDialog. RuleSetDialog is powerful but not that user friendly.

GetCouponPromotionRule and BuildBundlePromotionRule are two helper methods to build the Windows Workflow Rule objects based on the defined CouponPromotion and BundlePromotion. CodeDom expression trees are constructed to setup the WF condition and actions. CodeDom expression is an other topic and I am not going to expand it in detail here. Basically you need to build an expression for each statement but join them as an expression tree.

For CouponPromotion the logical condition is comparing the ShoppingCart's CouponCode property with the PromotionCode property defined in CouponPromotion by using CodeBinaryOperatorType.ValueEquality. The CodeBinaryOperatorExpression returns true if the values are the same. The action for a satisfied CouponPromotion condition is a property assignment using CodeAssignStatement to increase the ShoppingCart's Discount value. For BundlePromotion, the condition expression is a little different. The TotalCategoryAmountGreaterThan method defined in ShoppingCart class is used to determine the Workflow Rule's condition. This is achieved by CodeMethodInvokeExpression.

In our test code we create a shoppoing cart with five mock products and three promotions:
ShoppingCart1:
Product1: Name: Product1, Price: $100, Category: "Category1"
Product2: Name: Product2, Price: $200, Category: "Category1"
Product3: Name: Product3, Price: $300, Category: "Category1"
Product4: Name: Product4, Price: $400, Category: "Category1"
Product5: Name: Product5, Price: $500, Category: "Category1"
CouponCode: "123456789"
Promotion1: CouponPromotion $10 discount for coupon code "123456789"
Promotion2: BundlePromotion $100 discount if total amount for "Category1" is more than $1000
Promotion3: BundlePromotion $200 discount if total amount for "Category2" is more than $1000

We can see that both Promotion1 and Promotion2's conditions are matched, but Promotion3's condition is not. So the result shows that total $110 discount has been applied.

How about the performance? Let's do a simple test by increasing the ShoppoingCart's product number to 200, faking 200 CouponPromotion rules and 200 BundlePromotion rules:
    static void Main(string[] args)
{
ShoppingCart mockShoppingCart = new ShoppingCart();
Enumerable.Range(1, 200).ToList().ForEach(i => mockShoppingCart.SelectedProducts.Add(
new Product() { Name = "Product" + i.ToString(), Price = 100 * (i%10),
Category = "Category" + (i%10).ToString() }));
mockShoppingCart.CouponCode = "100";
Console.Write("\nShopping Cart before rules execution -- ");
Console.WriteLine("Total : ${0}\tDiscount : ${1}", mockShoppingCart.TotalAmount, mockShoppingCart.Discount);

RuleSet promotionRules = new RuleSet("Promotion Rules");
Enumerable.Range(1, 200).ToList().ForEach(i =>
{
string sequence = i.ToString(), category = "Category" + (i % 10).ToString();
CouponPromotion couponPromo = new CouponPromotion()
{ Name = "Coupon" + sequence, PromotionCode = sequence, Discount = 10 };
BundlePromotion bundlePromo = new BundlePromotion()
{ Name = "Bundle" + sequence, ProductCategory = category, TotalAmountThreshold = 1000, Discount = 100 };
promotionRules.Rules.Add(WFRuleBuilder.GetCouponPromotionRule(couponPromo));
promotionRules.Rules.Add(WFRuleBuilder.BuildBundlePromotionRule(bundlePromo));
});
Console.WriteLine("\nRules number: {0}", promotionRules.Rules.Count);

Stopwatch watch = new Stopwatch();
RuleValidation validation = new RuleValidation(typeof(ShoppingCart), null);
if (promotionRules.Validate(validation))
{
RuleEngine engine = new RuleEngine(promotionRules, validation);
watch.Start();
engine.Execute(mockShoppingCart);
watch.Stop();
}
else // Validation failed, print errors
{
StringBuilder errors = new StringBuilder();
foreach (ValidationError validationError in validation.Errors)
{
errors.Append(validationError.ErrorText);
}
Console.WriteLine("Validation errors:{0}", errors);
}

Console.WriteLine("Execution time in milliseconds: {0}", watch.ElapsedMilliseconds);

Console.Write("\nShopping Cart after rules execution -- ");
Console.WriteLine("Total : ${0}\tDiscount : ${1}", mockShoppingCart.TotalAmount, mockShoppingCart.Discount);
Console.Read();
}
The test result in an i5-CPU 8G memory Windows 2008 machine:



It shows the execution time for 400 rules is around 20-millisecond. In reality the rules would be more complex than our simple test case. But in general we can see that the speed of Windows Workflow Rules Engine is okay.

Wednesday, January 05, 2011

Good to be Lazy<T>?

Just read Dino Esposito's article Don’t Worry, Be Lazy about the new new Lazy<T> class in .NET Framework 4.0. For short it's a wrapper class to simplify your work for the "lazy loading" approach.

Sometimes we don't want to populate all related data for a complex object at one shot. Ideally, for the performance consideration, we should only retrieve data that are needed. A class is usually abstracted from business model, for instance a Customer object would have properties of ID, Name, Address, Purchased Orders, etc. Populating all these fields could be costly and may require a few expensive calls to backend system. It's possible that only a small portion of an object data is used when requesting a Customer object. Why those purchased orders data are filled when only Customer's name is in interest? Lazy<T> is designed to help for such scenarios: delay loading data until they are actually requested.

Following code example demos such usage. A User class has an Address property, and the Address data are only loaded once when Address property is accessed:

class Program
{
class Address
{
public string Location { get; set; }
public string GeoCode { get; set; }
public Address(string loc, string geo)
{
this.Location = loc;
this.GeoCode = geo;
}
}

class User // User has Address property which is lazily loaded
{
public string ID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }

public User(string id, string fName, string lName)
{
this.ID = id;
this.FirstName = fName;
this.LastName = lName;
_repoAddress = new Lazy<Address>(GetUserAddress);
}

private Lazy<Address> _repoAddress = null; // Lazy container loaded from repository
private Address _setAddress = null; // Container to setter
public Address Address
{
get
{
if (_setAddress == null) // Only get repository data if not being set
return _repoAddress.Value;
else
return _setAddress;
}
set
{
_setAddress = value;
}
}
private Address GetUserAddress() // Method to load the address data from repository
{
Console.WriteLine("...... Loading address for user {0} ......", this.ID);
return UserService.GetUserAddress(this.ID);
}
}

static List<Lazy<User>> _repoUserList = new List<Lazy<User>>(); // Fake User repository
static List<Lazy<Address>> _repoAddressList = new List<Lazy<Address>>(); // Fake Address repository
static void MockData() // Mock some sample data
{
for (int i = 1; i <= 10; i++)
{
string fakeID = i.ToString();
_repoUserList.Add(new Lazy<User>(() =>
{
Console.WriteLine("... Mocking User {0} ...", fakeID);
return new User(fakeID, "FName" + fakeID, "LName" + fakeID);
}));
_repoAddressList.Add(new Lazy<Address>(() =>
{
Console.WriteLine("... Mocking Address {0}...", fakeID);
return new Address("Location" + fakeID, "GeoCode" + fakeID);
}));
}
}

class UserService // Fake Service
{
public static Address GetUserAddress(string ID)
{
var addr = _repoAddressList.FirstOrDefault(a => a.Value.GeoCode == "GeoCode" + ID);
return (addr == null) ? null : new Address(addr.Value.Location, addr.Value.GeoCode);
}
public static User GetUserByID(string ID)
{
var user = _repoUserList.FirstOrDefault(u => u.Value.ID == ID);
return (user == null) ? null : new User(user.Value.ID, user.Value.FirstName, user.Value.LastName);
}
}

static void Main()
{
MockData();
User user = UserService.GetUserByID("5");
Console.WriteLine("The user's name: {0} {1}", user.FirstName, user.LastName);
Console.WriteLine("The user's address: {0} {1}", user.Address.Location, user.Address.GeoCode);
user.Address = new Address(user.Address.Location + "(new)", user.Address.GeoCode + "(new)");
Console.WriteLine("The user's updated address: {0} {1}", user.Address.Location, user.Address.GeoCode);
Console.Read();
}
}
Result:
... Mocking User 1 ...
... Mocking User 2 ...
... Mocking User 3 ...
... Mocking User 4 ...
... Mocking User 5 ...
The user's name: FName5 LName5
...... Loading address for user 5 ......
... Mocking Address 1 ...
... Mocking Address 2 ...
... Mocking Address 3 ...
... Mocking Address 4 ...
... Mocking Address 5 ...
The user's address: Location5 GeoCode5
The user's updated address: Location5(new) GeoCode5(new)
Above demo code searches a DAO User object from the mock data list, whose Address property is not loaded until its value gets accessed. Notice the private lazy<Address> property is only used in the getter method, and would be overridden by setter value so the custom value can be saved and passed back to service layer for further process.

A LINQ expression can be used to initiate the Lazy<T> as shown in the mock data population. We can see only the first 5 mocked Users were created because the fifth User was returned and the rest 5 mocked Users were not being accessed at all.

The last note is that all Lazy collections in above demo code are not thread-safe. To make Lazy<T> thread-safe, you need to pass a LazyExecutionMode parameter to its constructor, for detail refer to MSDN.

Monday, November 29, 2010

Powershell Scripts For WCF Site Deployment

A WCF service needs to be deployed to IIS in a web farm environment with 8 physical servers. To simplify the deployment work, I created Powershell scripts to do following job:
1. Check if powershell is installed or not. Prompt error and exit if powershell is not installed.
2. Check if IIS 7 powershell snap-in is installed or not. Install Powershell snap-in silently
if it's not installed.
3. Create application pool with desired identity
4. Create IIS Site using appPool created in step 3
5. Copy supporting files to related folders
6. Set permission for related folders
There are three scripts in total. The one to run is called Deploy.cmd:
@REM file: Deploy.cmd
@echo off
set PowershellExe="C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"
set IISSnapin="C:\Program Files\IIS\PowerShellSnapin\IIsConsole.psc1"

echo.
if not exist %PowershellExe% (
echo "Powershell 2.0 is not installed. Please install it and try again."
endlocal
)

%PowershellExe% -command "& .\InstallPowershellSnapin.ps1"

if not exist %IISSnapin% (
echo "IIS 7.0 Powershell Snap-in is not installed. Please install it and try again."
endlocal
)

%PowershellExe% -psconsolefile %IISSnapin% -command "& .\InstallWcfService.ps1"
The powershell script to install the IIS 7 Snap-in:
# file: InstallPowershellSnapin.ps1

Set-ExecutionPolicy Unrestricted

$LogFile = "DeploymentLog.log"
$SnapinInstallLog = "IIS7PowerShellSnappin.log"
$SnapinFile = "C:\Program Files\IIS\PowerShellSnapin\IIsConsole.psc1"
$SnapinInstaller = "iis7psprov_x64.msi"

Start-Transcript $LogFile -Append -Force

write-host "Checking IIS 7 Powershell Snapin..."

if (! (Test-Path $SnapinFile))
{
write-host "Install IIS 7 Powershell Snap-in..."
if ( !(Test-Path $SnapinInstaller))
{
write-error "`nCan not find IIS 7 Powershell Snap-in."
exit
}

trap {
write-error "`nInstallation of IIS 7 Powershell Snap-in failed."
}
msiexec.exe /i iis7psprov_x64.msi -qn /l* $SnapinInstallLog
start-sleep -second 10
}

Stop-Transcript
The last Powershell script to setup IIS:
# file: InstallWcfService.ps1

$MachineName = "Machine123"
$LogFile = "DeploymentLog.log"
$SnappinFile = "C:\Program Files\IIS\PowerShellSnapin\IIsConsole.psc1"

$DeploymentSourceFolder = "DeployFiles"
$SitePath = "C:\inetpub\WCFService"

$SiteName = "WCF Service"
$PortNumber = 80
$AppPoolName = "WCF Service Pool"
$AppPoolAccount = "Domain1\User1"
$AppPoolPassword = "SecretPassword"
$AppPoolType = 3

Start-Transcript $LogFile -Append -Force

write-host "Checking Site Path..."
if ( !(Test-Path $SitePath))
{
write-host "Creating Site Path" $SitePath
trap {
write-error "Error in creating site path:" $_.Exception.Message
}
mkdir $SitePath > $null

$SitePathPermission = Get-Acl $SitePath
$AccessRight = New-Object system.security.accesscontrol.
filesystemaccessrule($AppPoolAccount,"FullControl","Allow")
$SitePathPermission.SetAccessRule($AccessRight)
Set-Acl $SitePath $SitePathPermission
}

write-host "Checking application pool..."
$pcAppPool = Get-Item IIS:\AppPools\$AppPoolName
if ( !$pcAppPool)
{
Write-Host "Creating application pool..."
trap {
write-error "Error in creating application pool:" $_.Exception.Message
}
new-webapppool -name $AppPoolName > $null

Write-Host "Configuring application pool..."
$pcAppPool = Get-Item IIS:\AppPools\$AppPoolName
$pcAppPool.processModel.userName = $AppPoolAccount
$pcAppPool.processModel.password = $AppPoolPassword
$pcAppPool.processModel.identityType = $AppPoolType
$pcAppPool.managedRuntimeVersion = "v4.0"
$pcAppPool | Set-Item
}

write-host "Checking IIS Site..."
if ( !(get-website | where {$_.Name -eq $SiteName}))
{
Write-Host "Creating Site..."
trap {
write-error "Error in creating IIS site:" $_.Exception.Message
}
New-Website -name $SiteName -ApplicationPool $AppPoolName -PhysicalPath $SitePath
-port $PortNumber -HostHeader $MachineName > $null
}

Write-Host "Copying deployment files.."
copy-item $DeploymentSourceFolder\* $SitePath -recurse -force

Write-Host "Deployment completed."
Stop-Transcript

Friday, October 29, 2010

WCF 4.0 CommunicatinException

Found an exception sometimes in WCF 4.0 communication with wsHttpBinding binding in a test environment:

CommunictionException: "An error occurred while receiving the HTTP response to ....svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details."

InnerExcpetion: "The underlying connection was closed: An unexpected error occurred on a receive."

InnerInnerException: "An existing connection was forcibly closed by the remote host"

NativeErrorCode: 10054

Turn on WCF logging in the server and found following error message using Microsoft Service Trace Viewer:

"There was an error while trying to serialize parameter http://tempuri.org/:Get...Result. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. '. Please see InnerException for more details."

Update server's web.config as suggested:
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior>
<!-- ... -->
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</serviceBehaviors>
</behaviors>
<!-- ... -->
</system.serviceModel>
Now get another exception in the client:
NetDispatcherFaultException: The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:Get...Result. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. '. Please see InnerException for more details.
Inner Exception: "Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. "

Update client's app.config:
<system.serviceModel>
<client>
<endpoint behaviorConfiguration="ServiceBehavior" _A_B_C_Setting... />
</client>
<behaviors>
<endpointBehaviors>
<behavior name="ServiceBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647"/>
</behavior>
</endpointBehaviors>
</behaviors>
</system.serviceModel>
The exception is gone after configuration update.