Thursday, December 20, 2012

WinJS Data Storage and Protection in Windows 8 Store Apps

In my previous post I discussed about how to protect JavaScript source in Windows Store apps. Today I will discuss how to store and secure WinJS app data inside the client machine.

Data Storage and Access


The first question is how and where to save the data. This MSDN documentation is a good reference. There're a lot of options out there but you may end up using one of following methods to store configurations or settings in a Windows 8 machine:

1. WinJS.Application.local.writeText and WinJS.Application.local.readText. Both methods return Promise and they are used to write/read the specified text to/from the specified file. The saved content will be plain text and the content is physically stored in C:\Users\{UserName}\AppData\Local\Packages\{AppID}\LocalState folder. Code example:
    var appLocal = WinJS.Application.local;
    var mySettings = [ {Locale : "EN-CA"}, { Theme : "Classic" }, { LastAccessDate: "2012-12-12" }]; 
    appLocal.writeText("app.config", JSON.stringify(mySettings)).then(function () {
        appLocal.readText("app.config").then(function (data) {
            try {
                var retrievedSettings = JSON.parse(data);
                // do stuff
            }
            catch (ex) {
                // Exception handling 
            }
        });
    });
2. Windows.Storage.ApplicationData.current.localFolder. Same as using "ms-appdata:///Local/" protocol. The data storage location of this localFolder is actually the same as above but the API is different:
    var localFolder = Windows.Storage.ApplicationData.current.localFolder;
    var saveOption = Windows.Storage.CreationCollisionOption;
    var mySettings = {Locale : "EN-CA", Theme : "Classic", LastAccessDate: "2012-12-12"}; 
    localFolder.createFileAsync("myapp.config", saveOption.replaceExisting).then(
        function (file) {
            return Windows.Storage.FileIO.writeTextAsync(file, JSON.stringify(mySettings));
        }).done(function () { 
            localFolder.getFileAsync("myapp.config").then(function (file) {
                return Windows.Storage.FileIO.readTextAsync(file);
            }).done(function (data) {
                var retrievedSettings = JSON.parse(data);
                // do stuff
            });
        });
    }
3. Windows.Storage.ApplicationData.current.localSettings. This localSettings, a key/value pair container, is a bit easier to use because it's not implemented by Prmoise like above two methods. You can set and get values directly in a traditional JavaScript way. The data is stored in C:\Users\{UserName}\AppData\Local\Packages\{AppID}\Settings folder. As name suggested localSettings is ideal for saving settings or small amount of data but not good for big size of content:
    var localSettings = Windows.Storage.ApplicationData.current.localSettings;
    var mySettings = { Locale: "EN-CA", Theme: "Classic", LastAccessDate: "2012-12-12" };
    localSettings.values["myAppSetting"] = JSON.stringify(mySettings); // Assign value
    var data = localSettings.values["myAppSetting"]; // Retrieve value
    if (data) {
        try {
            var retrievedSettings = JSON.parse(data);
            // do stuff
        }
        catch (ex) {
            localSettings.values.remove("myAppSetting");
        }
    }
4. Windows.Storage.ApplicationData.current.roamingSettings/roamingFolder.. The roamingSettings API is pretty much the same as localSettings, and roamingFolder just like localFolder. The difference are:
  • Roaming store would automatically sync the local data to the user's profile in the cloud when the user login as a Microsoft account.
  • Roaming store's physical location is C:\Users\{UserName}\AppData\Local\Packages\{AppID}\RoamingState.
  • Roaming store's URI is "ms-appdata:///Roaming/"
  • Roaming store can only save maximum of 100K data.

Data Encryption and Decryption


In all above methods the saved data are not secure and they can be easily retrieved in other place. So how to protect sensitive data stored in the client machine? Encryption is straightforward answer. Encrypt your data if you don't want to expose them directly to the end user.

The asymmetric encryption is hard and not applicable for distributed single alone application due to the complexity of PKI system. Symmetric encryption such as AES is used in most cases. But client side symmetric encryption is not safe in general because the same encryption key is used in the client machine. The encrypted data can be decrypted in any other machine running the same app, and it's not hard to get the decrypted value. It would be more secure if the encryption key is associated with login user's identity such as SID in Windows machine. In that case the encrypted data can't be decrypted easily in other machine or different user in the same machine.

The problem is that the user SID is not achievable in JavaScript or Runtime component. Fortunately Windows Runtime environment provides Windows.Security.Cryptography library which includes mechanism to encrypt/decrypt data using a key associated with the current user, and the library is accessible from WinJS:
    var localSettings = Windows.Storage.ApplicationData.current.localSettings;
    var cryptography = Windows.Security.Cryptography;
    var cryptoBuffer = cryptography.CryptographicBuffer;
    var cryptoProvider = new cryptography.DataProtection.DataProtectionProvider("LOCAL=user");
            
    var mySettings = { Locale: "EN-CA", Theme: "Classic", LastAccessDate: "2012-12-12" };
    var bufferData = cryptoBuffer.convertStringToBinary(JSON.stringify(mySettings), cryptography.BinaryStringEncoding.utf8);

    cryptoProvider.protectAsync(bufferData).then(function (encryptedData) {
        var dataToBeSaved = cryptoBuffer.encodeToHexString(encryptedData);
        localSettings.values["appSecuredSetting"] = dataToBeSaved;
    }).then(function () {
        var encryptedHexData = localSettings.values["appSecuredSetting"];
        if (encryptedHexData) {
            try {
                var bufferData = cryptoBuffer.decodeFromHexString(encryptedHexData);
                cryptoProvider.unprotectAsync(bufferData).then(
                    function (decryptedBuffer) {
                        var decryptedData =
                            cryptoBuffer.convertBinaryToString(cryptography.BinaryStringEncoding.utf8, decryptedBuffer);
                        var retrievedSettings = JSON.parse(decryptedData);
                        // do stuff
                    },
                    function (err) {
                        // Decryption error handling
                    });
            } catch (ex) {
                // DecodeFromHexString error handling
            }
        }
    });;
The descriptor parameter of "LOCAL=user" passed to DataProtectionProvider is important. I tested other available parameters and this "LOCAL=user" parameter is the only one that would prevent other user from decrypting the data. When data is encrypted with this parameter, they can't be decrypted by any other user in the same machine or other machine. I also tried to decrypt the data in another machine by the same user name but it failed.

The real encryption key used by DataProtectionProvider is invisible to end user or developer. That's great because it makes super hard for attackers to reproduce that key. Conclusion is that by using Windows.Security.Cryptography library and passing "LOCAL=user" to DataProtectionProvider we could secure the data saved to a Windows 8 box.

Tuesday, December 18, 2012

Secure JavaScript Source for Windows 8 Store Apps

I demonstrated how to import an installed Windows JavaScript/HTML application in my previous post. JavaScript is an interpreted language and theoretically you can't prevent it from being reverse engineering. But Windows 8 store apps' architecture makes super easy for a user with administration right to see apps' source file by simply a few clicks. This is not acceptable and I agree with Justin Angel's opinion: that's a fundamental design flaw.

Originally JavaScript is designed to run inside browser. It's mainly for presentation layer and the application flow is driven by the web server. This has been evolved a bit recently due to the popularity of JavaScript, for instance the node.js framework developed for server environment. Now JavaScript becomes one of the main stream languages for developing Windows 8 store apps and the JavaScript code plays more important role there. Those apps' source is so easy to get from physical file system that it would attract people to do so. No need to jailbreak or root the Windows 8 devices, now even casual attackers can have an easy starting point simply by a few clicks. The worst thing is that most developers may not aware of that potential security hole. They simply trust Microsoft and do little or nothing to protect their application. That's why Justin was so easy to break various popular games built by C++, C#/XAML and HTML/JavaScript.

As time goes by I believe more and more measures will be introduced to address the security issues of Windows 8 store apps. For developers, what we can do is to secure our code and data. Following three methods can be used to improve the code security for Window 8 applications:

1. Move critical logic and data back to server side. This is kind of hybrid approach. The Windows 8 app runs locally but some pages or sometimes it would just act like an embedded browser.

2. Split some code into Windows Runtime Component using C#/C++/VB. The Runtime Component is compiled as a dll and can be called directly from JavaScript. For further protection we can obfuscate the code in Runtime Component.

3. Minimize/obfuscate JavaScript code. That's an easy but effective way to protect your code and your logic. Scott Hanselman has a great post going through a couple of tools to do this job. For example following batch file will go through all JavaScript in a solution folder and make a minimized version of it with .min.js extension (original js file unchanged) using AjaxMin:

dir /s /b *.js | find /v ".min.js" | find /v "\bin\" > jsminlist.txt
for /f "usebackq delims=" %%x in (`findstr /V min.js jsminlist.txt`) do (AjaxMinifier.exe "%%x" -o "%%~dpx/%%~nx.min.js" -clobber)
Note that the AjaxMin's executable AjaxMinifier.exe needs to be copied to the solution's root folder before running above script. After run the batch command, we simply exclude the original javascripts not ending with *.min.js, and include those minimized version of JavaScript ending with *.min.js, then update all html files to reference the new JavaScript.

Another topic is secure the data we stored in client machine. I will discuss that in my next post.

Update 2013-01:
1. when publishing the app if you get app validation error of "xxx.js is not properly UTF-8 encoded. Re-save the file as UTF-8 (including Byte Order Mark).", you simple the open the JavaScript file in Visual Studio and click File => Save xxxx.js As... from the menu, select "Save with Encoding" (default is "Save") from the popup window, and select "Unicode (UTF-8 without signature) - Codepage 65001" encoding format then click OK to save the file with proper format for publishing.

2. if get the "Optimized Binding References" error during app validation, something like "WinJS.Binding.optimizeBindingReferences = true is not set and data-win-bind was found in xxxx.html on line number xxx.", to resolve the problem you have to add "WinJS.Binding.optimizeBindingReferences = true;" right on top of your JavaScript if you have data binding in your page. If you get this error only after JavaScript minification, that means the Windows app certification kit doesn't recognize the obfuscated JavaScript and you need to try another tool to do the job. Using JSMin from Douglas Crockford would work for sure since it only removes the space and comments without any other code scrambling.

Sunday, December 16, 2012

An Exercise of Reverse Engineering Windows 8 JavaScript/HTML App

What's the buzz this week? Quite a lot of attention was drawn from a Nokia engineer Justin Angel, a former Microsoft developer who posted online how to hack various popular Windows 8 games in detail. Justin Angel's blog is offline now and even its Google cached page is not available any more. This once more raises people's concern about the Windows 8 application security. Actually Justin had wrote a blog post talking about reverse engineering Windows 8 app more than a year ago. I believe Justin has notified Microsoft about security concerns in the past year but Microsoft just didn't paid enough attention on it. Justin looked frustrated and he wrote: "If Microsoft doesn’t take it upon itself to fix these security attack vectors it’s not because it couldn’t, it’s because it chooses not to."

Justin then hacked some Windows 8 games and rang alarm bells once again. Hope Microsoft won't ignore the security issues this time. As a follow-up on Justin's work I am going to do some testing to reverse engineering a WinJS app. The goal is to run and debug the app using Visual Studio 2010 locally. I am doing this only for study purpose by no means any other.

The test environment I have is a Windows 8 64-bit virtual machine with Visual Studio 2012 and BingMap SDK installed. The app to be reversely engineered is called USAToday, one of my favorite apps also one of the top free apps under News category in Windows store. After installed the app you can see the physical HTML, JavaScript and CSS files under C:\Program Files\WindowsApps\USATODAY.USATODAY_1.2.0.0_neutral__wy7mw3214mat8 folder (google it if you can't see or open this folder). Following are steps I did to import USAToday files into VS2012.

Step 1. Open up VS2012 and create a JavaScript Windows Store project named "USATodayMock" using Navigation App template. The VS2012 solution looks like:



Step 2. Copy highlighted files and folders in C:\Program Files\WindowsApps\USATODAY.USATODAY_1.2.0.0_neutral__wy7mw3214mat8 as below to the VS2012 project:



After files copied to the VS2010 solution:



Step 3. Add project references: Windows Library for JavaScript 1.0, BingMap for JavaScript, and two dlls from USAToday source files:



Now the VS2012 solution can be compiled. When you run it in debug mode in VS2012 it will prompt you a warning, just click yes to override the existing app:



Then you will get an exception thrown by getResourceString method inside Microsoft.PlayerFramework.Js/js/PlayerFramework.js:



It shows missing resource string for "Microsoft.PlayerFramework.Js/resources/TimeFormatterTemplate". Apparently the original USAToday solution includes Microsoft Player Framework which requires some resource strings. To make it simple I just reverse engineer the compiled PRI resource file (resources.pri) come with Windows 8 app package, and directly add those resource strings to the solution. See next step for detail.


Step 4. Open Developer Command Prompt, go to app installation folder C:\Program Files\WindowsApps\USATODAY.USATODAY_1.2.0.0_neutral__wy7mw3214mat8, run "makepri dump" to build the resources.pri.xml file:



The exported resources.pri.xml looks something like:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<PriInfo>
  <ResourceMap name="USATODAY.USATODAY" version="1.0" primary="true">
    <Qualifiers></Qualifiers>
    <ResourceMapSubtree name="Bing.Maps.Javascript"></ResourceMapSubtree>
    <ResourceMapSubtree name="Files">
    <ResourceMapSubtree name="Microsoft.PlayerFramework.Js">
      <ResourceMapSubtree name="resources">
         <NamedResource name="TimeElapsedTitle" 
          uri="ms-resource://USATODAY.USATODAY/Microsoft.PlayerFramework.Js/resources/TimeElapsedTitle">
            <Candidate qualifiers="Language-EN-US" isDefault="true" type="String">
              <Value>{hour.integer}:{minute.integer(2)}:{second.integer(2)}</Value>
            </Candidate>
         </NamedResource>
       </ResourceMapSubtree>
     </ResourceMapSubtree>
   </ResourceMap>
</PriInfo>
Copying those resource strings is tedious. I created a simple console app that convert those player framework resource strings in resources.pri.xml to WinJS recognizable JSON format:
using System;
using System.Collections.Generic;
using System.Linq;
using System.IO;
using System.Xml.Linq;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            Dictionary<string, string> resources = new Dictionary<string, string>();
            XDocument xdoc = XDocument.Load("c:\\temp\\resources.pri.xml");
            var frameworkResources = xdoc.Descendants("ResourceMapSubtree")
                    .Where(e => (string)e.Attribute("name") == "Microsoft.PlayerFramework.Js")
                    .Elements("ResourceMapSubtree")
                    .Where(e => (string)e.Attribute("name") == "resources")
                    .Elements();
            foreach (var item in frameworkResources)
            {
                var key = item.Attribute("name").Value;
                var value = item.Descendants("Value").First().Value;
                resources.Add(key, value);
            }
            using (TextWriter tw = new StreamWriter("C:\\temp\\resources.resjson"))
            {
                tw.WriteLine("{");
                int resourceCount = resources.Count, counter = 1;
                foreach (var item in resources)
                {
                    if (counter++ < resourceCount)
                        tw.WriteLine(string.Format("\"{0}\" : \"{1}\",", item.Key, item.Value));
                    else
                        tw.WriteLine(string.Format("\"{0}\" : \"{1}\"", item.Key, item.Value));
                }
                tw.WriteLine("}");
            }
        }
    }
}
Above console app will create a resources.resjson file in C:\temp folder. Copy it to USATodayMock solution.

Step 5. Modify getResourceString method in Microsoft.PlayerFramework.Js/js/PlayerFramework.js, remove the prefix of "Microsoft.PlayerFramework.Js/resources/" for the resource keys since they are included in the project directly:
    function getResourceString(id) {
        /// <summary>Returns the resource string with the specified ID.</summary>
        /// <param name="id" type="String">The resource identifier.</param>
        /// <returns type="String">The resource string.</returns>

        if (id.indexOf("resources/") > 0)
            id = id.substring(id.lastIndexOf("/") + 1);

        var string = WinJS.Resources.getString(id);

        if (string.empty) {
            throw invalidResourceId;
        }

        return string.value;
    }
Click F5 and bang the USATodayMock app can be run and debug now!

Step 6. This step is to do a little more of test by modifying the default page. USAToday Win8 app shows weather in the right top corner:



Let's just change weather text to red and show some dummy text when you click it. Search following text inside default.html page under the root of the USATodayMock solution:
    <div class="banner">
        <div class="snapped-back"></div>
        <div class="app-logo"></div>

        <div class="weather">
            <div class="location"></div>
            <div class="icon-wrapper hidden"><img class="icon" /></div>
            <div class="temp"></div>
        </div>
    </div>
Then replace it with following HTML:
    <div class="weather">
        <div class="location" style="color:red;" onclick="showAlert();"></div>
        <div class="icon-wrapper hidden"><img class="icon" /></div>
        <div class="temp"></div>
    </div>

    <div id="divAlert" style="display:none; position:fixed; margin-top: 200px; margin-left: 150px;">
            <h1>App has been modified!</h1> 
    </div>
    <script type="text/javascript">
        function showAlert() {
            event.cancelBubble = true;
            WinJS.Utilities.query("#divAlert").setStyle("display", "block");
            WinJS.Utilities.query("#app-body").setStyle("display", "none");
        }
    </script>
Run the application again you will see the red location text:



When you click that red text (New York) the page will show the custom content instead of going to the location setting page:



So we see how easy it's to import an installed Windows 8 application into Visual Studio and run it directly. In my next post I will discuss some methods to secure the code and make it harder to be reversely engineered.

Saturday, December 08, 2012

Notes About Android Threading

  • Main thread, or UI thread, starts Android application, processes UI rendering and handles user interactions. Do NOT execute any long-running or heavy tasks inside main thread otherwise the UI would appear lagging and not responsive.
  • A Looper holds a message queue and implements infinite loop. It takes tasks one by one from the queue and executes them in sequence. When message queue is empty, the Looper is blocking and waiting for processing next queued task. A thread can associate with a Looper (only one to one relationship), then it becomes a Looper thread being able to process messages and tasks continuously, vs. regular thread that will die when completes its job in the run() method.
  • Handler is a bridge between Looper message queue and thread(s). Current thread or other threads can push runnable jobs to a Looper message queue by method handler.post() and its scheduled variance (postDelayed, postAtTime, etc.), or send a message to the queue by handler.sendMessage() method and its scheduled variance (sendMessageDelayed, sendMessageAtTime, etc.). Handler can also process the message by the Handler's handleMessage(Message) callback.
  • You can setup a thread with a Looper manually. Androdi also provides a handy class called HandlerThread for starting a new thread that already has a Looper attached. Not sure why it's not called LooperThread. It's a bit confusing as there's no Handler in a HandlerThread object. You always need to create Handler for a HandlerThread:
        HandlerThread  handlerThread = new HandlerThread("Thread name");
        handlerThread.start();
        Handler handler = new Handler(handlerThread.getLooper());
    
  • Main thread is a Looper thread. All UI interactions are pushed to main thread's Looper message queue and then are processed one by one. Configuration change request, such as orientation rotation, is just a special type of message sent to main thread's message queue.
  • Only main thread can update UI elements safely. Worker thread or background thread can use following means to work on UI elements:
    • Implement main thread Handler, then you can update UI inside handler.handleMessage() callback, and post UI-related task to handler.Post() method or their variance.
    • Use View.Post(Runnable). Android View objects have tied to a default Handler inside UI thread so you can post UI-related runnable jobs to it directly.
    • Use Activity.runOnUiThread(Runnable) method. Android Activity class has this runOnUiThread helper method to update the UI.
    • Use AsyncTask. The AsyncTask implementation takes the advantage of Thread pooling concept and provides a simple, understandable interface. Simply run background task inside the doInbackground() method, and run UI-related work inside the onPreExecute(), onProgressUpdate() and onPostExcute() methods. But be aware of the performance penalty of using it, see next.
  • AsyncTask thread has low priority of Process.THREAD_PRIORITY_BACKGROUND. With such low priority the total CPU consumption of all AysncTasks together is less than 10 percent of overall CPU power. This may cause some performance issue. You can change this behavior by changing its priority:
        new AsyncTaskClass().executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR); 
        //...
        class AsyncTaskClass extends AsyncTask<...> {
            //...
            protected Void doInBackground() {
                Thread.currentThread().setPriority(Process.THREAD_PRIORITY_DEFAULT);
                //...
            }
        }
    
  • On the other hand the HandlerThread uses Process.THREAD_PRIORITY_DEFAULT priority by default, but you can specify HandlerThread's priority from its constructor, e.g. setting a low priority background:
    HandlerThread bgThread = new HandlerThread("Background thread", Process.THREAD_PRIORITY_BACKGROUND);
    
  • The ExecutorService is more powerful and can manage a pool of threads. Go for it if you want to have full control of your threads.
  • Usually worker thread is created from Activity. But it can also run in Service context (thread object created inside Service) without UI connection. Note that Service still runs in main UI thread by default if it starts from the Activity, but Service is not bound to Activity's life-cycle. IntentService simply extends Service and implements an internal HandlerThread object, so you can guarantee the job passed to IntentService's onHandleIntent callback runs in a separate worker thread.
  • There're a few ways to deal with recurring or repeating tasks. One common approach is keep posting to Activity's handler by using Handler.postDelayed() inside the runnable task:
        int recurInSeconds = 5;
        Handler handler = new Handler();
        handler.postDelayed(runnable, 0); // start the task
        Runnable runnable = new Runnable() {
           @Override
           public void run() {
              doTask(); 
              handler.postDelayed(this, recurInSeconds * 1000); // repeat the task
           }
        };
    Without Activity context we can use following methods to achieve the goal:
    • AlarmManager.setRepeating()/setInexactRepeating() to repeatedly start a Broadcast/Service in which the job is handled. For longer sleep intervals this is the preferred mechanism. It's handled by system and is only triggered at the time arrives thus consuming less power.
    • ScheduledThreadPoolExecutor.scheduleWithFixedDelay()/scheduleWithFixedDelay() to repeatedly run a task.
      ScheduledThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(1);
      executor.scheduleAtFixedRate(runnable, 0, recurInSeconds, TimeUnit.SECONDS);
    • Use Timer.scheduleAtFixedRate() to do the recurring TimerTask. This is not the recommended way described from Android development guild.
  • Anders Göransson's Efficient Android Threading slides for DroidCon are very informative. It's the best reference on the topic of Android threading I have found so far.