Dev Bytes: Mobile

Showing posts with label Mobile. Show all posts

Tuesday, February 18, 2014

Windows 8 Bing Maps App Using JavaScript

A couple years ago I wrote a tutorial on how to work on Mircrosoft Virtual Earth. As an update I will create a simple Windows 8 food service locator Bing Map app in this post, which includes below features:

Search by location or by name
Show different pushpins based on service types
Dynamically update data when moving around the map
Show service detail when clicking on the pushpin

For testing purpose a mock data service is offering fake food service locations. The map view looks like following:

The custom pushpins shown on the map are copied courtesy of the free map icons online.

Environment Setup

First you need a Windows 8 or Windows 8.1 environment, VM or physical machine, and install Visual Studio 2012 or 2013 in the machine. Inside Visual Studio toolbar menu, select TOOLS=>Extension and Updates, then search and install "Bing Maps SDK for JavaScript".

You also need to setup a Bing Maps account from Microsoft. The detail of Bing Maps Account setup is described in MSDN.

Create Visual Studio Project

Ope Visual Studio, create a Blank App from JavaScript|Windows Store template, add a reference for "Bing Maps for JavaScript", and update default.html, default.js and default.css as following:

default.html:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <title>BingMap Test</title>

    <!-- WinJS references -->
    <link href="//Microsoft.WinJS.1.0/css/ui-dark.css" rel="stylesheet" />
    <script src="//Microsoft.WinJS.1.0/js/base.js"></script>
    <script src="//Microsoft.WinJS.1.0/js/ui.js"></script>
 
    <!-- Bing Maps reference -->
    <script type="text/javascript" src="ms-appx:///Bing.Maps.JavaScript//js/veapicore.js"></script>

    <!-- BingMapExample references -->
    <link href="/css/default.css" rel="stylesheet" />
    <script src="/js/default.js"></script> 
</head>
<body>
    <div class="mapPage">
        <div class="header">
            <!-- Segoe UI current location symbol -->
            <button id="btnCurrent">&#xE1D2;</button>
            <input id="txtSearch" type="text" />
            <progress id="progressDiv"></progress>
            <!-- Segoe UI search symbol -->
            <button id="btnSearch">&#xE000;</button>
        </div>
        <div id="mapDiv"></div>
    </div>
</body>
</html>

The Segoe UI symbols are listed in MSDN.

default.js:

(function () {
    "use strict";

    var map;            // Bing Map ojbect
    document.addEventListener("DOMContentLoaded", initialize, false);

    function initialize() {
        Microsoft.Maps.loadModule('Microsoft.Maps.Map', { callback: initMap });
        Microsoft.Maps.Events.addHandler(map, 'viewchangeend', mapViewChanged);

        btnCurrent.addEventListener("click", currentLocationClick);
        btnSearch.addEventListener("click", searchLocationClick);
    }

    function initMap() { // Initialize Bing map
        try {
            var mapDiv = document.getElementById("mapDiv");
            var mapOptions = {
                credentials: "xxxxxxx",  //Read from your Bing Maps Account
                center: new Microsoft.Maps.Location(43.813, -79.344), //TORONTO
                zoom: 13,
            };
            map = new Microsoft.Maps.Map(mapDiv, mapOptions);
        }
        catch (err)  {
            showError("Init Map error", err);
        }
    }

    // Repopulate map when it's moved
    function mapViewChanged() {
        // TODO
    }

    // Search by current location
    function currentLocationClick() { 
        // TODO
    }

    // Search servcie by name
    function searchLocationClick() { 
        // TODO
    }
})();

default.css:

.mapPage {
    width: 100%;
    height: 100%;
    display: -ms-grid;
    -ms-grid-columns: 120px 1fr 120px;
    -ms-grid-rows: 20px 40px 1fr 20px;
    position: absolute;
}

.mapPage .header {
 -ms-grid-column: 2;
 -ms-grid-row: 2;
    display: -ms-grid;
    -ms-grid-columns: auto 5px 1fr 5px auto;
}

.mapPage .header #btnCurrent {
    -ms-grid-column: 1;
    -ms-grid-row-align: center;
    font-family: "Segoe UI Symbol";
}
    
.mapPage .header #txtSearch {
    -ms-grid-column: 3;
    -ms-grid-row-align: center;
    width: 100%;
    border: none;
}

.mapPage .header #progressDiv {
    -ms-grid-column: 3;
    -ms-grid-column-align: end;
    -ms-grid-row-align: center;
}

.mapPage .header #btnSearch {
    -ms-grid-column: 5;
    -ms-grid-row-align: center;
    font-family: "Segoe UI Symbol";
}

.mapPage #mapDiv {
    -ms-grid-column: 2;
    -ms-grid-row: 3;
    position: relative;
    height: 100%;
    width: 100%;
}

The CSS3 grid-layout and fractional units are used to layout the page so it can auto-stretch to different screen size. One catch is that the map would occupy the whole screen unless you set its container to relative position.

Now the structure of the page is defined and you will see the map showing properly when debugging the app. Next we will work on the data model and mock data service.

Data Model and Mock Data Service

Our service location entity includes attributes of Geo location, service type, name, address, description, and service type. The Geo location pins the map location as a pushpin and the custom pushpin images will differ based on the service type. The name, address and description will be displayed in an infobox popup.

Mock service will return a list of service locations as well as the center location for the map. To generate locations in valid for a map, mock service needs to know the map boundary. Our prototype has two search types, by Geo location or by name query. For the Geo location search scenario the Geo location is also the map center in the mock service return:

(function () {
    "use strict";

    WinJS.Namespace.define("mockLocationService",
    {
        search: search  // Only expose one search method
    });

    /*
     * Proivde mock data for testing
     * Note: the caculation is based on North America and may not work in other location
     */
    function search(parameters) {
        var response = {};
        response.serviceLocations = [];

        if (parameters.latMin && parameters.latMax && parameters.longMin && parameters.longMax) {
            var latMin = parseFloat(parameters.latMin);
            var latMax = parseFloat(parameters.latMax);
            var longMin = parseFloat(parameters.longMin);
            var longMax = parseFloat(parameters.longMax);

            // recenter the map if search by location
            if (parameters.lat && parameters.long) {
                latMin = parameters.lat - (latMax - latMin) / 2;
                latMax = parameters.lat + (latMax - latMin) / 2;
                longMin = parameters.long - (longMax - longMin) / 2;
                longMax = parameters.long + (longMax - longMin) / 2;
            }

            // generate 10-20 mock service loacations 
            var count = 10 + Math.floor(Math.random() * 10);
            var mockData = generateMockData(latMin, latMax, longMin, longMax, count);
            for (var i = 0; i < mockData.length; i++) {
                var serviceLocation = mockData[i];
                response.serviceLocations.push(serviceLocation)
            }

            // also return a center location for mapping
            response.center = { lat: ((latMax + latMin) / 2), long: ((longMin + longMax) / 2) };
        }
        return response;
    }

    function generateMockData(latMin, latMax, longMin, longMax, count)
    {
        var services = [];
        for (var i = 1; i <= count; i++) {
            var lat = Math.random() * (latMax - latMin) + latMin;
            var long = Math.random() * (longMax - longMin) + longMin;
            var name = "Service location " + i;
            var address = i + " Queen street ...";
            var rand = Math.floor(Math.random() * 5) + 1;
            var type, address, description;
            switch (rand) {
                case 1:
                    type = MapService.ServcieType.Restaurant;
                    description = "Come and taste our famous dishes ...";
                    break;
                case 2:
                    type = MapService.ServcieType.Vegetarian;
                    description = "Looking for healthy vete food? ...";
                    break;
                case 3:
                    type = MapService.ServcieType.Buffet;
                    description = "No idea what to eat? ...";
                    break;
                case 4:
                    type = MapService.ServcieType.Bar;
                    description = "Have a few beers and other drinks here ...";
                    break;
                case 5:
                    type = MapService.ServcieType.Foodtruck;
                    description = "Call us and we will bring your our nice food ...";
                    break;
            }
            var servicePoint = { name: name, lat: lat, long: long, type: type, address: address, description: description };
            services.push(servicePoint);
        }
        return services;
    }

})();

Map Service Module

Map service module handles a few things: get current location, invoke location services, parse response data and create object to serve map pushpins and infobox popup:

(function () {
    "use strict";

    // Service types for mock service
    var ServcieType = { Restaurant: 1, Vegetarian: 2, Buffet: 3, Bar: 4, Foodtruck: 5 };

    function invokeLocationService(searchParameter, callBack, errorHandler, mapBounds) {
        if (!mapBounds) { // Get data from web service
            var searchUrl = "http://mylocationserver.com/search";
            WinJS.xhr({ url: searchUrl, data: searchParameter, responseType: "json" }).done(
                function (result) { // Service call completed
                    if (callBack) {
                        var data = JSON.parse(result.responseText);
                        callBack(data);
                    }
                },
                function (err) { // Service call error
                    if (errorHandler) {
                        errorHandler(err);
                    }
                });
        } else { // Get data from mock service
            if (callBack) {
                try {
                    // mapBounds values in north America, maynot work in other location
                    var latMin = mapBounds.getSouth();
                    var latMax = mapBounds.getNorth();
                    var longMin = mapBounds.getWest();
                    var longMax = mapBounds.getEast();
                    var parameters = { latMin: latMin, latMax: latMax, longMin: longMin, longMax: longMax };

                    if (searchParameter.lat && searchParameter.long) { // Search by location
                        parameters.lat = searchParameter.lat;
                        parameters.long = searchParameter.long;
                    } else { // Search by name
                        parameters.query = searchParameter.query;
                    }
                    var serviceLocations = mockLocationService.search(parameters);
                    callBack(serviceLocations);
                } catch (ex) {
                    if (errorHandler) {
                        errorHandler(ex);
                    }
                }   
            }
        }
    }

    // parse result JSON and return an array of service locaiton objects
    function parseServiceLocations(json) { 
        var serviceLocations = [];      
        for (var i = 0; i < json.serviceLocations.length; i++) {
            var service = json.serviceLocations[i];
            var newLocation = {};
            newLocation.location = new Microsoft.Maps.Location(service.lat, service.long);
            newLocation.name = service.name;
            newLocation.summary = service.description + "<br /><br /><b>Address: </b>" + service.address;
            newLocation.type = service.type;
            populateImageUrls(newLocation);
            serviceLocations.push(newLocation);
        }
        return serviceLocations;
    }

    // parse center location from result JSON
    function parseSearchCenterLocation(data) { 
        return new Microsoft.Maps.Location(data.center.lat, data.center.long);
    }

    // Populate pushpin and location image urls
    function populateImageUrls(serviceLocation) { 
        switch (serviceLocation.type) {
            case ServcieType.Restaurant:
                serviceLocation.pushpinUrl = "images/locationIcons/restaurant.png";
                break;
            case ServcieType.Vegetarian:
                serviceLocation.pushpinUrl = "images/locationIcons/vegetarian.png";
                break;
            case ServcieType.Buffet:
                serviceLocation.pushpinUrl = "images/locationIcons/buffet.png";
                break;
            case ServcieType.Bar:
                serviceLocation.pushpinUrl = "images/locationIcons/bar.png";
                break;
            case ServcieType.Foodtruck:
                serviceLocation.pushpinUrl = "images/locationIcons/foodtruck.png";
                break;
        }
    }

    // create a pushpin for a service location object
    function getServicePointPushpin(serviceLocation) {
        var pushpin = new Microsoft.Maps.Pushpin(serviceLocation.location,
                {
                    icon: serviceLocation.pushpinUrl,
                    width: 32,
                    height: 37,
                    anchor: new Microsoft.Maps.Point(16, 18)
                });
        pushpin.name = serviceLocation.name;
        pushpin.summary = serviceLocation.summary;
        return pushpin;
    }

    // create a pushpin for current location
    function getCurrentLocationPushpin (location) {
        var pushpin = new Microsoft.Maps.Pushpin(location,
                {
                    icon: "/images/locationIcons/current.png",
                    width: 32, height: 37,
                    anchor: new Microsoft.Maps.Point(16, 18)
                });
        pushpin.name = "Your current location";
        pushpin.summary = "latitude: " + location.latitude + " longitude:" + location.longitude;
        return pushpin;
    }

    // Location service helper
    var CurrentLocation = WinJS.Class.define(
        // Constructor
        function (disableCache) {
            this.doCache = !disableCache;
        },
        // Instance members
        {
            getCurrentLocationAsync: function () {
                var self = this;
                return new WinJS.Promise(function (c, e, p) {
                    if (CurrentLocation.geoLocator == null) {
                        CurrentLocation.geoLocator = new Windows.Devices.Geolocation.Geolocator();
                    }
                    if (CurrentLocation.geoLocator) {
                        if (self.doCache && CurrentLocation.location) {
                            c(CurrentLocation.location); // complete the call back with cached data
                        } else {
                            return CurrentLocation.geoLocator.getGeopositionAsync().then(
                                function (pos) { // get position data handler
                                    var loc = new Microsoft.Maps.Location(pos.coordinate.latitude, pos.coordinate.longitude);
                                    CurrentLocation.location = loc; // Cache local position
                                    c(loc);
                                }.bind(this),
                                function (ex) { // location service error
                                    e(ex);
                                }.bind(this));
                        }
                    }
                });
            }
        },
        // Static members
        { 
            geoLocator: null,
            location: null
        }
    );

    // Expose MapService
    WinJS.Namespace.define("MapService",
    {
        ServcieType: ServcieType,
        CurrentLocation: CurrentLocation,
        invokeLocationService: invokeLocationService,
        parseServiceLocations: parseServiceLocations,
        parseSearchCenterLocation: parseSearchCenterLocation,
        getServicePointPushpin: getServicePointPushpin,
        getCurrentLocationPushpin: getCurrentLocationPushpin,
    });

})();

A couple of notes about the MapService module:

The mapbounds parameter is unnecessary for real service calls, it's only used by mock service so that it has a coordinate to generate valid service points inside the map.
Locaiton service uses native Geolocator to retrieve current location, and caches the data for future usage to improve the performace. The cache feature can turn off when instantiating the CurrentLocation object if that's desired.
The Bing Maps' default pushpin is a 25x35 size image pining at the middle bottom of the image. We need to use pushpin's anchor property to adjust the pin point if using custom images with different size or different pining location.
Infobox popup supports some HTML tags inside the message, e.g. <br /> line break and <b> bold fonts for "Address" are used in our example

UI Interaction

Now we are ready to refine the UI and implement the user interactions. First add mock service and map service javascript references in the default.html. Following is the screen-shot of the final solution inside Visual Studio:

In general there are three user actions: click current location button to search services around; move map to search nearby locations; type some keyword and click search button to do search name. The first two cases are basically the same as they both search by location. The updated default.js is listed below and it's quite self explanatory:

<script type="text/javascript">
(function () {
    "use strict";

    var map;            // Bing Map ojbect
    var currentLocPin;  // Pushpin for current location
    var locationServie; // Location serverice instance 
    var pushpinLayer;   // A layer to contain all pushpins
    var infoboxLayer;   // A layer to contain the infobox
    var resetMapCenter; // Reset the map center location if is true

    document.addEventListener("DOMContentLoaded", initialize, false);

    function initialize() {
        Microsoft.Maps.loadModule('Microsoft.Maps.Map', { callback: initMap });
        Microsoft.Maps.Events.addHandler(map, 'viewchangeend', mapViewChanged);

        btnCurrent.addEventListener("click", currentLocationClick);
        btnSearch.addEventListener("click", searchLocationClick);
        showProgress();
    }

    function initMap() { // Initialize Bing map
        try {
            var mapDiv = document.getElementById("mapDiv");
            var mapOptions = {
                credentials: "xxxxxxx",  //Read from your Bing Maps Account
                center: new Microsoft.Maps.Location(43.813, -79.344), //TORONTO
                zoom: 13,
            };
            map = new Microsoft.Maps.Map(mapDiv, mapOptions);

            // layers to maintain the pushpins and infobox popup
            pushpinLayer = new Microsoft.Maps.EntityCollection();
            infoboxLayer = new Microsoft.Maps.EntityCollection();
            map.entities.push(pushpinLayer);
            map.entities.push(infoboxLayer);
        }
        catch (err)  {
            showError("Init Map error", err);
        }
    }

    // Repopulate map when its moved
    function mapViewChanged() {
        resetMapCenter = false;
        if (map.getZoom() > 10) {          
            var mapCenter = map.getCenter();
            var parameters = { lat: mapCenter.latitude, long: mapCenter.longitude };
            searchLocationService(parameters);
        }
    }

    // Search by current location
    function currentLocationClick() { 
        resetMapCenter = true;
        if (!locationServie) {
            locationServie = new MapService.CurrentLocation();
        }
        locationServie.getCurrentLocationAsync().then(
            function (loc) {
                currentLocPin = MapService.getCurrentLocationPushpin(loc);
                searchLocationService({ lat: loc.latitude, long: loc.longitude });
            },
            function (err) {
                showError("Location service error", err);
            });
    }

    // Search servcie by name
    function searchLocationClick() { 
        resetMapCenter = true;
        var searchInput = txtSearch.value;
        if (searchInput) {
            var parameters = { query: escape(searchInput) };
            searchLocationService(parameters);
        }
    }

    // Invoke map service by MapService module
    function searchLocationService(searchParameter) { 
        showProgress();
        try {
            var mapBounds = map.getBounds(); // Map Bounds are only used by mock service
            MapService.invokeLocationService(searchParameter, handleSearchResult, handleSearchError, mapBounds);
        } catch (err) {
            showError("Error occurs", err);
        }
    }

    // Show error message dialog
    function showError(title, err) { 
        var msg =  (err && err.message) ? err.message : err;
        var msgBox = new Windows.UI.Popups.MessageDialog(msg, title);
        msgBox.showAsync();
    }

    // Handle search error
    function handleSearchError(err) {
        showError("Map Service error", err);
        hideProgress();
    }

    // Search Result handling call back
    function handleSearchResult(data) { 
        var serviceLocations = MapService.parseServiceLocations(data);

        if (resetMapCenter) {
            var searchCenter = MapService.parseSearchCenterLocation(data);
            var locations = [];
            for (var i = 0; i < serviceLocations.length; i++) {
                locations.push(serviceLocations[i].location);
            }
            var origCenter = map.getCenter();
            var origBound = map.getBounds();
            var mapBound = Microsoft.Maps.LocationRect.fromLocations(locations);
            map.setView({ center: searchCenter, bounds: mapBound });
        }

        updatePushpins(serviceLocations);

        hideProgress();
    }
    
    // Shwo progress
    function showProgress() {
        progressDiv.style.display = "block";
    }

    // Hide progress
    function hideProgress() {
        progressDiv.style.display = "none";
    }

    // Update pushpins on the map
    function updatePushpins(serviceLocations) { 
        infoboxLayer.clear();
        pushpinLayer.clear();

        for (var i = 0; i < serviceLocations.length; i++) {
            var loc = serviceLocations[i];
            var pin = MapService.getServicePointPushpin(loc);
            pin.name = loc.name;
            pin.summary = loc.summary;
            pushpinLayer.push(pin);
            Microsoft.Maps.Events.addHandler(pin, 'click', showInfobox);
        }

        if (currentLocPin) {
            pushpinLayer.push(currentLocPin);
        }
    }

    // Display infobox when pushpin is clicked
    function showInfobox(e) { 
        if (e.targetType == "pushpin") {
            infoboxLayer.clear();
            var pin = e.target;
            var pinLocation = pin.getLocation();
            var infoboxOptions = {
                showCloseButton: true,
                //offset: new Microsoft.Maps.Point(0, 5),
                showPointer: true,
                zIndex: 10,
                title: pin.name,
                description: pin.summary,
            };

            var infobox = new Microsoft.Maps.Infobox(pinLocation, infoboxOptions);
            infoboxLayer.push(infobox);
        }
    }
})();
</script>

The whole project can be downloaded from GitHub.

Friday, November 01, 2013

WinJSLog - A Logging and Reporting JavaScript Module For Windows 8 Store Apps

Background

Usually developers use console.log() or WinJS.Log() function to log error and debug information. But that only helps during debugging in local machine. How about those users who installed your app and had errors or experienced crashes? Microsoft has Quality reports in Windows 8 app's dashboard if its telemetry service is enabled, but that function is quite limited, and it only collects crash data when the user accepts the so called Customer Experience Improvement Program (CEIP). It would be great to have errors and custom data reported to your own server so you can do some investigation at real-time. In this post I will have some discussion on this topic and present an implementation to log and report crashes, errors, and potentially any info you want for a Windows 8 store app.

Why logging and reporting are important for mobile apps?

Is your Mobile app working nicely in different devices? Are there any unexpected issues happening in different environment? The customer feedback is likely your main source of investigation. But that's not something you as a developer should rely on. Some issues may never surface, and people would just uninstall the app when they are not happy on it. Unless for some royal users, no many people would spend their precious time to write feedback or report to help you to improve your application.

Even you are lucky enough to have a nice user having a polite comment like this: "This's a good app! However sometimes it crashes when I am on editing mode. Please fix it.", you have no idea what's the real problem is. You may end up spending countless time just to reproduce the issue in specific situation.

So you want to monitor your app's running issues in real-time, and you want to get as much info when crash or error occurs. When logging service is on board, you may want more data to analyze the customer usage. Then it becomes an analytics framework and you have another communication channel to your app (carefully not to abuse using it).

All that sound to be essential to maintain the app quality for a distributed mobile app. But surprisingly I couldn't find a public available solution online. So I decided to write my own, and I called it WinJSLog. The goal of the implementation is light, effective, and easy to use.

Why an app crashes?

I have discussed WinJS exceptions in a previous post. Basically the unhandled exception will result in app crash by default. Crashes is the worst user experience ever because the running app is suddenly disappear without any prompt and reason to end user. A better experience would be redirect the user to a landing page. People may get confused too "why I am switched to a different place?", but at least the app maintains the running condition, giving user a chance to do other stuff or retry the previous action.

Unhandled exception in Windows 8 can be caught in Windows.Application.onerror global event. Following code snippet shows how to register an event handler to redirect user to a specific page when unhandled exception occurs:

    Windows.Application.onerror = function (e) {
        showErrorPage();
        return true; // telling WinJS that the exception was handled, don't terminate
    }

What data to collect?

Errors like "Object reference is null" is not so helpful. As a development you want as much info as possible to identify an issue, such as the stack trace which provides more context about the error. The device and environment variance could make some issue super hard to figure out. A problem may only happen in one specific device, in one specific language or in one specific screen size.

My implementation includes the error message detail, as well as following context info:

application version
language
device maker
device model
screen size
suspend and resume history
page navigation history
method invocation history

Above context information could help us to pin point the issues occurring in certain environment. Ideally the memory usage info should be included but I have not figured out how to get that data from WinJS libraries yet.

What's the data format?

Data are collected as key-value JavaScript objects, and JSON is used to transfer the logged data to different media, like file system or cross the network. JSON was born for such usage and it's a total no-brainer. Following JSON data is an example of crash log sent from a Windows 8 test app running in a VMWare virtual machine:

{
    "os" : "Windows 8",
    "version" : "1.0.0.1",
    "manufacturer" : "VMware, Inc.",
    "model" : "VMware Virtual Platform",
    "lang" : "en-CA",
    "screen" : "1680x1050",
    "orientation" : "landscape",
    "timezone" : "-5", 
    "latitude" : "43.666698",
    "longitude" : "-79.416702",
    "logtime" : "November 1, 2013 9:42:56 AM", 
    "pagetrace" : "home[9:42:27AM]=>user[9:42:41AM]=>test[9:42:48AM]",
    "level" : "crash", 
    "log": [ 
            {
             "source" : "promise", 
             "message" : "'username' is undefined", 
             "description" : "'username' is undefined",
             "stacktrace" : "ReferenceError: 'username' is undefined\n 
                at ready (ms-appx://testApp/pages/test/test.js:85:13)\n 
                at Pages_ready (ms-appx://microsoft.winjs.1.0/js/base.js:4511:29)\n
                at notifySuccess (ms-appx://microsoft.winjs.1.0/js/base.js:1404:21)\n 
                at enter (ms-appx://microsoft.winjs.1.0/js/base.js:1091:21)\n
                at _run (ms-appx://microsoft.winjs.1.0/js/base.js:1307:17)\n
                at _completed (ms-appx://microsoft.winjs.1.0/js/base.js:1275:13)\n
                at notifySuccess (ms-appx://microsoft.winjs.1.0/js/base.js:1404:21)\n
                at enter (ms-appx://microsoft.winjs.1.0/js/base.js:1091:21)\n
                at _run (ms-appx://microsoft.winjs.1.0/js/base.js:1307:17)\n
                at _completed (ms-appx://microsoft.winjs.1.0/js/base.js:1275:13)", 
             "level" : "crash", 
             "time":"2013-11-01T14:42:56.438Z"
           }
        ]
}

In next post I will present a simple logging server implementation to handle such requests.

When to send the logged data? What happen if no Internet access?

Crash or unhandled exception is considered serious issue, and the error detail should be sent to the logging server immediately when it happens. For regular error messages or debugging info we could accumulate them and send them out once current page is completed or a different page is visited. There's also a timer process to check and cleanup the pending logs in background (2 minutes by default and is configurable), so the logged message could be delivered even the user stay on one page for a long time.

The logged data will be stored to file system if Internet service is not available. When the application starts up, it will check if such log file(s) exist, if so the log file(s) will be read and send to logging server if Internet is available. Usually there're many operations during the application booting up. To avoid the action congestion, it's a good idea to defer to run the check-and-send action a little bit later than the application launch (30 seconds by default). If the Internet access is not available, the background process will also periodically recheck the network status (2 minutes by default) and send out the file logs when Internet access becomes available.

Other consideration

You app may target to some specific features such as map and camera stuff, and the location and camera info would be good for you to diagnose the issue. In such case, the location and camera data should be obtained right in the spot when they are used, and store them in a global variable like sessionState, so the logging code can read it directly without a redundant code executed again, which makes the logging service lighter and less intrusive.

What happens if logging server requires authentication? As I discussed in WinJS Authentication, it would be better to allow anonymous access for such general logging service. Hard-coded logging username/password is not safe, and it's not feasible to let user to type the username and password. Would it be a potential DOS/DDOS attach when anonymous access is open? No service in Internet can avoid that, but firewall in front of your servers should take care of that threat. However we can still do a bit more to protect ourself in the code level, such as not logging abnormal data with extremely large size of URL or message.

How about the errors inside logging service? In general any errors or potential errors within logging and reporting code should be silently skipped. Optionally those errors can be sent to the logging server, or send to a different server if errors occur during sending data to logging server.

How to use the code?

The WinJSLog module exposes following functions:

    function fatal(error)                // log crash
    function error(description, error)   // log error
    function warning(error, description) // log warning
    function page(pageName)              // navigate to a page
    function method(methodName)          // invoke a method
    function registerLogging()           // register logging
    function unregisterLogging()         // unregister logging

First include the winjslog.js file inside default.html or your custom page, then register the logging service before app.start():

(function () {
    "use strict";

    WinJS.Binding.optimizeBindingReferences = true;

    var app = WinJS.Application;
    var nav = WinJS.Navigation;

    app.addEventListener("activated", function (args) {
      ...// skip for abreviation
    }); 
   
    app.onerror = function (e) {
        Logging.fatal(e);
        Logging.page("homepage");
        nav.navigate("/pages/home/home.html");
        
        return true; // the app will terminate if false
    };

    Logging.registerLogging("http://myloggingserver/");
    
    app.start();

    ...

In above code an onerror event handler is registered to catch the unhandled exception to avoid the app's crash, log the crash by Logging.fatal(e), then redirect to home page. Note that the action of redirecting to home page is also logged as a navigation path by Logging.page() function. Following code snippet shows how to log a function invocation and its potential error message:

    function doComplicatedTask() {
        Logging.method("doComplicatedTask");
        try {
            ... // job with potential error
        } catch (e) {
            logging.error("Error occur!", e);
        }
    }

Alternatively you can log lower level warning, info and debug message by using Logging.warning(), Logging.info() and logging.debug() functions respectively.

Source code of WinJSLog.js

Update: the source code now is available at github and you can get the latest version there.

(function () {
    "use strict";

    var logs = [], pages = [], methods = []; // log data
    var loggingServer, loggingEnabled, debugEnabled; // service setting
    var version, manufacturer, model; // app context
    var currentLevel = 'debug', levels = { 'debug': 1, 'info': 2, 'warning': 3, 'error': 4, 'crash': 5 };

    // register logging service
    function registerLogging(serverUrl, doDebugging, deferRunInSeconds, recheckInSeconds) {
        if (!serverUrl) {
            throw 'registerLogging error: logging server not defined.';
        }

        loggingEnabled = true;
        loggingServer = serverUrl;
        debugEnabled = doDebugging || false;
        deferRunInSeconds = deferRunInSeconds || 30;
        recheckInSeconds = recheckInSeconds || 60;

        // Register event handler for suspend, resume and relaunch (terminated then restarted)
        WinJS.Application.addEventListener("activated", restoreSessionData);
        Windows.UI.WebUI.WebUIApplication.addEventListener("suspending", suspend);
        Windows.UI.WebUI.WebUIApplication.addEventListener("resuming", resume);

        // defer to run the check-log-file-and-send process
        WinJS.Promise.timeout(deferRunInSeconds * 1000).then(function () {
            cleanupLogs(recheckInSeconds);
        });
    }

    // unregister logging service
    function unregisterLogging() {
        loggingEnabled = false;
        WinJS.Application.removeEventListener("activated", restoreSessionData);
        Windows.UI.WebUI.WebUIApplication.removeEventListener("suspending", suspend);
        Windows.UI.WebUI.WebUIApplication.removeEventListener("resuming", resume);
    }

    // restore logging data after app relaunch from terminated state
    function restoreSessionData(args) {
        if (args && args.detail && args.detail.kind === Windows.ApplicationModel.Activation.ActivationKind.launch) {
            if (args.detail.previousExecutionState === Windows.ApplicationModel.Activation.ApplicationExecutionState.terminated) {
                var sessionState = WinJS.Application.sessionState;
                if (sessionState.logPages)
                    pages = sessionState.logPages;
                if (sessionState.logMethods)
                    methods = sessionState.logMethods;
                if (sessionState.logLogs)
                    logs = sessionState.logLogs;
                if (sessionState.logLevel)
                    currentLevel = sessionState.logLevel;
            }
        }
    }

    // log suspending event and store log data into session
    function suspend() {
        if (loggingEnabled) {
            var pageEntry = { 'time': new Date(), 'page': 'suspending' };
            pages.push(pageEntry);
            var sessionState = WinJS.Application.sessionState;
            sessionState.logPages = pages;
            sessionState.logMethods = methods;
            sessionState.logLogs = logs;
            sessionState.logLevel = currentLevel;
        }
    }

    // log resuming event
    function resume() {
        if (loggingEnabled) {
            var pageEntry = { 'time': new Date(), 'page': 'resuming' };
            pages.push(pageEntry);
        }
    }

    // log an event: navigate to a page
    function pageLog(pageId) {
        if (loggingEnabled) {
            try {
                processMemoryLogs();
            } catch (e) { }
            var pageEntry = { 'time': new Date(), 'page': pageId };
            pages.push(pageEntry);
        }
    }

    // log an event: invoke a method
    function methodLog(methodName) {
        if (loggingEnabled) {
            methods.push(methodName);
        }
    }

    // log a crash; a crash or unhandled exception can be caught by WinJS.Application.onerror event
    function crashLog(err) {
        if (loggingEnabled) {
            setLevel('crash');
            var errWrapper = getErrorObject(err);
            errWrapper.level = "crash";
            errWrapper.time = new Date();
            logs.push(errWrapper);
            try {
                processMemoryLogs();
            } catch (e) { }
        }
    }

    // log an error
    function errorLog(description, err) {
        if (loggingEnabled) {
            setLevel('error');
            logs.push(getLogObject('error', description, err));
        }
    }

    // log a warning message
    function warningLog(description, err) {
        if (loggingEnabled && debugEnabled) {
            setLevel('warning');
            logs.push(getLogObject('warning', description, err));
        }
    }

    // log an info message
    function infoLog(description) {
        if (loggingEnabled && debugEnabled) {
            setLevel('info');
            logs.push(getLogObject('info', description));
        }
    }

    // log a debug message
    function debugLog(description) {
        if (loggingEnabled && debugEnabled) {
            setLevel('debug');
            logs.push(getLogObject('debug', description));
        }
    }

    // build a log object
    function getLogObject(level, description, err) {
        var logObject = getErrorObject(err);
        if (logObject.description) {
            logObject.description = logObject.description + description;
        } else {
            logObject.description = description || '';
        }
        logObject.level = level || 'unknown';
        logObject.time = new Date();
        return logObject;
    }

    // build an error object
    function getErrorObject(err) {
        var errObject = {};
        if (err) {
            if (err.detail && typeof err.detail === 'object') {
                var detail = err.detail;
                if (detail.promise) {
                    errObject.source = "promise";
                }
                if (detail.errorMessage) {
                    errObject.message = detail.errorMessage;
                    if (detail.errorLine)
                        errObject.codeline = detail.errorLine;
                    if (detail.errorUrl)
                        errObject.sourcUrl = detail.errorUrl;
                } else if (detail.error && typeof detail.error === 'object') {
                    errObject.message = detail.error.message || 'unknown';
                    if (detail.error.description)
                        errObject.description = detail.error.description;
                    if (detail.error.stack)
                        errObject.stacktrace = detail.error.stack;
                } else {
                    errObject.message = detail.message || 'unknown';
                    if (detail.description)
                        errObject.description = detail.description;
                    if (detail.number)
                        errObject.codeline = detail.number;
                    if (detail.stack)
                        errObject.stacktrace = detail.stack;
                }
            } else {
                errObject.message = err.message || err.exception || err;
            }
        }
        return errObject;
    }

    // determine the highest log level for current log entry
    function setLevel(level) {
        if (levels[level] > levels[currentLevel]) {
            currentLevel = level;
        }
    }

    // periodically check the memory logs and storage logs, and send logs to server if Internet is available
    function cleanupLogs(recheckInseonds) {
        if (loggingEnabled) {
            processMemoryLogs();
            processFileLogs();
            setTimeout(function () {
                cleanupLogs(recheckInseonds);
            }, recheckInseonds * 1000);
        }
    }

    // construct log message and send to server if Internet is available, otherwise save it to local storage
    function processMemoryLogs() {
        if (logs.length > 0) {
            var data = getContext();
            var date = new Date();
            data.logtime = date.toLocaleString() + ' [' + date.toISOString() + ']';
            if (pages.length > 0) {
                var pagetrace = pages.map(function (item) {
                    if (item.time && item.time.toLocaleTimeString)
                        return item.page + "[" + item.time.toLocaleTimeString().replace(' ', '') + ']';
                    else
                        return item.page + "[" + item.time + ']';
                }).join(' => ');
                data.pagetrace = pagetrace;
            }
            if (methods.length > 0) {
                data.methodtrace = methods.join(' => ');
            }
            data.level = currentLevel;
            data.log = logs.slice(0); //(logs.length == 1) ? logs[0] : logs.slice(0);

            if (isConnectedToInternet()) {
                sendLogsToServer(JSON.stringify(data));
            } else {
                saveLogsToFile(data);
            }
        }

        // clean up the logs
        methods = [];
        logs = [];
        currentLevel = 'debug';
    }

    // read all saved log files and send them to server if Internet is available
    function processFileLogs() {
        if (isConnectedToInternet()) {
            var localFolder = Windows.Storage.ApplicationData.current.localFolder;
            localFolder.getFilesAsync().then(function (files) {
                files.forEach(function (file) {
                    if (file && file.displayName && file.displayName.indexOf("logs") == 0) {
                        Windows.Storage.FileIO.readTextAsync(file).then(function (text) {
                            sendLogsToServer(text);
                        }).then(function () {
                            file.deleteAsync();
                        }).done(function () { }, function (err) { });
                    }
                });
            });
        }
    }

    // save a log entry to file system if Internet is not available
    function saveLogsToFile(obj) {
        var fileName = "logs.txt";
        var content = JSON.stringify(obj);
        var localFolder = Windows.Storage.ApplicationData.current.localFolder;
        var saveOption = Windows.Storage.CreationCollisionOption;
        localFolder.createFileAsync(fileName, saveOption.generateUniqueName).then(
            function (file) {
                return Windows.Storage.FileIO.writeTextAsync(file, content);
            }).done(function () {
                console.log("Log saved");
            }, function (error) {
                console.log("Log saved error");
            });
    }

    // send log message to logging server
    function sendLogsToServer(jsonData) {
        WinJS.xhr({
            type: "post",
            url: loggingServer,
            headers: { "Content-type": "application/json" },
            data: jsonData
        }).done(function completed(c) {
            console.log("log sent");
        },
        function error(e) { // One more try? send to different server? or silently skip?
            console.log("log sent error");
        });
    }

    // get current application context
    function getContext() {
        if (!version) {
            var appVersion = Windows.ApplicationModel.Package.current.id.version;
            version = appVersion.major + "." + appVersion.minor + "." + appVersion.build + "." + appVersion.revision;
            try {
                var deviceInfo = new Windows.Security.ExchangeActiveSyncProvisioning.EasClientDeviceInformation();
                manufacturer = deviceInfo.systemManufacturer;
                model = deviceInfo.systemProductName;
            } catch (e) {
                manufacturer = 'unknown';
                model = 'unknown';
            }
        }
        var context = {};
        context.version = version;
        context.manufacturer = manufacturer;
        context.model = model;
        context.os = "Windows 8";
        context.lang = navigator.appName == "Netscape" ? navigator.language : navigator.userLanguage;
        context.screen = screen.width + "x" + screen.height;
        context.orientation = getOrientation();
        context.timezone = (-(new Date()).getTimezoneOffset() / 60).toString();
        return context;
    }

    // determine current orientation
    function getOrientation() {
        var orientation = "unknown";
        switch (Windows.Graphics.Display.DisplayProperties.currentOrientation) {
            case Windows.Graphics.Display.DisplayOrientations.landscape:
                orientation = "landscape";
                break;
            case Windows.Graphics.Display.DisplayOrientations.portrait:
                orientation = "portrait";
                break;
            case Windows.Graphics.Display.DisplayOrientations.landscapeFlipped:
                orientation = "landscapeFlipped";
                break;
            case Windows.Graphics.Display.DisplayOrientations.portraitFlipped:
                orientation = "portraitFlipped";
                break;
        }
        return orientation;
    }

    // check if Internet access is available
    function isConnectedToInternet() {
        var connectivity = Windows.Networking.Connectivity;
        var profile = connectivity.NetworkInformation.getInternetConnectionProfile();
        if (profile) {
            var connected = (profile.getNetworkConnectivityLevel() == connectivity.NetworkConnectivityLevel.internetAccess);
            return connected;
        } else {
            return false;
        }
    }

    WinJS.Namespace.define("Logging", {
        registerLogging: registerLogging,
        unregisterLogging: unregisterLogging,
        page: pageLog,
        method: methodLog,
        fatal: crashLog,
        error: errorLog,
        warning: warningLog,
        info: infoLog,
        debug: debugLog
    });
})();

Thursday, May 02, 2013

Implementing Camera Switch, Resolution Change and Option Popup with MediaCapture API in WinJS

WinJS provides CameraCaptureUI and MediaCapture API to access camera. The CameraCaptureUI has predefined UI but if you want more control such as custom UI, MediaCapture is the only option.

David Rousset's posts, Tutorial Series: using WinJS & WinRT to build a fun HTML5 Camera Application for Windows 8 is great resource to learn how to work with MediaCapture API. But some common features available on CameraCaptureUI are not included in David's example. After some research and testing I implemented camera switching, resolution selection and camera option popup with MediaCapture API.

Change camera:
An option to let user to switch another camera if multiple cameras available. The key is set the MediaCaptureInitializationSettings.videoDeviceId property, and pass MediaCaptureInitializationSettings to initialize the MediaCapture object. We can use Windows.Devices.Enumeration.DeviceInformation.finaAllAsync method to loop through all video capture devices and get all cameras' device ID.
Change resolution :
User can change camera's resolution for photo taking. We use MediaCapture.videoDeviceController.getAvailableMediaStreamProperties method to get a list of available resolutions for the camera tied to the MediaCapture object, and use MediaCapture.videoDeviceController.setMediaStreamPropertiesAsync method to set the camera resultion.
Open other options dialog:
A button to open camera's option dialog by calling Windows.Media.Capture.CameraOptionsUI.show(mediaCapture) method.

Following is code example implementing above three features. Although the code is targeting to photo functions for demo purpose, it applies to video capturing as well. Refer to David's posts for more details on how to work on videos using MediaCapture API.

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <title>CustomCameraPage</title>

    <!-- WinJS references -->
    <link href="//Microsoft.WinJS.1.0/css/ui-dark.css" rel="stylesheet" />
    <script src="//Microsoft.WinJS.1.0/js/base.js"></script>
    <script src="//Microsoft.WinJS.1.0/js/ui.js"></script>

    <!-- CustomCameraPage references -->
    <link href="/css/default.css" rel="stylesheet" />
    <script src="/js/default.js"></script>
</head>
<body>
    <div id="cameraPage">
        <video id="cameraPreview"></video>
        <img id="imgPhotoTaken" style="display:none;"/>
        <div class="footer">
            <button id="btnSave" title="Save a photo">Save Photo</button>
            <select id="selectResolutions" title="Camera Resolutions"></select>
            <span id="cameraOption" title="Set camera option">&#xe15e</span>
            <span id="cameraSwitch" title="Switch to another camera">&#xe124</span>
            <button id="btnTake" title="Take a photo">Take Photo</button>
        </div>
    </div>
</body>
</html>

CSS:

#cameraPreview, #imgPhotoTaken { width: 100%; height: 100%; }
#cameraPage .footer { position: absolute; bottom: 0px; left: calc(50% - 200px); }
#cameraPage .footer > * { background-color: rgba(0,0,0,0.5); color: #ffffff; margin-left: 10px; }
#cameraPage .footer  { font-size: 1.4em; }

JavaScript:

// For an introduction to the Blank template, see the following documentation:
// http://go.microsoft.com/fwlink/?LinkId=232509
(function () {
    "use strict";

    WinJS.Binding.optimizeBindingReferences = true;

    var app = WinJS.Application;
    var activation = Windows.ApplicationModel.Activation;

    var mediaCapture;
    var cameraSettings;
    var allCameras = [];
    var cameraResolutions = [];
    var liveMode = true;
    var mediaType = Windows.Media.Capture.MediaStreamType;
    var tempFile = "photo.jpg";
    var tempFolder = Windows.Storage.ApplicationData.current.localFolder;
    var fileSaver = new Windows.Storage.Pickers.FileSavePicker();

    app.onactivated = function (args) {
        if (args.detail.kind === activation.ActivationKind.launch) {
            if (args.detail.previousExecutionState !== activation.ApplicationExecutionState.terminated) {
                // TODO: This application has been newly launched. Initialize
                // your application here.
            } else {
                // TODO: This application has been reactivated from suspension.
                // Restore application state here.
            }
            args.setPromise(WinJS.UI.processAll());

            initPage();
        }
    };

    app.start();

    // Page initialization
    function initPage() {
        fileSaver.suggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.picturesLibrary;
        fileSaver.fileTypeChoices.insert("JPG image", [".jpg"]);
        getCamera().then(function (camera) {
            if (camera) {
                wireControls();
                if (allCameras.length <= 1) {
                    cameraSwitch.disabled = "disabled";
                }
                startCamera(camera);
            } else {
                showError("No camera found.", "Error");
            }
        },
        function (error) {
            onCameraError(error);
        });
    }

    // UI element event binding 
    function wireControls() {
        btnSave.addEventListener("click", saveImage);
        btnTake.addEventListener("click", takePhoto);
        cameraSwitch.addEventListener("click", switchCamera);
        cameraOption.addEventListener("click", setCameraOtherOption);
        selectResolutions.addEventListener("change", setCameraResolution);
    }

    // Creating a new mediaCapture object based on a selected camera, then get all its available resolutions;
    // Return a Promise when completed
    function startCamera(camera) {
        if (camera) {
            cameraSettings = new Windows.Media.Capture.MediaCaptureInitializationSettings();
            cameraSettings.photoCaptureSource = Windows.Media.Capture.PhotoCaptureSource.videoPreview;
            cameraSettings.streamingCaptureMode = Windows.Media.Capture.StreamingCaptureMode.video;
            cameraSettings.videoDeviceId = camera.id;
            mediaCapture = new Windows.Media.Capture.MediaCapture();
            mediaCapture.addEventListener("failed", onCameraError, false);
            return mediaCapture.initializeAsync(cameraSettings).then(function () {
                mediaCapture.setPreviewRotation(Windows.Media.Capture.VideoRotation.none);
                return getCameraResolutions();
            }).then(function () {
                cameraPreview.src = URL.createObjectURL(mediaCapture, { oneTimeOnly: true });
                cameraPreview.play();
            },
            function (error) {
                showError(error);
            });;
        } else {
            return WinJS.Promise.wrapError("No camera to start");
        }
    }

    // Release resource of live video preview and mediaCapture object
    function releaseCamera() {
        cameraView.src = null;
        mediaCapture = null;
    }

    // Loop through all cameras
    // Return a Promise with a default camera
    function getCamera() {
        allCameras = [];
        return new WinJS.Promise(function (c, e) {
            var deviceInfo = Windows.Devices.Enumeration.DeviceInformation;
            return deviceInfo.findAllAsync(Windows.Devices.Enumeration.DeviceClass.videoCapture).then(function (devices) {
                if (devices.length > 0) { // Camera found
                    var defaultCamera = devices[0]
                    for (var i = 0; i < devices.length; i++) {
                        var device = devices[i];
                        allCameras.push(device);
                        // Default to embedded front camera
                        if (device.enclosureLocation && device.enclosureLocation.panel == Windows.Devices.Enumeration.Panel.front)
                            defaultCamera = device;
                    }
                    c(defaultCamera);
                } else { // No camera found
                    c();
                }
            },
            function (error) { // Error
                e(error);
            });
        });
    }

    // Switch to different camera if mulitple cameras available
    function switchCamera() {
        if (mediaCapture && allCameras && allCameras.length > 1) {
            var currentCameraId = mediaCapture.mediaCaptureSettings.videoDeviceId;
            var nextCamera = null;
            var cameraCount = allCameras.length;
            for (var i = 0; i < cameraCount; i++) {
                var cam = allCameras[i];
                if (cam.id === currentCameraId) {
                    if (i == (cameraCount - 1)) {
                        nextCamera = allCameras[0];
                    } else {
                        nextCamera = allCameras[i + 1];
                    }
                    break;
                }
            }
            if (nextCamera) {
                releaseCamera();
                startCamera(nextCamera);
            }
        }
    }

    // Take a photo, save it to temporary folder then display it on the screen
    // Return a Prmoise when completed
    function takePhoto() {
        if (mediaCapture) {
            if (!liveMode) { // Show the live video preview if clicking "Retake"
                toggleView();
                return WinJS.Promise.as();
            } else { // Save and display taken photo if clicking "Take"
                var saveOption = Windows.Storage.CreationCollisionOption;
                return tempFolder.createFileAsync(tempFile, saveOption.replaceExisting).then(function (file) {
                    var photoProperties = Windows.Media.MediaProperties.ImageEncodingProperties.createJpeg();
                    return mediaCapture.capturePhotoToStorageFileAsync(photoProperties, file).then(function (result) {
                        return tempFolder.getFileAsync(tempFile).then(function (newFile) {
                            if (newFile) {
                                var imageUrl = URL.createObjectURL(file, { oneTimeOnly: true });
                                imgPhotoTaken.src = imageUrl;
                                toggleView();
                            }   
                        });
                        
                    });
                });
            }
        }
    }

    // Toggle display to show taken photo or video live preview, and switch button text to "Take" or "Retake"
    function toggleView () {
        if (liveMode) { // when showing live video preview
            liveMode = false;
            btnTake.textContent = "Retake";
            imgPhotoTaken.style.display = "block";
            cameraPreview.style.display = "none";
            cameraPreview.pause();
            
        } else { // when showing taken image 
            liveMode = true;
            btnTake.textContent = "Take";
            cameraPreview.style.display = "block";
            imgPhotoTaken.style.display = "none";
            cameraPreview.play();
        }
    }

    // Get all resolutions of current MediaCapture object and populate resolutions option in UI 
    // Return a Promise when completed
    function getCameraResolutions() {
        if (mediaCapture) {
            cameraResolutions = [];
            while (selectResolutions.length > 0) {
                selectResolutions.remove(0);
            }

            var currentResolution = mediaCapture.videoDeviceController.getMediaStreamProperties(mediaType.videoPreview);
            var allResolutions = mediaCapture.videoDeviceController.getAvailableMediaStreamProperties(mediaType.videoPreview);

            // Save all unique resolutions
            var tempResolutions = [];
            for (var i = 0; i < allResolutions.length; i++) {
                var res = allResolutions[i];
                if (res.width < 600)
                    continue;
                var hasDuplicated = false;
                tempResolutions.forEach(function (item) {
                    if (item.width == res.width && item.height == res.height)
                        hasDuplicated = true;
                });
                if (!hasDuplicated)
                    tempResolutions.push(res);
            }

            // Sort all unique resolutions from low to high based on width
            tempResolutions.sort(function (item1, item2) {
                return item1.width == item2.width ? item1.height - item2.height : item1.width - item2.width;
            });

            // Save camera resolutions and populate resolution options in UI 
            for (var i = 0; i < tempResolutions.length; i++) {
                var res = tempResolutions[i];
                cameraResolutions.push(res);
                selectResolutions.add(new Option(res.width + "x" + res.height));
                if (currentResolution.width == res.width && currentResolution.height == res.height) {
                    selectResolutions.selectedIndex = i;
                }
            }
            return mediaCapture.videoDeviceController.setMediaStreamPropertiesAsync(mediaType.videoPreview, currentResolution);
        } else {
            return WinJS.Promise.as();
        }
    }

    // Switch to a different resolution on a camera
    function setCameraResolution() {
        if (mediaCapture) {
            try {
                var index = selectResolutions.selectedIndex;
                if (cameraResolutions.length > index) {
                    var resolution = cameraResolutions[index];
                    mediaCapture.videoDeviceController.setMediaStreamPropertiesAsync(mediaType.videoPreview, resolution).then(function () {
                        cameraPreview.src = null;
                        cameraPreview.src = URL.createObjectURL(mediaCapture, { oneTimeOnly: true });
                        cameraPreview.play();
                    },
                    function (error) {
                        onCameraError(error);
                    });
                }
            } catch (e) {
                showError(e);
            }
        }
    }

    // Popup camera option menu
    function setCameraOtherOption() {
        if (mediaCapture) {
            Windows.Media.Capture.CameraOptionsUI.show(mediaCapture);
        }
    }

    // When user select to save a photo 
    function saveImage() {
        if (liveMode) {
            takePhoto().then(function () {
                saveImageToStorage();
            });
        } else {
            saveImageToStorage();
        }         
    }

    // Open file picker and save photo to specific name and location
    function saveImageToStorage() {
        fileSaver.pickSaveFileAsync().then(function (file) {
            if (file) {
                tempFolder.getFileAsync(tempFile).then(function (tmpImage) {
                    tmpImage.copyAndReplaceAsync(file).done(function () {
                        console.log("Photo saved to " + file.path + " successfully.");
                    });
                });
            }
        });
    }

    // Error occurs during camera operation
    function onCameraError(error) {
        showError(error);
    }

    // Ddisplay error information
    function showError(error, title) {
        var message = (error && error.message) ? error.message : error;
        var msgPopup = new Windows.UI.Popups.MessageDialog(message);
        if (title)
            msgPopup.title = title;
        msgPopup.commands.append(new Windows.UI.Popups.UICommand("Ok"));
        msgPopup.showAsync();
    }
})();

The resolution option now is available for user to select:

When camera other options dialog pops up:

When a photo is taken, you have option to retake or save it:

Saturday, February 16, 2013

Android Thread Handling in Configuration Change

The Activity Recreating Issue During Configuration Change

When configuration change occurs in an Android device, e.g. rotating the screen from landscape to portrait mode, the Activity will be destroyed and recreated. This could introduce some issues if some tasks inside Activity are not completed during the configuration change. For instance a worker thread may still be running in background and leaking the memory during the configuration change. We assume that the worker thread here in discussion ties to Activity and will communicate back to Activity instance when it completes its task.

Let's take a look at following Activity code:

public class MainActivity extends Activity {
    private static final String TAG = MainActivity.class.getSimpleName();
    private static int instanceCount = 0;
    private Handler handler;
    private Thread thread;
    private TextView textView;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        instanceCount++;
        Log.d(TAG, "onCreate()");

        textView = (TextView)findViewById(R.id.textView1);
        textView.setText("Activity Instance " + String.valueOf(instanceCount));
            
        handler = new Handler() {
            @Override
            public void handleMessage(Message msg) {
                Log.d(TAG, "Handler thread - " + getThreadInfo());
            }
        };

        thread = new Thread(new Runnable() {
            @Override
            public void run() {
                Log.d(TAG, "Worker thread - " + getThreadInfo());
                try {
                    int count = 10;
                    while(count-- > 0) { // pause 10 seconds
                        Thread.sleep(1000); 
                    }
                    Log.d(TAG, "Worker thread sendMmessage to handler");
                    handler.sendEmptyMessage(0);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
        thread.start();
    }    
    
    @Override
    protected void onDestroy() {
        Log.d(TAG, "onDestroy()");
        super.onDestroy();
    }

    private static String getThreadInfo()
    {
        Thread currentThread = Thread.currentThread();
        String info = String.format("%1$s ID: %2$d Priority: %3$s",  
                currentThread.getName(), currentThread.getId(), currentThread.getPriority());
        return info;
    }
}

A separate thread sleeps for 10 seconds to simulate a long-run task, then updates a UI view by a handler. If the the screen is rotated within 10 seconds, the activity will be recreated, so as a new thread and a new handler. However the old thread is still running in background, consuming resource and leaking the memory. The old Activity object will not be garbage collected at the time of destroy since the handler and thread are referencing it. The view switches from "Activity Instance 1" to "Activity Instance 2", and the LogCat shows:

Disabling Dangling Thread

The easiest method to resolve the issue is set a flag when the activity is destroyed to control the stale thread:

public class MainActivity extends Activity {
    private boolean stopThread = false;
    //...
    
    @Override
    protected void onCreate(Bundle savedInstanceState) {
    //...
        thread = new Thread(new Runnable() {
            @Override
            public void run() {
                Log.d(TAG, "Worker thread - " + getThreadInfo());
                try {
                    int count = 10;
                    while(count-- > 0 && !stopThread) { // pause 10 seconds
                        Thread.sleep(1000); 
                    }
                    if (!stopThread) {
                        Log.d(TAG, "Worker thread sendMmessage to handler");
                        handler.sendEmptyMessage(0);
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
        thread.start();
        }
    }
    
    @Override
    protected void onDestroy() {
        Log.d(TAG, "onDestroy()");
        super.onDestroy();
        stopThread = true;
        handler.removeCallbacksAndMessages(null);
    }
    
    //...
 }

The LogCat logs:

Now the first worker thread is cancelled along with its partially completed task. To save the work by first thread, we can use onSaveInstanceState() callback to store the partial result, so later the second worker thread can use it as an initial start point, as described in this post.

Using Static Thread Object

The solution above is not perfect: multiple thread instances created during configuration change which is inefficient and expensive. We can use static thread variable to maintain one thread instance:

public class MainActivity extends Activity {
    private static final String TAG = MainActivity.class.getSimpleName();
    private static int instanceCount = 0;
    private static WorkerThread thread;
    private TextView textView;
    
    private Handler handler = new Handler() {
        @Override
        public void handleMessage(Message msg) {
            Log.d(TAG, "Handler thread - " + getThreadInfo());
        }
    };

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        instanceCount++;
        Log.d(TAG, "onCreate()");

        textView = (TextView)findViewById(R.id.textView1);
        textView.setText("Activity Instance " + String.valueOf(instanceCount));
        
        if (savedInstanceState != null && thread != null && thread.isAlive()) {
            thread.setHandler(handler);
        } else {
            thread = new WorkerThread(handler);
            thread.start();
        }
    }    
    
    @Override
    protected void onDestroy() {
        Log.d(TAG, "onDestroy()");
        super.onDestroy();
        handler.removeCallbacksAndMessages(null);
        if (thread.isAlive()) {
            thread.setHandler(null);
        }
    }

    private static String getThreadInfo()
    {
        Thread currentThread = Thread.currentThread();
        String info = String.format("%1$s ID: %2$d Priority: %3$s",  
                currentThread.getName(), currentThread.getId(), currentThread.getPriority());
        return info;
    }
    
    private static class WorkerThread extends Thread {
        private Handler handler;

        public WorkerThread(Handler handler) {
            super();
            this.handler = handler;
        }

        public void setHandler(Handler handler) {
            this.handler = handler;
        }

        @Override
        public void run() {
            Log.d(TAG, "Worker thread - " + getThreadInfo());
            try {
                int count = 10;
                while (count-- > 0) { // pause 10 seconds
                    Thread.sleep(1000);
                }
                if (handler != null) {
                    Log.d(TAG, "Worker thread sendMmessage to handler");
                    handler.sendEmptyMessage(0);
                }
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

Notice the extended WorkerThread class is also static to avoid memory leak, as in Java non-static inner and anonymous classes will implicitly hold an reference to their outer class. Now the LogCat logs:

As a side note, be cautious to use static variables within Activity to avoid memory leak. If you have to use the static variables, do not forget to cleanup the resources/references in the Activity.onDestroy() callback.

Using Fragment to Retain Thread

Another option, also the recommended way from Android Developer Guild, is to use Fragment with RetainInstance set to true to retain one instance of thread. The worker thread is wrapped into the non-UI Fragment:

public class ThreadFragment extends Fragment {
      private static final String TAG = ThreadFragment.class.getSimpleName();
      private Handler handler;
      private Thread thread;
      private boolean stopThread;

      public ThreadFragment(Handler handler) {
          this.handler = handler;
      }
      
      public void setHandler(Handler handler) {
          this.handler = handler;
      }
      
      @Override
      public void onCreate(Bundle savedInstanceState) { 
        Log.d(TAG, "onCreate()");
        super.onCreate(savedInstanceState);

        setRetainInstance(true); // retain one Fragment instance in configuration change
        
        thread = new Thread(new Runnable() {
            @Override
            public void run() {
                Log.d(TAG, "Worker thread - " + MainActivity.getThreadInfo());
                try {
                    int count = 10;
                    while(count-- > 0 && !stopThread) { // pause 10 seconds
                        Thread.sleep(1000); 
                    }
                    if (handler != null) {
                        Log.d(TAG, "Worker thread sendMmessage to handler");
                        handler.sendEmptyMessage(0);
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
        thread.start();
      }

      @Override
      public void onDestroy() {
        Log.d(TAG, "onDestroy()");
        super.onDestroy();
        handler = null;
        stopThread = true;
      }
 }

The Fragment feature was added from Android 3.0 Honeycomb. For older versions you need to include the Android Support package (android-support-v4.jar) to get the Fragment work. With Fragment setup, the main activity will dynamically create or activate existence of ThreadFragment:

public class MainActivity extends Activity {
    private static final String TAG = MainActivity.class.getSimpleName();
    private static int instanceCount = 0;
    private ThreadFragment fragment;
    private TextView textView;
    
    private Handler handler = new Handler() {
        @Override
        public void handleMessage(Message msg) {
            Log.d(TAG, "Handler thread - " + getThreadInfo());
        }
    };

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        instanceCount++;
        Log.d(TAG, "onCreate()");

        textView = (TextView)findViewById(R.id.textView1);
        textView.setText("Activity Instance " + String.valueOf(instanceCount));
        
        FragmentManager fm = getFragmentManager();
        fragment = (ThreadFragment) fm.findFragmentByTag("thread");

        if (fragment == null) {
            fragment = new ThreadFragment(handler);
            fm.beginTransaction().add(fragment, "thread").commit();
        } else { // retained across configuration changes
            fragment.setHandler(handler);
        }
    }    
    
    @Override
    protected void onDestroy() {
        Log.d(TAG, "onDestroy()");
        super.onDestroy();

        fragment.setHandler(handler);
        handler.removeCallbacksAndMessages(null);
    }

    public static String getThreadInfo()
    {
        Thread currentThread = Thread.currentThread();
        String info = String.format("%1$s ID: %2$d Priority: %3$s",  
                currentThread.getName(), currentThread.getId(), currentThread.getPriority());
        return info;
    }
}

The LogCat result:

Thread Safety

The code snippets demoed about are not thread-safe. To make the code thread-safe, we can set the variable as volatile and wrap the setting inside a synchronized method so that only one thread updates the values at any time:

public class ThreadFragment extends Fragment {
    //...
    private volatile Handler handler;
    private volatile boolean stopThread;  
    //...
    
    public void setHandler(Handler handler) {
        synchronized( this.handler ) {
            this.handler = handler;
        }
        if (handler == null){
            requestStop();
        }
    }      

    public synchronized void requestStop() {
        stopThread = true;
    }
    
    thread = new Thread(new Runnable() {
        @Override
        public void run() {
           //...
           synchronized (handler) {
                if (handler != null) {
                    //...
                    handler.sendEmptyMessage(0);
                }
            }
        }
    }
      
    @Override
    public void onDestroy() {
        //...
        requestStop();
        handler = null;
    }
    
    //...
}

Above code avoids the scenarios like the work thread goes into the block after checking handler is not null, but right at that moment the handler is set to null by the main thread. However I am not so sure if such implementation is necessary. Unlike server side services that may be invoked by multiple callers at the same time, Android apps run locally so this kind of race conditions would rarely occur.