Packaging an ExtJS4 application using Node-Webkit part 3

This is a continuation of Part 1 and Part 2

How to install Node modules in your app

At this point you might want to install a node module and extend the demo. For example, you might want to make use of the neDB module. This is a way to persist data locally. There are a number of modules that do that. This one has no other dependencies so is a good candidate for our demo.

To install the module, open a command line shell in the root of your development app. The process below will install that module into the node_modules folder. I assume you have set a PATH to your NodeJs install folder so npm can be found.

1. From your command prompt:

npm install nedb --save

2. Open resources/js/NWK-requires.js and add

NWK.Datastore = require('nedb');

Add some code to a new button and handler in Main.js to add and find some docs then test. eg:

{
    xtype: 'button',
    text: 'neDb',
    handler: function(btn, e) {
        NWK.mydb = new NWK.Datastore({
            filename: 'mydb',
            autoload: true
        });

        var docs = [{
            mykey: '2f4vn9l1NGW',
            data: 'blah blah'
        }, {
            mykey: '2f4vn9l1rAB',
            data: 'Boo hoo'
        }];

        NWK.mydb.insert(docs, function(err, newdoc) {
            console.log('INSERT callback')
            console.log(err);
            console.log(newdoc);
        })

        NWK.mydb.find({
            mykey: '2f4vn9l1rAB'
        }, function(err, docs) {
            console.log('FINDING')
            console.log(err);
            console.log(docs);
        })
    }
}

Packaging an ExtJS4 application using Node-Webkit part 2

In Part 1 of this tutorial we generated an ExtJS4 app and got it running in the node-webkit context. Now we want to build, package and deploy our demo app.

Build

1. Build using sencha cmd as usual. eg From your command prompt:

sencha app build production

2. You also need to copy some files to the production folder that sencha cmd doesnt know about. I have done this for you in the demo files but if you have changed anything you will need to do this yourself. Copy these to the app root of the production build – the same folder as index.html
node-modules folder
package.json
mydocs folder (our sample files to launch via your app)

You might want to edit the package.json file to set the “toolbar” property to false as an example of how you can have your production deployed version configured differently than your development version. You could also have different window constraints etc. And, of course, in your production app you would probably turn off the debugging window! ie in package.json set toolbar:false and remove the code from Applications.js.

3. Test the production version by double clicking the nw.bat file in the production build app root. (Again, adjust the path in the batch file to suit your NWK install).

Package

There are a number of ways to package your app in node-webkit for Windows, OSX and Linux. I developed a batch file to do this for Windows which you will find in the included build/production/AppNWK folder.

For my system you need 7-zip since I am using it’s command line options to automate the process. So get that and install it if you dont already have it.

1. Copy the required files from your node-webkit installation to the AppNWK/nwFiles folder. You only need to do this once (but update these if you install a new version of node-webkit). The files currently in that folder are the ones that are required to be packaged with Windows apps for v0.10.2 of node-webkit. I left them there as a guide.

2. Double click the package.bat batch file. This will copy the production build app files to the AppNWK folder and package it all into a zip file called nwkTest.zip.

3. The resulting nwkTest.zip file is your packaged app. Create a new folder somewhere, unzip the file and double click the nwkTest.exe file to launch your app!

Part 3 shows how to install extra NodeJs modules using the npm (Node Package Manager)

Packaging an ExtJS4 application using Node-Webkit part 1

In a recent post I explored whether node-webkit could be used to package an ExtJS4 application for the desktop. I found that it could. I then went on to package the app that I am currently building which has dozens of js files. It is all working really well!

This post details the process I am using by starting with “sencha generate app App” and ending with a packaged nwkTest.exe standalone. It includes installation instructions as well as how to debug etc etc. Some caveats:

  1. I have only done this for Windows. OSX and Linux users could adapt what I am doing and it would be great if you could report back the amended steps.
  2. My idea was to build a “hybrid” app – one that will work in a normal browser and have extra functionality when run in the node-webkit (NWK) package – eg access to the local file system. You dont have to do that – you could easily build a NWK only version.
  3. This tutorial will not teach you about ExtJS, ExtDirect or NodeJs, and only a little about node-webkit itself. It is designed for those that are on top of those technologies and who want to package their apps for the desktop.
  4. I use ColdFusion server-side so you might need to adjust the simple CF scripts to suit your server architecture if you want to amend the demo.
  5. There are a few “hacks” to keep the demo simple that you might not do in a production version.

Introduction

If you dont know anything about node-webkit you might find the info in my first post useful. If you have suggestions for improvements in what follows I am keen to hear them. For this tutorial I am using node-webkit version 0.10.2 and ExtJS 4.2.1.

All the files mentioned below are available here. There seems to be a lot to do when you read this post but it isnt as bad as it looks. I am being as detailed as I can so hang in there.

Step 1 – Generate an ExtJS4 app.

I will assume you know how to do this or you probably wouldnt be reading this post. For the exercise I used:

sencha generate app App {path to webroot}/nwktest

Nothing special here. Of course, make sure the default app is working in your web browser before proceeding.

 Step 2 – Install node-webkit

Download for your OS from GitHub.

Step 3 – Install NodeJs

Download NodeJs from here. I found the easiest was to install it to C:\nodejs to avoid issues of file permissions. See my previous post for details. When you install NodeJs you get the npm package manager as well. This is necessary so that you can install any extra node packages you might want. Again, see the previous post for more details.

It makes it easier if you add a PATH to your nodejs install folder eg C:\nodejs\; The instructions for this demo assume you have set such a path.

Step 4 – Copy the sample files to your nwktest app

Copy all the sample files to your test app. I will explain each of the extra files below. Note that the sample file set does NOT contain the ExtJS library or config files which is why you need to generate your app first. See step 1.

Step 5 – Test the browser version

At this point, you should refresh the browser version of the app to make sure everything works so far. The first three buttons should return a message that the operation was not allowed since these are all NWK specific operations.

The “RPC request using ExtDirect” works in the browser except that since you might not be using ColdFusion I have added two files that return hardcoded JSON responses to simulate a real remoting service. When running the app under NWK it uses a real remoting service on a live CF server.

The “Ajax request using Ext Ajax” will work since the test app has been configured to return a hard coded string from postto.cfm. If you wish to test with your own server side script, upload your script to your server then amend the Ajax request config in Main.js file to point to your script.

Step 6 – Test the demo using NWK

I explain all the ins and outs of this below. At this point however, you can run your app in NWK. What I do is use a batch file (included). Actually, I have two files: nw.bat and nwLocal.bat. The latter includes the commandline switch to tell my app to serve files from localhost. For more details on that, see app/Global.js below.

For now, edit these batch files and adjust them to suit your nwk installation folder.

Then, to run the demo, double click either one. If all is working you should see the familiar app in the node-webkit window and the debugging window open above it. Click the buttons in the centre panel to test. Which tests work depends upon whether you have elected to serve files from the local or remote server:

  1. If you elect the remote server (nw.bat) all buttons should work. The remote server is in Western Australia as the messages will indicate.
  2. If you elect to run using a local server (nwLocal.bat) you will obviously need to adjust the examples to suit your server, and possibly to point to your own remoting service if that is something you need to test. See Global.js and DirectAPI.js

Important info

1. Once you package your app using node-webkit, or execute it within node-webkit for testing, node-webkit uses the file:// protocol to load the *.js files (ie including your ExtJS app files). This means that any Ajax or ExtDirect type calls you make to your server to load the app itself (Ext.Loader), or to get data for your components, will be made in the context of the file:// protocol, not the http:// protocol. That is an obvious problem in terms of getting resources from a remote server. So, the solution I use is to make sure that all urls configured in your components are absolute rather than relative. ie they start with http:// (or https:// obviously). I configure that in my Global.js file which I explain below.

2. Local testing and sencha cmd builds: I go into this in more detail below. Suffice to say that you can easily run your unbuilt, unminified ExtJS app in node-webkit for testing and debugging. Indeed it has become my preferred way of testing since it is really fast to launch because the app files are loaded using the file:// protocol (no web server required). Then, for deployment (detailed below), you can easily include the production build of your app and resources in NWK.

3. Debugging: As you will see below I have included some code to automatically open the debugging window which is the Chromium debugger. Not everything works however. For example you cant get a menu by right clicking an item so you can’t “Replay XHR” in the Network tab. However, breakpoints, console.log() and the debugger; statement etc all work as normal. Breakpoints and debug settings survive an app reload but are all lost once you restart the app from the batch file (naturally enough).

Tip 1: On the main nwk app window toolbar you have 2 reload icons. The one on the right reloads the debugging window and if you click it you get an error because we are opening it automatically. Ignore that icon. The one on the left, however, reloads the app itself (which is what you want). So, if you have a bug in the initialisation code you can set a breakpoint then reload the app via the left reload icon.

Tip 2: If you get any exceptions thrown, you will need to restart via the batch file, not via a reload. The debugger “break on exception” works as long as the exception happens after you have been able to click the break button in the debugger. If not, you will need to use debugger statements to stop just before the crash point.

Tip 3: If your bug relates to an Ext file loader issue or syntax error (eg a missing bracket) you get very little info to help pinpoint your error since NWK crashes immediately. So, here I reload the unminified version of the app using a web browser where you do get helpful diagnostics to solve the problem, then go back to the NWK version to continue testing.

Descriptions of the files in the demo:

app/view/Main.js

The buttons you clicked above are in app/view/Main.js. This is the default file generated by sencha cmd to which I have added some buttons and handlers to the center region in order to test the functionality. Of course, in your real app you would use the standard MVC patterns which works fine. I just wanted to keep this demo as simple as possible.

Ext.define('App.view.Main', {
    extend: 'Ext.container.Container',
    requires: [
        'Ext.tab.Panel',
        'Ext.layout.container.Border'
    ],

    xtype: 'app-main',

    layout: {
        type: 'border'
    },

    items: [{
        region: 'west',
        xtype: 'panel',
        title: 'west',
        width: 150
    }, {
        region: 'center',
        xtype: 'tabpanel',
        dockedItems: [{
            xtype: 'toolbar',
            dock: 'top',
            items: [{
                    xtype: 'button',
                    text: 'About node-webkit',
                    handler: function(btn, e) {
                        var nwkVersion = 'This version of the app isnt using node-webkit';
                        if (NWK.process) {
                            nwkVersion = 'node-webkit version=' + NWK.process.versions['node-webkit'];
                        }
                        Ext.Msg.alert('About node-webkit', nwkVersion);
                    }
                }

                , {
                    xtype: 'button',
                    text: 'Launch a file',
                    handler: function(btn, e) {
                        if (NWK.gui) {
                            // Launch by extention. eg .txt .doc .xls etc
                            // ie the OS configured application for the relevant extention.
                            // Note that openItem('Test1.txt') will work if the file
                            // is in the app root folder. 
                            // BUT openItem('mydocs/Test1.txt') or openItem('./mydocs/Test1.txt')
                            // WONT WORK because NWK cant correctly resolve the path.
                            // Hence the use of the convenience property NWK.homePath
                            // which is set in resources/js/NWK-requires.js
                            NWK.gui.Shell.openItem(NWK.homePath + 'mydocs/Test1.txt');
                        } else {
                            Ext.Msg.alert('Problem', 'Sorry, you cant do that in a browser')
                        }
                    }
                }

                , {
                    xtype: 'button',
                    text: 'Browse docs folder',
                    handler: function(btn, e) {
                        if (NWK.fsexplorer) {
                            var startPath = NWK.homePath + "mydocs";
                            NWK.fsexplorer.readdir(startPath, function(err, path, details) {
                                if (err) {
                                    Ext.Msg.alert('Error', err.message);
                                } else {
                                    var msg = '';
                                    Ext.Array.forEach(details, function(file) {
                                        msg += Ext.String.format('<br>{0}: {1}, Size:{2}', (file.isDir ? 'Dir' : 'File'), file.name, file.size);
                                    })
                                    Ext.Msg.alert('Files in ' + startPath, msg);
                                }
                            });
                        } else {
                            Ext.Msg.alert('Problem', 'NWK.fsexplorer isnt found.')
                        }
                    }
                }

                , {
                    xtype: 'button',
                    text: 'RPC request using ExtDirect',
                    handler: function(btn, e) {
                        if (typeof(RPC) === 'object') {
                            // Ask the server for it's time using ExtDirect.
                            // This works for both browser and nwk versions.
                            // However, the nwk version needs the override in
                            // app/overrides/Connection.js
                            RPC.Util.getdate('dddd, d/m/YYYY', function(res) {
                                Ext.Msg.alert('Message', 'The server time is ' + res + '<br> ');
                            }, console);
                        } else {
                            Ext.Msg.alert('Problem', 'Sorry, remoting isnt available.')
                        }

                    }
                }

                , {
                    xtype: 'button',
                    text: 'Ajax request using Ext Ajax',
                    handler: function(btn, e) {

                        // Server side script. Start with relative path which will work
                        // for the browser version only if the script is on the same server
                        // as the app. ie the normal way of doing things in browser version, 
                        // use the local host or server host as appropriate.
                        var url = "postto.cfm";

                        // However, for NWK we need to fully qualify the URL
                        // so we are using a http/s: protocol instead of the local file: protocol
                        //if (NWK.isPresent) url = App.Global.getServerURL() + url;

                        // BUT, for this test app, we will fully qualify the browser version as well
                        // so you can test the browser version of the app without needing to 
                        // make an appropriate server side script.
                        url = App.Global.getServerURL() + url;

                        var dataToSend = Ext.JSON.encode({
                            test: 'test data'
                        });

                        Ext.Ajax.request({
                            url: url,
                            method: 'POST',
                            params: {
                                data: dataToSend
                            },
                            callback: function(options, success, response) {
                                Ext.Msg.alert('Ext Ajax', 'Request to<br>' + url + ' returned:<br>' + response.responseText + '<br> ');
                            }

                        });
                    }
                }


            ]
        }],
        items: [

            {
                title: 'Center Tab 1',
                html: 'Hello node-webkit demo app.'
            }
        ]
    }]
});

package.json

Node-webkit requires that you include a package.json file in the root of your app. There are many configuration options for this file. The following is a small example that we will use for this demo. One “gotcha” is that while you can put comments in this file as far as NWK goes, when you use the package manager (npm, see below) it throws an error if there are comments in the package.json file.

{
    "name": "nwkTest",
    "main": "index.html",
    "window": {
        "toolbar": true,
        "frame": true,
        "width": 1000,
        "height": 600,
        "min_width": 500,
        "min_height": 200,
        "max_width": 1400,
        "max_height": 800
    }
}

resources/js/NWK-requires.js

This file creates a namespace called NWK that helps keep your code tidy and allows you to check whether required node modules have been loaded. Note that you can call it whatever you want. In your app’s index.html file, include

<script src="resources/js/NWK-requires.js"></script>

as per the included example files.

/**
 * Node-Webkit detection and requires.
 * Create a global namespace for node-webkit.
 * Detect if node-webkit is running and set isPresent accordingly.
 * If NWK is present, require the nodejs modules you need and save them to the global namespace.
 */
var NWK = {
    // Just a convenient self-documenting property to know
    // whether we are in a node-webkit environment or not
    isPresent: (typeof(process) === 'object' && process.versions['node-webkit']) ? true : false
};



if (NWK.isPresent) {
    // Save the node web-kit process object to the namespace 
    // (for consistency using the NWK namespace in your app)
    NWK.process = process;
    // Tip: There is a lot of useful info in the process object.
    // console.log(process);

    // Use the namespace to hold the required modules
    // These come with node-webkit via NodeJs
    NWK.gui = require('nw.gui');
    NWK.path = require('path');
    NWK.fs = require('fs');

    // This is a sample custom node module for the purposes of this test app.
    // FileSystem explorer. See node_modules/fsexplorer.js
    NWK.fsexplorer = require('fsexplorer');

    // For convenience, save the path to the home folder of this app
    // using the path module.
    // There doesnt seem to be a consistent way to know the actual folder that 
    // the app is launched from. We need a different method depending on whether 
    // the app has been packaged or not. 
    // Hence this "hack":
    if (process.execPath.toLowerCase().search("nw.exe") == -1) {
        // This is what we need in the packaged app which was are assuming
        // is NOT launched from nw.exe
        NWK.homePath = NWK.path.dirname(process.execPath) + '/';
    } else {
        // This is what we need in the unpackaged app that we are 
        // assuming IS launched from nw.exe
        NWK.homePath = NWK.path.resolve("./") + '/';
    }


    // Do we want to use the localHost resources to test or use a live server?
    // Look for useLocalHost as a custom command line argument to nw.exe
    // which is set in the nwLocal.bat file.
    // If present, set the useLocalHost flag. You can then use that in your app
    // to use local host resources or live server resources for http requests.

    // Assume live server
    NWK.useLocalHost = false;

    // Get the command line arguments to the nw.exe 
    NWK.args = NWK.gui.App.argv;

    // If 'useLocalHost' command line parameter is present set the property to true
    if (NWK.args.indexOf('useLocalHost') != -1) NWK.useLocalHost = true;

    // See also app/Global.js
}

app/Global.js singleton

Often ExtJS developers end up with a Global singleton as a way to manage development and server environments. This app does that too. When using NWK we need to make some adjustments to what we would normally do for browser environments. You might have suggestions for a better way to handle this. You will probably need to adjust these properties to suit your server as indicated below.

// For testing in a browser. Are we running on localHost?
var onLocalHost = (window.location.host.substring(0, 4) == '127.' || window.location.host.substring(0, 9) == 'localhost');

Ext.define('App.Global', {
    singleton: true,

    config: {
        onLocalHost: onLocalHost,
        webHost: window.location.protocol + '//' + window.location.host,
            // Set these to suit your server:
        webRoot: (onLocalHost) ? '/sites/nwktest/' : '/nwktest/',
        serverURL: '' // Set in constructor below.              
    }, // config

    constructor: function(config) {
            this.initConfig(config);

            // Adjustments for node-webkit
            if (NWK.isPresent) {
                // Adjust the webroot etc based on whether 
                // we are using localhost to get server resources, or use the live server.
                // NWK uses the file system protocol so window.location.protocol
                // returns file:// rather than http:// and onlocalhost is irrelevant

                // Reset these since they wont be accurate when set above
                // for a browser

                // Do we want to access the localHost server or the live server?
                // See resources/js/NWK-requires.js for details
                this.setOnLocalHost(NWK.useLocalHost);

                // Set the web root 
                // Set these to suit your server:
                var webRoot = (NWK.useLocalHost) ? '/sites/nwktest/' : '/nwktest/';
                this.setWebRoot(webRoot);

                // Set the correct web host
                // Set these to suit your server:
                var host = (NWK.useLocalHost) ? 'http://127.0.0.1:8501' : 'http://murrah.com.au';
                this.setWebHost(host);
            }

            // Set the url of the server (localhost or live http or live https)
            this.setServerURL(this.getWebHost() + this.getWebRoot());

            // Now, when we need a url in our app (browser or NWK version),
            // we can use App.Global.getServerURL() 

            return this;
        } // constructor

});

app/Application.js

We need to add a few things here. Firstly, we require the relevant files. I will discuss the App.* ones separately. In the init() method below we can adjust the configuration of NWK (defined in package.json) to suit our development environment and there are some examples of NWK methods we can use to adjust the main window.

Ext.define('App.Application', {
    name: 'App',

    extend: 'Ext.app.Application',

    requires: [
        'Ext.window.MessageBox',
        'Ext.direct.*',
        'Ext.Ajax',
        'App.Global',
        'App.overrides.Connection',
        'App.DirectAPI'
    ],

    views: [],
    controllers: [],
    stores: [],

    init: function() {

        // If NWK.gui is defined we are running in the node-webkit environment
        // See packages/js/NWK-requires.js
        if (NWK.gui) {
            var win = NWK.gui.Window.get();

            // Open the debugging window if it isnt already open.
            if (NWK.useLocalHost && !win.isDevToolsOpen()) {
                var devWin = win.showDevTools('', false);
                // eg Move the debugging window to another monitor
                // devWin.moveTo(1940,50);

                devWin.moveTo(300, 250); // left, top
                devWin.resizeTo(1000, 600); // width, height
            }
            // eg changing the zoom level (equivalent of ctrl+ in browser)
            if (win.zoomLevel == 0) {
                win.zoomLevel = 1;
            }

            // eg move the main window
            // left, top
            win.moveTo(100, 50);

            // eg set the main window title
            win.title = "NWK-ExtJS package test";

        }
    }
});

app/overrides/Connection.js

This is an override that is necessary if you use Ext.Direct with NWK (which I normally dont use). For the purposes of testing whether NWK was a good candidate for packaging EXTJS apps I made a simple remoting service (thanks to Bruce Lomasky for his help). Since NWK is running on your local computer and you are trying to access resources on a remote server you run into cross origin issues. The override seems to solve those issues.

/**
 * Override Ext.data.Connection to enable cors in ExtDirect for node-webkit
 * Not used for AJAX requests.
 *
 */
Ext.define('App.overrides.Connection', {
    override: 'Ext.data.Connection',

    constructor: function(config) {
        config = config || {};

        // Override to set default parameters so that for node-webkit we can
        // POST to the live web server. 
        // ie Do a Cross Domain post via the normal ExtJS requests in ExtDirect.
        // Your server side remoting must set the "Access-Control-Allow-Origin" header.
        //  
        // eg for ColdFusion you must use something like:
        //      response = getPageContext().getResponse();
        //      response.setHeader("Access-Control-Allow-Origin","*");
        // or 
        //      <cfheader name="Access-Control-Allow-Origin" value="*"> 
        // 
        // Note: If you set these properties when running in a browser context (ie not node-webkit)
        // you can do cross domain posting in that context.
        // However, I couldnt yet get the ExtDirect to do a cross domain post in browser mode.
        // I dont normally use ExtDirect so my knowledge is sketchy here.
        // ExtDirect DOES work for node-webkit environments though, as well as for Ajax requests.

        if (NWK.isPresent) {
            config.cors = true;
            config.useDefaultXhrHeader = false;
        }

        Ext.apply(this, config);

        this.callParent(arguments);
    }
});

app/DirectAPI.js

Again, this is only required if you are using Ext.Direct. What this does is makes the initial call to the remoting service to get the RPC API. The extra bit for NWK is that we need to ensure that we are using absolute urls. For this demo in browser mode the remoting “service” is just a couple of JSON files pretending to be CFM pages. I did that so you can run the demo in a browser without needing to actually set up the service. For the NWK version, however, the remoting service is a real service on a live CF server so you can test that it actually does work.

Ext.define('App.DirectAPI', {
    requires: ['Ext.direct.*', 'Ext.Ajax']
}, function() {

    /* NOTES:
    For this to work in a Node-webkit context:
    1.  If you use relative paths in your urls, xmlHttpRequest 
    	(used by ExtJS to do Ajax calls (including remote)) will supply the
    	protocol. For browser environments the protocol will be http or https 
    	but for NWK it will be file://
    	So, the Ajax call will therefore try to locate the resource on the 
    	local file system, not the web server.
    	For that reason we need to fully qualify the url when using NWK and ExtJS.
    	Hence the use of the NWK.webRoot variable when setting the URL below
    	(see resources/js/NWK-requires.js)

    2.  Api.cfm returned data must be prefixed with an Ext.ns(); so the global 
    	namespace for the ExtDirect is created. eg we are expecting Ext.ns('RPC') 
    	in this example, which is what our Api.cfm defines.

    3.  In this test app: remoting wont work for the browser version unless 
    	you have implemented remoting
    	on the server that you are running the app on. 
    */
    var hostUrl = App.Global.getServerURL();
    var url = hostUrl + "servicebus/API.cfm";

    var Loader = Ext.Loader,
        wasLoading = Loader.isLoading;

    //Load the API config from the server
    Loader.loadScriptFile(
        // The url of the Api.cfm
        url,
        // onLoad listener 
        Ext.emptyFn,
        // onError listener
        function(errMsg, synchronous, scope) {
            console.log(errMsg);
            //throw errMsg
        },
        // Scope
        null,
        // Synchronous. We must have the Api config before we continue.
        true
    );

    Loader.isLoading = wasLoading;

    // For node-webkit all http requests MUST use the http protocol (ie not file://)
    // so we are fully qualifying the Router's url. 
    // eg instead of just 'servicebus/Router.cfm' it needs to be something like:
    // 'http://mywebserver/myapp/servicebus/Router.cfm'
    // See comment above.
    if (NWK.isPresent) {
        // Get the url of the remote service
        var directUrl = RPC.REMOTING_API.url.toLowerCase();

        // Set the fully qualified url to the service		
        if (directUrl.indexOf(hostUrl.toLowerCase()) == -1)
            RPC.REMOTING_API.url = hostUrl + directUrl;
    }

    // If we have a valid RPC config,
    // pass the remoting config to the ExtDirect manager
    // whether or not in the browser or NWK environment
    if (typeof(RPC) === 'object') Ext.direct.Manager.addProvider(RPC.REMOTING_API);
});

node_modules/fsexplorer.js

node-webkit uses nodejs, so you can use pretty much any NodeJs module in your app. There is a repository of NodeJs modules here: npmjs.org which you can install using the package manager that comes with NodeJs  (npm). More on that below.

Or, you can write your own packages. I have supplied a very simple, bare bones package that I wrote that reads the contents of a local file system folder. It is just intended to show how as simply as possible. It is not a good example of best practice!

/**
 * A custom sample node module that handles native file interactions.
 *
 * This is a bit quick and dirty. It is just as a simple example of what a
 * custom node module can look like so we have something to use for our test.
 *
 */
// Require the NodeJs fs (filesystem) module
// See http://nodejs.org/api/fs.html
var fs = require('fs');

function FSExplorer() {

    /**
     * Read the contents of a local folder / directory.
     * Simple example. Does not recurse.
     */
    function readdir(path, callback) {

        // Start by getting the status of the folder 
        fs.lstat(
            // The path to the folder
            path,
            // The callback. Err will be null if no errors
            function(err, stat) {
                if (err) {
                    return callback(err)
                }
                // Have we got a directory ?
                if (stat.isDirectory()) {
                    // Ok, read the directory
                    fs.readdir(
                        path,
                        function(err, files) {

                            // Get the data for files and folders in this directory 
                            var fileDetails = [];
                            files.forEach(function(file, i) {
                                    var abspath = path + '/' + file;
                                    var fileStat = fs.statSync(abspath);

                                    var details = {
                                        name: file,
                                        size: fileStat.size,
                                        created: fileStat.birthtime,
                                        modified: fileStat.mtime,
                                        isDir: (fileStat.isDirectory())
                                    }

                                    fileDetails.push(details);
                                })
                                // Return the details as an array
                                // along with the path we started with
                            return callback(null, path, fileDetails);
                        }
                    );

                } else {
                    return callback(new Error("path: " + path + " is not a directory"));
                }
            }
        );

    }

    this.readdir = readdir;
}

module.exports = new FSExplorer;

Step 7 – Building and packaging your app

Ok. At this stage you should have a working test version. Time to actually build it and package it. See Part 2.

Some final comments:

  1. While I have been using this system for a few weeks on a reasonably large app and have included everything I have learned in the process, there will of course be bits of ExtJS that I havent tested yet. If you find any such issues please post your adjustments.
  2. There might be some security issues that I dont know about when using this packaging method. Please post if you know of anything that needs to be taken care of.

Explorations of using node-webkit + ExtJS4 to build a desktop application

Since the demise of Sencha Desktop Packager and the non-arrival of any examples or clues about how TideKit might actually work I decided to have a look at node-webkit and see whether or not it would work as a desktop packager for ExtJS4. What I say below might work with ExtJS5 but I havent upgraded yet since it is still quite new.

Node-webkit is an executable (one each for Win, OSX, Linux). It is a chromium webkit browser wrapped by a “shell” that adds JavaScript libraries so that you can access the local file system, memory, native windows and menus, system tray, etc etc of the computer it is running on. Hence, it creates the opportunity to turn HTML5/JS/CSS into a desktop application. The extra JavaScript is provided by NodeJS (well the V8 JS engine actually, but via NodeJS libraries). Hence the name “node-webkit”.

The first observation is that there was a lack of any “Using node-webkit with ExtJS4″ information. Lots of little bits and clues but no overall approach. So, I thought I would share my exploration to make the trail easier for the next person.

The second observation is that you can make a desktop app with just node-webkit. However, unless you want to re-invent the wheel in terms of making Http requests or direct remoting (for example) you will also need to install NodeJS separately so you can use the NPM packager to easily download the various NodeJS libraries that you might want to use. There are nearly 90,000 of them!

And, of course, you will need to be familiar with ExtJS4 and Sencha Cmd. I wont be explaining any of that.

Useful resources

These are the things that helped me get my head around the concept:

https://www.youtube.com/watch?v=lNpjx4BJlH4 – A series of 6 videos for beginners. Very helpful.

https://www.youtube.com/watch?v=XmHh4RrlpIk – An introduction by Roger Wang, the guy that made node-webkit.

http://code.tutsplus.com/tutorials/introduction-to-html5-desktop-apps-with-node-webkit–net-36296

http://www.amazon.com.au/Windows-Desktop-App-Creation-node-webkit-ebook/dp/B00EEYFFVC/ – a short, inexpensive eBook that covers the main aspects clearly. Great for beginners. Even though the book is for Windows users, OSX and Linux users will also find the info useful.

(I will add more resources as I find them).

Installation

Installing node-webkit went without a hitch. Just download the installer for your OS from the GitHub page and follow the dots.

First tests

I then used the above mentioned resources to make my first desktop app. I followed the tuts introductory tutorial. That all worked without a hitch. Very excited by now.

With ExtJS4:
To test the ExtJS compatability I created a new ExtJS app using Sencha Cmd generate, built the default production app, then ran it in node-webkit and ….. it worked! More excitement.

Some learnings here:

1. To test your app you basically pass the folder containing your app (ie the folder with the main html file in it) to nw.exe. You *can* drag and drop it but I found the fasted way was to create a batch file (I am using Win7) in the same folder as the app and then run that. The batch file contents (for my machine) looks like:
..\..\node-webkit-v0.10.2-win-ia32\nw ./
which basically says (go back up to the folder containing the nw.exe and run it with the current folder as the parameter. You could also add the node-webkit folder to your PATH (on windows) to avoid needing the path in the batch file.
Edit: And/or of course maybe rename the folder to (say) NW so the folder or path doesnt need updating when you update your version of node-webkt!

Executing nw opens the window and displays your app (as you would have seen in the video series above.

2. Note that my initial testing built the ExtJS app first then passed that to NW. Also at this stage there is no NodeJS library being used, It is just my ExtJS app. It gets a bit more complicated when you want to pass the unbuilt version (so you can debug your app via the NW chrome debugger). More on that below – I am still exploring all that.

NodeJS

Now, installing NodeJS so I can require the appropriate extra libraries. This wasn’t as simple initially. The default installation folder is C:\Program files\nodejs. That seems fair enough. However, even though I am logged in as Adminstrator on my computer file permission problems meant that I couldn’t run npm (the package installer). See this post for details of errors. My solution was to uninstall NodeJS and reinstall in C:\nodejs. That worked perfectly and I could run npm without errors.

Using NodeJS packages (modules)

To use the downloaded NodeJS packages you need to install them in the node_modules folder at the root of your app. It must be called node_modules. So, one way to install a module is to run npm install {package name} from a command line in the root of your app’s folder. The node_modules folder will be created if necessary and the module will be saved in that folder. For me I just ran  C:\nodejs\npm install httpreq from a terminal / command line (to load the httpreq module, for example).

More tests: Adding Node to ExtJS4

This is preliminary. Starting with the default ExtJS app generated by Sencha Cmd I added the following:
1. The require() statements which I placed above Ext.define(‘App.view.Main’, {:

var gui = require('nw.gui');
var httpreq = require('httpreq');

2. a toolbar to the centre Tab panel with some test buttons

{
        region: 'center',
        xtype: 'tabpanel',
        dockedItems: [{
            xtype: 'toolbar',
            dock: 'top',
            items: [{
                xtype: 'button',
                text: 'Open file',
                handler: function(btn, e) {
                    // Test opening a Word Doc
                    console.log('Pressed open file');
                    gui.Shell.openItem('Test1.doc');
                }
            }, {
                xtype: 'button',
                text: 'Http request',
                handler: function(btn, e) {
                    // Test making a http request
                    // Using https://www.npmjs.org/package/httpreq
                    var url = "http://www.google.com";
                    console.log(url);
                    httpreq.get(url, function(err, res) {
                        if (err) return console.log(err);
                        console.log(res);
                    });
                }
            }]
        }],
        items: [

            {
                title: 'Center Tab 1',
                html: 'Hello first app'
            }
        ]
    }

I then built the production app and loaded it into NW as described above and yes! it works!

Next steps

My next task is to work out the best way to integrate ExtJS4 and node-webkit so that it is possible to run the uncompressed version of my app in NW so I can debug it using the chrome developer tools in NW. I will report back when I have done that!

Edit: Done. See Part 1

CFPOP – deleting an email when there is a comma in the UID

It seems that a problem has existed with the CFPOP delete action for a number of versions and it has not been fixed. The problem occurs when trying to delete a single email (or a list of emails) from the mail server where the mail server has included a comma in the UID. eg “1382396111.12731.mydomain.net,S=965″

The CF docs for the CFPOP Delete action says: “UID or a comma-separated list of UIDs to get or delete. Invalid UIDs are ignored.”

So, the abovementioned UID gets split into 2 uids (notice that it has a comma in it) and of course the email is not located so cant be deleted. CFPOP does not include a DELIMITERS parameter to use with the UID paramater which would solve the problem, I think. eg

 
<cfset myUIDlist = "1382051456.15221.mydomain.net,S=965|1382051456.18000.mydomain.net,S=966" />  
<cfpop server = "********" username = "********" password = "******** 
	action = "Delete" uid="#myUIDlist#" delimiter="|" >

Fortunately, there is a solution – use POP CFC instead of CFPOP! Thanks to Paul Vernon.

So, here is a sample page to demonstrate a way to use POP CFC.

<cfscript>
/**
 * Example of reading, processing then deleting individual emails from the pop mail server
 * using the POP CFC custom component.
 * http://popcfc.riaforge.org/
 */

	// To make it easier to rescope this example (local, variables ... whatever)
variables.s = {};

			// Initialise the custom POP CFC component. Used instead of CFPOP
			// Set your values here
s.popAccount = createObject("component", "pop").init(myPopServerHostname, myUsername, myPassword)

			// Specify a non-comma separator for the UID lists. 
			// You must do this in order for the list functionality
			// of POP CFC to work (including deletions)
s.mySeparator = "|";
s.popAccount.setProperty("separator",s.mySeparator);

			// Get an array of the UIDs of the messages in the mail account.
			// ie retrieve them from the mail server
			// This is a necessary step to solve the comma-in-uid problem
s.UIDArray = s.popAccount.getUIDList();

if (arrayLen(s.UIDArray)) {
			// Convert the array to a list
			// Keep the list to use for deletions
	s.UIDList = ArrayToList(s.UIDArray, s.mySeparator);

			// Get an array of the raw messages - one entry per UID supplied
			// This gets the raw email data from the pop server for whichever
			// emails are in the UIDList
	s.msgSrcArray = s.popAccount.GetRawMessageByUID(s.UIDList);

			// Turn the array into a query of email structures
	s.messagesQry = s.popAccount.parseRawMessage(s.msgSrcArray);

			// Loop over the array of email messages
	for( s.i=1; s.i <= s.messagesQry.recordcount; s.i=s.i+1) {

			// Process the message as you wish
			// ....
		
		writeOutput("<br/>" & s.messagesQry.subject[s.i]);


			// Now, delete this email from the server 
			// Note: you can also delete all the emails in your query 
			// in one go after the loop. See below.

			// The UID is NOT present in the messagesQry. Therefore,
			// you need to look up the UID by list position.
			// Obviously, you can also selectively delete single messages 
			// based on your criteria.

		s.uid = listGetAt(s.UIDList, s.i, s.mySeparator);
		s.popAccount.deleteMessagesByUID(s.uid);

	}

			// Uncomment to delete all messages in the list in one go, 
			// instead of one by one as above
	//s.popAccount.deleteMessagesByUID(s.UIDList);
}

</cfscript>

POP CFC has lots of other useful functions too.

Cheers,
Murray

Sublime Text – exclude / include search folders using regex

I recently switched from Eclipse to Sublime Text 3 and am very happy I did. It is fast to load, lots of extensions, active community.

One thing that I couldnt figure out was how to exclude folders from the search within files (project). I mostly code in JS (Sencha Ext/Touch) and ColdFusion and keep my JSDuck API Docs etc in the same project folder as my source files. That means that when I search within files in my project I end up with matches from the docs folder, which is not what I want. Also, and probably more importantly, when using Sencha Cmd to build production apps there is a /build folder which I do not want to search in since that is the “compiled” output.

There is a forum post about using -/my/path/to/folderToExclude (ie prefix the path with a – ). http://sublimetext.userecho.com/topic/97052-find-in-files-exclude-directories/

For Windows (which I use) you need backslashes instead of forward slashes and the syntax becomes odd (IMJ). At the end of that post was an example of using regex in the Where input box. This was a revelation.

So, to exclude the various folders within my project that I never want to search in, my where box looks like:

-*/docs/*,-*/build/*,-*/ext/*,-*/sass/*

which means search in all folders and subfolders in the project except these ones. Brilliant!

By the same method you can be specific about which folder to include. eg:

*/app/* will find a match in all “app” folders (and sub folders of course) eg myappname/app and myappname/build/app. However I want the first one but not the second one (build). So:

*/myappname/app/*

does the trick, only searching in folder (and sub folders of) myappname/app.

And, of course you can combine exclude and include:

*/myappname/app/*,-*/controller/*

ie only search in myappname/app/ but NOT in myappname/app/controller (or any sub folder with ‘controller’ in the path)

Finally, dont forget that the Where box is cached and previous contents are available by clicking the down arrow at the end of the input box. So, you can easily switch between saved search strings.

All very nifty!

ColdDuck – beautiful documentation for ColdFusion CFCs

Background

I have been a consumer of the Sencha ExtJS documentation for a while now and wanted to be able to use JSDuck for my ColdFusion CFCs. But how? While I am a JSDuck novice I figured that trying to make JSDuck produce documentation directly from ColdFusion CFCs was likely to result in tears (mine!). So, after some thought, I came up with another strategy that users of other programming languages might want to consider.

My CF to JSDuck strategy

Mark Mandel’s ColdDoc can be extended to produce different kinds of output by utilising a “strategy” cfc. It comes with one that produces the JavaDoc style HTML format. Strategies utilise the CF ComponentMetaData. So, I made my own strategy that takes ColdFusion CFCs and makes a pseudo-app in JavaScript code that is annotated using JSDuck formatting. Of course it auto-documents functions etc and picks up the ‘hint’ attributes where it finds them and uses all that to produce the raw material for rich documentation. It also works with CF ORM CFCs (although this part could be made even richer than it is at present).

Video overview

Here is a really quick video to give an overview of what ColdDuck does: http://www.screencast.com/t/5p4JqbvNR

Documentation

The ColdDuck package includes the documentation. You can also view it here http://murrah.com.au/coldduck/docs

Sample output

Here is the sample SuperBlog app documented with ColdDoc in JavaDoc format

Here is the sample SuperBlog app documented with ColdDuck in JSDuck format

A full implementation of JSDuck style documentation is here http://docs.sencha.com/extjs/4.1.3/

Almost all of that JSDuck functionality is available for you to use for your ColdFusion projects. I say “almost” because there are a few JSDuck features that are JavaScript specific (eg view JS class source).

Installation

Get it at GitHub https://github.com/murrah/ColdDuck

I hope you find this useful and fun. Please leave your comments below.

Thanks,
Murray