The Karma Project: Code Less, Teach More

January 15, 2010

Get Started with Jake, a Rake clone in JavaScript

Filed under: News — Tags: , , , , — bryanwb @ 8:33 am

In my last post, I gave an introduction to narwhal, a standard library for JavaScript. This time I will explore Jake, a Rake clone in JavaScript. I have been using it for the repetitive tasks in the Karma project like generating documentation, cleaning up temporary files, and checking out the latest version and packaging it for deployment. You may want to take a look at my Jakefile before you work through this tutorial.

What Jake doesn’t have

  • XML files
  • Special syntax

Jake only has pure JavaScript. This is a good thing. (mental) Context switching is expensive, particularly for utility scripts you might only look at once every few months.

Why use *-ake?

Managing a software project, especially web development projects, I find myself needing to do a number of repetitive and mind-numbing tasks that often, but not always, need to be done in a certain order. When packaging a new release of Karma, I usually want to check out the latest code from several repositories, put all the files in a temporary directory, remove unnecessary files, minify the source, and copy common JavaScript and CSS files to a common directory. Sometimes, I just want to check out the latest code. Sometimes I just want to minify the source and not move around the JavaScript and CSS files.

Here are the Tasks

  1. Check out files from version control
  2. Copy files to a temporary build/ directory
  3. Copy Common JS and CSS to a common directory, change references to new paths
  4. Remove files not needed for deployment to the server like docs, tests, utilities
  5. Minify JS and CSS files
  6. Optionally, compress project files into a tarball or zip file

Here is how those same tasks map to Jake tasks

  1. Check out files from version control ==> jake checkout
  2. Copy files to a temporary build/ directory  ==> jake build-dir
  3. Copy Common JS and CSS to a common directory, change references to new paths  ==>jake move-common-files
  4. Remove files not needed for deployment to the server like docs, tests, utilities ==> jake repack
  5. Minify JS and CSS files ==> jake minify
  6. Optionally, compress project files into a tarball or zip file  ==> jake package

Now some of these tasks depend on previous tasks so I shouldn’t be able to run them unless the previous tasks have been completed. But if they have been completed, I don’t want to rerun them. This is what buildmasters call <em>build dependencies</em>.

There is a task I haven’t yet specified, <code> jake build </code> which depends on tasks 1-6. I will show you an example build task later in the tutorial. Now let’s go back to creating basic tasks.

First Steps with Jake

Next, let’s set up jake. Narwhal has a package manager called tusk, which is comparable to ruby’s gem and Python’s easy_install.

$ tusk install jake
you@computer$ jake
No Jakefile found (looking for: jakefile, Jakefile, jakefile.js, Jakefile.js, jakefile.j, Jakefile.j)

Jake, like its siblings Rake and Make, requires a Jakefile that contains jake tasks.

Here is a very simple Jakefile

//import the jake module
//This is equivalent to python's "import" statement
var  JAKE = require("jake");
// JAKE.task('taskname', [ dependencies ], function(){});
JAKE.task('hello', function(){
      print('hello world');
});

Like Rakefiles, Jakefiles contain pure JavaScript. You should see a number of commands that are specific to narwhal. It is beyond the scope of this tutorial to cover those. I recommend you take a look at the narwhal documentation here.

To execute this simple hello task, run the following in the same directory as the Jakefile

$ jake hello
hello

Now let’s do something more interesting. I frequently need to regenerate the documentation for karma.js using a long shell command that I can never remember all the switches for. I use the excellent jsdoc-toolkit.

JAKE.task('docs', function(){
        var path = './tools/jsdoc-toolkit';
         if(FILE.exists(path)){
             var cmd = 'java -jar ' + path + '/jsrun.jar ' +
                 path + '/app/run.js ' + './js/karma.js -d=docs/ ' +
                 '-t=tools/jsdoc-toolkit/templates/jsdoc/';
             OS.system(cmd);
         } else {
             print("The folder ./tools/jsdoc-toolkit isn't present " +
                   "you need it to generate documentation");
         }

});

May prefer passing an array to OS.system rather than a string.

var cmd = [
   'java', '-jar', path + '/jsrun.jar',
   path + '/app/run.js', './js/karma.js',
   '-d=docs/', '-t=tools/jsdoc-toolkit/templates/jsdoc/'
];
OS.system(cmd);

One task that I run quite quickly is “jake clean” which I use to get rid all the temporary files created by text editors. Importing the submodule jake/clean automatically adds the “clean” task to your Jakefile. The next step is to add regular expressions that I want cleaned. The syntax for regular expressions here is not what I am used to. If may look odd to you too.

var CLEAN_LIB = require('jake/clean');  //adds the tasks clean and clobber
var CLEAN = CLEAN_LIB.CLEAN;
//include the temporary files created by text editors
CLEAN.include('**/#*#', '\.#*' , '**/\.tmp*',"**/\.*\.*\.swp");
CLEAN.exclude('\.git');    //don't touch my .git directory!

The regular expressions in the CLEAN.include() statement may look odd and that is because they are not regular expressions at all. Rather they are glob expressions.

Now, let’s run the clean task

$ jake clean

Here I tell the clobber task to delete my entire build-dir/ directory

var CLOBBER = CLEAN_LIB.CLOBBER;
CLOBBER.include('**/build-dir');
CLOBBER.exclude('\.git');

Often during a build process you find yourself checking if a particular file or directory exists and creating it if it doesn’t. Jake’s filedir() method is great shorthand for this. JAKE.filedir() which runs the task’s action if a) the target file (passed as the task name) doesn’t exist, or b) any of the dependencies’ timestamps are newer than the target’s timestamp.

filedir ("build-dir", ["debug", "release"], function()
{
    FILE.mkdirs("build-dir");
});

Now let’s go back to the 6 build tasks I outlined earlier in this article. Let’s create the build meta-task. We can specify dependencies for the build task as an array for the second argument to Jake.task. The dependencies will be run before the the build task if they are not up-to-date.

JAKE.task('build',['checkout', 'build-dir', 'move-common-files', 'repack', 'minify', 'package'],  function()
{
    /* do any finishing touches */
});

I have just scratched the surface of what Jake can do. You may want to look through my Jakefile for ideas. Cappuccino’s Jakefile is also a great example of what can be done with Jake.

Jake is a great tool and narwhal is a very useful platform. While quite young, narwhal is quite a stable and feature-rich platform. The biggest drawback that I have found working with narwhal is that documentation is quite lacking. Hopefully this will be resolved shortly.

Advertisements

Getting Started with Narwhal, a Standard Library for JavaScript

Filed under: News — Tags: , , , , , — bryanwb @ 2:38 am

I have been writing system administration scripts for a couple of years now, first with just regular Bash and later Python. In terms of bash scripting, I have to admit that while my invocations of my favorite tools, such as sed and find, have become more complicated my use of shell scripting programming statements has remained pretty basic. I don’t write shell scripts on a daily basis so in the weeks between writing scripts I manage to re-forget how to write a proper for loop and how to use getopts.

Sadly, this same applies to Python as I am no longer programming in Python on a regular basis. My old setup.py scripts are starting to look unfamiliar. Like a large and growing portion of web developers out there, I spend more than 90% of my time writing JavaScript and the other 10% a mix of PHP, Java, ruby, Python, etc. About 6 months ago I discovered the CommonJS project which seeks to create a standard library for JavaScript and Narwhal implementation of that developing library. There are many great uses of narwhal but my current favorite is the jake tool, which is clone of ruby’s popular rake tool. I have been using it for the repetitive tasks in the Karma project like generating documentation, cleaning up temporary files, and checking out the latest version and packaging it for deployment. I will cover Jake in my next tutorial.

“But JavaScript is a toy language!” you declare. “It isn’t meant for serious stuff like system administration.” Let me break it to you softly, by historical accident JavaScript may now be the most popular programming language and boasts some of the fastest run-times for a dynamic language. It isn’t any less-suited to utility scripting than its more august cousins Ruby and Python.

Setting up narwhal

Download and extract the http://github.com/280north/narwhal/zipball/master or http://github.com/280north/narwhal/tarball/master archive, or

$ git clone git://github.com/280north/narwhal.git

You should append the following text to your .bashrc file and then open a new terminal

export PATH=$PATH:~/narwhal/bin

Run “narwhal” or “js” (they are equivalent).

You need the java5 or java6 JDK to run narwhal which by default runs on top of Rhino, a JavaScript implemented on the Java JVM. Narwhal doesn’t play well with OpenJDK, so we need to get the JDK from sun. Below are instructions for getting Sun’s JDK and setting it up on Ubuntu Linux.

First add these lines to your /etc/apt/sources.list

deb http://us.archive.ubuntu.com/ubuntu/ jaunty multiverse
deb http://us.archive.ubuntu.com/ubuntu/ jaunty-updates multiverse

Then run

$ sudo aptitude update
$ sudo aptitude install sun-java6-jdk     # alternately, sun-java5-jdk

You may want to remove the extra lines from your sources.list after you have finished installing the package.

Next you may need to tell Ubuntu explicitly to use Sun’s JDK

$ sudo update-alternatives —config java

# There are at least 2 alternatives which provide `java’. 

Selection Alternative 

+ * 1 /usr/lib/jvm/java-6-openjdk/jre/bin/java 
2 /usr/lib/jvm/java-1.5.0-sun/jre/bin/java 

Press enter to keep the default[*], or type selection number:
# Choose #2

To test your install just run narwhal from the command line

you@computer:/$ narwhal
Rhino 1.7 release 3 PRERELEASE 2009 12 12
js> print("hello")
hello
js>

We can run our little “Hello World” as a script

// hello.js
print('hello world');


$ narwhal hello.js
hello

Narwhal has a number of command line options which you can list with narwhal --help

A Note on Narwhal Engines
Narwhal is a standard library for JavaScript, that can run on several different JavaScript implementations. So far it runs on top of Rhino, google’s V8, and Webkit’s JavaScriptCore. I believe that there are plans to run on top of the awesome node.js in the near future. You may find a Rhino a bit slow for your tastes but don’t let that put you off of Narwhal.

If you are using a Mac you can easily install the JavaScriptCore engine which is part of Webkit. It is approximately 10 times faster than rhino. I have not yet gotten it running successfully on linux. Narwhal has a package manager called tusk, which is comparable to ruby’s gem and Python’s easy_install.

$ tusk install narwhal-jsc
$ cd packages/narwhal-jsc
$ make
 # or "make webkit" if you want to use the jsc context from a webkit instance, 
# and thus have access to the DOM and other APIs
 

From here on, you can run narwhal-jsc by simply running “narwhal-jsc” or you can make it your default engine for narwhal by adding NARWHAL_ENGINE=jsc in your .bashrc or whatever file you use to control your bash shell settings.

That’s it for now, stay tuned for a tutorial on Jake, a rake clone. If you aren’t familiar with make or rake, they are excellent tools for automating repetitive administration tasks.

Create a free website or blog at WordPress.com.