Now that things are settling down from my Mining the Social Web hiatus, I’m hoping to find some folks from the Dojo community who would be interested in helping me to take an editing pass through the book, clean up outstanding errata, and get the book a little more up to date as part of an effort to transition maintenance of the book to the community through the Dojo Foundation. In other words, I’m trying to work out some arrangements that would provide a free online version of Dojo: TDG that’s continually updated and expanded as Dojo itself evolves. If you’re interested in helping, please get in touch!

An example of one of the zillions of charts you can generate

This implementation is part of a more involved magazine article on hacking open data that will be republished here once it’s been run its course in print.

From the advisory:

If you have Dojo 0.4 through Dojo 1.4 installed on your site, you are strongly encouraged to read all of this message and take immediate action.

We recently had a security review done on the Dojo codebase, and some issues were discovered. Most of the issues were in test files or related PHP files, but there were some issues discovered with a few files used by modules. You are strongly encouraged to remove some files immediately to give yourself the best protection. While we are not aware of any specific exploits, we take security issues very seriously and we encourage you to take quick preventative action.

Click to view the demo

The data set I settled on is what appears to be a simple csv file of all USAID distributions. Since the spreadsheet was quite large, I coded up a small Python program to slice and dice the numbers in order to get familiar with the data. One of the first things I noticed is that the dollar figures all appear to be in “raw” form — meaning, that if there was a distribution in 1946 for $1,000 that figure is obviously not adjusted for inflation; it’s in “1946 dollars.” So, I harvested some CPI data from the Bureau of Labor Statistics and attempted to create a minimal inflation calculator so that I could get a better sense of the overall trends.

You can download my PyDev project if you are interesting in hacking on the data a little bit yourself. The one thing I’ll warn you about if you use the PyDev project in Eclipse is that you’ll probably need to change the project properties to point to your own modules, which may be installed into different locations. If you’re not familiar with PyDev, you may be better off just using easy_install to pick up any missing modules and run from the command line.

After I was satisfied that I understood the data reasonably well, I added a couple of functions to emit the CPI and USAID data as JSON for easy consumption by the ComboBox widgets that drive the chart.

It turns out that most of the demo is related to getting the proper data structures hacked out and managing the events that reload the chart. The code snippet to actually build the chart is pretty self-explanatory:

// create and configure the chart
chart = new dojox.charting.Chart2D("chart");
chart.setTheme(dojox.charting.themes.Minty);
chart.addAxis("x", {
    natural: true,
    includeZero: true,
    fixUpper: "minor",
    labelFunc : function(o) {
        return parseInt(o) + usaid.getMinYear()-1;
    }
});
chart.addAxis("y", {
    vertical: true,
    natural: true,
    includeZero: true,
    fixUpper: "minor",
    labelFunc : function(o) {
        return dojo.currency.format(o, {currency : "USD", places : 0});
    }
});
chart.addPlot("default", {type: "Columns", gap: 2});
//chart.addPlot("grid", {type: "Grid", hMinorLines: true, vMinorLines: true});
legend = new dojox.charting.widget.Legend({chart: chart}, "legend");

// add some nice action for the chart when you hover over columns
new dojox.charting.action2d.Highlight(chart, "default", {
    highlight: "green"
});
new dojox.charting.action2d.Tooltip(chart, "default", {
    text : function(o) {
        return ("FY" + (parseInt(o.x)+usaid.getMinYear()) + " : " +
               dojo.currency.format(o.y, {currency : "USD", places : 0}));
    }
});

The event-driven logic demo simply uses a couple of additional functions to remove and load a series into the chart though the Chart2D’s addSeries and removeSeries functions like so:

/* Add a series to the chart */
chart.addSeries("Afghanistan/Title II" 10000, {stroke: {color: "black", width: 1}});

/* Remove a series from the chart */
chart.removeSeries("Afghanistan/Title II");

The rest is of the work in putting this together really is just non-charting-related-book-keeping-cruft.

On a data related note, the one thing that kept coming to mind as I hacked this out is that it would be nice if there were a way to annotate the data.gov catalog with links to projects that slice and dice the data, transform the data into more useful forms, provide visualization tools for the data, etc. After all, the next logical step once you have the data is actually building stuff with the data. For example, if you wanted to hack on the USAID data too, you would probably benefit from the work I’ve already done in converting it to a useful JSON format, coding up an inflation calculator (that is driven by the CPI data I had to go find), and you might even want to improve upon the interactive chart that plots it all out.

If there is interest, I’m considering a series of follow-on posts where I’ll add some improvements to the chart like plotting multiple series, viewing normalized dollar totals by country, etc.

Also as promised, I removed the PHP dependencies for some of the server stuff and replaced it with a simple python server that uses cherrypy. You should hopefully have no problems getting the sample server to run. You should have no problems finding stuff on the net to help you get that setup if you have problems, but feel free to comment here as well. Additionally be sure to skim the README in there.

You can get the materials here.

Thanks again to everyone who attended. I hope that you were exposed to some of the great tools that Dojo can bring to the user experiences you are creating and that these slides are sufficiently succinct that they are a help to you. While they don’t cover everything in my book, they really do cover a lot of it…more than enough to get you pretty darn proficient.

Finally, remember that we’re having the dojo.beer() tomorrow here in San Jose at 7pm. Please don’t miss it.

As I was finishing up my slides for the 3 hour Dojo tutorial at OSCON tomorrow, I remembered that I hadn’t posted the source for the Linux Journal article on Dojo’s fab DataGrid that ran in the June 2009 issue. Fortunately for me, Linux Journal posted the source online for this one, so I’ll just go ahead and point you there instead of reproducing that work. It’s laid out nicely, so I hope you enjoy.

If you’re at OSCON, please come out to the dojo.beer() on Wednesday at 7pm and hang out with some of us. I’m here all week, so feel free to look me up if you want to chat about anything.

Anyone want to get together for a dojo.beer() event at OSCON next week? Basically, a dojo.beer() event is just a little get together to talk some shop about what’s happening with Dojo, Ajax, the open web, etc. whilst refreshing oneself with dinner and a few beers in good company.

Any preferences for Wednesday, July 22 versus Thursday, July 23?

Recommendations for venue in the San Jose area?

Please, chime in below or throw out a holla to @dojotdg and/or @dojobeer.

It’s a bit more diverse than some drive-by-tutorials in that it digs into DojoX just a little bit and mentions the build system, so even if you’ve been using some of the core tools for a while now, you may still learn a few things from a quick read.

In between the time when the article was finished and now, Dojo has since updated to version 1.3 so you’ll unfortunately see a few CDN URLs that point to version 1.2; however, upgrading the URLs shouldn’t break anything. The latest and greatest is usually a great place to start, so I always encourage folks to stay up to date.

If you’re coming to OSCON this summer, be sure to drop by my 3 hour tutorial for a closer look at many of the great tools that Dojo provides.