Posts Tagged ‘data.gov’

March 9th, 2010 Comments Off
Tags: , , ,

Category: Articles

USAID Data Charted with Dojo

Dojo’s charting infrastructure has come quite a long way since I’ve looked at it several months ago, and since the first edition of Dojo: TDG didn’t include coverage of charting, I thought it might be fun to pick up an interesting data set from data.gov and wire together a modest demo. (And yes, the next edition of the book will definitely include coverage of charting and many other subprojects in DojoX.)

Click to view the demo

Click to view the demo

The data set I settled on is what appears to be a simple csv file of all USAID distributions. Since the spreadsheet was quite large, I coded up a small Python program to slice and dice the numbers in order to get familiar with the data. One of the first things I noticed is that the dollar figures all appear to be in “raw” form — meaning, that if there was a distribution in 1946 for $1,000 that figure is obviously not adjusted for inflation; it’s in “1946 dollars.” So, I harvested some CPI data from the Bureau of Labor Statistics and attempted to create a minimal inflation calculator so that I could get a better sense of the overall trends.

You can download my PyDev project if you are interesting in hacking on the data a little bit yourself. The one thing I’ll warn you about if you use the PyDev project in Eclipse is that you’ll probably need to change the project properties to point to your own modules, which may be installed into different locations. If you’re not familiar with PyDev, you may be better off just using easy_install to pick up any missing modules and run from the command line.

After I was satisfied that I understood the data reasonably well, I added a couple of functions to emit the CPI and USAID data as JSON for easy consumption by the ComboBox widgets that drive the chart.

It turns out that most of the demo is related to getting the proper data structures hacked out and managing the events that reload the chart. The code snippet to actually build the chart is pretty self-explanatory:

// create and configure the chart
chart = new dojox.charting.Chart2D("chart");
chart.setTheme(dojox.charting.themes.Minty);
chart.addAxis("x", {
    natural: true,
    includeZero: true,
    fixUpper: "minor",
    labelFunc : function(o) {
        return parseInt(o) + usaid.getMinYear()-1;
    }
});
chart.addAxis("y", {
    vertical: true,
    natural: true,
    includeZero: true,
    fixUpper: "minor",
    labelFunc : function(o) {
        return dojo.currency.format(o, {currency : "USD", places : 0});
    }
});
chart.addPlot("default", {type: "Columns", gap: 2});
//chart.addPlot("grid", {type: "Grid", hMinorLines: true, vMinorLines: true});
legend = new dojox.charting.widget.Legend({chart: chart}, "legend");

// add some nice action for the chart when you hover over columns
new dojox.charting.action2d.Highlight(chart, "default", {
    highlight: "green"
});
new dojox.charting.action2d.Tooltip(chart, "default", {
    text : function(o) {
        return ("FY" + (parseInt(o.x)+usaid.getMinYear()) + " : " +
               dojo.currency.format(o.y, {currency : "USD", places : 0}));
    }
});

The event-driven logic demo simply uses a couple of additional functions to remove and load a series into the chart though the Chart2D’s addSeries and removeSeries functions like so:

/* Add a series to the chart */
chart.addSeries("Afghanistan/Title II" 10000, {stroke: {color: "black", width: 1}});

/* Remove a series from the chart */
chart.removeSeries("Afghanistan/Title II");

The rest is of the work in putting this together really is just non-charting-related-book-keeping-cruft.

On a data related note, the one thing that kept coming to mind as I hacked this out is that it would be nice if there were a way to annotate the data.gov catalog with links to projects that slice and dice the data, transform the data into more useful forms, provide visualization tools for the data, etc. After all, the next logical step once you have the data is actually building stuff with the data. For example, if you wanted to hack on the USAID data too, you would probably benefit from the work I’ve already done in converting it to a useful JSON format, coding up an inflation calculator (that is driven by the CPI data I had to go find), and you might even want to improve upon the interactive chart that plots it all out.

If there is interest, I’m considering a series of follow-on posts where I’ll add some improvements to the chart like plotting multiple series, viewing normalized dollar totals by country, etc.