More fun with arrays in protovis

In my short tutorial on working with data and protovis I briefly covered some standard javascript and protovis methods to work with arrays. The more I work with Protovis, the more I am convinced that efficient array manipulation is key to achieving just about anything with the library. So, I would like to go into more detail in some javascript methods for building, processing and testing arrays that can really be helpful.

Going through arrays: map and forEach

I said rapidly that the map method was very useful in protovis especially used in combination with pv.range. But that's not very fair to map() to be treated this lightly. Protovis canon examples do not use many traditional loops such as for or while statements. One reason for that is that many constructs in protovis are de facto loops: when we pass an array to protovis as a data file, to create a bar chart for instance (or panels, pie wedges, you name it), it will go through each element of the array to create individul bars (panels, wedges...), to position them, style them, and so on and so forth. This is why it is so important to have your data elements in the best possible shape when you first pass them to protovis. It makes the rest of your code much nicer. Remember our early example:
var categories = ["a","b","c","d","e","f"];
var vis = new pv.Panel()
  .width(150)
  .height(150);

vis.add(pv.Bar)
  .data([1, 1.2, 1.7, 1.5, .7, .3])
  .width(20)
  .height(function(d) d * 80)
  .bottom(0)
  .left(function() this.index * 25)
  .anchor("bottom").add(pv.Label)
    .text(function() categories[this.index]);

vis.render();
This is not ideal to have values and categories in two separate places, because one could be changed without updating the other. So let's try to use map to create one single variable.
var categories = ["a","b","c","d","e","f"];
var data=[1, 1.2, 1.7, 1.5, .7, .3].map(function(d,i) {return {value:d, key: categories[i]};});
Let's look at our map() method here. First, it's right after an array. It will run against this array, so it will perform an operation on each element of this array, and the result will be another array of the same size with the outcome of each operation in the same order. The next thing here is a function with two arguments: d and i. Again the naming is arbitrary, call them what you want. And they are both optional. But it has to be a function. pv.range(3).map(3) will not return [3,3,3], you need to write pv.range(3).map(function() 3). The first argument refers to the current item of the array map is working on. So 1, then 1.2, etc. If the array is more complex, and is an array of arrays or similar, the current element can be itself an array, an object or anything. It doesn't have to be a number. Here, we want to create an array of associative arrays where the value handle corresponds to the values of the array, and where key corresponds to the category name. So we start our output by "{value: d,". This puts the value of each array element in sequence where we need it to be. The second argument corresponds to the index of the current item in the array, so - 0, 1, 2 etc. This is not unlike using "this.index" in other parts of protovis. This helps us getting the right category name, the one in the same position as the value we are fetching. So we write "key: categories[i]}". The rest of the code can then be changed to :
vis.add(pv.Bar)
  .data(data)
  .width(20)
  .height(function(d) d.value * 80)
  .bottom(0)
  .left(function() this.index * 25)
  .anchor("bottom").add(pv.Label)
    .text(function() d.key);
vis.render();
Now how about forEach()? forEach works in a very similar way to map(), the difference is that it doesn't output an array. It's just a function that runs on each element of the array. It can be used to perform an operation a number of times corresponding to the length of that array, for instance.

Testing arrays

There are some times when you need to know whether some or all the elements of your array fulfill a condition. And some other times, you need to be able to extract a subset of your array also on a conditional basis. Now, that would be possible using forEach or map methods as above, but fortunately javascript provides simpler means to achieve that.

Testing a condition on an array at once

There are two methods that do that: every() and some(). every() will return true is the condition is true for, well, every element of the array. some() will return true if the condition is true for at least one element of the array. So, they can also be used to tell if the condition is false for at least one element of the array, or all elements of the array respectively. This is how they work:
[0,1,2].every(function(d) {return d;})
// will return false: 0 is false, 1 is true and 2 is true.
[0,1,2].every(function(d) {return (d<3);})
// will return true. All elements are less than 3.

[0,1,2].some(function(d) {return d;})
// will return true. 1 is true, so at least one element in the array is true.

Creating conditional subsets of an array

It is also possible to get only the elements that fit a condition using the filter() method.
[0,1,2].filter(function(d) {return d;})
// this will return [1,2]. 0 is evaluated as "false".
[1,2,3,4,5].filter(function(d) {return (d>2);})
// this will return [3,4,5].
Of course, the more complex the array, the more interesting these functions get. With the barley array from part 4:
 barley.filter(function(d) {return d.variety=="Manchuria";}
/* this will return: 
  [{"yield":27,"variety":"Manchuria","year":1931,"site":"University Farm"},
   {"yield":48.86667,"variety":"Manchuria","year":1931,"site":"Waseca"},
   {"yield":27.43334,"variety":"Manchuria","year":1931,"site":"Morris"},
   {"yield":39.93333,"variety":"Manchuria","year":1931,"site":"Crookston"},
   {"yield":32.96667,"variety":"Manchuria","year":1931,"site":"Grand Rapids"},
   {"yield":28.96667,"variety":"Manchuria","year":1931,"site":"Duluth"},
   {"yield":26.9,"variety":"Manchuria","year":1932,"site":"University Farm"},
   {"yield":33.46667,"variety":"Manchuria","year":1932,"site":"Waseca"},
   {"yield":34.36666,"variety":"Manchuria","year":1932,"site":"Morris"},
   {"yield":32.96667,"variety":"Manchuria","year":1932,"site":"Crookston"},
   {"yield":22.13333,"variety":"Manchuria","year":1932,"site":"Grand Rapids"},
   {"yield":22.56667,"variety":"Manchuria","year":1932,"site":"Duluth"}]*/

Visualizing arrays

(without plotting them, of course) When you are manipulating arrays, turning them into maps, performing roll-ups and sorts, you may want to get a glimpse of the array. But, unless it's a single, one-dimensional array, firebug or chrome debugger will represent it as a cryptic [ > Object, > Object, > Object ]. Not being able to follow step by step what's happening to an array makes understanding the data functions much more difficult. Fortunately, you can use the JSON.stringify() method.
JSON.stringify(barley)
/*returns: 
"[
{"yield":27,"variety":"Manchuria","year":1931,"site":"University Farm"},
{"yield":48.86667,"variety":"Manchuria","year":1931,"site":"Waseca"},
{"yield":27.43334,"variety":"Manchuria","year":1931,"site":"Morris"},
{"yield":39.93333,"variety":"Manchuria","year":1931,"site":"Crookston"},
{"yield":32.96667,"variety":"Manchuria","year":1931,"site":"Grand Rapids"},
{"yield":28.96667,"variety":"Manchuria","year":1931,"site":"Duluth"},
{"yield":43.06666,"variety":"Glabron","year":1931,"site":"University Farm"},
{"yield":55.2,"variety":"Glabron","year":1931,"site":"Waseca"},
{"yield":28.76667,"variety":"Glabron","year":1931,"site":"Morris"},
{"yield":38.13333,"variety":"Glabron","year":1931,"site":"Crookston"},
{"yield":29.13333,"variety":"Glabron","year":1931,"site":"Grand Rapids"},
{"yield":29.66667,"variety":"Glabron","year":1931,"site":"Duluth"},
{"yield":35.13333,"variety":"Svansota","year":1931,"site":"University Farm"},
{"yield":47.33333,"variety":"Svansota","year":1931,"site":"Waseca"},
{"yield":25.76667,"variety":"Svansota","year":1931,"site":"Morris"},
{"yield":40.46667,"variety":"Svansota","year":1931,"site":"Crookston"},
{"yield":29.66667,"variety":"Svansota","year":1931,"site":"Grand Rapids"},
{"yield":25.7,"variety":"Svansota","year":1931,"site":"Duluth"},
{"yield":39.9,"variety":"Velvet","year":1931,"site":"University Farm"},
{"yield":50.23333,"variety":"Velvet","year":1931,"site":"Waseca"},
{"yield":26.13333,"variety":"Velvet","year":1931,"site":"Morris"},
{"yield":41.33333,"variety":"Velvet","year":1931,"site":"Crookston"},
{"yield":23.03333,"variety":"Velvet","year":1931,"site":"Grand Rapids"},
{"yield":26.3,"variety":"Velvet","year":1931,"site":"Duluth"},
{"yield":36.56666,"variety":"Trebi","year":1931,"site":"University Farm"},
{"yield":63.8333,"variety":"Trebi","year":1931,"site":"Waseca"},
{"yield":43.76667,"variety":"Trebi","year":1931,"site":"Morris"},
{"yield":46.93333,"variety":"Trebi","year":1931,"site":"Crookston"},
{"yield":29.76667,"variety":"Trebi","year":1931,"site":"Grand Rapids"},
{"yield":33.93333,"variety":"Trebi","year":1931,"site":"Duluth"},
{"yield":43.26667,"variety":"No. 457","year":1931,"site":"University Farm"},
{"yield":58.1,"variety":"No. 457","year":1931,"site":"Waseca"},
{"yield":28.7,"variety":"No. 457","year":1931,"site":"Morris"},
{"yield":45.66667,"variety":"No. 457","year":1931,"site":"Crookston"},
{"yield":32.16667,"variety":"No. 457","year":1931,"site":"Grand Rapids"},
{"yield":33.6,"variety":"No. 457","year":1931,"site":"Duluth"},
{"yield":36.6,"variety":"No. 462","year":1931,"site":"University Farm"},
{"yield":65.7667,"variety":"No. 462","year":1931,"site":"Waseca"},
{"yield":30.36667,"variety":"No. 462","year":1931,"site":"Morris"},
{"yield":48.56666,"variety":"No. 462","year":1931,"site":"Crookston"},
{"yield":24.93334,"variety":"No. 462","year":1931,"site":"Grand Rapids"},
{"yield":28.1,"variety":"No. 462","year":1931,"site":"Duluth"},
{"yield":32.76667,"variety":"Peatland","year":1931,"site":"University Farm"},
{"yield":48.56666,"variety":"Peatland","year":1931,"site":"Waseca"},
{"yield":29.86667,"variety":"Peatland","year":1931,"site":"Morris"},
{"yield":41.6,"variety":"Peatland","year":1931,"site":"Crookston"},
{"yield":34.7,"variety":"Peatland","year":1931,"site":"Grand Rapids"},
{"yield":32,"variety":"Peatland","year":1931,"site":"Duluth"},
{"yield":24.66667,"variety":"No. 475","year":1931,"site":"University Farm"},
{"yield":46.76667,"variety":"No. 475","year":1931,"site":"Waseca"},
{"yield":22.6,"variety":"No. 475","year":1931,"site":"Morris"},
{"yield":44.1,"variety":"No. 475","year":1931,"site":"Crookston"},
{"yield":19.7,"variety":"No. 475","year":1931,"site":"Grand Rapids"},
{"yield":33.06666,"variety":"No. 475","year":1931,"site":"Duluth"},
{"yield":39.3,"variety":"Wisconsin No. 38","year":1931,"site":"University Farm"},
{"yield":58.8,"variety":"Wisconsin No. 38","year":1931,"site":"Waseca"},
{"yield":29.46667,"variety":"Wisconsin No. 38","year":1931,"site":"Morris"},
{"yield":49.86667,"variety":"Wisconsin No. 38","year":1931,"site":"Crookston"},
{"yield":34.46667,"variety":"Wisconsin No. 38","year":1931,"site":"Grand Rapids"},
{"yield":31.6,"variety":"Wisconsin No. 38","year":1931,"site":"Duluth"},
{"yield":26.9,"variety":"Manchuria","year":1932,"site":"University Farm"},
{"yield":33.46667,"variety":"Manchuria","year":1932,"site":"Waseca"},
{"yield":34.36666,"variety":"Manchuria","year":1932,"site":"Morris"},
{"yield":32.96667,"variety":"Manchuria","year":1932,"site":"Crookston"},
{"yield":22.13333,"variety":"Manchuria","year":1932,"site":"Grand Rapids"},
{"yield":22.56667,"variety":"Manchuria","year":1932,"site":"Duluth"},
{"yield":36.8,"variety":"Glabron","year":1932,"site":"University Farm"},
{"yield":37.73333,"variety":"Glabron","year":1932,"site":"Waseca"},
{"yield":35.13333,"variety":"Glabron","year":1932,"site":"Morris"},
{"yield":26.16667,"variety":"Glabron","year":1932,"site":"Crookston"},
{"yield":14.43333,"variety":"Glabron","year":1932,"site":"Grand Rapids"},
{"yield":25.86667,"variety":"Glabron","year":1932,"site":"Duluth"},
{"yield":27.43334,"variety":"Svansota","year":1932,"site":"University Farm"},
{"yield":38.5,"variety":"Svansota","year":1932,"site":"Waseca"},
{"yield":35.03333,"variety":"Svansota","year":1932,"site":"Morris"},
{"yield":20.63333,"variety":"Svansota","year":1932,"site":"Crookston"},
{"yield":16.63333,"variety":"Svansota","year":1932,"site":"Grand Rapids"},
{"yield":22.23333,"variety":"Svansota","year":1932,"site":"Duluth"},
{"yield":26.8,"variety":"Velvet","year":1932,"site":"University Farm"},
{"yield":37.4,"variety":"Velvet","year":1932,"site":"Waseca"},
{"yield":38.83333,"variety":"Velvet","year":1932,"site":"Morris"},
{"yield":32.06666,"variety":"Velvet","year":1932,"site":"Crookston"},
{"yield":32.23333,"variety":"Velvet","year":1932,"site":"Grand Rapids"},
{"yield":22.46667,"variety":"Velvet","year":1932,"site":"Duluth"},
{"yield":29.06667,"variety":"Trebi","year":1932,"site":"University Farm"},
{"yield":49.2333,"variety":"Trebi","year":1932,"site":"Waseca"},
{"yield":46.63333,"variety":"Trebi","year":1932,"site":"Morris"},
{"yield":41.83333,"variety":"Trebi","year":1932,"site":"Crookston"},
{"yield":20.63333,"variety":"Trebi","year":1932,"site":"Grand Rapids"},
{"yield":30.6,"variety":"Trebi","year":1932,"site":"Duluth"},
{"yield":26.43334,"variety":"No. 457","year":1932,"site":"University Farm"},
{"yield":42.2,"variety":"No. 457","year":1932,"site":"Waseca"},
{"yield":43.53334,"variety":"No. 457","year":1932,"site":"Morris"},
{"yield":34.33333,"variety":"No. 457","year":1932,"site":"Crookston"},
{"yield":19.46667,"variety":"No. 457","year":1932,"site":"Grand Rapids"},
{"yield":22.7,"variety":"No. 457","year":1932,"site":"Duluth"},
{"yield":25.56667,"variety":"No. 462","year":1932,"site":"University Farm"},
{"yield":44.7,"variety":"No. 462","year":1932,"site":"Waseca"},
{"yield":47,"variety":"No. 462","year":1932,"site":"Morris"},
{"yield":30.53333,"variety":"No. 462","year":1932,"site":"Crookston"},
{"yield":19.9,"variety":"No. 462","year":1932,"site":"Grand Rapids"},
{"yield":22.5,"variety":"No. 462","year":1932,"site":"Duluth"},
{"yield":28.06667,"variety":"Peatland","year":1932,"site":"University Farm"},
{"yield":36.03333,"variety":"Peatland","year":1932,"site":"Waseca"},
{"yield":43.2,"variety":"Peatland","year":1932,"site":"Morris"},
{"yield":25.23333,"variety":"Peatland","year":1932,"site":"Crookston"},
{"yield":26.76667,"variety":"Peatland","year":1932,"site":"Grand Rapids"},
{"yield":31.36667,"variety":"Peatland","year":1932,"site":"Duluth"},
{"yield":30,"variety":"No. 475","year":1932,"site":"University Farm"},
{"yield":41.26667,"variety":"No. 475","year":1932,"site":"Waseca"},
{"yield":44.23333,"variety":"No. 475","year":1932,"site":"Morris"},
{"yield":32.13333,"variety":"No. 475","year":1932,"site":"Crookston"},
{"yield":15.23333,"variety":"No. 475","year":1932,"site":"Grand Rapids"},
{"yield":27.36667,"variety":"No. 475","year":1932,"site":"Duluth"},
{"yield":38,"variety":"Wisconsin No. 38","year":1932,"site":"University Farm"},
{"yield":58.16667,"variety":"Wisconsin No. 38","year":1932,"site":"Waseca"},
{"yield":47.16667,"variety":"Wisconsin No. 38","year":1932,"site":"Morris"},
{"yield":35.9,"variety":"Wisconsin No. 38","year":1932,"site":"Crookston"},
{"yield":20.66667,"variety":"Wisconsin No. 38","year":1932,"site":"Grand Rapids"},
{"yield":29.33333,"variety":"Wisconsin No. 38","year":1932,"site":"Duluth"}
]"*/
No matter the manipulations you inflict to an array you will always be able to make it reveal its innards by using this.
 

One thought on “More fun with arrays in protovis

  1. Regarding Visualizing arrays, you can always use console.dir to get both Firebug and Chrome Dev tools (and IE with the console open, Opera DragonFly and Safari) to output all children of a given object. It’s also formatted more nicely than JSON.stringify.
    Ei. console.dir(barley)

Leave a Reply