D3.js

How to make a d3js force layout stay within the chart area - even with multiple components

For a post on analysing networks of corporate control, I wanted to create some network graphs with d3.js. The new edition of Scott Murray’s great book on d3.js, which is updated to version 4, contains a good example to get you started. However, I was still struggling with some practical issues, as the chart below illustrates (reload the page to see the problem develop).

A large part of the graph drifts out of the chart area, and the problem only gets worse on a mobile screen. But I figured out some sort of solution.

As Murray explains, you can vary the strength value of the force layout. Positive values attract, negative values repel. The default value is –30. You could set d3.forceManyBody().strength(-3) to create a more compact graph.

Of course, the ideal setting will depend on screen size. You could vary the strength value according to screen width. While you’re at it, you may also want to vary the radius of nodes and the stroke-width of edges. For example with something like this:

if(w > 380){
    var strength = -3;
    var r = 3;
    var sw = 0.3;
}
else{
    var strength = -1;
    var r = 3;
    var sw = 0.15;
}

Now this may make the graph more compact, but it doesn’t solve one specific problem: components not connected to the rest of the chart will still drift out of the chart area. In my example, there are four components: a large one, and three pairs of nodes that are only connected to each other and not to the rest of the graph.

The way in which I dealt with this was to create four different graphs and attach the small components to a forceCenter at the margin of the chart area. For example, d3.forceCenter().x(0.1 * w).y(0.9 * h)) will put one of them in the bottom left corner. Here’s the result:

It’s still a lot of code - I can’t help feeling there should be a more efficient way to do this. Also, it’s slightly weird that the small components immediately freeze, whereas the large one takes its time to develop into its final shape. And the text labels could be improved. But at least it seems to work.

Embedding D3.js charts in a responsive website - a better solution

I often use D3.js to create charts which I embed on my website (the chart below is included merely as an illustration; it was copied from here). Normally you set the width and height of the embedded page in the embed code, but with a responsive layout it’s not so simple. The challenge is to adapt the iframe width to varying screen sizes and change the height so that the chart still fits.

After struggling with this issue for quite a while I thought I had come across the solution and wrote an article about it. However, this solution has two problems:

  • You have to define the aspect ratio in both the embed code and the D3 code; ideally, you shouldn’t have to do that in more than one place;
  • More importantly, it doesn’t take the height into account of any title and captions that are not part of the D3-created svg. You could handle this by making the title and caption part of the svg itself, but this is a bit awkward, especially with multiline captions.

A while ago, I came across a different approach which uses HTML5’s postMessage. The embedded page posts a message containing it’s own height to the parent page. The parent page picks up the message and changes the iframe height accordingly.

A smart variant has the embedded page not only send its height, but also its url to the parent page. That way, you can identify the corresponding iframe by its src attribute and thus make sure the right iframe gets updated - which is nice if you have more than one iframe on a web page.

Here’s how it works. In the D3 code, set the width of the chart to the width of the div the svg is attached to and use the aspect ratio to calculate the chart height. Also add the following code to the embedded page. It will send its height and url to the parent page:

function sendHeight() {
  var height = $('body').height();
  window.parent.postMessage({
    'height': height,
    'location': window.location.href
  }, "*");
}
 
$(window).on('resize', function() {
  sendHeight();
}).resize();

And here’s the code for the parent page. It will pick up the message, identify the corresponding iframe and update its height (note that Drupal requires jQuery instead of $):

window.addEventListener('message', function(event) {
    if (event.origin !== 'https://dirkmjk.nl') return;
    var data = event.data;
    var height = data.height + 32;
    jQuery('iframe[src^="' + data.location + '"]').css('height', height + 'px');
}, false);

In the second line, the domain should be replaced with the domain where the embedded page is hosted (the line checks for the origin of the posted message for security reasons).

I haven’t extensively checked this but it works on iOS and Android. Since it uses postMessage, it will not work on some older browsers. Then again, D3.js won’t work on some older browsers either.

Credits go to thomax and Jan Werkhoven.

Datawrapper’s policy on bad graphs

Datawrapper is a tool that lets you turn a dataset into a decent-looking chart within minutes. In an interview, co-founder Mirko Lorenz said Datawrapper is designed to prevent people making misleading graphs:

With Datawrapper, we try to make it as hard as possible to take data and create misleading charts with it. For example, it’s not possible to create bar charts with cropped axes. From time to time, users ask us to add this feature, but we never have and we never will. (via)

This may sound a bit paternalistic but it makes sense: Datawrapper’s philosophy is to offer a simple, robust way to quickly create a chart. If you don’t like the limitations, learn to code D3.js.

But Lorenz’ remark made me curious: would there be more design options, besides bar charts with cropped axes, that Datawrapper deems unacceptable? And are they limited to chart designs that are outright misleading, or do they more generally ban designs that result in ineffective or inaccurate data communication? Here’s an exploration of Datawrapper’s bad graph policy.

Y-axis not starting at zero
Datawrapper disapproves of y-axes that don’t start at zero in bar and column charts, but it allows them in line charts. I think this is consistent with the consensus on the topic.[1]

Spaghetti chart

I’m using the term spaghetti chart in the non-technical sense, meaning a chart with many lines that create an indecipherable mess.[2] Datawrapper doesn’t ban spaghetti charts.

Pie chart
Long the chart type we all loved to hate, the pie chart has recently been sort of rehabilitated. I think many people would now agree that pie charts are a legitimate way to represent proportions. That said, 3D and exploding pie charts are still suspect. Datawrapper allows pie (and donut) charts, but doesn’t seem to allow 3D or exploding pie charts.

3D
Using perspective to create a 3D effect will make it difficult to compare the sizes of elements in a chart. Fortunately Datawrapper doesn’t seem to allow any type of 3D chart.

Stacked bar chart
The rehabilitation of the pie chart coincided with a renewed critique of stacked bar charts: «basic bar charts are clearly better than pie charts, but stack them and they’re worse!». Which, by the way, doesn’t mean that it’s always wrong to use stacked bar charts.[3] Datawrapper allows them.

Dual y-axes
Some charts have have a secondary y-axis, so different scales can be used in one chart (here’s an awkward example, source). There may be situations where this is defensible, but in general it shouldn’t be considered good practice. Datawrapper doesn’t seem to allow this.

Pictograms instead of bars
Some designers try to jazz up bar charts using pictograms instead of bars, forgetting to take into account that if you double the height of the pictogram, its area increases fourfold. The distortion is even worse when the pictograms are drawn to appear three-dimensional. Datawrapper doesn’t seem to allow replacing bars with pictograms.


  1. The most well-known example of y-axes not starting at zero are cropped or truncated axes which start at a value higher than zero, but there are also examples of axes starting at a negative value. Edward Tufte points this out in The Visual Display of Quantitative Information, using a chart from an annual report as an illustration: «A careful look at the middle panel reveals a negative income in 1970, which is diguised by having the bars begin at the bottom at approximately minus $4.200.000».  ↩

  2. You can make a spaghetti chart interactive, for example let users click a label and the corresponding line will be highlighted. But this may still be an awkward solution, especially on mobile.  ↩

  3. «They can be useful when the point is to show that a value is the sum of other values, but you’re only interested in comparing the totals. They also work if you only need to show one section and can make that the one on the bottom. Then the bars are comparable and work well. But just throwing values into a stacked bar chart is a bad idea», Robert Kosara argued. Here’s how Dutch minister Jeroen Dijsselbloem messed up.  ↩

Is it still ok to ridicule pie charts

Workers without job security as a percentage of all working people in the Netherlands. The pink slice shows the percentage in 2003; the red slice how much this has increased since. Data Statistics Netherlands, chart dirkmjk.nl. Relaunch animation.

In a series of articles that caused a bit of a commotion among chart geeks, Robert Kosara summarised the findings of a number of studies on pie charts. In one of the articles, he observes:

Pie charts are generally looked down on in visualization, and many people pride themselves on saying mean things about them and the people who use them.

I guess I’m one of those people who look down on pie charts. Sure, I’m not as outspoken as the respected Edward Tufte, who famously wrote that «the only worse design than a pie chart is several of them». I’m not always against pie charts and I’ve even experimented with animated pie charts to illustrate change in a proportion. But I’m not above making lame jokes about pie charts either. My rule of thumb would be: don’t use pie charts - unless you can come up with a good reason why you should use one in a particular situation.

Kosara describes a number of studies in which he measured how accurately people interpret pie charts and other charts showing a proportion, e.g. 27%. According to his findings, exploded pie charts are doing worse than regular pie charts (phew!) and square pie charts are doing better. Interestingly, a stacked bar chart appears to be doing worse than a regular pie chart (note that a stacked bar chart depicting a single proportion amounts to something that looks like a progress bar).

It’ll be interesting to see how this holds up in future studies. But for now, the finding that (stacked) bar charts are doing worse than pie charts may come as a bit of a shock, for there appears to be a sort of consensus that bar charts are generally better than pie charts. Question is, better at what?

Workers without job security as a percentage of all working people in the Netherlands. Data Statistics Netherlands, chart dirkmjk.nl.

A bar chart is quite good at showing that the level of workers without job security in the Netherlands was higher in 2015 than in 2014. But which chart type is better at showing how much the share has increased between 2003 and 2015? Until recently I would have said «the bar chart» without hesitation, but now I’m not so sure anymore.

That said - I think it’s still ok to ridicule 3D exploded pie charts.

Robert Kosara summarises his findings here and here. The recent studies were done in collaboration with Drew Skau; an older study in collaboration with Caroline Ziemkiewicz. The Tufte quote is from his book The Visual Display of Quantitative Information. The charts above show workers with permanent jobs and a fixed number of hours per week, as a percentage of all working people in the Netherlands (not just employees), source CBS.

Embedding D3.js charts in a responsive website

UPDATE - better approach here.

For a number of reasons, I like to use D3.js for my charts. However, I’ve been struggling for a while to get them to behave properly on my blog which has a responsive theme. I’ve tried quite a few solutions from Stack Overflow and elsewhere but none seemed to work.

I want to embed the chart using an iframe. The width of the iframe should adapt to the column width and the height to the width of the iframe, maintaining the aspect ratio of the chart. The chart itself should fill up the iframe. Preferably, when people rotate their phone, the size of the iframe and its contents should update without the need to reload the entire page.

Styling the iframe

Smashing Magazine has described a solution for embedding videos. You enclose the iframe in a div and use css to add a padding of, say, 40% to that div (the percentage depending on the aspect ratio you want). You can then set both width and height of the iframe itself to 100%. Here’s an adapted version of the code:

<style>
.container_chart_1 {
    position: relative;
    padding-bottom: 40%;
    height: 0;
    overflow: hidden;
}
 
.container_chart_1 iframe {
    position: absolute;
    top:0;
    left: 0;
    width: 100%;
    height: 100%;
}
</style>
 
<div class ='container_chart_1'>
<iframe src='https://dirkmjk.nl/2016/embed_d3/chart_1.html' frameborder='0' scrolling = 'no' id = 'iframe_chart_1'>
</iframe>
</div>

Making the chart adapt to the iframe size

The next question is how to make the D3 chart adapt to the dimensions of the iframe. Here’s what I thought might work but didn’t: in the chart, obtain the dimensions of the iframe using window.innerWidth and window.innerHeight (minus 16px - something to do with scrollbars apparently?) and use those to define the size of your chart.

Using innerWidth and innerHeight seemed to work - until I tested it on my iPhone. Upon loading a page it starts out OK, but then the update function increases the size of the chart until only a small detail is visible in the iframe (rotate your phone to replicate this). Apparently, iOS returns not the dimensions of the iframe but something else when innerWidth and innerHeight are used. I didn’t have that problem when I tested on an Android phone.

Adapt to the iframe size: Alternative solution

Here’s an alternative approach for making the D3 chart adapt to the dimensions of the iframe. Set width to the width of the div that the chart is appended to (or to the width of the body) and set height to width * aspect ratio. Here’s the relevant code:

var aspect_ratio = 0.4;
var frame_width = $('#chart_2').width();
var frame_height = aspect_ratio * frame_width;

The disadvantage of this approach is that you’ll have to set the aspect ratio in two places: both in the css for the div containing the iframe and in the html-page that is loaded in the iframe. So if you decide to change the aspect ratio, you’ll have to change it in both places. Other than that, it appears to work.

Reloading the chart upon window resize

Then write a function that reloads the iframe content upon window resize, so as to adapt the size of the chart when people rotate their phone. Note that on mobile devices, scrolling may trigger the window resize. You don’t want to reload the contents of the iframe each time someone scrolls the page. To prevent this, you may add a check whether the window width has changed (a trick I picked up here). Also note that with Drupal, you need to use jQuery instead of $.

width = jQuery(window).width;
jQuery(window).resize(function(){
    if(jQuery(window).width() != width){
        document.getElementById('iframe_chart_1').src = document.getElementById('iframe_chart_1').src;
        width = jQuery(window).width;
    }
});

In case you know a better way - do let me know!

FYI, here’s the chart used as illustration in its original context.

Pages