Interactive charts - are Dygraphs or Plotly alternatives for D3?

There are quite a few Javascript libraries that you can use to create interactive graphs (with the added bonus that your graphs look crisp: somehow my PNG images always end up looking blurry). Some of these libraries are based on D3.js and are intended to make coding with D3 easier:

The sheer number of D3 based charting tools gives a good indication of how much people love the D3’s functionality, and yet actually hate coding with D3 directly.

Personally, I don’t hate coding with D3. Still, because I only use it sporadically, I’m constantly figuring out how things worked again (unsurprisingly, this involves a lot of code recycling and a lot of googling). So I wouldn’t be averse to an easier alternative for creating basic line graphs that don’t really require the advanced capabilities of D3.

The other day, I posted an article on how newspapers use the word illegal, which included two rather dense spaghetti graphs (I’m using the term in the non-technical sense of the word). I added dropdowns to link labels to lines. Labour relations scholar Maarten Hermans suggested I take a look at Dygraphs instead. A few days later, I read at Flowing Data that the Plotly.js library is being open sourced. So let’s give them a try (note that I haven’t checked how they do on mobile devices).


As far as I can tell, Dygraphs can only create line graphs. Below is a Dygraphs version of one of the spaghetti graphs from my article on the word illegal.

I have no problem with their somewhat dull colour scheme, although I’d probably change that. Note that you can click-and-drag to zoom in on a part of the chart and double-click to zoom out again.

This chart type isn’t really suitable for a graph with this many lines: when you hover your mouse over the graph, the labels obscure much of the graph. You could move the labels outside the graph area, but then you’d have to reserve quite a bit of space to accomodate them.

So I also did a version of a simpler graph, from an article in which I traced the use of the word machine for bicycle.

In this chart I made some modifications (line colour, position of the labels). I found that making these changes is relatively straightforward and well-documented. I’m not entirely happy yet with how the labels look, but other than that I think the graph looks pretty much OK.


Same approach with Plotly: here’s a version of the spaghetti graph from the article on the word illegal:

Plotly has similar click-and-drag / double-click functionality to zoom in and out as Dygraphs (it works slightly different in that you can also zoom in vertically). I think the way they show the labels to the right of the chart is OK. If you click them, the line associated with the label disappears and reappears, so you can easily find the line associated with a label. It would be even better if clicking a label would make the associated line toggle between grey and coloured.

I’m not happy with what happens when you hover your mouse over the graph. You get a menu to the top that offers functionality that strikes me as superfluous. And the labels that pop up are obviously a mess with a spaghetti graph like this one.

So here’s a Plotly version of the simpler bicycle graph:

That’s better, although I still haven’t figured out how to get the title to left-align (CSS doesn’t seem to work and here they only explain how to set font-family and size). More importantly, I still think there’s too much stuff popping up when you hover your mouse over the graph.


I can see myself using Dygraphs on occasion if I need a simple line graph. As for Plotly, I don’t like all the clutter in their charts and while I’m sure you can get rid of that, I’m afraid that would defeat the purpose of making it easier to create graphs.

Come to think of it, it seems that quite a few data visualisation libraries, and I’m not just thinking of Plotly here, haven’t really outgrown the «look at all the neat tricks I can do» phase. Until they move in the direction of Edward Tufte’s more minimalistic approach to data visualisation, I think I’ll rather keep struggling with D3.

Immigrants, filesharing and wiretaps: How newspapers use the word illegal

People should mind their language: an apparently neutral term like immigration has gotten xenophobic overtones as a result of its frequent use in combination with illegal, James Gingel argued in the Guardian. As an illustration, he pointed out that illegal, when typed into a Google search box, will likely get autocompleted to illegal immigrant or illegal immigration.

Earlier, the Guardian had been criticised for using the term illegal immigrant, among other things because it’s dehumanising. David Marsh of the Guardian Style Guide agreed. (The Style Guide itself takes a rather technical position on the matter: «… there is no such thing as an illegal asylum seeker … An asylum seeker can become an illegal immigrant only if he or she remains in Britain after having failed to respond to a removal notice.»)

Personally, I’d be in favour of reappropriating the term illegal immigrant - but it’s not for me to tell other people what strategy to use.

So how does the Guardian use the word illegal? I counted the words that follow the word illegal in their articles. I ignored stop words and in most cases I used stemming to lump together words like download, downloads, and downloading (see Method below).

The chart shows that the term illegal is most often used in combination with immigrant and variants. Other than that, it appears that illegal filesharing is a 2009 thing and that illegal phone [hacking] was an issue in 2011. Unsurprisingly, the expression illegal war started being used in 2003. By the way, what’s the status of that trial?

There’s also a bit of a peak in mentions of illegal thing in 2000. This can be attributed to a series of interviews in which one of the standard questions was «What was the last illegal thing you did?» The answers are somewhat boring, with the exception of «Shot a man in Reno just to watch him die» (a reference to Johnny Cash, of course).

The Guardian’s search API is largely limited to articles that appeared after 1998. For a longer term perspective, let’s turn to the New York Times, which offers access to the lead paragraphs of articles dating back to it’s origin in 1851.

That’s weird: expressions with the term illegal seem to have been rare until the 1970s. Either that, or I made an error in my analysis of the NYT data. I checked their own Chronicle tool, which confirms that the term illegal wasn’t used very much before the 1970s.

Again, the term illegal is mainly used in combination with aliens (1980s) and immigrants (2000s), but such uses seem to have dropped in the 2010s. My guess would be that this has to do with the growing importance of the «Latino vote», which means that politicians can no longer evoke negative images of immigrants without risking electoral consequences.

Speaking of vote: the expression illegal vote is one of the rare uses of the term illegal in the early days of the New York Times. Illegal voting appears to have been a recurrent concern in 19th century New York, as illustrated by a report from 1888:

Notwithstanding the widespread reports to the contrary and the wholesale issue of warrants for the arrest of illegal voters yesterday’s election in King’s County passed off without unusual excitement.

Tracking the use of the expression illegal strike provides an interesting insight into American social history: wildcat teachers’ strikes in the 1960s, broader public sector strikes in the 1970s and Reagan’s brutal standoff with air traffic controllers in the 1980s. Despite the progressive reputation it enjoys today, the New York Times often sided with law and order, for example in this 1962 report:

It was not a day New York City could be proud of. Half of the city’s 40,000 public school teachers had chosen an outlaw course and stayed away from their classrooms in an illegal strike. (If you’re wondering why public sector workers resorted to illegal strikes, read this article.)

The 1970s saw a modest peak in the use of the expression illegal wiretaps, often in connection with Watergate. In an article from 1974, the question was raised «whether President Nixon may have knowingly used claims of national security to cloak illegal wiretaps and other illegal surveillance». How modern.

So here’s my preliminary, non-scientific conclusion: newspapers appear to use the term illegal mainly to talk about immigrants, but when those in power really mess up, their actions will occasionally be called illegal too.


I used the search APIs of the Guardian and the New York Times to search for articles with the search term illegal. I counted the words following the term illegal, using the Python nltk library to exclude English stop words and to reduce words to their stem. A practical matter is that stemming will reduce both immigrant and immigration to immigr. Since some of the arguments against using the expression illegal immigrant do not apply to illegal immigration, it makes sense to differentiate between immigration and immigrant. Therefore, I separately counted occurances of the expression illegal immigrant[s]. Here’s the code.


Nieuwe term geleerd: dumbbell plot ofwel haltergrafiek. Een korte uitleg met voorbeelden is hier te vinden. Zonder de naam te weten had ik er zelf ook al eens een gemaakt, zie hieronder (hoort bij dit artikel).

Overigens wordt er in de discussie op Twitter verwezen naar een grafiek over verschillen tussen de salarissen van mannen en vrouwen naar gelang de Amerikaanse universiteit waar ze ingeschreven zijn geweest (hier en hier; gebaseerd op NYT?). Ik vind dat eerlijk gezegd niet zo’n sterke grafiek; doordat de x-as bij $60.000 begint, zie je niet groot het relatieve verschil is (hoeveel procent verdienen mannen meer dan vrouwen).

Vierde plaats in de NPO app-competitie

Sinds enige tijd kan je via een API programmagegevens van de publieke omroep opvragen en er is een app-competitie georganiseerd om de mogelijkheden van de API te promoten. Een heuse app is wat hoog gegrepen, maar een web app mocht ook. Dus toen ik naar aanleiding van dit artikel nog eens op de competitie werd gewezen dacht ik: what the heck, ik stuur gewoon iets op.

Hier is de app te vinden. En dit is de omschrijving waarmee ik hem heb ingediend:

De app gaat over de rol van de publieke omroep in de samenleving. Wat zegt het dat thema’s als TTIP of het basisinkomen pas relatief laat aandacht krijgen van de NPO, terwijl het zoekgedrag bij Google suggereert dat die onderwerpen al een tijdje in de belangstelling stonden? Maakt de NPO zich wel voldoende los van de gevestigde denkkaders? En wat valt er te zeggen over onderwerpen waar de NPO juist relatief vroeg al aandacht aan besteedde, zoals de Arabische Lente? Uiteraard kan de app deze vragen over de rol van de NPO niet beantwoorden. Het is niet meer en niet minder dan een hulpmiddel bij de discussie.

OK, wereldschokkend is het niet, maar het was een leuke klus om de webpagina aan de praat te krijgen (met een combinatie van PHP en D3.js). Ondertussen is de uitslag bekend: nummer 4. Cool. En vooral ook cool dat NPO en Hack de Overheid actief hun open data promoten.

Startup ratio

Wethouder Kasja Ollongren wil hoogopgeleide vluchtelingen naar Amsterdam lokken om het startup-klimaat in de stad te versterken. Het plan leidde tot enige ophef, maar voor degenen die de wethouder op Twitter volgen kan de link met startups ook weer niet helemaal als een verassing zijn gekomen. Ze twittert regelmatig over dit onderwerp; momenteel gaat één op de drie tweets van haar over startups.