champagne anarchist | armchair activist

OpenStreetMap

New Python package for downloading and analysing street networks

stationsplein

The image above shows square mile diagrams of cyclable routes in the area around the Stationsplein in Amsterdam, the Hague, Rotterdam and Utrecht. I made the maps with OSMnx, a Python package created by Geoff Boeing, a PhD candidate in urban planning at UC Berkeley (via).

Square mile diagrams are a nice gimmick (with practical uses), but they’re just the tip of the iceberg of what OSMnx can do. You can use it to download administrative boundaries (e.g. the outline of Amsterdam) as well as street networks from Open Street Map. And you can analyse these networks, for example: assess their density, find out which streets are connections between separate clusters in the network, or show which parts of the city have long or short blocks (I haven’t tried doing network measure calculations yet).

Boeing boasts that his package not only offers functionality that wasn’t (easily) available yet, but also that many tasks can be performed with a single line of code. From what I’ve seen so far, it’s true: the package is amazingly easy to use. All in all, I think this is a great tool.

How much delay for cyclists is caused by traffic lights

Road segments near traffic lights

The other day I posted an article on how much time cyclists lose at traffic lights in Amsterdam. Someone asked if I can calculate what percentage of total time lost by cyclists is caused by traffic lights. Keep in mind that delays can be caused by traffic lights, but also by crossings without traffic lights, crowded routes and road surface.

Here’s an attempt to answer the question, although I must say it’s a bit tricky. Again, I’m using data from the Fietstelweek (Bicycle Counting Week), during which over 40,000 cyclists shared their location data. This time I’m using the data about links (road segments). For each link, they provide the number of observations, average speed and relative speed.

With this data, it should be possible to estimate what share of total delays occurs near traffic lights. But what is near? It’s to be expected that the effect of traffic lights is observable at some distance: people slow down while approaching a traffic light and it takes a while to pick up speed again after. But what threshold should you use to decide which segments are near a traffic light?

One way to address this is to look at the data. I created a large number of subsets of road segments that are within increasing distances from traffic lights, and calculated their average speed. For example, segments that are within 50m from a traffic light have an average speed of about 16 km/h. The larger group of segments that are within 150m have an average speed of about 17 km/h.

Judging by the chart, it appears that the effect of traffic lights is diminishing beyond, let’s say, 150m. You could use this as a threshold and then calculate that delays near traffic lights constitute nearly 60% of all delays.

However, there’s a problem. Even if a delay occurs within 150m of a traffic light, the traffic light will not always be the cause of that delay. I tried to deal with this by estimating a net delay, which takes into account how much delay normally occurs when cyclists are not near a traffic light (in fact, I used two methods, that have quite similar outcomes). Using this method, it would appear that over 20% of delays are caused by traffic lights.

Now, I wouldn’t want to make any bold claims based on this: these are estimates based on assumptions and simplifications (in fact, if you think there’s a better way to do this I’d be interested). That said, I think it’s fair to say that average bicycle speeds appear to be considerably lower near traffic lights and that it’s plausible that this may be the cause of a substantial share of delays for cyclists.

UPDATE - I realise that the way I wrote this down sort of implies that you could reduce delay for cyclists by perhaps 20% just by removing traffic lights, but that would of course be a simplification.

Method

I used Qgis to process the Fietstelweek data. I used the clip tool to select only road segments in Amsterdam. I had Qgis calculate the distance of each segment and extract the nodes, which I needed to get the coordinates of the start and end points. Further processing was done with Python.

The dataset contains a relative speed variable (it is capped at 1, which means that it only reflects people cycling slower than normal, not faster). A relative speed of 0.8 would mean that people cycle at 80% of their normal speed. I calculated total delay at segments this way:

number of observations * (1 - relative speed) * distance / speed

You can then calculate delay at segments near traffic lights, as a percentage of the sum of all delays.

I tried to get an idea of how much of delay is actually caused by traffic lights, by estimating net delay. For this, I needed net relative speed. I used two methods to estimate this: 1. divide the relative speed of a segment by the median relative speed of all segments that are not near a traffic light; and 2. divide the speed of a segment by the median speed of all segments that are not near a traffic light.

Python code here.

Beautiful relief maps from Stamen

At Nathan Yau’s Flowing Data, I read that Stamen Terrain maps are now available globally, not just for the United States. Stamen uses Open Street Map and their tiles can be used with Leaflet.

One thing about relief maps is that they rub it in how embarrassingly flat the Netherlands are. For example, here’s the Oude Holleweg near Nijmegen (the nearby Van Randwijckweg was in this year’s Giro). It’s just a 70 meter height difference, but with gradients around 14% it’s one of the most challenging climbs in the Netherlands. Apparently, cyclists are not allowed to ride it downhill because that’s considered dangerous (update: I checked, it’s true, descending is not allowed). But if you look at the map, it doesn’t seem all that impressive.

Be that as it may, the Stamen Terrain maps «sure are pretty to look at», as Yau put it. Even when there’s hardly any elevation to show.

Power and buzz: Analysing trade union HQ locations by closeness to power and by convenience store score

When Hans Spekman ran for chairman of the Dutch Social-Democrat party in 2011, he said he wanted to move the party’s headquarters from the posh office at the Herengracht in Amsterdam to a «normal district, a neighbourhood where things happen, like Bos en Lommer». Bos en Lommmer is a multicultural neighbourhood in the west of the city, in transition from deprived to gentrified.

I agree with Spekman (at least on this matter) and I think his ideas about locations should also apply to trade union headquarters. Out of curiosity I decided to analyse the headquarters locations of European trade unions, using two criteria. First: closeness to power, operationalised pragmatically as the walking distance from the union office to the national parliament. And second: the liveliness of the neighbourhood. For measuring this I propose the convenience store score, which assumes that the number of convenience stores within half a kilometer gives a rough indication of how lively a neighbourhood is. Convenience stores could be for example 7-Eleven or AH to go stores and some ethnic shops will also be classified as convenience stores.

The chart below shows the scores for each union. You can also see the locations of union offices, parliaments and convenience stores on an interactive map, but note that the map may take a while to load - it’s not very suitable for viewing on a smartphone.

The median union headquarters is within 2km walking distance from parliament. For about three-quarters of unions, the distance is below 5km. The general pattern thus seems to be that unions have their national offices close to the institutions of political power. There are exceptions though. Officials of the major Dutch federations FNV and CNV would have to walk 15 to 68km to reach parliament. And sometimes the distance is even longer: a Basque union has its HQ in Bilbao; a Turkish union in Istanbul and Polish union Solidarnosz has its HQ near the port of Gdansk, where it originated. But all in all, the large Dutch unions are quite exceptional in that they don’t have their headquarters near the centre of political power.

As for liveliness: the median number of convenience stores within half a kilometer from union headquarters is 2, but about one in three unions have no convenience stores nearby at all. Some of the most lively union office locations are in countries like Romania, Hungary and Bulgaria. Other examples are CFDT (France), TUC (UK), SAK (Finland) and UGT (Spain). Dutch unions are at the other end of the spectrum and have rather dull headquarters locations - judging by the convenience store score.

So where should a union be? I’d say that influencing the government is one of the tasks unions should be doing, and an important one at that. However, this doesn’t depend on having a headquarters close to parliament, but rather on the ability to mobilise workers. I’d argue that the convenience store score is a far better criterium to judge headquarters locations by.

In case you were wondering: Spekman was successful in his bid for the chairmanship of the Social-Democrat party. The party’s headquarters is still at the Herengracht, though: it turned out the lease doesn’t expire until 2018.

Full disclosure: I work at the FNV, at the former FNV Bondgenoten location.

Method

This analysis turned out to be quite a bit more challenging than I initially thought, but it was very instructive. I’m especially happy that I now have a basic understanding of the Overpass API that you can use to retrieve Open Street Map data. OSM has always been a bit of a black box to me but the Overpass API turns out to be a valuable tool.

Measuring neighbourhood characteristics

Initially I wanted to use Eurostat regional stats to analyse neighbourhood characteristics, but Eurostat doesn’t have data beyond the NUTS 3 level (I should’ve known). Level 3 areas may comprise entire cities and are useless for analysing neighbourhoods, so I had to look for alternatives.

Subsequently, I tried getting the name of the smallest area a location is in using the Mapit tool (based on Open Street Map). I thought I might then be able to construct a Wikipedia url by adding the name to https://en.wikipedia.org/wiki/. This turned out to work pretty well, not least because Wikipedia is quite good at handling different variants of geographical names. However, while Wikepedia articles tend to be informative, they do not contain a lot of uniform statistical information. Often population, area and population density will be included, but not much beyond that. In addition, the fact that the size of the areas varies poses problems. For example, the population density of a small area cannot be meaningfully compared to the density of a large area. In the end I did add the Wikipedia links to the popups on the map, but I continued looking for other ways to analyse neighbourhood characteristics.

One of the measures I ended up using is closeness to power, operationalised as the walking distance to the national parliament (in countries with a bicameral parliament, I used the location of the lower house). This was a pragmatic choice. An alternative would have been to use the location of ministries, but then I’d have to come up with a way to pick the relevant ministry.

For measuring the liveliness of a neighbourhood, I used the number of convenience stores within half a kilometer, using data from Open Street Map. Obviously there are some limitations to this method. For example, some countries will be mapped in more detail than others. Also, there will be inconsistencies in how shops are classified (cf this discussion in Dutch about how to classify stores of chains like Blokker).

Obviously, the convenience store score has not been properly validated. I’m not even sure whether objective measures of a neighbourhood’s liveliness exist. I checked this list of «coolest» neighbourhoods in Europe and all but one (Amsterdam Noord) have convenience stores nearby, but then again coolness isn’t the same as liveliness (I guess a neighbourhood can be uncool yet lively). Furthermore, being on a list of cool neighbourhoods isn’t necessarily an indicator of coolness.

Ideally I think a proper assessment of the convenience store score should include a comparison with measurements of criteria derived from Jane Jacob’s The death and life of great American cities: mixed primary uses, short blocks, buildings of various ages and density. I guess it should be possible to measure some of these with OSM data (especially the first two). However, that would require a deeper understanding of OSM classifications than I currently have.

Getting the data

While some of the data was obtained by good old-fashioned googling, some of it could be automated.

The starting point for the analysis was the list of affiliates of the European Trade Union Confederation (ETUC). Note that this includes unions in non-EU countries such as Turkey. Also note that I use the word union but most are in fact union federations (the FNV is a bit more complicated; a recent merger has partly done away with the federation structure).

The ETUC doesn’t seem to have a list of addresses on their website. They do provide urls for most of their affiliates. Still, looking up addresses was a bit of an adventure, especially for countries which use non-Latin alphabets (let me know if you find any errors).

For walking distances I used the Bing API. In a number of cases Bing couldn’t find a walking route or the distance seemed wrong. In those cases I manually looked up the distance in Google Maps. Here’s a sample url for getting information from the Bing API (replace KEY with API key).

I used the Overpass API (demo) of Open Street Map to get all nodes within 500m from the union HQs, which I used for counting the number of convenience stores. I also used the API for getting the coordinates of all convenience stores in all countries where the ETUC has affiliates. Here’s a sample url for getting all nodes within 500m of a location, and here for getting all convenience stores in a country.

A few unions are missing in the final results because of missing data. For example, I couldn’t figure out what the main office of the Belgian ACV is and I couldn’t find the exact location of the parliament of Malta (somewhere along Republic Street, Valletta).

Calculating scores

I calculated scores as either walking distance to parliament in kilometers or the number of nearby convenience stores. In both cases I took the log10 of the value + 1. To arrive at a 0 to 10 scale, I multiplied by 10 and divided by the maximum score for each variable. For the distance to power measure I converted the score to 10 minus the score, so that a higher score means closer to power.

Mapping

I used Leaflet and D3.js to map the locations of HQs, parliaments and convenience stores. There are over 60,000 convenience stores in the dataset. This turned out to be a bit too much and the browser all but crashed. I found this script that deals with exactly this problem. While I managed to figure out what I needed to change to make the script work with my data, I’m afraid I don’t fully understand how it works. It’s still too slow for mobile, though.

Cycling against traffic #2

The other day I posted something about cycling against traffic which, it has been claimed, is allowed on 85% of oneway streets in Brussels. I tried to find out the percentage for Amsterdam using Open Street Map, but found that the relevant information is often missing. Or so I thought.

I posted a question on the OSM forum (here and here) and got various helpful answers. Basically, I shouldn’t have looked just for oneway:bicycle=no tags, but also for cycleway=opposite (and perhaps a few more). I was also directed to a web page where cycling tags can be shown on a map.

So does Amsterdam allow cycling against traffic on anyway near the 85% of oneway streets reported for Brussels, if you include the cycleway=opposite tags? Well, no. Then again, looking at a similar map of Brussels, it doesn’t really look like they do any better. Of course, one shouldn’t jump to conclusions:

  • It depends on the part of the city you look at. In Amsterdam, cycling against traffic is more often allowed in the city centre and some other parts like Oost; in Brussels is appears to be more spread out over the city,
  • Perhaps local Open Street Map contributors have different mapping habits.

That said, I was getting curious as to the basis for the 85% claim for Brussels. I found a report from 2010 published by cyclists’ organisation Gracq, which said that 75% of oneway streets in Brussels had sens unique limité (which is apparently a legal requirement on suitable oneway streets). Gracq had contacted local governments by telephone to collect the data.

Pages