Scooter-free cycle tracks in Amsterdam

Amsterdam plans to ban scooters from most cycle tracks. Currently, cycle tracks are still used by the so-called snorfiets category which has a speed limit of 25km/h - although most ride (much) faster. The measure will make cycle tracks safer for cyclists, and it will also result in cleaner air on cycle tracks.

The city has produced a map showing the new snorfiets regime. By my calculations, snorfietsen will have to use the road on a total of 180km (blue on the map) and they will be banned from another 71km of routes where there’s only a cycle track (marked in red). However, they’ll still be allowed to use about 93km of cycle tracks (green), at least for now.

Busy cycle tracks

To decide where to ban scooters from the cycle track, the city used data from the Fietstelweek, the large-scale initiative to collect smartphone location data from cyclists. This is interesting, since governments have complained that the number of participants in the Fietstelweek is declining (they started experimenting with Strava data instead).

Perhaps that’s why the city of Amsterdam used the 2016 edition of the Fietstelweek (rather than 2017) to assess how busy cycle tracks are. It used its own traffic counts to validate the Fietstelweek data. The Mathematics Centre of the University of Amsterdam deemed the method used ‘reliable and suitable’.

I’ve created a map to show how the new snorfiets regime compares to Fietstelweek data. The width of the pink lines corresponds to the number of cyclists who used a route; the green dotted line shows where snorfietsen will still be allowed to use the cycle track.

It appears that the city has done a decent job at avoiding the busiest cycling routes (in some cases, this is because these routes don’t have separate bicycle tracks to begin with).

That said, some problems remain. One example is the cycle track along the IJ north of Central Station, which can be very busy and where some snorfiets riders overtake other traffic in a dangerous way. And there’s the Amsterdamse Brug and Schellingwouder Brug (the bridges to the northeast of Amsterdam), where cycle tracks are too narrow for snorfietsen.

Making all cycle tracks scooter-free

In the future, the city intends to ban scooters from all mandatory cycle tracks within the A10 Ring Road. Obviously, they want to do this without compromising the safety of snorfiets riders. This will be easier on routes where car speed is already low.

The map below shows the current average car speed on major roads. The green lines indicate where snorfiets riders will initially be allowed to continue using the cycle track.

Of course I don’t know what the situation is when you’re reading this, but likely some of the highest car speeds (on sections where snorfietsen will initially be allowed to use the cycle track) will be on the Gooiseweg, entering the city centre from the southeast, and the aforementioned Amsterdamsebrug and Schellingwouderbrug. On those bridges, many motorists exceed the speed limit. The city wants to change the road design to invite lower speeds before banning snorfietsen from the cycle track.

Elsewhere, it should be relatively easy to make the remaining cycle tracks scooter-free. Of course, a more practical solution would be to abolish the snorfiets category altogether.

If you wish to respond to the city’s plans, you can use this form. The deadline is 24 September.

UPDATE 6 April 2019 - The map showing car speeds doesn’t work anymore because the map of the city of Amsterdam where I got the data from is offline.

Method

I used Qgis to create the first map and Leaflet for the second. I used GeoPandas and Shapely to calculate lengths. On the first map, the width of the pink may be slightly distorted due to varying distances between cycle routes in opposite directions.

Is tourist dispersion working? An analysis of Lonely Planet maps

Discussion

For more than fifteen years, Amsterdam has been trying to convince tourists to visit areas outside the city centre. There is a concern that the inner city is approaching the limit of how many tourists it can handle.

To explore the effect of these policies, I analysed changes in the maps in Lonely Planet guides. Over the past years, sights have been added in areas outside of the inner city - mostly areas that had already been affected by gentrification. Still, the large majority of sights are still in the traditional tourist areas, in the city centre and some parts of the Zuid district.

It appears that the effect of tourist dispersion policies is modest at best - and not nearly enough to compensate for the growth of tourism. Reducing the impact of tourism may well require a different approach - for example targeting hotel capacity and low-cost flights to Schiphol Airport.

Dispersion policies

In its coalition agreement, the new city government said that the positive aspects of tourism are increasingly overshadowed by its negative effects, putting the liveability of some neighbourhoods at risk. One of the ways to deal with this is spreading tourists over the city (and the surrounding region). Amsterdam is to be primarily a place where people live and do business, and only in the second place a tourist destination.

The idea to disperse tourists is not new. In 2016, Amsterdam launched a campaign to promote areas outside the inner city. Interestingly, the campaign caused a bit of a controversy when politicians noticed the Nieuw-West district had been left out of a promotional map. Amsterdam Marketing responded that ‘in our professional opinion’, the district is currently ‘less suitable to be offered as a primary alternative to the city centre’. They argued that neighbourhoods must first be embraced by locals, which suggests that city marketing follows gentrification.

In 2009, Amsterdam planned to promote the eastern parts of the city as ‘the new (2nd) Museum Quarter’; the Northern IJ Waterfront as ‘Creative City’, the Westerpark as a variation on Berlin’s ‘Kulturbrauerei’; the Eastern Harbour Area as Docklands; de Pijp as ‘Quartier Latin’ and Oud-West as ‘Notting Hill’.

And as early as 2001, the tourism board warned that the inner city had almost reached the limit of how many tourists it can handle. «But where should they go? To IJburg for architecture; fun shopping at the Arena Boulevard in Zuidoost and visit the former GVB tram depot in Oud-West.»

A common denominator of the campaigns is that they target repeat visitors. As the tourism board explained in 2001, «we don’t want to send first-time foreign visitors to the outskirts».

Lonely Planet maps

To get an idea of the impact of these policies, I analysed changes in the sights shown on maps in Lonely Planet guides (for caveats, see Method below). If tourists turn to new parts of the city, you’d expect these areas to show up on those maps. Further, in 2009, the tourism board started seinding information about sights outside the city centre to publishers of travel guides. «Inclusion in the guides is not guaranteed, but this often happens.»

2006 is a bit of an outlier. A number of sights outside of the city centre were added, only to disappear again in the next edition (see below, Sights that were dropped). If you zoom in on specific neighbourhoods, you’ll notice more changes. For example:

  • A number of sights in Oost were added in 2012: Oosterpark (including De Schreeuw, Slavery Memorial and Spreeksteen), Dappermarkt and Frankendael;
  • In 2018, a number of sights in Noord were added, including some at the former NDSM Wharf, EYE Film Museum and Nieuwendammerdijk.

The table below shows the percentage of sights per district:

District 2000 2006 2012 2016 2018
Stadsdeel Centrum 84 74 78 78 75
Stadsdeel Zuid 10 12 11 11 10
Stadsdeel Oost 1 3 6 5 6
Stadsdeel Noord 2 1 0 0 5
Stadsdeel West 3 5 4 5 4
Stadsdeel Nieuw-West 0 1 0 0 0
Stadsdeel Zuidoost 0 1 0 0 0
Wijk 00 Amstelveen 0 3 0 0 0

There has been an increase in especially Oost and Noord, but the large majority of sights are still in Centrum and in Zuid (which includes the Museumplein).

Sights that were dropped

In each edition, new sights are added and others are dropped. The latter category includes sights that don’t exist anymore, such as the Netherlands Media Art Centre, the Vakbondsmuseum (trade union museum) and temporary locations of the Stedelijk Museum. Other sights apparently fell out of grace with the authors.

The authors of the various editions have their own preferences and interests. For example, Andrew Bender, author of the 2006 edition, appears to be a bit of a health enthousiast. He added many sports facilities and fitness centres, which explains why his edition had more sights outside the city centre. Most of these were dropped in the next edition. In 2012, Karla Zimmerman and Sarah Chandler added many hofjes (~almshouses). Again, most of them didn’t make the next edition.

Method

I used the following editions of the Lonely Planet Amsterdam guide:

2000: Rob van Driesum, Nikki Hall
2006: Andrew Bender
2012: Karla Zimmerman, Sarah Chandler
2016: Catherine Le Nevez, Karla Zimmerman
2018: Catherine Le Nevez, Abigail Blasi

I analysed sights in the legends of the maps at the end of the guides. The maps also include categories like eating, drinking, sleeping and entertainment. I focused on sights, reckoning that this category would likely present less problems when you want to geocode information from old maps. Note that the classification of especially the 2000 edition is somewhat different from later editions.

It’s possible that errors occured in geocoding or in copying data from the guides. If you spot any errors, please let me know.

Obviously, Lonely Planet maps are not a perfect measure of tourism dispersion. On the other hand, if there had been major shifts in the areas tourists visit, it seems rather unlikely they wouldn’t be reflected in the sights Lonely Planet shows on its maps.

Tags: 

Converting Election Markup Language (EML) to csv

Note that the map above isn’t really a good illustration here because I used a different data source to create it.

Getting results of Dutch elections at the municipality level can be complicated, but what if you want to dig a little deeper and look at results per polling station? Or even per candidate, per polling station? For elections since 2009, that information is available from the data portal of the Dutch government.

Challenges

The data is in Election Markup Language, an international standard for election data. I didn’t know that format and processing the data posed a bit of a challenge. I couldn’t find a simple explanation of the data structure, and the Electoral Board states that it doesn’t provide support on the format.

For example, how do you connect a candidate ID to their name and other details? I think you need to identify the Kieskring (district) by the contest name of the results file. Then, find the candidate list for the Kieskring and look up the candidate’s details using their candidate ID and affiliation. But with municipal elections, you have to look up candidates in the city’s candidate list (which doesn’t seem to have a contest name).

Practical tips

If you plan to use the data, here are some practical tips:

  • Keep in mind that locations and names of polling stations may change between elections.
  • If you want to geocode the polling stations, the easiest way is to use the postcode, which is often added to the polling station name (only for recent elections). If the postcode is not available or if you need a more precise location, the lists of polling station names and locations provided by Open State (2017, 2018) may be of use. Use fuzzy matching to match on polling station name, or perhaps you could also match on postcode if available. Of course, such an approach is not entirely error-free.

Further, note that the data for the 2017 Lower House election is only available in EML format for some of the municipalities. I guess this has something to do with the fact that prior to the election, vulnerabilities had been discovered in software to count the votes, so they had to count the votes manually.

Python script

Here’s a Python script that converts EML files to csv. See caveats there.

UPDATE 23 February 2019 - improved version of the script here.

Tags: 

The orientation of Amsterdam’s streets

Eight days from now, Amsterdam will have a new metro line traversing the city from north to south. But what about the orientation of the city’s streets?

Geoff Boeing - who created a Python package for analysing street networks using data from OpenStreetMap - just published a series of polar histograms of American and ‘world’ cities. Amsterdam isn’t among them, but Boeing made his code available, so I used that to create charts for the largest cities in the Netherlands.

While the pattern isn’t nearly as monotonous as in most American cities, I’m still surprised how many streets in Amsterdam run from north to south or from east to west. The Hague has a strong diagonal orientation; Rotterdam doesn’t seem to have a dominant orientation and Utrecht is a bit in between.

With Boeing’s code, you can also do the analysis specifically for roads that are accessible to cyclists, but for Amsterdam that doesn’t make much difference since most roads are.

Discussion

15 July 2018 - There was some really interesting discussion on Twitter in response to my post from last Friday (I use Twitter names to refer to people; most sources are in Dutch).

Curved streets

Both Sanne and Egon Willighagen asked how the chart treats curved streets. I have to admit I hadn’t checked, but the docstring of the add_ege_bearings function explains that it calculates the compass bearing of edges from origin node to destination node, so that implies that streets are treated as if they were straight lines.

Is that a problem? Probably not for many US cities, for they seem to have few curved streets. As for Amsterdam: most people’s mental image of the city is probably dominated by the curved canals of the city centre. However, many neighbourhoods consist of grids of more or less straight streets. So perhaps curved streets have little impact on the analysis after all.

Length versus surface

Hans Wisbrun argues that the chart type is nice, but also deceptive. The number of streets is represented by the length of the wedges, but one may intuitively look at the surface, which increases with the square of the length. In a post from 2013 (based on a tip from Ionica Smeets), he used a chart by Florence Nightingale to discuss the problem.

Rogier Brussee agrees, but argues that a polar chart is still the right choice here, because what you want to show is the angle of streets.

In a more general sense, I think the charts are an exploratory tool that’ll give you an idea how street patterns differ between cities. If you really want to understand what the wedges represent, you’ll have to look at a map.

Beach ridges

That’s what Stephan Okhuijsen did. He noted that the chart for The Hague appears to reflect the orientation of the city’s coastline. Not quite, Christiaan Jacobs replied. The orientation of the city’s streets is not determined by the current coastline, but by the original beach ridges.

I don’t know much about geography (or about The Hague for that matter), but a bit of googling suggests Jacobs is right. See for example this map (from this detailed analysis of one of The Hague’s streets), with the old sand dunes shown in dark yellow.

See also links to previous similar work in this post by Nathan Yau (FlowingData).

Gentrification mapped

The map makers of the City of Amsterdam have created a map that shows the Neighbourhood Street Quota or BSQ. The BSQ plays a key role in a highly controversial reform that is eroding the city’s social ground lease policy, but that’s not the topic of this article. For now, I’m interested in the BSQ as an indicator of land value.

As the city government puts it, «the high BSQs are found at popular locations in the city and the low BSQs at less popular locations in the city» (for details see Method, below). Unsurprisingly, the centrally located Centrum and Zuid districts have high BSQs and the peripheral areas have low BSQs.

More interesting is how the BSQ has changed. The city government has provided data for thousands of streets or street segments, for 2014 and 2016. Of course, this is a short time period and the patterns may or may not reflect longer-term developments.

The chart below shows the distribution of BSQs for flats (as opposed to single-family dwellings) for 2014 and 2016.

The peak has moved to the right, as the median value has risen from 28 to 38. For political reasons, the BSQ can never be lower than 5 or higher than 49, which explains the large number of streets with a value of 5 or 49. This implies that rises in BSQ don’t fully reflect how much land values have risen.

The map below shows how much BSQs for flats have risen in different parts of Amsterdam. I omitted streets with low or high BSQs where substantial changes in BSQ may have been hidden by the upper and lower limits. At the high end, this applies to the Canal Belt and much of the Zuid District. At the lower end, this applies to many peripheral areas including almost the entire Zuidoost District.

Red streets indicate an increase of the BSQ by more than a half; orange streets an increase by less than a half and the rare green streets a decrease of the BSQ. There are some red areas outside the ring road: mainly the IJburg expansion to the east; some parts of Nieuw-West; and Buitenveldert. Buitenveldert is a neighbourhood south of the Zuidas business district with a growing number expats and students among its residents.

Within the ring road, BSQs are rising in areas that are often associated with gentrification, such as the Kolenkit in West, the Vogelbuurt in Noord and the Indische Buurt in Oost. Perhaps more surprising is Betondorp, a low-income area with many older residents, described in 2015 as «one of the few neighbourhoods in Amsterdam not yet affected by the advance of gentrification». If the BSQ is an indication, that may be about to change.

Method

A list (pdf) of BSQs for 2016 and 2014 was recently sent to the city council. The BSQs are referred to as 2018 and 2017, but are based on data from 2016 and 2014 respectively (or to be more precise: the ‘2017 BSQ’ uses data from 2015 or 2014, whichever is lowest). The map created by the City of Amsterdam uses the ‘2017 BSQ’.

For each house, the municipality calculates an individual land quota using the formula: land value / (land value + theoretical cost of rebuilding the house). The land value is obtained by subtracting the rebuilding cost from the total value of the house (WOZ).

Subsequently, BSQs are calculated as the average land quota per street (or street segment if a street traverses multiple neighbourhoods). This is done separately for single-family dwellings and flats.

The interpretation of the BSQ is a bit tricky: one should expect higher land values to be reflected in higher BSQs, but the exact relationship will depend on the value of the building and whether that also responds to changes in land value (for example, because more expensive materials are used).

In my analysis, I only used BSQs for flats, and only the streets or street segments for which a BSQ is available for both 2014 and 2016 (thus excluding new urban expansions).

For the map, I also excluded streets where an increase of the BSQ by less than half may be hidden by the lower or upper limit of the BSQ: those with a 2014 value of 5 and a 2016 value of less than 8; and those with a 2014 value above 32 and a 2016 value of 49.

In creating the map I also ignored long streets that traverse multiple neighbourhoods and that therefore have been separated into multiple segments. Constructing street segments from line geometries representing the entire street seemed like a lot of work (perhaps there’s a simple way to do this, but I couldn’t find it).

I used Tabula to extract data from the original pdf; this Python script to process the data, create a csv for the chart and create a shapefile for the map; D3.js for the chart and Qgis to create the map (using Open Street Map map data and Stamen Toner Lite for the background).

Tags: 

Dutch governments consider using Strava data

Strava is a popular app to record bicycle rides. For some years, the company has been trying to sell its data to local governments for traffic planning. NDW, a platform of Dutch governments including the city of Amsterdam, has bought six months’ worth of Strava data to give it a try.

The switch to Strava may mean the end of the Fietstelweek, an annual one-week effort to collect bicycle data from thousands of volunteers. In the past, I’ve used Fietstelweek data to analyse waiting times at traffic lights. The Fietstelweek received funding from the same governments that are now experimenting with Strava data.

One reason why they are looking for alternatives is that the number of Fietstelweek participants is lower than they’d like. They seem to have a point. Consider for example the map below, which shows bicycle routes to and from Amsterdam Central Station.

As such, it’s an interesting map. Unsuprisingly, it seems that intensity is highest near the bicycle parking facilities. Main access routes appear to be the Geldersekade (with the sometimes chaotic crossing with Prins Hendrikkade) and the Piet Heinkade. It seems that people cycling to and from Central Station are somewhat more likely to live in the eastern part of the city.

There’s one caveat though: the numbers are small. Even the busiest segments represent at most 40 rides. One loyal Fietstelweek participant recording her commute during the entire week could literally change the map.

Strava has far larger numbers, but its data raises different kinds of questions. Strava calls itself ‘the social network for athletes’ and wants to know if you use a road bike, a mountain bike, a TT bike or a cyclocross bike (no option ‘other’ available). So how representative is Strava data of people who use their city bike for commutes and other practical purposes?

Strava’s response to such questions is that they’re trying to make the app less competition-focused and more social, with Facebook-like features. This should help them collect data about ‘normal’ bike rides. They have also argued that «especially in cities, those with the app tended to ride the same routes as everyone else».

But is that really true? Strava’s heatmap (choose red and rides) for Amsterdam could perhaps be interpreted as a combination of recreational rides (Vondelpark, Amstel) and cyclists trying to get in or out of the city as quickly as possible (plus quite a few people who recorded their laps at the Jaap Eden ice skating rink as bicycle rides).

Perhaps you could find a way to filter out ‘lycra’ rides and end up with a sufficient number of ‘normal’ rides. Then again, almost three-quarters of bicycle rides in the Netherlands are under 3.7 km, and I suspect very few of those short rides end up on Strava.

There’s also a socio-economic aspect. It has been argued that Strava is used most by people living in wealthier neighbourhoods, which aren’t necessarily the neighbourhoods most in need of better cycling infrastructure.

Of course, bicycle use is unequal in the first place, which is also reflected in Fietstelweek data. The map below shows the start and end points of rides for Amsterdam.

Density is highest in the area within the ring road and south of the IJ. The number of trips per 1,000 residents also correlates with house values: more bicycle trips start or end in affluent neighbourhoods. As said, this probably reflects actual patterns in bicycle use and not a problem of the data.

To summarise, Fietstelweek has smaller numbers than one would like, while Strava data raises questions about representativeness. One way for Strava to help answer these questions would be to make a subset of its Amsterdam data available as open data.

This Python script shows how the analysis was done.

The Digital City

Amsterdam has a new coalition agreement. The paragraph on democratisation and the Digital City was well received - a London-based researcher from Amsterdam liked the plans so much she decided to translate them into English.

The new coalition wants to create a democratic version of the smart city. Citizens should be in control of their data. The city will support co-operations that provide an alternative to platform monopolists. An information commissioner will see to it that the principles ‘open by default’ and ‘privacy by design’ are implemented.

The agreement also lists a number of issues the city is (or was) already working on:

  • City council information will be opened up. In 2015, the city council asked to make documents such as council meeting reports, motions, written questions etc available as open data. Since, some of that data has been made available through Open Raadsinformatie, but as yet no solution has been found for offering all council information in a machine readable form.
  • Freedom of information requests (Wob requests) will be published. Amsterdam started publishing decisions on Wob requests earlier this year; so far three have been [published][wob].
  • Interestingly, Amsterdam wants to use open source software whenever possible. Over ten years ago, Amsterdam planned a similar move, and its plans were sufficiently serious to needle Microsoft. In 2010, the plans foundered on an uncooperative IT department.

All in all, a nice combination of new ambitions and implementation of ‘old’ plans.

Dutch government drops 3D pie charts

The Dutch government has replaced pie charts with bar charts in it’s annual reports, someone noted on Twitter (via @bokami). Pie charts aren’t always a bad choice - contrary to the view of some adherents of the stricter school in data visualisation. But 3D pie charts are really hard to justify and it’s a bit awkward they were still used one year ago.

The charts are from the 2016 and 2017 reports of the Department of Social Affairs and Employment.

Tags: 

How to use Python and Selenium for scraping election results

A while ago, I needed the results of last year’s Lower House election in the Netherlands, by municipality. Dutch election data is available from the website of the Kiesraad (Electoral Board). However, it doesn’t contain a table of results per municipality. You’ll have to collect this information from almost 400 different web pages. This calls for a webscraper.

The Kiesraad website is partly generated using javascript (I think) and therefore not easy to scrape. For this reason, this seemed like a perfect project to explore Selenium.

What’s Selenium? «Selenium automates browsers. That’s it!» Selenium is primarily a tool for testing web applications. However, as a tutorial by Thiago Marzagão explains, it can also be used for webscraping:

[S]ome websites don’t like to be webscraped. In these cases you may need to disguise your webscraping bot as a human being. Selenium is just the tool for that. Selenium is a webdriver: it takes control of your browser, which then does all the work.

Selenium can be used with Python. Instructions to install Selenium are here. You also have to download chromedriver or another driver; you may store it in /usr/local/bin/.

Once you have everything in place, this is how you launch the driver and load a page:

from selenium import webdriver
 
URL = 'https://www.verkiezingsuitslagen.nl/verkiezingen/detail/TK20170315'
 
browser = webdriver.Chrome()
browser.get(URL)

This will open a new browser window. You can use either xpath or css selectors to find elements and then interact with them. For example, find a dropdown menu, identify the options from the menu and select the second one:

XPATH_PROVINCES = '//*[@id="search"]/div/div[1]/div'
element = browser.find_element_by_xpath(XPATH_PROVINCES)
options = element.find_elements_by_tag_name('option')
options[1].click()

If you’d check the page source of the web page, you wouldn’t find the options of the dropdown menu; they’re added afterwards. With Selenium, you needn’t worry about that - it will load the options for you.

Well, actually, there’s a bit more to it: you can’t find and select the options until they’ve actually loaded. Likely, the options won’t be in place initially, so you’ll need to wait a bit and retry.

Selenium comes with functions that specify what it should wait for, and how long it should wait and retry before it throws an error. But this isn’t always straightforward, as Marzagão explains:

Deciding what elements to (explicitly) wait for, with what conditions, and for how long is a trial-and-error process. […] This is often a frustrating process and you’ll need patience. You think that you’ve covered all the possibilities and your code runs for an entire week and you are all happy and celebratory and then on day #8 the damn thing crashes. The servers went down for a millisecond or your Netflix streaming clogged your internet connection or whatnot. It happens.

I ran into pretty similar problems when I tried to scrape the Kiesraad website. I tried many variations of the built-in wait parameters, but without any success. In the end I decided to write a few custom functions for the purpose.

The example below looks up the options of a dropdown menu. As long as the number of options isn’t greater than 1 (the page initially loads with only one option, a dash, and other options are loaded subsequently), it will wait a few seconds and try again - until more options are found or until a maximum number of tries has been reached.

MAX_TRIES = 15
 
def count_options(xpath, browser):
 
    time.sleep(3)
    tries = 0
    while tries < MAX_TRIES:
 
        try:
            element = browser.find_element_by_xpath(xpath)
            count = len(element.find_elements_by_tag_name('option'))
            if count > 1:
                return count
        except:
            pass
 
        time.sleep(1)
        tries += 1
    return count

Here’s a script that will download and save the result pages of all cities for the March 2017 Lower House election, parse the html, and store the results as a csv file. Run it from a subfolder in your project folder.

UPDATE 23 February 2019 - improved version of the script here.

Notes

Dutch election results are provided by the Kiesraad as open data. In the past, the Kiesraad website used to provide a csv with the results of all the municipalities, but this option is no longer available. Alternatively, a download is available of datasets for each municipality, but at least for 2017, municipalities use different formats.

Scraping the Kiesraad website appears to be the only way to get uniform data per municipality.

Since I originally wrote the scraper, the Kiesraad website has been changed. As a result, it would now be possible to scrape the site in a much easier way, and there would be no need to use Selenium. The source code of the landing page for an election contains a dictionary with id numbers for all the municipalities. With those id numbers, you can create urls for their result pages. No clicking required.

Tags: 

The pay gap between CEO and workers

The Dutch Corporate Governance Code has been revised. As a result, Dutch listed companies have started to report their internal pay ratios. These pay ratios are often thought of as the gap between CEO pay and what other workers in the firm get paid.

How can you use these ratios? One way is to create a Fat Cat Calendar. The inspiration here is the British phenomenon of Fat Cat Day: the day when CEOs have earned as much as their employees will earn in an entire year.

The calendar shows that by 2 January at quarter to five in the afternoon, the Heineken CEO had earned more than his workers over all of 2017.

Some firms are missing from the calendar. For example, Euronext says it’s complicated to calculate a pay ratio, because they operate in different countries (but then, this would apply to most listed companies). Shell has calculated a pay ratio, but they don’t report how high it is - only how it compares to the pay ratios of a selection of other companies.

Note that a pay ratio is not a simple and straightforward figure. Below is an exploratory analysis of some of the issues involved (see Method section below for caveats).

Heineken

Heineken, which has by far the highest reported pay ratio in my sample (215), argues this is a consequence of their business model.

First, Heineken does a lot of business in ‘emerging markets with widely different pay levels and structures compared to the Netherlands and Europe’. In other words, many workers are in low-wage countries. The underlying suggestion seems to be that you don’t have to pay these workers the same wages as their colleagues in Europe.

Second, Heineken has ‘a large number of breweries and sales forces in-house worldwide, which adds to the variety of pay within the Company’ (the treatment of Heineken sales staff in Africa is controversial, but that’s a different matter). Aside from the question whether Heineken are paying their brewery and sales workers enough, they do have a point here.

Many companies have simply outsourced their low-paid workers. It’s a bit arbitrary to compare CEO pay to employees of the firm but not to the outsourced workers who clean their offices, serve their lunches, fix their computers, and manufacture and sell their products.

Third, Heineken says that pay ratios can be very volatile because of the substantial bonuses that go up and down. One might argue that this is not a problem of the pay ratio, but a problem of the composition of CEO pay.

Anyone can calculate a ratio

According to the Corporate Governance Code, firms should report the ‘ratio between the remuneration of the management board members and that of a representative reference group determined by the company’. This leaves ample room for firms to decide how to calculate the ratio. For that reason, comparing ratios between firms is a bit tricky.

In practice, many firms calculate the pay ratio as the ratio between total CEO pay and total staff costs per FTE. This is information that is normally included in the annual report, so you can calculate the pay ratio yourself.

I calculated pay ratios for a sample of companies listed in Amsterdam and compared these to the pay ratios these companies reported in their annual report. The chart below shows how my calculated pay ratios compare to the reported pay ratios.

In many cases, my calculated pay ratios are quite similar to the pay ratios reported by the companies. It’s interesting to look into some of the examples where this is not the case, and why this might be:

  • Some CEOs were appointed during 2017 and therefore weren’t paid an entire year’s remuneration. I didn’t correct for this, so my calculated pay ratio is too low in these cases. An example is AkzoNobel, which correctly reported a higher pay ratio than the one I calculated.
  • Randstad probably used only corporate employees and not ‘candidates’ (temp workers) to calculate their pay ratio. This would explain why they arrived at a smaller gap between CEO and workers than the one I calculated.
  • Other companies also use a subset of their employees for their calculation. Assuming these are relatively well-paid employees, the gap between CEO and workers will appear smaller that way. An example is OCI, which used only employees in Europe and North-America as a reference group. Unilever takes this approach one step further by comparing CEO pay to their various UK and Dutch management work levels. This resulted in a range of pay ratios, each far smaller than the one I calculated.
  • A few companies use the average remuneration of the entire executive board, rather than CEO remuneration, to calculate the pay ratio. Assuming the CEO earns more than the other board members, this will also make the gap with workers appear smaller. An example is AMG, which argues that using average board remuneration is appropriate ‘given the collective management responsibility of the Management Board members’ (this is somewhat ironic, for ‘collective responsibility’ was apparently not a key consideration when they decided to pay the CEO 50% more than the other board members).

Discussion

I think it would be fair to say that the requirement to report pay ratios is a failed attempt at transparency. Since firms can use whatever method they want to calculate the ratio, comparisons across firms are problematic. Meanwhile, anyone can calculate ratios from data that is already available. While this isn’t entirely unproblematic (see Method section below), the ratios you calculate will probably be more consistent across firms than the reported ratios.

Instead of requiring firms to report a pay ratio, it would make more sense to require them to report CEO pay, staff costs and staff numbers in a more consistent and transparent way.

Further, it’s somewhat arbitrary to compare CEO pay only to employees of the firm, and not outsourced workers such as cleaners. A fair measure of inequality should look beyond the employees of the firm. One option is to compare CEO pay to the median income in a country; another is the norm which states that CEO pay should not be higher than 20 times the legal minimum wage. Of course, a limitation of these approaches is that they don’t capture international inequality.

The highest CEO-to-minimum wage ratio for the firms in my sample is 577 (Unilever). By 1 January 2017 at quarter past three in the afternoon, the CEO of Unilever had earned a the minimum wage for an entire year. If you want to narrow this gap, there are broadly two ways to do it: show more restraint in CEO remuneration, and raise the minimum wage.

Method

I calculated ‘Fat Cat Day’ by simply adding one year, divided by the pay ratio, to 1 January 2017:

d1 = dt.datetime.strptime('2017-01-01', '%Y-%m-%d')
d2 = dt.datetime.strptime('2018-01-01', '%Y-%m-%d')
year = d2 - d1
 
dates = defaultdict(list)
for i, row in df.iterrows():
    if pd.notnull(row.ratio_reported_17):
        ratio = row.ratio_reported_17
        fcd = d1 + year / ratio
        date = dt.datetime.strftime(fcd, '%Y-%m-%d')
        time = dt.datetime.strftime(fcd, '%H:%M')
        company = row.company
        item = {
            'time': time,
            'company': company
        }
        dates[date].append(item)

Note that the British High Pay Centre has a far more elaborate method to calculate Fat Cat Day, which considers how many hours CEOs work, whether or not they work weekends, etcetera. While this is very conscientious, I don’t think it’s necessary. CEOs are paid on an annual basis, and if they work less than a full year, they’ll generally receive a (more or less) proportional share of their annual pay, regardless of the actual number of hours worked.

I googled for annual reports of companies listed on the Amsterdam stock exchange (in a few cases, I also looked up a separate remuneration report). I didn’t always find one, which can be for a number of reasons: perhaps I didn’t look hard enough; perhaps companies hadn’t filed their report yet; or perhaps they filed it with the company register but didn’t publish it online. As a quick filter, I disregarded reports that don’t contain the term pay ratio. All in all, this is a rather pragmatic sample, which may not be representative of all Amsterdam-listed companies (for example, it’s conceivable that firms with stronger roots in the Netherlands are more inclined to comply with the Corporate Governance Code). For exploratory purposes, I think that’s ok.

I calculated the pay ratio dividing total CEO pay by the average total staff costs per FTE. This may sound straightforward, but it isn’t:

  • I had to manually copy data from pdfs, so errors can’t be excluded;
  • Different methods may be used to calculate CEO pay (I used the total amount as reported by the company, without checking the method they used to calculate it);
  • It’s not always clear which categories of workers are included in staff costs and staff numbers;
  • If possible I used FTE for staff numbers, but sometimes only headcount is reported and some reports don’t specify what unit they used;
  • If possible I used the average number of staff, but sometimes only the number of staff at the end of the year is reported;
  • I didn’t annualise CEO pay for CEOs who were appointed during 2017. It might seem simple to do so (total pay * 365 / number of days worked), but in some cases CEO pay appears to contain elements that are not dependent on the number of days worked (e.g., the annual incentive at Philips Lighting).

All but two companies in my sample use EUR as presentation currency. I pragmatically used the average exchange rate for 2017 as reported by OCI to convert USD to EUR.

For the minimum wage, I used the average of the rates per 1 January and 1 July 2017, and added 8% holiday pay.

The ‘20 times minimum wage’ norm has been ascribed to trade union FNV, but that’s not technically correct.

The data I used can be found in this csv file. Please let me know if you find any errors.

Tags: 

Pages