Amsterdam, evaluated by cyclists

With the Ping if you care programme, the city of Amsterdam has collected about thirty thousand reports from cyclists. Participants had a button on their handlebars for reporting positive and negative aspects of their cycle route. The gps of their phones connected their reports to a location and afterwards, participants specified online what the report was about.

The city is using the reports to plan improvements, such as shorter waiting times at traffic lights. The programme has resulted in an interesting dataset and I had the opportunity to do an exploratory analysis for cyclists’ organisation Fietsersbond. The analysis comes with a few caveats, as explained at the end of the article.

I created a map that shows the geographical distribution of reports, by topic. It shows for example that, unsurprisingly, ‘attractive / atmospheric areas’ are found mainly in parks and along rivers and canals. However, some topics received so many reports that it’s hard to see the forest for the trees. Therefore, I tried to find a method to analyse patterns in these reports.

Problematic crossings

A case in point are problematic crossings (‘limited overview’ and ‘complicated crossing / intersection’). I have clustered reports about these topics by taking one report; looking up all other reports within 20 metres; then looking up all other reports within 20 metres of those reports, etcetera. The map below shows all clusters with more than 15 reports. The grey circles show where the largest clusters are.

There are many reports about problematic crossings on roads with busy car traffic, like the Stadhouderskade; at locations where traffic making a turn conflicts with other road users, like the eastern side of the Nieuwe Amstelbrug; and at the Haarlemmermeercircuit, where a controversial new design caused many complaints.

Waiting at traffic lights

Using the same method, I created a map of complaints about ‘(too) long waiting time at traffic lights’.

If you zoom in, you’ll notice that some reports are located off the road and sometimes even in a canal. That’s not really surprising, given that gps on phones isn’t perfect. This is also one of the reasons why it’s not always clear which traffic light a report is about. For example, at the entrance of the Vondelpark, most reports likely concern the crossing of the Stadhouderskade, but some are probably about crossing the Hobbemastraat.

Notice that reports appear to be positioned on a number of horizontal lines. I think this may have a practical cause: the coordinates appear to have been stored in fields with space for 6 digits. In Amsterdam, the value for longitude is around 4.9. With one digit before the decimal point, there’s room for 5 decimal places. The value for latitude is around 52.4; with two digits before the decimal point, there’s room for just 4 decimal places. In practical terms, this means that positions on the y-axis have been rounded at units of about 10 metres.

Bad road surface

The method I used above is suitable for analysing clusters around a specific location, such as a crossing, but less suitable for analysing reports that are associated with a road or part of a road. For such topics, I opted for a different approach, linking reports to roads (incidentally, that’s not as simple as it sounds). I used a road network extracted from Open Street Map. For pragmatic reasons, I merged all road segments (ways) with the same name.

The map below shows reports about bad road surface. I filtered reports linked to roads with more than 10 reports and more than 5 reports per km. A consequence of this approach is that longer roads will more easily pass the criteria. An alternative would have been to only use a ‘reports per km’ criterion, but then short roads with just a small number of reports would get included.

It’s still a busy map, but I think it provides more insight than a map showing all reports about bad road surface.

Scooters

I used the same approach to map concentrations of reports about scooters (‘conflict moped on cycle path’).

Complaints about scooters appear to occur often on main routes through the city, although this will be partly due to the fact that longer streets will more easily pass the criteria. I’ll return to the topic of scooters below.

Fast cycle routes

The Ping if you care programme was used to collect not just complaints, but also positive experiences. The map below shows routes suitable for uninterrupted cycling (‘pleasant continued cycling (no stopping)’ and ‘broad spacious cycle path’)

According to participants, fast cycle routes are in parks, along rivers and along the Sarphatistraat, which has been turned into a fietsstraat (a street designed specifically with cyclists in mind).

More about scooters

The data can also be used to analyse developments over time. This is particularly interesting for complaints about scooters. In the Netherlands, mopeds with blue registration plates (‘snorfietsen’) are allowed to use the cycle lane, but a new regulation has come into force in Amsterdam recently, banning scooters from most cycle lanes. The chart below shows the share of reports about scooters, by week.

The new regulation came into force on Monday 8 April; that was the beginning of week 15. From that moment, there’s been a considerable increase in the share of reports that are about scooters. Initially, many scooter riders ignored the new regulation, causing irritation among cyclists.

Not all cycle lanes are now scooter-free and a considerable number of reports about scooters was made on roads that do not have separate cycle lanes. The chart below shows reports about scooters on cycle lanes where scooters are now banned, as a share of all reports about scooters.

The pattern isn’t as clear-cut, but it appears that, since 8 April, reports about scooters increasingly are about cycle lanes where scooters are banned.

Explore the reports

As I indicated above, I’ve created a map where you can inspect all reports by topic.

I also created a version where you can first select certain routes and then a report category. For car (auto), bus, bicycle (fiets), tram and pedestrian (voetganger), it shows different categories of routes considered important by the municipality (the so-called plus and hoofd networks and the corridor for cars). If you select SNOR, you’ll see the scooter regime in force since 8 April: cycle lanes where scooters must use the street (blue), where they aren’t allowed anymore (orange) and where they can still use the cycle lane (green). You could for example display the SNOR routes in combination with reports about scooters (‘conflict moped on cycle path’).

There’s also an official website where you can see the Ping if you care data, offering different options for analysing the data.

Caveats

Some caveats apply:

• Participants are probably not entirely representative of all Amsterdam cyclists. For example, the lowest and highest age groups appear to be underrepresented, and they may encounter specific problems when cycling in Amsterdam.
• As indicated, gps on phones will have a margin of error.
• A concentration of complaints can mean a number of things. Perhaps the problem is very annoying. Perhaps many cyclists use that route. Or perhaps many cyclists who participate in Ping if you care. Perhaps one person made a series of complaints about the same topic.
• In summarising the data, I made some choices, e.g. on distance thresholds. Other choices might have resulted in somewhat different results.

Code

I used Python to analyse the data, using geopandas for geographical analysis and osmnx for downloading Amsterdam’s street network from Open Street Map.

Below is the code for creating clusters of reports.

import os
pdf_folder = 'data/pdf'
csv_folder = 'data/csv'
base_command = 'java -jar tabula-0.9.2-jar-with-dependencies.jar -n -p all -f TSV -o {} {}'
for filename in os.listdir(pdf_folder):
    pdf_path = os.path.join(pdf_folder, filename)
    csv_path = os.path.join(csv_folder, filename.replace('.pdf', '.csv'))
    command = base_command.format(csv_path, pdf_path)
    os.system(command)

And below is the code for linking locations to nearby streets.

def create_subset(point, lines, length):
    """Create subset """
    x = point.x
    y = point.y
    x_min = x - length
    x_max = x + length
    y_min = y - length
    y_max = y + length
    return lines[
        (lines.avg_x > x_min) &
        (lines.avg_x < x_max) &
        (lines.avg_y > y_min) &
        (lines.avg_y < y_max)
    ]

def nearest_line(point, lines):
    """Find line nearest line to point"""
    lines = lines.copy()
    lines['distance'] = lines.geometry.map(lambda x: point.distance(x))
    lines = lines.sort_values(by='distance')
    for i, row in lines.iterrows():
        return i, row.distance
    return None, None


def find_street(i):
    """Find nearest street"""
    point = pings_m.loc[i, 'geometry']
    for j in [10, 100, 1000, 5000]:
        subset_named = create_subset(point, osm_m_named, j)
        line, distance = nearest_line(point, subset_named)
        if pd.notnull(distance) and distance < THRESHOLD:
            return i, line, distance
        subset_not_named = create_subset(point, osm_m_not_named, j)
        line, distance = nearest_line(point, subset_not_named)
        if pd.notnull(distance) and distance < THRESHOLD:
            return i, line, distance
    return i, None, None