Technical

Worldwide GPS tracks with OpenStreetMaps for urban design analysis

I work as an architect, and one of the data available to us when masterplanning and early phases of an urban design project is GPS track activity. Knowing where people drive, where people walk, and cycle and recreate allows us to make decisions such as where to define architectural axes, where to place retail, and how to extend public transport and pedestrian walkways.

One of the resources available is from a company known as Strava, who runs a proprietary fitness social network, where fitness buffs can track their movements via GPS devices (which can be as simple as your phone) and compare cycling routes, distances run, and so on. Primarily used by runners and cyclists, these GPS logs are voluntarily uploaded to Strava, who then aggregates all the data and resells it to urban design parties, known as their “Strava Metro” initiative.

Publicly without purchasing any data, Strava also hosts a global GPS heatmap where you can visually see the fire of activity by runners and cyclists. Zooming in shows you right down to where people run down various streets. As a high-level overview, this is a great graphic and can immediately pinpoint activity. It also is a fantastic feat of engineering, processing 5 terabytes of raw input data. That’s big data!

Strava heatmap example

Of course, just recently Strava decided to stop showing this public heatmap at high zoom levels and locked it behind a paywall. Thankfully, there are alternatives.

In a previous post, I introduced an open-data initiative known as OpenStreetMaps. Strava is largely based on OpenStreetMaps and uses it as a base layer embedded into MapBox, and also has a fork of the OSM iD editor called “Strava Slide”, to allow people to edit routes based off strava GPS data tracks. However, OSM itself has many active GPS track contributors (used for various purposes, such as mapping new routes and calibrating the map), and we can use this open data in lieu of the proprietary product offered by Strava. Below, we see the world’s GPS tracks from the perspective of OSM visualised by Pascal Neis.

OSM GPX tracks

Before I get into the specifics of getting GPS data, I’d like to show you what data is in a GPS track. Here’s some GPS tracks visualised with JOSM. We can see things like speed, direction, and sometimes, elevation, if it is recorded. See those red segments of the line? Those are traffic lights!

GPS track velocity visualised with JOSM

OSM has an API and Planet GPS extract available to download GPS data. The Planet GPX is rather unwieldy, and is also very outdated (from 2013). The API is not the best either, in that it only returns 5,000 GPS points per query, and doesn’t quantify the total pages of results, so that you can’t really tell with one query how many points you need to fetch in advance. However, if you query the API and put a page number higher than what is available, it won’t return any points. So using a binary search you can find out how many pages to extract.

For the area of Sydney, there are roughly 750 pages of results, so that means just under 4 million GPS points. Here’s a heat map visualisation of it I made using QGIS (but JOSM also has a heat map visualisation feature). You won’t need huge supercomputers processing the data, either.

Sydney, Australia GPS activity heat map

Here’s another of Manhattan, New York.

Manhattan, New York, GPS activity heat map

We can couple this visualisation with other OSM data such as all public transport nodes. In this case, railway tracks, bus routes, ferry routes, cycling routes, train stations, and bus stops are shown. This is also created with QGIS.

Sydney public transport GPS analysis

There are a few pros and cons to using this GPS data. The pro is that it’s more general purpose: it’s not only used by runners and cyclists, it’s used by regular people (well, GIS geeks) doing everyday things like shopping. Unfortunately, OSM isn’t that widely used, and so the data is relatively sparse. In remote areas perhaps no-one has walked that route, or only a few people have. So you don’t get a sense of what they’re doing. The GPS data is also not processed, so you’ll have to do your own cleaning: especially in the city where GPS data goes a bit haywire with all the tall buildings.

Have fun and happy mapping!

Life & much, much more

OpenStreetMaps – an open-source Maps application

Recently I’ve been interested in an initiative known as OpenStreetMaps. Launched in 2004, OpenStreetMaps is the open-source equivalent of Google Maps, and functions largely like how Wikipedia does (and in fact was inspired by Wikipedia) – it’s a map of the world drawn completely by volunteers and open-source enthusiasts.

OpenStreetMaps world map

You might’ve already seen OSM in action. Below it’s used by default in the privacy-friendly search engine DuckDuckGo, other wiki-based projects like WikiVoyage, and many games use it as a base layer, such as Pokemon Go.

PokemonGo uses OpenStreetMaps as a base layer

You’ve probably used Google Maps before and have it installed on your phone to help you drive to places with the GPS. You may have also played with Bing Maps which essentially does the same thing. At first glance OpenStreetMaps is purely a clone: you can zoom in and out, look at street names and see buildings, and have it tell you how to drive to a destination. It’s not that exciting, and isn’t worth talking about.

However if you were a user of OSM, occasionally you might notice areas of the map where volunteers have gone above and beyond to draw details of the environment that other maps will not. Things like individual driveways, articulated building outlines, kerbside grass, wheelchair accessible walkways and kerb ramps, and individual bush and tree locations, fences, and parking niches. Zooming in we can identify storm drains, streetlamps, water taps and park benches. This level of detail is possible because the map is created by people who are genuinely interested and express a love and care in their work. The example below is in Brisbane, Australia, largely by a fellow called ThorstenE.

OpenStreetMaps example in Brisbane, Australia

Where OSM really excels is as an open-data resource. Usually, you are only limited to raster map images produced by Google Maps and Bing, but aren’t allowed to access the underlying database of geographic and vector information. In contrast, because the data in OSM’s database is free for everyone, specialist maps can easily be created. Take for example the extensive mapping of skiing and snowboarding tracks in Oslo, Norway provided by OpenSnowMap

OpenSnowMaps

… alternatively there is the Whitewater rafting map in the UK …

Open Whitewater Rafting maps

… and the OpenCycleMap which maps the world’s bicycle routes, and shows the incredible culture of pedestrian and cycling friendly urban planning in the Netherlands.

OpenCycleMaps example

OSM also helps lead the way in humanitarian mapping. When a flood, fire, earthquake or other natural disaster occurs, existing maps provided by Google and Bing are no longer current. Mappers need to create new maps to allow disaster relief teams to coordinate their efforts, target houses for rescues efficiently, or to know what routes relief organisations can take to navigate the terrain. This work is done by the excellent Humanitarian OpenStreetMap Team. It also includes non natural disasters, such as mapping demographics and environmental issues related to poverty elimination, gender equality, refugee response strategies, public disease outbreaks, clean energy, and water and sanitation. As one current example, right this minute the Monsoon rains have caused severe flooding in the Kurunagala and Puttalam districts of Sri Lanka. A map is being prepared so that first respondents and aid agencies can deliver relief supplies. A grid of zones with their mapping progress is updated in real time below.

Humanitarian OpenStreetMaps Team map in Nepal

As an open-source creation, it doesn’t data-mine your activity so you can use it as a Maps application without privacy concerns, you can download the raw vector so you can use it offline on your phone, and has a conservative approach to licensing data that allow people who want to embed OSM technologies in their own creations in a much more flexible manner. If you feel strongly about supporting privacy-aware applications (especially after the Cambridge Analytica scandal), and encouraging communities that aren’t motivated by profit, OpenStreetMaps should be something to consider. There are over 1,000,000 mappers who have contributed to OSM, and you can become one of those too.

One of the most amazing things about OSM is that whereas mapping the world is an inherently complex process, it has managed to make it easy and fun and doable by anybody who knows how to draw a rectangle with their mouse. Most of the other open-source initiatives have a high learning curve and lots of technical prerequisites, but OSM is completely the opposite. Just zoom into your city on OSM.org and click the Edit button on the top left. It will give you a short tutorial that lets you draw new roads and buildings within minutes. The thought that has gone into the user-friendliness of this online map editor is absolutely incredible.

OpenStreetMaps iD web-based editing software

I’ll talk about OSM a bit more in upcoming posts, and share some of the more interesting technical sides of things.