CategoryMaps

How to extract highways and subway lines from OpenStreetMap as a shapefile

It’s possible to use Overpass Turbo to extract any object from the OpenStreetMap “planet” and convert it from a GeoJSON or KML file to a shapefile for manipulation and analysis in GIS.

Say you want the subway lines for Mexico City, and you can’t find a GTFS file that you could convert to shapefile, and you can’t find the right files on Sistema de Transporte Colectivo’s website (I didn’t look for it).

Here’s how to extract the subway lines that are shown in OpenStreetMap and save them as a GIS shapefile.

This is my second tutorial to describe using Overpass Turbo. The first extracted places of worship in Cook County. I’ve also used Overpass Turbo to extract a map of campgrounds

Extract free and open source data from OpenStreetMap

  1. Open the Overpass Turbo website and, on the map, search for the city from which you want to extract data. (The Overpass query will be generated in such a way that it’ll only search for data in the current map view.)
  2. Click the “Wizard” button in the top toolbar. (Alternatively you can copy the code below and paste it into the text area on the website and click the “Run” button.)
  3. In the Wizard dialog box, type in “railway=subway” in order to find metro, subway, or rapid transit lines. (If you want to download interstate highways, or what they call motorways in the UK, use “highway=motorway“.) Then click the “build and run query” button.
  4. In a few seconds you’ll see lines and dots (representing the metro or subway stations) on the map, and a new query in the text area. Notice that the query has looked for three kinds of objects: node (points/stations), way (the subway tracks), relation (the subway routes).
  5. If you don’t want a particular kind of object, then delete its line from the query and click the “Run” button. (You probably don’t want relation if you’re just needing GIS data for mapping purposes, and because routes are not always well-defined by OpenStreetMap contributors.)
  6. Download the data by clicking the “Export” button. Choose from one of the first three options (GeoJSON, GPX, KML). If you’re going to use a desktop GIS software, or place this data in a web map (like Leaflet), then choose GeoJSON. Now, depending on what browser you’re using, a couple things could happen after you click on GeoJSON. If you’re using Chrome then clicking it will download a file. If you’re using Safari then clicking it will open a new tab and put the GeoJSON text in there. Copy and paste this text into TextEdit and save the file as “mexico_city_subway.geojson”.
Overpass Turbo screenshot 1 of 2

Screenshot 1: After searching for the city for which you want to extract data (Mexico City in this case), click the “Wizard” button and type “railway=subway” and click run.

Overpass Turbo screenshot 2

Screenshot 2: After building and running the query from the Wizard you’ll see subway lines and stations.

Overpass Turbo screenshot 3

Screenshot 3: Click the Export button and click GeoJSON. In Chrome, a file will download. In Safari, a new tab with the GeoJSON text will open (copy and paste this into TextEdit and save it as “mexico_city_subway.geojson”).

Convert the free and open source data into a shapefile

  1. After you’ve downloaded (via Chrome) or re-saved (Safari) a GeoJSON file of subway data from OpenStreetMap, open QGIS, the free and open source GIS desktop application for Linux, Windows, and Mac.
  2. In QGIS, add the GeoJSON file to the table of contents by either dragging the file in from the Finder (Mac) or Explorer (Windows), or by clicking File>Open and browsing and selecting the file.
  3. Convert it to GeoJSON by right-clicking on the layer in the table of contents and clicking “Save As…”
  4. In the “Save As…” dialog box choose “ESRI Shapefile” from the dropdown menu. Then click “Browse” to find a place to save this file, check “Add saved file to map”, and click the “OK” button.
  5. A new layer will appear in your table of contents. In the map this new layer will be layered directly above your GeoJSON data.
Overpass Turbo screenshot 4

Screenshot 4: The GeoJSON file exported from Overpass Turbo has now been loaded into the QGIS table of contents.

Overpass Turbo screenshot 5

Screenshot 5: In QGIS, right-click the layer, select “Save As…” and set the dialog box to have these settings before clicking OK.

Query for finding subways in your current Overpass Turbo map view

/*
This has been generated by the overpass-turbo wizard.
The original search was:
“railway=subway”
*/
[out:json][timeout:25];
// gather results
(
// query part for: “railway=subway”
node["railway"="subway"]({{bbox}});
way["railway"="subway"]({{bbox}});
relation["railway"="subway"]({{bbox}});
/*relation is for "routes", which are not always
well-defined, so I would ignore it*/
);
// print results
out body;
>;
out skel qt;

How to make a map of places of worship in Cook County using OpenStreetMap data

The screenshot shows the configuration you need to find and download places of worship in Cook County, Illinois, using the Overpass Turbo website.

If you’re looking to make a map of churches, mosques, synagogues and other places of worship, you’ll need data. The Yellow Pages won’t help because you can’t download that. And Google Maps doesn’t let you have a slice of their database, either. That’s where OpenStreetMap comes in. It’s a virtual planet that anyone can edit and anyone can have for free.

First we need to figure out what tag people use to identify these places. Sometimes on OSM there are multiple tags that identify the same kind of place. You should prefer the one that’s either more accurate (and mentioned as such in the wiki) or widespread.

The OSM tag info website says that editors have added over 1.2 million places of worship to the planet using “amenity=place_of_worship”.

Now that we know which tag to look for, we need an app that will help us get those places, but only within our desired boundary. Open up Overpass Turbo, which is a website that helps construct calls to the Overpass API, which is one way to find and download data from OSM.

In the default Overpass Turbo query, there’s probably a tag in brackets that says “[amenity=drinking_fountain]”. Change that to say “[amenity=place_of_worship]” (without the quotes). Now change the viewport of the map to show only the area in which you want Overpass Turbo to look for these places of worship. In the query this argument is listed as “({{bbox}})”.

The map has a search bar to find boundaries (cities, counties, principalities, neighborhoods, etc.) so type in “Cook County” and press Enter. The Cook County in Illinois, United States of America, will probably appear first. Select that one and the map will zoom to show the whole county in the viewport.

Now that we’ve set the tag to [amenity=place_of_worship] and moved the map to show Cook County we can click “Run”. In a few seconds you’ll see a circle over each place of worship.

It’s now simple to download: Click on the “Export” button and click “KML” to be able to load the data into Google Earth, “GeoJSON” to load it into a GIS app like QGIS, or “save GeoJSON to gist” to create an instant map within GitHub.

The best way to store directions on your smartphone is low tech

map of southern illinois as seen in the OsmAnd app

OsmAnd has great offline mapping features but it was tedious to ensure I had all of the maps at the desired zoom levels for the three-city bike ride in southern Illinois (pictured).

My friend is going to pick up a unique bicycle in Ohio and ride it back to Chicago. He designed a good route on Google Maps but now he needs to save it to his smartphone so he doesn’t have to constantly load directions and use data and waste battery life.

I gave him these instructions:

The best way to get a mobile view of the route is to use the Google Maps print feature and save it as a PDF. Then transfer that PDF to your phone through the Dropbox app. Then, in the Dropbox app, mark the PDF file as a favorite so that it’s stored offline, onto the phone.

There’s probably an app that can do what he wants, but I don’t know about it. There are hundreds of “maps” apps to sort through in each the App Store for iOS and the Play Store for Android.

In fact, I’ve downloaded OsmAnd, an offline maps app, for my Android tablet. I installed it and tried to learn how to use it in order to follow a downstate, intercity bike camping route. The app, though, required that you zoom in to each part of the map you wanted to store and then press “download”.

 

I spent 30 minutes downloading parts of the map, manually panning to the next section, before I decided to instead obtain one of the Illinois Department of Transportation’s regional bike maps and just draw it on there and write out a “cue sheet” (turn by turn directions).

Two things I don’t like about TIF expenditures in Chicago

Chicago Cityscape's TIF Projects map

I built a map of most Chicago TIF projects that you can filter on the fly. Type in any keyword, alderman’s name, or neighborhood and the map will re-center and zoom to the results.

1. Millions of dollars ($14.4 to be exact) has been or will be given to rich corporations, like Home Depot, to build massive stores with huge roofs and parking lots far away from where people live so everyone has to drive there. It’s highly unlikely they don’t mitigate stormwater runoff (except through temporary storage in a retention pond) or treat any of the water on site, contributing to local flooding and clogged pipes.

According to the project descriptions, property tax payers in these four TIF districts have partially subsidized the construction of over 1,903 car parking spaces and the associated ills of expansive asphalt areas and motorized traffic.

2. A massive subsidy was approved – $96 million – for McCaffery Interests’s Lakeside development on the former U.S. Steel South Works plant to build a mixed-use tower of 250 apartments in an area that has weak transit access and will take decades to fully fill out. We should instead be spending this kind of money building housing in already developed parts of the city (where there’s already amenities, or infrastructure for amenities – the Rezko land comes to mind).

What’s interesting about the Lakeside TIF project approval is that the containing TIF district, “Chicago Lakeside Development Phase 1”, has collected zero property tax revenue because there is no property in it!

Trolley on the future Lake Shore Drive

A tour bus drivers on the Lakeside development. Photo by Ann Fisher.

There are some projects I like, though. TIF has been used frequently to build affordable housing, housing for seniors, and housing for people who need assistance. 78 out of 380 projects mention the word “affordable”.

The City Hyde Park building, designed by Studio Gang Architects, will have 20% of its residential units designated as “affordable”, for families (of varying sizes) earning up to 60 percent of the area median income. The city standard is 10 percent but developers are also able to pay an “in lieu” fee so they don’t have to build the affordable units and instead can offer those units at market rates.

Other projects have a majority of affordable units.

Chicago’s TOD rule is the only reason multi-family is being built in neighborhoods

This is the ordinance that says residential developments have to provide 0.5 car parking spaces per home, and that the minimum home size can be smaller.

How many units? At least 1,500. Here’re the 19 buildings I know about that are being built within 600 and 1,200 feet* of a Chicago Transit Authority ‘L’ station – the only areas, essentially, where multi-family housing can be developed.

Why can’t dense housing be built elsewhere? Because the most desirable living areas in Chicago – along retail streets in Logan Square, North Center, Lincoln Park, Lakeview, and West Town – are zoned for single-family use. (And ad-hoc zoning districts taking the place of community land use planning.)

How do I know popular neighborhoods are zoned for single-family use? Because Daniel Hertz’s new Simplified Chicago Zoning Map makes it easy to see. Yep, even along those dense business districts and even outside the train stations.

Do the single-family home zones contain single-family homes now? Absolutely not! Much of the buildings in areas zoned for single-family homes have everything but! The particular view of the map that Hertz uses in his blog post shows that even adjacent to CTA stations, and within 1 block, there are only single-family zones (in red). There are many multi-family buildings in these red zones.

Red areas are zoned for single-family homes only.

Red areas are zoned for single-family homes only. View the map.

What ends up happening there? Teardowns. And the Lakeview Chamber of Commerce finds believes that non-matching zoning – it matches neither the existing uses nor the needs for the neighborhood – and teardowns are going to cut into consumer spending on its lively retail streets. Lakeview is seeing a population change to families which tend to have less disposable income.

More housing in a popular neighborhood means more shoppers, more property taxes, more “boots on the ground”, more “pedestrian congestion” in front of our local businesses.

Doesn’t the ordinance make station-adjacent parcels friendly to multi-family housing because of the TOD ordinance? Yes, and no. As Hertz points out, “virtually every sizable development involves a zoning variance or planned development process that goes beyond the zoning you’ll see on the map”.

The TOD ordinance is 19 months old and working exactly as intended, building more housing next to train stations, and giving more people the opportunity to have access to affordable transportation. So it needs an upgrade to be able to do more. Since, in Chicago, zoning is our land use plan, we need the best kind of zoning rules and this is one of the best.

Imagine what the TOD ordinance could do if it were expanded. Think, making the parking requirement relief and allowing different unit sizes by-right instead of going through an arduous and expensive zoning change process. Then, expanding the rule to include more than just 600 feet (which is less than a block) from a train station – people walk several blocks to get to CTA stations, and bike even more. And, beefing up the affordable housing requirements.

Let’s do this, Commissioner Andrew Mooney. Let’s do this, housing advocates. Let’s do this, transit advocates. I’m looking at you, Latin United Community Housing Association (LUCHA), Logan Square Neighborhood Association (LSNA), We Are/Somos Logan Square, Pilsen Alliance, Metropolitan Planning Council (MPC), Active Transportation Alliance, and the Center for Neighborhood Technology (CNT).

* The distance depends on existing Pedestrian Street zoning. If the property is on a designated Pedestrian Street then the station can be up to 1,200 for the ordinance to apply, double the normal 600 feet.

Not all demolition activity is alike on Chicago’s North and South Sides

According to building permit activity in the past 30 days there are more demolitions on the North Side than on the South Side, but most of those will have new buildings.

According to building permit activity in the past 30 days there are more demolitions on the North Side than on the South Side, but most of those will have new buildings. Map by Chicago Cityscapen using Mapbox satellite imagery.

I review the new permits each day on Chicago Cityscape, both in the list and map views, to get a sense of what’s being built around the city. This keeps me informed so I can tweet interesting permits, respond to people’s questions several times a week, and even help out on a Moxie tour of Motor Row and the Cermak Green Line station by mention new construction and renovation permits I’ve seen pop up nearby.

This week I was poking around the Demolitions Tracker and saw that, not unusually, there were a lot of houses being demolished on the South Side. So I checked out a 30-day view of demolitions in the full-screen map that shows the entire city.

While the North Side has more demolitions going on than the South Side, most of them are part of teardowns – the demolished buildings are getting replaced or allowing for expansion. Demolitions on the South Side are much less likely to have associated new construction projects than the North Side.

Tina Fassett Smith tweeted back:

Use Turf to perform GIS functions in a web browser

Turf's merge function joins invisible buffers around each Divvy station into a single, super buffer.

Turf’s merge function joins invisible buffers around each Divvy station into a single, super buffer –all client-side, in your web browser.

I’m leading the development of a website for Slow Roll Chicago that shows the distribution of bike lane infrastructure in Chicago relative to key and specific demographics to demonstrate if the investment has been equitable.

We’re using GitHub to store code, publish meeting notes, and host discussions with the issues tracker. Communication is done almost entirely in GitHub issues. I chose GitHub over Slack and Google Groups because:

  1. All of our research and code should be public and open source so it’s clear how we made our assumptions and came to our conclusions (“show your work”).
  2. Using git, GitHub, and version control is a desirable skill and more people should learn it; this project will help people apply that skill.
  3. There are no emails involved. I deplore using email for group communication.*

The website focuses on using empirical research, maps, geographic analysis to tell the story of bike lane distribution and requires processing this data using GIS functions. Normally the data would be transformed in a desktop GIS software like QGIS and then converted to a format that can be used in Leaflet, an open source web mapping library.

Relying on desktop software, though, slows down development of new ways to slice and dice geographic data, which, in our map, includes bike lanes, wards, Census tracts, Divvy stations, and grocery stores (so far). One would have to generate a new dataset if our goals or needs changed .

I’ve built maps for images and the web that way enough in the past and I wanted to move away from that method for this project and we’re using Turf.js to replicate many GIS functions – but in the browser.

Yep, Turf makes it possible to merge, buffer, contain, calculate distance, transform, dissolve, and perform dozens of other functions all within the browser, “on the fly”, without any software.

After dilly-dallying in Turf for several weeks, our group started making progress this month. We have now pushed to our in-progress website a map with three features made possible by Turf:

  1. Buffer and dissolving buffers to show the Divvy station walk shed, the distance a reasonable person would walk from their home or office to check out a Divvy station. A buffer of 0.25 miles (two Chicago blocks) is created around each of the 300 Divvy stations, hidden from display, and then merged (dissolved in traditional GIS parlance) into a single buffer. The single buffer –called a “super buffer” in our source code – is used for another feature. Currently the projection is messed up and you see ellipsoid shapes instead of circles.
  2. Counting grocery stores in the Divvy station walk shed. We use the “feature collection” function to convert the super buffer into an object that the “within” function can use to compare to a GeoJSON object of grocery stores. This process is similar to the “select by location” function in GIS software. Right now this number is printed only to the console as we look for the best way to display stats like this to the user. A future version of the map could allow the user to change the 0.25 miles distance to an arbitrary distance they prefer.
  3. Find the nearest Divvy station from any place on the map. Using Turf’s “nearest” function and the Context Menu plugin for Leaflet, the user can right-click anywhere on the map and choose “Find nearby Divvy stations”. The “nearest” function compares the place where the user clicked against the GeoJSON object of Divvy stations to select the nearest one. The problem of locating 2+ nearby Divvy stations remains. The original issue asked to find the number of Divvy stations near the point; we’ll likely accomplish this by drawing an invisible, temporary buffer around the point and then using “within” to count the number of stations inside that buffer and then destroy the buffer.
Right-click the map and select "Find nearby Divvy stations" and Turf will locate the nearest Divvy station.

Right-click the map and select “Find nearby Divvy stations” and Turf will locate the nearest Divvy station.

* I send one email to new people who join us at Open Gov Hack Night on Tuesdays at the Mart to send them a link to our GitHub repository, and to invite them to a Dropbox folder to share large files for those who don’t learn to use git for file management.

This is why we need more people editing OpenStreetMap

Unmapped homes in the Irving Park community area

These homes were built after the City of Chicago’s building footprints dataset was created (2010?). Ian Dees imported the dataset in 2012. Many of the buildings that you can now see on Bing Maps have not been present on Bing’s satellite imagery since at least 2012.

1. OpenStreetMap is the world’s most complete free map, to which anyone can contribute their “ground truth” data (the location of wells and convenience stores, road names, and whether Lula Café at 2537 N Kedzie Boulevard in Logan Square has outdoor seating).

2. OpenStreetMap is used by thousands of non-profit and non-governmental organizations, corporations, apps, and people daily to locate themselves, locate others, get directions, and find places.

3. Nearly every map is out of date the moment it is published, including online, “current” maps like Google Maps, Bing Maps, their competitors, and OpenStreetMap.

4. Bing Maps provides its satellite imagery to OpenStreetMap editors – you and me – so that we can trace (copy) things on the planet to be things on the map. Google Maps doesn’t allow tracing (copying).

5. Bing updated its satellite imagery for Chicago (and probably a lot of other places) within the last six weeks…and there are hundreds of objects that aren’t yet mapped in OpenStreetMap. In Chicago most of these buildings are newly constructed houses.

Those hundreds of houses now need to be added to OpenStreetMap, with addresses, to complete the buildings collection in Chicago, and to expand the gazetteer (an address book) of places in Chicago.

I’m glad you want to help me do it! Here are two helpful things you can do:

  1. Start tracing the buildings yourself (here’s how new mappers can get started), or
  2. Leave notes at buildings which aren’t yet mapped so that map editors like myself know where to look to trace buildings.

Update: There’s a bonus third thing you can do, and that’s come to the next MaptimeCHI event on Thursday, February 26th, at the Chicago Community Trust (225 N Michigan, 22nd floor). RSVP for Anatomy of a Web Map. The Trust will also provide food and beverages. I’ll be there to teach new mappers and assist generally.

Adding notes is extremely helpful

You can contribute without editing by adding notes describing new things, or identifying problems with existing things. Click the “Add a note” button on OpenStreetMap.org.

Why architects should learn OpenStreetMap

I’m teaching OpenStreetMap 101 at the first MaptimeCHI.

Architects will learn that OpenStreetMap can be used as a data source when developing projects and as a basis for designing custom maps in project publications (website, anthology, monograph, client presentations).

This meeting is about getting an introduction to OpenStreetMap and learning to make your first edit in the “Wikipedia of maps”.

Thursday, July 17th, from 6-8 PM
Thoughtworks office
200 E Randolph St

RSVP on EventBrite.

Here are two examples of how architects could use OpenStreetMap data.

Example 1 of how to use OpenStreetMap. Instead of publishing a screenshot of Google Maps in your documents or website, create a custom design map like this without having to spend so much time tweaking it in Illustrator. This map was created by Stamen Design using TileMill.

Example 1 of how to use OpenStreetMap. Instead of publishing a screenshot of Google Maps in your documents or website, create a custom design map like this without having to spend so much time tweaking it in Illustrator. This map was created by Stamen Design using TileMill.

And the second.

Willow Creek Church on OpenStreetMap: After

Here’s one example where OpenStreetMap could be useful. Let’s say you’re working on a site plan for Willow Creek Church in South Barrington and you need a general layout of the parking lot. 1. You can get it from OpenStreetMap because it’s already there. 2. You can draw it in OpenStreetMap yourself (to benefit all other OSM users) and then extract it as a shapefile.

Maptime is time for mapmaking and it’s taking the country by storm.

How Chicagoans commute map: An interview with the cartographer

Chicago Commute Map by Transitized

A screenshot of the map showing Lakeview and the Brown, Red, Purple and Purple Line Express stations.

Shaun Jacobsen blogs at Transitized.com and yesterday published the How Chicagoans Commute map. I emailed him to get some more insight on why he made it, how, and what insights it tells about Chicago and transit. The map color-symbolizes census tracts based on the simple majority commuting transportation mode.

What got you started on it?

It was your post about the Census data and breaking it down by ZIP code to show people how many homes have cars. I’ve used that method a few times. The method of looking up each case each time it came up took too long, so this kind of puts it in one place.

What story did you want to tell?

I wanted to demonstrate that many households in the city don’t have any cars at all, and these residents need to be planned for as well. What I really liked was how the north side transit lines stuck out. Those clearly have an impact on how people commute, but I wonder what the cause is. Are the Red and Brown Lines really good lines (in people’s opinions) so they take them, or are people deciding to live closer to the lines because they want to use it (because they work downtown, for example)?

The reason I decided to post the map on Thursday was because while I was writing the story about a proposed development in Uptown and I wanted  information on how many people had cars around that development. As the map shows, almost all of Uptown is transit-commuting, and a lot of us don’t even own any cars.

What data and tools did you use?

I first used the Chicago Data Portal to grab the census tract boundaries. Then I grabbed all of the census data for B08141 (“means of transportation to work by number of vehicles available”) and DP04 (“selected housing characteristics”) for each tract and combined it using the tract ID and Excel’s VLOOKUP formula.

Read the rest of this interview on Web Map Academy.

© 2017 Steven Can Plan

Theme by Anders NorénUp ↑