CategoryGIS

Get out of Googleville: my presentation on web mapping

Alternate headlines: Google Maps versus OpenStreetMap; why OpenStreetMap is better than Google Maps

I presented to the Chicago GIS Network Meetup group on February 5,2013, about alternatives to Google when it comes to mapping on the web. I created the presentation and outline a couple hours before giving it and came up with this slideshow with three frames.

Googleville 1 of 3

Google Maps and its data is a one-way street (or many one-way streets). Google will take data but won’t give it back.

Googleville 2 of 3

Google Maps has all of these features, but they’re easier to manipulate when you use an alternative. Alternatives like: MapBox, TileMill, OpenLayers, OpenStreetMap (made easy with JOSM), GeoCommons – I’m sure there are plenty more.

Googleville 3 of 3

OpenStreetMap is the Wikipedia of online mapping and geographic data. Considering switching to OSM.

Converting shapefiles to GeoJSON, and other format conversions

To develop the Chicago Bike Map app, I had a problem I thought would be simple to solve: load train lines into a Leaflet-powered map. I had the train lines stored as a polyline shapefile but Leaflet can only read the GeoJSON format or a string of geographic coordinates representing lines.

I eventually found a solution (I can’t remember how) and I need to share it with you. The converter can do more than ESRI shapefiles to GeoJSON. It can reproject the data in the conversion. It can convert from several formats to several other formats.

The site is called MyGeodata Converter. You upload a ZIP file of geographic files – .shp and its companion files (.prj, .dbf, .shx), .kml, and .gpx. Let’s take the Chicago Transit Authority train lines shapefile straight from the City of Chicago’s open data portal. It downloads as a zipped collection of a shapefile and its buddies and we can take this file straight to the Converter and upload it. The Converter will unzip it and read the data; it will even identify the projection system (for Chicago-based geographic data, its common to use NAD83 Illinois StatePlane East FIPS 1201 Feet (SRID 102671, the same as SRID 3435).

The Converter will convert to one of the following formats, with same or new projection; accepts SQL statements to extract a subset of data:

  • ESRI shapefile
  • GML
  • KML, KMZ
  • GeoJSON
  • Microstation DGN
  • MapInfo File
  • GPX
  • CSV

How I created a map of Illinois Amtrak routes in TileMill in less than 30 minutes

This interactive map was created for a Grid Chicago article to show the cities and Amtrak routes mentioned. Click and drag it around or hover your mouse on the red train station markers. 

Want to create a map like that and publish it on your own website? It’s easy. I’ll show you how to do it in less than 30 minutes. First, download the following files:

All shapefiles are from the United States Department of Transportation, Bureau of Transportation Statistics’s National Transportation Atlas 2012 edition except for Illinois places, which comes from the Census Bureau’s TIGER project.

At the end of this tutorial, you’ll have a good introduction on how to find geographic data, build a map with TileMill, style the map, and publish it for the public. Your map will not look like mine as this tutorial doesn’t describe how to add labels or use the hover/info feature.

Tutorial to make Amtrak Illinois map

  1. Unzip the four ZIP files you downloaded and move their contents into a folder, like /Documents/GIS/Amtrak Illinois/shapefiles. This is your project folder.
  2. Install TileMill and open it.
  3. Set up a project. In the Projects pane, click “New Project”. In the filename field, title it “amtrak_illinois”. Ensure that the checkbox next to “Default data” is checked – this shows a world map and helps you get your bearings (but it’s not absolutely necessary).
  4. Get familiar with TileMill’s layout. Your new project will open with the map on the left side and your Carto style code on the right side. There are four buttons aligning the left edge of your map. From top to bottom they are: Templates, Font list, Carto guide, and Layers.
  5. Add a layer. We’re going to add the four shapefile layers you downloaded. Click the “Layers” button and then click “Add layer”. In the ID field, type in “amtrak_routes”. For Datasource, browse to your project folder and find “amtrak.shp” – this file has the Amtrak route lines. Then click “Done”. Click “Save & Style”.
  6. Style that layer. When you click “Save & Style” after adding a layer, your attention will be called to the Carto style code on the right side of TileMill. A section of code with the “amtrak_routes” #selector will have been inserted with some default colors and styles. If you know CSS, you will be familiar with how to change the Amtrak routes line styles. Change the “line-color” to “#000”. After “line-color”, add a new line and insert “line-opacity: 0.5;”. This will add some transparency to the line. Press the “Save” button above the code.
  7. Add remaining layers. Repeat Step 5 and add 3 more layers: “amtrk_sta.shp” (ID field: “amtrak_stations”), “state.shp” (ID field: “states”), and “tl_2012_17_place.shp” (ID field: “illinois_cities”).
  8. Hide bus stations. The Amtrak stations layer shows bus and ferry stations as part of Amtrak’s Thruway connections. You probably don’t want to show these. In your Carto style code, rename the #selector from “#amtrak_stations” to “#amtrak_stations[STNTYPE=’RAIL’]”. That makes the following style code only apply to stations with the “rail” type. Since there’s no style definition for things that aren’t of that type, they won’t appear.

Screenshot of my map.

Prepare your map for uploading

TileMill has many exporting options. You can save it as MBTiles and publish the map for free using MapBox (TileMill’s parent), or you can export it as image files (but it won’t be interactive), or you can display the map using the Leaflet JavaScript map library (which I use for the Chicago Bike Map app). This tutorial will explain how to export MBTiles and upload to MapBox, the server I’m using to display the map at the top of this page.

  1. Change project settings. To upload to MapBox, you’ll have to export your project as MBTiles, a proprietary format. Click the “Export” button above your Carto style code and click “MBTiles”. You’ll be asked to provide a name, description, attribution, and version. Input appropriate text for all but version.
  2. Adjust the zoom levels. Adjust the number of zoom levels you want (the more you have the longer it takes to export and upload your project, and you might exceed MapBox’s free 50 MB account limit). My map has zoom levels 8-11.
  3. Adjust the bounds. You’ll then want to draw your bounds: how much of the map’s geographic extents you want to export. Zoom to a level where you can see the entire state of Illinois in your map. Hold down the Shift key and drag a box around the state, plus a buffer (so viewers don’t fall of your map when they pan to the edges).
  4. Export your map. Click Export and watch the progress! On a four-year-old MacBook it took less than one minute to export the project.
  5. Bring the export to your project folder. When export finishes, click the “Save” button and browse to your project folder. Click the file browser’s save button.
  6. Upload to MapBox. Login to MapBox’s website and click “Upload Layer”. Browse to your project folder, select the .mbtiles folder, and click “Upload file”. Upon a successful upload, your map will display.
  7. Embed it in your website. Click the “Share” button in the upper left corner of your map and copy the embed code. Paste this into the HTML source code of a webpage (or in a WordPress post) and save that (I’m not going to provide instructions on how to do that).

Now you know how to find geographic data, build a custom map using the TileMill application, begin to understand how to style it, and embed your map for the public on a website or blog.

N.B. I was originally going to use QGIS to build a map and then publish a static image before I realized that TileMill + MapBox (the website) can build a map but publish an interactive feature instead of a static image. I’m happy I went that route. However, I did use QGIS to verify the data and even create a new shapefile of just a few of the key train stations on the Lincoln Service (the centerpiece of my Grid Chicago article).

How to split a bike lane in two and copy features with QGIS

A screenshot of the splash image seen on users with iPad retina displays in landscape mode. 

To make the Chicago Offline Bike Map, I need bikeways data. I got this from the City of Chicago’s data portal, in GIS shapefile format. It has a good attribute table listing the name of the street the bikeway is on and the bikeway’s class (see below). After several bike lanes had been installed, I asked the City’s data portal operators for an updated shapefile. I got it a month later and found that it wasn’t up-to-date. I probably could have received a shapefile with the current bikeway installations marked, but I didn’t have time to wait: every day delayed was one more day I couldn’t promote my app; I make 70 cents per sale.

Since the bikeway lines were already there, I could simply reclassify the sections that had been changed to an upgraded form of bikeway (for example, Wabash Avenue went from a door zone-style bike lane to a buffered bike lane in 2011). I tried to do this but ran into trouble when the line segment was longer than the bikeway segment that needed to be reclassified (for example, Elston Avenue has varying classifications from Milwaukee Avenue to North Avenue that didn’t match the line segments for that street). I had to divide the bikeway into shorter segments and reclassify them individually.

Enter the Split Features tool. QGIS is short on documentation and I had trouble using this feature. I eventually found the trick after a search that took more time than I expected. Here’s how to cut a line:

  1. Select the line using one of the selection tools. I prefer the default one, Select Features, where you have to click on the feature one-by-one. (It’s not required that you select the line, but doing so will ensure you only cut the selected line. If you don’t select the line, you can cut many lines in one go.)
  2. Toggle editing on the layer that contains the line you want to cut.
  3. Click Edit>Split Features to activate that tool, or find its icon in one the toolbars (which may or may not be shown).
  4. Click once near where you want to split the line.
  5. Move the cursor across the line you want to split, in the desired split location.
  6. When the red line indicating your split is where you desire, press the right-click mouse button.

Your line segment has now been split. A new entry has been added to the attribute table. There are now two entries with duplicate attributes representing that together make up the original line segment, before you split it.

This screenshot shows a red line across a road. The red line indicates where the road will be split. Press the right-click mouse button to tell QGIS to “split now”.

After splitting, open the attribute table to see that you now have two features with identical attributes. 

Copying features in QGIS

A second issue I had when creating new bikeways data was when a bikeway didn’t exist and I couldn’t reclassify it. This was the case on Franklin Boulevard: no bikeway had ever been installed there. I solved this problem by copying the relevant street segments from the Transportation (roads) shapefile and pasted them into the bikeways shapefile. New entries were created in the attribute table but with blank attributes. It was simple to fill in the street name, class, and extents.

Chicago bikeways GIS description

Bikeway classes (TYPE in the dataset) in the City of Chicago data portal are:

  1. Existing bike lane
  2. Existing marked shared lane
  3. Proposed on-street bikeway
  4. Recommended bike route
  5. Existing trail
  6. Proposed off-street trail
  7. Access path (to existing trail)
  8. Existing cycle track (also known as protected bike lane)
  9. Existing buffered bike lane

It remains to be seen if the City will identify the “enhanced marked shared lane” on Wells Street between Wacker Drive and Van Buren street differently than “existing marked shared lane” in the data.

Last minute vote on bicycle data visualization project

The United States Department of Transportation is holding a data visualization competition and Chicago bike crash locations are one of the topics.

From Michael Carney:

I started this project because, basically, I thought it would be interesting to make a map of bike crash locations around Chicago and present it to my GIS class at UIC. Sebastian Lew and I collaborated for several months and it evolved into something more. Using ArcGIS, we were able to symbolize streets by level of overall crash intensity, calculate crashes per mile on streets, perform hot spot analysis to identify areas with high numbers of crashes, compare ridership levels with crash levels, examine temporal trends in crash activity, and perhaps most importantly, assess the effectiveness (at least at a basic level) of current bicycle infrastructure. At the very least I hope our project can help you plan a safe route to work, at the most I hope it can be used by policymakers and planners when considering how and where to expand Chicago’s bike infrastructure.

Vote today onlyView the project.

Are protected bike lanes going in the right places?

Bike crash map of Ogden, Milwaukee, Chicago

Common bike-car crash locations in West Town. The bottom blue circle identifies Ogden/Milwaukee, where there is a yellow trap for northbound, left-turning motorists (from Milwaukee to Ogden) that makes them run into southbound bicyclists who have a green light.

My contribution to a discussion on The Chainlink, Are protected bike lanes going in the right places?

Kelvin, Milwaukee/Ogden/Chicago is the intersection along Milwaukee Avenue with the highest number of bicycle crashes. I created this table and map to show them, using data from 2007-2009.

The blue rings on the map are called, in GIS parlance, “buffers” and are circles used to select things (in this case, bike crashes) within a certain distance of the circle center. In this map I used 50 feet radius buffers (100 feet diameter). While this distance encompasses the intersection from center to all four curbs, it doesn’t encompass the crashes that happened just outside the buffer that were still most likely influenced by the intersection (like drivers’ turning movements).

I am working on a project with three friends to create a better map and “crash browser”. I mentioned it in the last story on Grid Chicago in this post. For this project, we are using 200 feet radius (400 feet diameter) buffers to ensure we encompass the entire intersection and the area in which it still has an effect. This also grabs the bike lane “pinch points”, places where a bike lane doesn’t start until 100-200 feet beyond the intersection.

I am also concerned with the strategy and approach CDOT is using to choose locations. It’s not transparent; at MBAC, CDOT said they were choosing locations “without controversy and that could be implemented quickly”.

Read more about Kinzie Street, Chicago’s first protected bike lane, and my other thoughts on protected bike lanes

How to upload shapefiles to Google Fusion Tables

It is now possible to upload a shapefile (and its companion files SHX, PRJ, and DBF) to Google Fusion Tables (GFT).

Before we go any further, keep in mind that the application that does this will only process 100,000 rows. Additionally, GFT only gives each user 200 MB of storage (and they don’t tell you your current status, that I can see).

  1. Login to your Google account (at Gmail, or at GFT).
  2. Prepare your data. Ensure it has fewer than 100,000 rows.
  3. ZIP up your dataX.shp, dataX.shx, dataX.prj, and dataX.dbf. Use WinZip for Windows, or for Mac, right-click the selection of files and select “Compress 4 items”.
  4. Visit the Shape to Fusion website. You will have to authorize the web application to “grant access” to your GFT tables. It needs this access so that after the web application processes your data, it can insert it into GFT.
  5. If you want a Centroid Geometry column or a Simplified Geometry column added, click “Advanced Options” and check their checkboxes – see notes below for an explanation.
  6. Choose the file to upload and click Upload.
  7. Leave the window open until it says it has processed all of the rows. It will report “Processed Y rows and inserted Y rows”. You will be given a link to the GFT the web application created.

Sample Data

If you’re looking to give this a try and see results quickly, try some sample data from the City of Chicago data portal:

Notes

I had trouble many times while using Shape to Fusion in that after I chose the file to upload and clicked Upload, I had to grant access to the web application again and start over (choose the file and click Upload a second time).

Centroid Geometry – This creates a column with the geographic coordinates of the centroid in a polygon. It lists it in the original projection system. So if your projection is in feet, the value will be in feet. This is a function that can easily be performed in free and open source QGIS, where you can also reproject files to get latitude and longitude values (in WGS84 project, EPSG 4326). The centroid value is surrounded in the field by KML syntax “<Point><coordinates>X,Y</coordinates></Point>”.

Simplified Geometry – A geometry column is automatically created by the web application (or GFT, I’m not sure). This function will create a simpler version of that geometry, with fewer lines and vertices. It also creates columns to list the vertices count for the simple and regular geometry columns.

My essential QGIS plugins

Plugins for QGIS I use most often.

All of these can be installed automatically by QGIS. Click on Plugins>Fetch Python Plugins. Then search for the plugin, click on its name, and click Install Plugin. Few plugins require a restart.

  • MMQGIS – Great for working with CSV files; also merges layers (even if they have differing attributes); has various other useful functions, including converting string data to float data. Has Voronoi diagram function (takes a long time to process).
  • fTools – Replicates some of the most basic geographic tools in ArcGIS, like Clip, Dissolve, and Reproject. Can also add X/Y values to point attribute tables that are missing them (if you want latitude/longitude, you must reproject into a coordinate reference system first, like WGS84 [EPSG: 4326]). Unfortunately, there’s little information on what each fTools function does. Below are descriptions:
    • Extract Nodes – Create a point at each intersection of vertices.
    • Basic Statistics – Generate arithmetic statistics for fields, same as statistics function in ArcGIS. Great for quickly understanding the extent of values in a field (especially numeric values), like mean, max, min, standard deviation, and number of unique values.
    • Nearest Neighbour Analysis – More details here.
    • Geoprocessing Tools>Dissolve – Combine features based on a shared attribute. For example, all features with an identical STREET_TYPE be combined into a single feature. For example, all “Avenues” will become one feature and all “Boulevards” will become a second feature. Only works on polygon layers.
    • Descrição em português
  • Table Manager
  • Open Layers – Embed Google, Yahoo, Bing, and OpenStreetMap layers in your map. See my example.

Better bike crash map now available for Chicago

I met Derek at a get together for “urban geeks” last Tuesday where he told me he was making a filterable/searchable version of my Chicago bike crash map using the Google Fusion Tables API. It essentially allows you to perform SQL-like queries to show different results on the map than one view. It’s possible to do this yourself if you open the bike crash map in the full Google Fusion Tables interface (do that now).

You can use it now!

New bike crash map, click through to view

Derek’s map has the benefit of great interface to drill down to the data you want. You can select a day, a surface condition, and the injury type. To download the data yourself, you’ll still have to access the full Fusion Tables interface.

Door lane photo and graphic by Gary Kavanagh in Santa Monica, California.

And since the data is the same as my original map, crash reports involving motor vehicle doors are not included. Here’s why doorings are excluded.

Free online GIS tools: An introduction to GeoCommons

Read my tutorial on how I created the pedestrian map with GeoCommons. Read on for an introduction to GeoCommons and online GIS tools.

GeoCommons, like Google My Maps and Earth, is part of the “poor man’s GIS package.” It’s another tool that provides (few) of the functions that desktop GIS software offers. But it excels at making simple and somewhat complex maps.

I first used GeoCommons over a year ago. I started using it because it would convert whatever data you uploaded into another format that was probably more useful. I mentioned it in this article about converting files. For example, if you have a KML file, you can upload it and export it as a shapefile for GIS programs, or a CSV file to load into a table editor or spreadsheet application.

After creating the Chicago bike crash maps using Google Fusion Tables, I wanted to try out another map-making web application, one that provided more customization and prettier maps.

I found that web application and created a version of the bike crash maps, with several other data layers, in GeoCommons. I overlaid bike counts and bikeways so you can observe some relationships between each visual dataset. My latest map (screenshot below), created Wednesday, shows pedestrian counts in downtown Chicago overlaid with CTA and downtown Metra stations, as well as the 48 intersections with the most pedestrian collisions (from this UNC study, PDF).

Screenshot of pedestrian count map described above.

How these online GIS tools can be useful to you

I bet there’s a way you can use Google Fusion Tables and GeoCommons for your job or project. They’re extremely simple to use: they can take in data from the spreadsheets you’re already working on and turn them into themed reference maps. With mapping, you can do simple, visual analysis that doesn’t require statistical software or knowledge.

Imagine plotting your client list on a map and grouping them by age to see if perhaps your younger clients tend to live in the same neighborhoods of town, or if they’re more diverse (should you do this, keep the map private, something that you can’t do in GeoCommons – yet).

You may also find it useful if you want to create a route for your salespeople or for visiting church members at their homes. Plot all the addresses on a map, then manually filter them into different groups based on the clusters you see. With Google Fusion Tables, you can easily add a new column with the GROUP information and apply a numbered or lettered group and then re-sort.

Other things you can do in GeoCommons

  • Merge tables with geography – I uploaded two datasets: a table containing census tract IDs and demographic information for Cook County I downloaded from the American FactFinder 2; and a shapefile containing Cook County census tracts boundary information. After merging them, I could download a NEW shapefile that contained both datasets.
  • Make multi-layer maps
  • Symbolize based on frequency/rate
  • Convert data – This is by far the most useful feature. It imports “shapefiles (SHP), comma separated values (CSV), Keyhole Markup Language (KML), and GeoRSS” and exports “Shapefile, CSV, KML, GeoRSS Atom, Spatialite, and JSON” (from the GeoCommons user manual).

Read my tutorial on how I created the pedestrian map with GeoCommons.

© 2017 Steven Can Plan

Theme by Anders NorénUp ↑