Category: Information

Reminder about open data and Obama’s Open Government Directive

Quickly after taking office, President Obama issued a memorandum about open government and opening government data. Then came the Open Government Directive* which said:

To the extent practicable and subject to valid restrictions, agencies should publish information online in an open format that can be retrieved, downloaded, indexed, and searched by commonly used web search applications. An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.

Essentially, the executive government (er, Obama Administration) adopts the presumption of openness, that distributing public data is the default position and action to take.

Don’t squat on the data. Don’t fret over how people will view or manipulate the data – this is not your concern. Don’t delay its release. If you do this, you are a frigid dataist and I will remember this.

Photo of visual note taking at an open data seminar by Karen Quinn.

*The Directive has a little more backbone than the original memorandum: “This memorandum requires executive departments and agencies to take the following steps toward the goal of creating a more open government.”

Thank you to Tech President.

More data goodness for Chicago: TIFs, vacant and abandoned buildings

Derek Eder emailed me to tell me about two web applications he created based on Google Fusion Tables and its API (application programming interface, basically a question and answer program for designers and programmers to interact with).

He created searchable/filterable maps for TIF districts (tax increment financing, the Chicago mayor’s pet project bank account) and vacant and abandoned buildings. Both use data straight from the City of Chicago.

Screenshot of the Derek Eder’s TIF district web application.

Essentially, the web applications work like this (in case you want to build one yourself):

  • Load the data into Google Fusion Tables (this is very easy)
  • Build a custom interface on your own website (not so easy)
  • Hook into the Fusion Tables API to load the data into your custom interface

As for me, I might look into building a custom interface on my website, but right now I’m going to create a pedestrian crash map for Chicago using Polymaps, a Javascript library. I specifically want to use the k-Means Clustering to show crash hotspots. We already know where they are based on a 2007 report from the University of North Carolina – see that map here.

These markings are intended to reduce the number of pedestrian crashes by increasing the walking person’s visibility.

Measuring gas prices and bicycling trips

From the Chicago Tribune: Gas prices continued to rise Monday, driven higher for nearly two weeks straight by the turmoil in Libya, with analysts expecting prices to keep climbing.

Active Transportation Alliance asks, “How can we make the gas price bubble permanent?” -Essentially the same topic I write about below.

I was thinking ever since I first read in the Chicago newspapers that gas will hit $4 per gallon this year (it already has in the City) that there’s a relationship between the price of gas and the number of people on bicycling or the number of trips people make on their bicycles.

As the price of gas rises, so does the number of people out bicycling on the streets. As the price of gas falls, bicycling declines as well.

Chart from GasBuddy.com showing average gas prices in Chicago for the past 3 years.

The data available to us doesn’t necessarily support this hypothesis, but the data available* is nearly worthless. Gas prices were over $4 per gallon in 2008. That was when Chicago started seeing tons of people on the street on their bicycles. The local Fox News affiliate interviewed Mike Amsden, a city planner at the Chicago Department of Transportation (CDOT), about the bike counts (first in five years) in a news segment about the influence of $4.65 and a “major peak, almost 350% in pedal pushers this year.”

Several newspapers published articles about the palpable increase in cycling, including a Time Out issue called “Bike Love” with messenger Jeff Perkins on the cover and interviewing 7 local cyclists inside. All of them published “how to get out and ride”-type articles. But despite the many new riders on the street in 2008, few came back the next year!

This graphic describes my point about gas prices up, bike trips up; gas prices down, bike trips down (but perhaps ending at a rate a little higher than where it started).

2009 came and the gas prices dropped – the modern heyday of Chicago cycling was gone. 2008 saw the highest numbers at 2 of 3 locations also counted in 2003, although the difference in study months makes the comparison suspect. I hope that 2011 is the start of annual and accurate counts of bicycling in Chicago.

But it’s reasonable to expect that some of the new people riding their bikes instead of taking expensive car trips will stick with it the following year, even as gas prices decline. Let’s keep these riders bicycling year after year, encouraging more to stay on the bike path than would normally otherwise with strategies like more urban-appropriate infrastructure (separated and protected bike lanes; secure bike parking at workplaces and train stations; traffic calming/slower traffic) as well as enforcement of laws that protect cyclists.

Let’s concentrate less on the “insane”  numbers of people cycling on Milwaukee Avenue at Ohio Street (3,121 bikes on September 15, 2009) and more on how to raise the number of people cycling on our other streets. Milwaukee Avenue doesn’t need anymore attention (except for its intersections). Getting people off Milwaukee and safely and efficiently onto east-west and north-south routes should be the priority. -Photo shows Halsted/Grand/Milwaukee, just 300 feet southeast of the Ohio count location.

*Available data

The American Community Survey (ACS) 3-year estimate for 2006-2008 tells us that 1.0% of working Chicagoans 16+ took their bikes to work (nevermind the tinny sample size that makes this data near worthless – it’s the only thing we have*). The 3-year estimate before (2005-2007) says 0.9% took their bikes to work. Not much of a peak or increase! For 2007-2009, the data shows 1.1% cycled to work.

Also ignore the fact that the ACS only asks about the mode you spent the most distance on. It does not collect data on multi-mode trips. So if you bike 3 miles to the train and the train is 30 miles to your destination, the ACS would only record “public transportation.”

Weighting people’s experiences in route choice

An iPhone app is not a substitute for a paper map*, good signage on your bikeway network, or someone just telling you, “Turn right on Church, right on Chambers, left on Reade” to get to the bike shop where you left your water bottle.

At the bike shop I asked about how to get to the Williamsburg bridge so I could go “home” to Brooklyn. After looking at the map, he said, “Oh, take Grand.” -He then told me how to get to Grand.

The Williamsburg bridge. I took this one even though the Manhattan bridge was probably closer to my “home” because I hadn’t yet ridden on it!

I did. It worked. It was excellent. I even passed by the Doughnut Plant (which I had forgotten about visiting).

Doughnut Plant makes really tasty donuts. I wouldn’t get them too often, though, because each one costs $3.

Not only did I receive a “tried and true” route suggestion, I got it faster than any automated route devising device would have generated one.

Each month I’m asked by people how to get somewhere in Chicago. We have so many resources these days but we often still rely on the spoken interaction to get us to our destination.

*I’ve read or heard people suggest that “someone should make” an app that puts the bike map on their smartphone. I don’t think this app would be very useful or easy to use. But a paper map is both – and almost always free.

Free online GIS tools: An introduction to GeoCommons

Read my tutorial on how I created the pedestrian map with GeoCommons. Read on for an introduction to GeoCommons and online GIS tools.

GeoCommons, like Google My Maps and Earth, is part of the “poor man’s GIS package.” It’s another tool that provides (few) of the functions that desktop GIS software offers. But it excels at making simple and somewhat complex maps.

I first used GeoCommons over a year ago. I started using it because it would convert whatever data you uploaded into another format that was probably more useful. I mentioned it in this article about converting files. For example, if you have a KML file, you can upload it and export it as a shapefile for GIS programs, or a CSV file to load into a table editor or spreadsheet application.

After creating the Chicago bike crash maps using Google Fusion Tables, I wanted to try out another map-making web application, one that provided more customization and prettier maps.

I found that web application and created a version of the bike crash maps, with several other data layers, in GeoCommons. I overlaid bike counts and bikeways so you can observe some relationships between each visual dataset. My latest map (screenshot below), created Wednesday, shows pedestrian counts in downtown Chicago overlaid with CTA and downtown Metra stations, as well as the 48 intersections with the most pedestrian collisions (from this UNC study, PDF).

Screenshot of pedestrian count map described above.

How these online GIS tools can be useful to you

I bet there’s a way you can use Google Fusion Tables and GeoCommons for your job or project. They’re extremely simple to use: they can take in data from the spreadsheets you’re already working on and turn them into themed reference maps. With mapping, you can do simple, visual analysis that doesn’t require statistical software or knowledge.

Imagine plotting your client list on a map and grouping them by age to see if perhaps your younger clients tend to live in the same neighborhoods of town, or if they’re more diverse (should you do this, keep the map private, something that you can’t do in GeoCommons – yet).

You may also find it useful if you want to create a route for your salespeople or for visiting church members at their homes. Plot all the addresses on a map, then manually filter them into different groups based on the clusters you see. With Google Fusion Tables, you can easily add a new column with the GROUP information and apply a numbered or lettered group and then re-sort.

Other things you can do in GeoCommons

  • Merge tables with geography – I uploaded two datasets: a table containing census tract IDs and demographic information for Cook County I downloaded from the American FactFinder 2; and a shapefile containing Cook County census tracts boundary information. After merging them, I could download a NEW shapefile that contained both datasets.
  • Make multi-layer maps
  • Symbolize based on frequency/rate
  • Convert data – This is by far the most useful feature. It imports “shapefiles (SHP), comma separated values (CSV), Keyhole Markup Language (KML), and GeoRSS” and exports “Shapefile, CSV, KML, GeoRSS Atom, Spatialite, and JSON” (from the GeoCommons user manual).

Read my tutorial on how I created the pedestrian map with GeoCommons.