TagSmart Chicago Collaborative

Outta left field: I recreated the city’s contractor listing website

The site looks good and works quickly on mobile devices.

LicensedChicagoContractors.com looks good and works quickly on mobile devices.

I’m working on a secret project to get something installed on the public way. The process to find out how to do it is as arduous as getting it done because you never finish learning the process. Every time you think you’ve figured something out, there’s something else.

To get the secret project installed I need a licensed contractor. Not only do a need a licensed contractor, but they must have the license to do work in the public way (versus doing work at your private property).

The Chicago Department of Buildings publishes a continually updated list of licensed contractors on its website but it’s annoying to use. There’s no search, no permanent links, and if you leave the window open long enough this weird session manager kicks in and stops you from browsing to the next page of results.

I asked my followers on Twitter the best way to scrape the data. The ever-amusing Dan O’Neill, who leads the Smart Chicago Collaborative (which hosts the Chicago Crash Browser), recommended just copying and pasting all 10 pages. That would work fine for the first time, but I might need to do it a second time when the data updates. Nick Bennett jumped in and used Selenium, a tool that automates web browsers. He said, “it’s inefficient but for a small job like that I figured why bother with something faster”.

I imported the data into a MySQL table and ran through some of my “standard” data cleaning methods (like trimming leading and trailing spaces, removing odd characters, and extracting good information into other columns, like phone numbers and ZIP codes).

With PHP – my favorite web language – I created a single page website that loads all 3,930 licensed general contractors extremely fast, loads the DataTables JavaScript library to enhance the table with search and sort. I used Bootstrap to make a responsive design meaning it adjusts to fit multiple screen sizes including smartphones and tablets.

I call it LicensedChicagoContractors.com.

The new website still doesn’t solve my problem of finding a company that can do work in the public way – I’m still working on this. The last online dataset I could find is on the city’s old http://egov.cityofchicago.org domain, and was cached by the Internet Archive’s Wayback Machine on January 25, 2010. Ideally this information – plumbers, public way, and general contractors – should be posted on the City’s data portal.

Where do Divvy riders go?

Divvys

Divvy bikes fit people of almost all sizes. Photo by Mike Travis (mikeybrick).

Divvy released the 2013 trip data on Tuesday for their data challenge, and presented alongside me the data, basic system operations info, and existing visualizations and apps, at a Divvy data-focused Open Gov Hack Night I put together at the weekly meeting. Thank you Chris Whitaker at Smart Chicago Collaborative for writing the meeting recap.

I “ran the numbers” on some selected slices of the data to post on Twitter and they range from the useless to useful! I’m using the hashtag #DivvyData.

  • Average trip distance of members in 2013 is estimated to be slightly shorter than casuals: 1.81 miles versus 1.56 miles – tweet
  • Bike 321 has traveled the furthest: 989 miles. Beat the next bike by 0.2 miles – tweet
  • Women members on average took longer trips (but fewer trips overall) on @DivvyBikes than men in 2013. – tweet
  • The average trip distance of 759,788 trips (by members and casuals) in 2013 is an estimated 1.68 miles. – tweet
  • In 2013, 79.05% of member trips were by men and 20.95% by women. – tweet
  • On average in 2013, 24-hour pass holders (whom I call casuals) made trips 2.5x longer (time wise) than members. – tweet
  • Damen/Pierce Divvy station (outside the Damen Blue Line station) is most popular in Wicker Park-Bucktown – data

And other stats, presented as embedded tweets:

 

 

© 2019 Steven Can Plan

Theme by Anders NorénUp ↑