Tagdata collection

Working with ZIP code data (and alternatives to using sketchy ZIP code data)

1711 North Kimball Avenue, built 1890

This building at 1711 N Kimball no longer receives mail and the local mail carrier would mark it as vacant. After a minimum length of time the address will appear in the United States Postal Service’s vacancy dataset, provided by the federal Department of Housing and Urban Development. Photo: Gabriel X. Michael.

Working with accurate ZIP code data in your geographic publication (website or report) or demographic analysis can be problematic. The most accurate dataset – perhaps the only one that could be called reliably accurate – is one that you purchase from one of the United States Postal Service’s (USPS) authorized resellers. If you want to skip the introduction on what ZIP codes really represent, jump to “ZIP-code related datasets”.

Understanding what ZIP codes are

In other words the post office’s ZIP code data, which they use to deliver mail and not to locate people like your publication or analysis, is not free. It is also, unbeknownst to many, a dataset that lists mail carrier routes. It’s not a boundary or polygon, although many of the authorized resellers transform it into a boundary so buyers can geocode the location of their customers (retail companies might use this for customer tracking and profiling, and petition-creating websites for determining your elected officials).

The Census Bureau has its own issues using ZIP code data. For one, the ZIP code data changes as routes change and as delivery points change. Census boundaries needs to stay somewhat constant to be able to compare geographies over time, and Census tracts stay the same for a period of 10 years (between the decennial surveys).

Understanding that ZIP codes are well known (everybody has one and everybody knows theirs) and that it would be useful to present data on that level, the Bureau created “ZIP Code Tabulation Areas” (ZCTA) for the 2000 Census. They’re a collection of Census tracts that resemble a ZIP code’s area (they also often share the same 5-digit identifiers). The ZCTA and an area representing a ZIP code have a lot of overlap and can share much of the same space. ZCTA data is freely downloadable from the Census Bureau’s TIGER shapefiles website.

There’s a good discussion about what ZIP codes are and aren’t on the GIS StackExchange.

Chicago example of the problem

Here’s a real world example of the kinds of problems that ZIP code data availability and comprehension: Those working on the Chicago Health Atlas have run into this problem where they were using two different datasets: ZCTA from the Census Bureau and ZIP codes as prepared by the City of Chicago and published on their open data portal. Their solution, which is really a stopgap measure and needs further review not just by those involved in the app but by a diverse group of data experts, was to add a disclaimer that they use ZCTAs instead of the USPS’s ZIP code data.

ZIP-code related datasets

Fast forward to why I’m telling you all of this: The U.S. Department of Housing and Urban Development (HUD) has two ZIP-code based datasets that may prove useful to mappers and researchers.

1. ZIP code crosswalk files

This is a collection of eight datasets that link a level of Census geography to ZIP codes (and the reverse). The most useful to me is ZIP to Census tract. This dataset tells you in which ZIP code a Census tract lies (including if it spans multiple ZIP codes). HUD is using data from the USPS to create this.

The dataset is documented well on their website and updated quarterly, going back to 2010. The most recent file comes as a 12 MB Excel spreadsheet.

2. Vacant addresses

The USPS employs thousands of mail carriers to delivery things to the millions of households across the country, and they keep track of when the mail carrier cannot delivery something because no one lives in the apartment or house anymore. The address vacancy data tells you the following characteristics at the Census tract level:

  • total number of addresses the USPS knows about
  • number of addresses on urban routes to which the mail carrier hasn’t been able to delivery for 90 days and longer
  • “no-stat” addresses: undeliverable rural addresses, places under construction, urban addresses unlikely to be active

You must register to download the vacant addresses data and be a governmental entity or non-profit organization*, per the agreement** HUD has with USPS. Learn more and download the vacancy data which they update quarterly.

Tina Fassett Smith is a researcher at DePaul University’s Institute of Housing Studies and reviewed part of this blog post. She stresses to readers to ignore the “no-stat” addresses in the USPS’s vacancy dataset. She said that research by her and her colleagues at the IHS concluded this section of the data is unreliable. Tina also said that the methodology mail carriers use to identify vacant addresses and places under change (construction or demolition) isn’t made public and that mail carriers have an incentive to collect the data instead of being compensated normally. Tina further explained the issues with no-stat.

We have seen instances of a relationship between the number of P.O. boxes (i.e., the presence of a post office) and the number of no-stats in an area. This is one reason we took it off of the IHS Data Portal. We have not found it to be a useful data set for better understanding neighborhoods or housing markets.

The Institute of Housing Studies provides vacancy data on their portal for those who don’t want to bother with the HUD sign-up process to obtain it.

* It appears that HUD doesn’t verify your eligibility.

** This agreement also states that one can only use the vacancy data for the “stated purpose”: “measuring and forecasting neighborhood changes, assessing neighborhood needs, and measuring/assessing the various HUD programs in which Users are involved”.

Data collection issues in police database design

Please don’t park on sucker (er, sign) poles. And use a good lock. When your bike is stolen, no one but you will care. And the police won’t even know. 

When I asked the Chicago Police Department for statistics on how many bicycles are reported stolen each year, the response was that statistics couldn’t be provided because the database can’t be filtered on “bicycle theft”. I called the police officer responding to my FOIA request to learn more. He said that bike thefts are only categorized as “property thefts under $300” or “property thefts over $300”.

It would be possible to search through the database using keywords, but that would have been an unreasonable use of the officer’s and department’s time in accordance with FOIA laws (FOIA meaning Freedom of Information Act, known as FOIL in other states).

What a poorly designed database.

The Federal Bureau of Investigation (FBI) has Uniform Crime Reporting standards that police departments nationwide voluntarily adopt. It’s mainly a way for the FBI to collect statistics across the country and I assume this program is necessary because of a state’s right to impose its own standards. That sounds fine by me, but it shouldn’t impede collecting data to assist the Department of Transportation, and the Police Department itself, to combat bike theft. The Bike 2015 Plan, released in 2005 and adopted by the Mayor’s Bicycle Advisory Council (MBAC), includes several strategies to reduce bike theft in Chicago.

The FBI has a standard reporting code for bike thefts: 6Xf. Its definition is “the unlawful taking of any bicycle, tandem bicycle, unicycle, etc.”

The Chicago Police Department should upgrade its database to code property theft of bicycles as a “bike theft”. For now, though, advocates and activists will have to rely on the homegrown Chicago Stolen Bike Registry*. If you don’t know the extent of the problem, it’ll be hard to develop any solution.

The Bike 2015 Plan called for a report, by 2006, to be written that determined the “amount and types of bike theft”. We’re in 2012 – where’s that report?

Thinking about New York City

New York City’s Police Department (NYPD) has similar, but more grievous, database problems. Right now the New York City Council is holding a hearing to figure out why the NYPD isn’t investigating reckless drivers who’ve killed people walking and cycling. And I read this about it:

 Vacca’s first question to Deputy Chief John Cassidy, the NYPD Chief of Transportation, was about speeding, and how often drivers caught speeding are charged with reckless endangerment. The answer came not from Cassidy, but from Susan Petito, an NYPD attorney, who politely explained that they simply don’t know, because reckless endangerment charges “are not segregated in the database” and can’t be easily found. Via Gothamist.

Hmm, seems like the same issue the Chicago Police Department has.

* The Stolen Bike Registry has its own issues, which is often because of “user error”, wherein people who submit reports to it don’t provide much detail, especially as from where the bike was stolen.

© 2019 Steven Can Plan

Theme by Anders NorénUp ↑