Category: Uncategorized

Two-flat journal 5: Costs

Updated May 3, 2021, to add more insight from Robinson Meyer (The Atlantic) as to why lumber prices are so high.

My architect and I are still working on plans, slowly but surely. Read my previous entry, Two-flat journal #4, to understand why that seems to be taking awhile.

There is something else on my mind as we work toward the goal of a gut-rehabbed two-flat: How much this whole project is going to cost.

I’ve talked to several contractors, engaged a structural engineer to specify and design the new steel beam in the basement, and obtained quotes for all new windows from four manufacturers.

One contractor happily gave me an estimate, based on incomplete plans, that was about $220,000. That price could go down with more specific plans and instructions, as the estimate had variability based on unknowns, and it doesn’t include the cost of purchasing the windows. More likely, I think the price will go up due to material costs.

How much windows might cost

All window quotes I’ve obtained include installation by the manufacturer’s selected installers, which has a benefit from some companies, mainly that the maker will guarantee the installation for a period of time.

The first floor living room window is currently double hung with a mullion and a tripartite transom window above it (with stained glass, that’s been covered on the inside). This window will become a “Chicago window” with a centered picture window flanked by casement windows.

I would share the quotes with you but I don’t think they would be very helpful at this point because I haven’t evaluated each of the quotes on the quality of the window. For example, one of the window quotes was three times higher than the next highest quote, but the maker guarantees installation for 10 years and is a higher-quality window. But what is the factor of difference in quality, is it three times? And how valuable is a 10-year installation warranty? It’s unlikely I would need to avail that benefit and the three times difference in price means I could replace all of the windows *again* two times for the same price! (Assuming prices didn’t increase between now and that future moment.)

The 15 new windows, according to the four quotes, will cost anywhere from $12,000 to $46,000. I should mention that the highest quote doesn’t include any discounts or special offers, as those will be offered once I re-engage the estimator and ask for one.

Look at windows 1.1 and 1.3, which look like bay windows. These are both Chicago windows, which have a center picture (fixed) window flanked by two operable windows (casement windows, in this case).

There are a couple of opportunities to reduce window costs. I could convert more of the casement windows to be double hung windows (which I don’t want to do as I prefer single hung windows), or I could change the window opening size. A couple of the window openings are taller than most of the window makers have in their standard window design, so an upper transom (fixed) window would be required. However, changing the window opening size may end up shifting costs to a different plan of adding bricks and adjusting walls.

Another way to reduce the window cost would be to use models that are less energy efficient, but I also don’t want to do that. I’ve insisted that every window be Energy Star certified – this is about the only certification standard that I understand, and it’s common across most window makers in the Chicago area. (There are also Passive House and Passivhaus certified windows, and companies that import higher-quality and more efficient windows from European manufacturers, but I haven’t bothered with any of those because I assume the prices will be even higher.)

Lumber and other construction materials

That lumber prices have more than doubled over prices a year ago is well known if you read real estate industry news media, or if you’ve shopped for wood at Menards to build a couple of benches for some nicer outdoor space or installed a new porch.

My gut rehab will require a lot of plywood (to replace the subfloor), “soft lumber” replacement studs, and some replacement joists.

The St. Louis Federal Reserve maintains “FRED”, an amazing website with interactive charts to explore economics statistics, including lumber. The pricing information comes from the Bureau of Labor Statistics and their Producer Price Indexes.

The chart for the plywood price index below shows very stable pricing in fall and winter 2019-2020, and then in May 2020 prices start climbing and the index increased by 100 points to March 2021.

U.S. Bureau of Labor Statistics, Producer Price Index by Commodity: Lumber and Wood Products: Plywood [WPU083], retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/WPU083, April 25, 2021.

BLS has monthly detailed reports so you can find data about more than the products FRED has charts for. Let’s dig in to the March 2021 report (indexes mean that the pricing represents percentage changes based on 100% being the price when the index was established):

  • “Softwood dressed 2-inch lumber, 2 inches in nominal thickness only, not edge worked” (a.k.a. 2×4 studs) (index established June 2012):
    • March 2020: 205.1
    • March 2021: 324.1 (this means that the price has increased by 119% year over year, a more than doubling of price)
  • “Softwood plywood products: rough, sanded, and specialties” (index established December 2011):
    • March 2020: 139.0
    • March 2021: 242.9 (again, this means that the price has increased by 103.9%, doubling the price)

In addition to general demand being much higher, there are other reasons why lumber costs so much more right now, according to Robinson Meyer writing in The Atlantic last week.

Since 2018, a one-two punch of environmental harms worsened by climate change has devastated the lumber industry in Canada, the largest lumber exporter to the United States. A catastrophic and multi-decade outbreak of bark-eating beetles, followed by a series of historic wildfire seasons, have led to lasting economic damage in British Columbia, a crucial lumber-providing province. Americans have, in effect, made a mad dash for lumber at the exact moment Canada is least able to supply it.

[snipped]

“There are people who say, ‘Climate change isn’t affecting me,’” Janice Cooke, a forest-industry veteran and biology professor at the University of Alberta, told me. “But they’re going to go to the hardware store and say, ‘Holy cow, the price of lumber has gone up.’”

[snipped]

It has lost 2.5 billion board feet of annual production capacity since 2019, enough to shift prices in a North American market of 70 billion annual board feet, Jalbert said.

Read Robinson’s full article to see how the bark-eating beetles overwhelmed the forests of British Columbia and the northern forest belt in Canada and why their rampage is fueled by climate change.


The same contractor, when they checked in with me recently, said that the prices of other construction materials had gone up, too.

In the same Producer Price Index report, it looks like wood doors and door frames went up 29.2% from March 2020 to March 2021; metal windows are up 7.1%, double hung wood windows are up 6.8% and wood casement windows are up 5.1%; wood moldings are up 16.9%.

I didn’t see any notable price increases in plumbing materials or kitchen cabinets – all were close to inflation. The PPI doesn’t have vinyl window products, or I don’t know under which category it falls.

Two-flat journal 4: How’s the renovation going?

I get this question several times a week, and I want to give everyone who asks a clear and accurate response without ignoring them (which I sometimes do). This blog post is how I can do that efficiently, and be more organized and mindful about the progress.

tl;dr:

  • The renovation part of the renovation project hasn’t started.
  • The preparation part of the renovation project has been going on since August 2020.
  • I do not know when we’ll be done, or when we’ll have a permit, or when construction will start.

“Why is it taking so long?” is the question that people don’t ask aloud, but I guess that my friends are like me and curious.

It’s taking a long time because my architect and I don’t spend a whole lot of time working on the project. It’s all relative, of course. We spend about 10 hours a week on this (at most), and that includes a lot of learning and research.

^ Not my house 🙂 These three mini-split outdoor heat pump condenser units are for someone else’s three-flat that’s described below. Each has a heating capacity of 36,000 BTUs, and can perform at 80 percent capacity when the outdoor air temperature is -13°F. Read more about this tour on my @ChiBuildings Twitter thread.

We both have full-time jobs, have other things to work on, and our experience in this arena is limited. My architect, who prefers to stay out of the blogging spotlight, is extremely experienced in multi-family new construction residential architecture, though. We also rely heavily on the input of friends and neighbors who live in similar houses and have experience dealing with the construction idiosyncrasies of old buildings in Chicago. I also rely on the input of other architects who are experts in designing renovated and new construction one, two, and three-unit houses (thank you K.D. and P.M.).

This is a learning process for both of us. There are so many decisions to make, independently and together. We both want to be deeply involved in understanding what it takes to buy and gut rehab a house that I will live in. If we do it well this time, it would make it easier for either one of us to do it again for a different house.

What we’ve done in the past two weeks

  • I invited a fourth general contractor into the house to take a look. I am not getting estimates yet from these four GCs because the plans / permit drawings aren’t far enough along. Each GC, however, has reviewed the current plans prior to the visit so it’s easier to point and explain what’s going to change.
    • One of the big things that will change is replacing the wooden center beam in the basement to a steel I-beam, and changing out some of the joists. This will require temporarily shoring (supporting) the joists. Each GC has had a slightly different opinion on the procedure. But we get to hear each procedure and come up with the one we think is best.
  • I received and organized three quotes for all-new windows, as well as researched and modified the window schedule. “Organizing” the quotes means adding a new column to my window schedule spreadsheet with each manufacturer’s pricing so I can compare them. Part of window research is figuring out the type of window I want for each room and opening and assessing the quality of the manufacturer. For example:
    • I am learning that casement windows are more energy efficient because they create a better seal when closed and locked than hung windows.
    • I prefer single-hung windows to double-hung windows, but some manufacturers don’t make single-hung windows.
    • Every manufacturer has different maximum window heights. One of the openings is 80 inches tall and, for one manufacturer, the maximum height for a casement window is 72 inches and the maximum height for their hung windows is 75 inches. Thus, the estimator added transom windows to all openings that exceeded their maximums. That’s fine, but then it adds a new decision point: What proportion should be operable windows and what proportion should be transom windows?
While discussing window openings and installation details, my architect realized we didn’t collect enough data when we measured it last fall. We went to my house last week to get a closer look, take some new measurements, and inspect the conditions of the sills. The last GC had pointed out that some of the sills should be replaced.

  • My architect continued talking to the structural engineer I hired to make sure the engineer’s drawing has the right details and lists the specifications we asked him to write out on the drawing.
  • Settled on an HVAC system. For a couple months I’ve been researching mini-splits off and on. I got an estimate for equipment + installation from one company, which was much higher than I expected. I combined their projections for required heating capacity with projections from others, as well as CoolCalc.com, to settle on a lower capacity.
    • I started looking at more retailers who will sell mini-split systems directly to the consumer, or the GC, to get grounded in pricing and availability. Then, when I was done specifying the schedule, my architect took a look at the designs and the cut sheets, and figured out where the indoor units should go. He recommended choosing the cassette style that gets mounted within the ceiling between two joists. It costs more, but it looks a lot better since there’s not a big box hanging on the wall in the living room and kitchen.
    • Last Friday my architect and I visited a “near Passive House” standard three-flat being built in Pilsen. The developer, who will live in one of the units, also used mini-splits, from Mitsubishi. It was cool to see them installed (on the roof) and how they were connected to an air handler that serves as the blower to push the air through a traditional duct and out via an “energy recovery ventilator” (in the winter, the warmth of the outgoing air is transferred to the fresh incoming air so the machinery has to expend less energy warming up cold air).

While I was writing this, another of the four GCs that toured the house checked in. I told them that after this week we’ll have made significant progress since my architect has the week off of work.

What we’re working on next

  • Last week, my architect started drawing a window detail in the plans because there are good ways of installing windows and bad ways. I don’t fully understand them, so I’m not going to write about them here, but he did explain them to me and pointed out some good examples in his house and some bad examples in someone else’s house.
  • Finish the “panelboard schedule”. This diagram shows each circuit that will be in a breaker box, and labels them according to the circuits in the electrical drawings. Not every circuit breaker will be identical, as some circuits – like the ones for the water heater and mini-splits – need 30 or 40 amp breakers.
  • I am going to research the construction type’s fire protection requirements by reading the Chicago Building Code and noting the required fire ratings for the basement ceiling, the ceiling between floors, the interior front door’s rating, the roof rating, the rating for the light well wall, and the rating requirement for load bearing walls.

What I’m worried about

All of the decisions that have to be made. See the three points about windows above. I’ve got three quotes for the windows, all based on about the same schedule. But after I receive the fourth quote that I’m expecting this week, my desire for certain windows may change. I may want more casement windows, or different proportions of transom windows and casement-picture-casement window layouts.

The cost of everything. I have significant savings but I will still need to borrow some money. I need to start researching the universe of options and the availability of options. On top of this, lumber pricing has doubled since the start of the COVID-19 pandemic.

I shoveled the walks every time it snowed. This is significant because I don’t live here so I took the 82 Kimball/Homan bus from where I live to where I don’t live. I enjoyed it because it got me out of the house when it was cold, and it gave me some moments of exercise.

Two-flat owner journal 3: Choosing the right water heater

Read all of the “Two-flat owner journal” entries

This journal entry is all about the several hours of time I spent researching which water heaters to buy for my two-flat. While my architect works on drawing the plans, I am doing a lot of research to answer questions for him, so he knows what appliances are going to be hooked up to the house’s mechanical systems.

I started researching water heaters on a cold Saturday in January so I could fill out the “appliance schedule” for the project. My first journal entry was about my new distaste for Peoples Gas. Since then, my view has evolved and we (my architect and I) are designing an all-electric house.

SCROLL DOWN FOR THE RESEARCH – Read on while I discuss my thoughts about electricity and natural gas as a power source for homes.

Discussion

Let me step back a moment. I also have a “distaste” (a weird word to use when talking about something that would literally kill you if you drank it) for fossil fuels, too. When I approached this research task I thought that an electric tankless water heater had some kind of inherent efficiency over an electric or natural gas-powered tank type.

But it doesn’t, and that’s partly dependent on how much water a household uses. Tankless water heaters are sometimes marketed as providing “unlimited” hot water, which scared me. Since I don’t go to an office anymore (because of COVID-19 and because I quit my office-attached job), I take a shower every other day. And they’re long. Just imagine that your shower never ran out of hot water. I might not get out. So that’s a personal critique of the type of water heater.

(Another con of tankless water heaters: When the electric is off, water cannot be heated; with a tank water heater, there is always residual hot water in the tank, which can still flow. Additionally, electric tankless water heaters have special electrical box requirements because they draw so much electricity.)

Let’s talk about the fuel. Natural gas is cheap right now – to purchase. But it has awful costs elsewhere, namely its contribution to pollution and carbon dioxide emissions when burnt. Burning it in the home also releases additional gases, which is why I think you should run your range hood vent/exhaust when you cook food on the stovetop.

Gas stoves emit a host of dangerous pollutants, including particulate matter, formaldehyde, carbon monoxide, and nitrogen dioxide. 

Mother Jones, “How the Fossil Fuel Industry Convinced Americans to Love Gas Stoves”, by Rebecca Leber, FEBRUARY 11, 2021

Going electric in the kitchen is pretty easy, I think. I didn’t do much research and I picked out all Energy Star-certified appliances. For cooking I chose a range with an induction cooktop and an electric convection oven (which means there’s a fan inside to blow the heat around for even cooking). Induction cooktops, while being a really efficient user of energy, have cooking benefits: Water reaches a boil faster, the surface is easy to clean, and there are fewer burns because the surface doesn’t get hot.

Creating an all-electric house is pretty easy, actually, until you get to the heating and cooling part, and knowing how to heat a house with electricity in the very cold climate of northern Illinois requires even more research.

Water heaters are actually easy to figure out (after 5 hours of research) because, in the end, all you do is plug them in to a 240 volt receptacle and connect to the already-existing water pipes.

I am still in the middle of researching electric heating and cooling and I’ve opened a conversation with two HVAC contractors (one that sells Carrier and one that sells Mitsubishi).

Water heater research

I originally tweeted all of this on January 16, 2021, because I like using Twitter as a sounding board.

I have researched five types of water heaters because I want to fully understand the purchase price and energy price of each.

  1. Natural gas, tank
  2. Natural gas, tankless
  3. Electric, tank
  4. Electric, tank (hybrid w/heat pump)
  5. Electric, tankless 

Based on my research of manufacturers’ reported “Energy Guide” stickers (the yellow stickers required by federal law) for these five types, the Electric, tank (hybrid w/heat pump) (option 4) is far and away the most efficient water heater. 

The Electric, tank (hybrid w/heat pump) has an annual operating cost of $104. What is this thing? It takes the standard tank, uses electric heating elements (think of the wires inside your bread toaster), and extracts heat from the air in your house with the heat pump. That’s pretty amazing: There is free heat to be extracted from the air.

The Electric, tank (hybrid w/heat pump) has less than half the operating cost of the next lowest type: Natural gas, tankless.

Rheem (a water heater manufacturer, which also owns the Richmond brand) says that an electric tank water heater with hybrid heat pump uses less electricity than an old-fashioned incandescent light bulb.

Note that the electrical prices in the yellow “Energy Guide” sticker is 12 cents per kWh, and ComEd just charged me ~7 cents per kWh, so the annual operating costs of electric are less than the stickers say. 

Next, as part of the “total cost of ownership” (well, minus maintenance) I added in the purchase price. The Electric, tank (hybrid w/heat pump) costs $1,400. That’s 1.5-4x more expensive than the other types of water heaters!

However, I’ve since found that Rheem/Richmond makes a couple cheaper models that don’t have high-tech features, so the purchase price is anywhere from $1,000 (if you can find that model) to $1,175.

I calculated the “5 year cost of ownership” price for water heaters of each type because it makes sense to distribute the purchase price over a period of its lifetime. I could have easily made this a 10-year amortization since water heaters come with 6 to 12-year warranties. I interpret the length of the warranty as the manufacturer’s assessment as to how durable they’ve created the machine.

How they compare

Of the five types, I compared 11 machines (view the comparison chart).

The cheapest water heaters over five years are: 

  1. Electric tankless
    $1,479 (caveats in that annual energy cost was extrapolated because a direct Energy Guide sticker wasn’t found)
  2. Electric, tank (hybrid w/heat pump)
    $1,695 (does not include federal tax credit)
  3. Natural gas, tank
    $1,999

Remember, though, that the Energy Guide stickers for the electric water heaters use a 41.7 percent higher energy cost than ComEd currently charges, and ComEd offers hourly pricing so the price can be much, much lower per kWh and the prices for the electric water heaters are EVEN LOWER. (Thank you for pointing this out, Troy.)

Guess what…the price of the Electric, tank (hybrid w/heat pump) can come down even more because of (1) a ComEd rebate, and (2) federal tax credit worth 10 percent of the purchase price + installation, up to $300.

Bonus: By getting electric you are part of the carbon-free future. 

Further discussion

Now that I’ve convinced you that an Electric, tank (hybrid w/heat pump) water heater is the cheapest option, here are a couple of other things that came up in the Twitter conversations.

  • Including utilities (i.e. electricity cost) in the rent for the rental apartment in my two-flat benefits both me and the tenant. The tenant has a fixed and predictable energy cost and fewer bills to pay, while I am able to charge a bit more than I predict the cost will be in order to pay back the higher upfront costs of the water heater and the Energy Star-certified appliances (as well as the heating and cooling system).
  • The calculations might be different if I looked into having individual tankless water heaters at the point of demand, or using them as “boosters”. Tankless water heaters come in a variety of capacities and energy outputs (measured by how much energy it takes to increase the temperature from how cold the water is when it enters the house or heater to the desired temperature). One could be added to the kitchen, the bathroom, and next to the laundry, and sized for the differing demands of each location.
  • A complex system could be built that is programmed to buy energy from ComEd when the hourly cost is the lowest and use the power from the battery when the hourly cost is highest.
  • A couple people asked about geothermal. I looked into it and I wrote it off quickly: A drilling rig needs to access the yard to drill multiple horizontal wells. The garage blocks that from happening. However, an apartment building renovation in Rogers Park designed by PMP Architects is converting its heating system to use geothermal energy.

2020 year in review

Rear view of the two prefab single-unit detached houses and their backyard ADUs
I was biking around in Denver and I came across these identical modular houses with backyard houses under construction. The City of Denver posts building permit plans online, which is fantastic because it makes design more transparent. Architects from around the world can be inspired by this project.
  • Due to the (ongoing) COVID-19 pandemic, I started working from home on March 12, 2020. I visited the office once in the fall for an hour to perform a minor task.
  • My small business, Chicago Cityscape, hired Casey Smagala to run business development for the real estate information website.
  • I bought a two-flat in East Garfield Park in the summer after touring dozens of houses. I had first toured this house, and met the owner, Carl, who lived there, in February. I toured it a second time that month, with my friend, Kevin. After several more months of touring houses and realizing that I couldn’t afford an already-renovated house, I made an offer to Carl in May. The house closed in July.
  • Friends in Denver asked me and my friend, R, to come visit and camp, hike, and bike in and around Aspen in August. Denver also has allowed accessory dwelling units (ADUs) for years, so I talked to a local housing organizer and biked around the city looking at backyard houses.
  • I didn’t move into my two-flat because it turned out to need more renovation than I had believed. I also started a journal, which has one post so far.
  • I left my job as of December 31, 2020, so that I could focus on developing and growing Chicago Cityscape.
  • By the end of the year, I published eight blog posts. I didn’t have a goal; two of them were based on books I was reading: (1) Before the Lake Street elevated (now the Green Line) was built, a monorail was proposed (via a book about the history of the CTA’s predecessors)! (2) I read Beryl Satter’s book about her father, a landlord in Chicago, “Family Properties”, and I biked by one of the extant buildings.
  • My “hot air balloon” was used for some cool projects that needed aerial photography (both co-produced by Paola Aguirre and her consulting firm, Borderless Studio). I filmed some the massive mapmaking happening at the former Overton school in Bronzeville – which can only be seen from the sky – for an AIA film challenge; I also snagged some clips of the Pink Line ‘L’ going in and out of the California Ave station to visualize an area being studied for a corridor revitalization.

The high rises of the Bronzeville lakefront

3600 S King Dr

I shot this aerial photo yesterday from the 3600 block of South Dr. Martin Luther King Jr. Drive.

The “towers in the park” are three groups of apartment buildings (none are public housing).

From north (background) to south (foreground):

  • Prairie Shores (5 identical buildings that are very hard to see)
  • Lake Meadows (4 identical buildings behind the associated one-story shopping center plus multiple buildings to the east, closer to the lakefront)
  • T.K. Lawless Gardens (3 identical buildings, that appear the tallest because they’re the closest, 746 units, and 54 townhouses not seen)

John Warren Moutoussamy (an architecture graduate of Illinois Institute of Technology who studied under Ludwig Mies van der Rohe) designed Lawless Gardens, according to this IIT Magazine article.

Two-flat owner journal 1: Peoples Gas charges a lot of money for no gas

One of the first things I did after I bought a two-flat in July was contact Peoples Gas and Comed to ensure utilities were in my name, and that the utility connections would not be interrupted.

A few days later I decided that I wasn’t going to move in, because I wanted to make a good amount of changes and the best time to do that would be when nobody is living there. “Good amount of changes” turned into “gut rehab”. One of my friends is an architect and we (mostly him) are drawing permit plans right now.

It wasn’t until a week ago (8 weeks since I bought the house) that I realized there’s no reason to be paying for Peoples Gas to maintain a connection when I’m not using natural gas.

I’m writing this journal entry to exclaim how expensive it is to just “leave the gas line connected”.

It costs $50 per month per unit to have the privilege of possibly purchasing the delivery of natural gas through a pipe. Both units used 0 therms in the longest-period bill I received. (I received three bills, only one of which was for 30 days.)

The bill for the only 30-day period Peoples Gas served one of the units in my two-flat. Both units are unoccupied until after the gut rehab is completed. See the “customer charge”, which is the charge just to have an account open and for the potential to use gas.

I’ve got to pay $50 per unit for no gas.

I visited a three-flat under construction in Pilsen on Friday, and talked to the developer, Brent. He described how he’s following high-efficiency building wall standards to create a “tight envelope” (one in which very little air can leak) so that the tenants can “receive the comfort they’re paying for”. When it comes to setting the thermostat, the air delivered by HVAC machines should match that exactly, no more, no less. No oversized furnaces pushing too much heated air because so much of the air leaks through the walls and windows.

And, as a way to control costs, Brent will not connect a natural gas pipe to the building, mostly because of the expensive and default customer charge that persists even when no gas is used. A VRF (variable refrigerant flow) and heat pump machines will be entirely powered by electricity to serve the tenant’s heating and cooling needs.

Brent said that the tight building envelope coupled with the high-efficiency HVAC means that it’s more cost effective to use electricity to heat a house than natural gas.

After our meeting, I looked again at my final bill from Peoples Gas (I closed the account two days prior) and understood what Brent was saying about controlling costs. With an electric water heater and an electric range, there’s no need to have any gas connection.

I will probably have to keep the gas at my two-flat, to power the furnaces, because I don’t have the expertise or financial resources to renovate an existing building to have a tight enough envelope to make electrically-generated heat more cost effective than gas-generated heat.

Update January 13, 2021: I turned off the gas and closed the accounts so I don’t have to waste any money while I’m not living there (a gut rehab still needs to happen).

To keep water pipes from freezing and bursting I cleared the vast majority of water lines and added an electric pipe heating cable to keep the remaining sections warm.

The two water service pipes (I don’t know why there are two) have an electric pipe heating cable and are wrapped in foam insulation. The cable has a thermostat that touches the pipe and starts heating when the pipe drops to 38°F. The pipe is heated until the thermostat detects ~46°F.

What is this place? Des Plaines “park” edition

Screenshot shows the “park” in Des Plaines, Illinois.

I was methodically reviewing features on OpenStreetMap that could benefit from additional attributes, namely missing names and cities. The information on OpenStreetMap feeds into Chicago Cityscape so that the real estate information service I created can show people looking up addresses the locations of nearby amenities, including parks.

This particularly large park in Des Plaines had no name and no city, so I started investigating. From the overhead imagery view, it looked to me like a landfill.

Since there was a forest preserve to the east, I looked in the Forest Preserve District of Cook County’s nice interactive map for any properties near Beck Lake. Nothing found.

Overhead imagery of the so-called “park”. Source: Mapbox

I looked on Google Maps Street View for some insight and there was no name. (Note that data from Google Maps cannot be used to amend OpenStreetMap because of Google Maps having a data license that isn’t compatible with OSM’s openness requirements).

Next I looked at the Cook County parcel map that Chicago Cityscape has to identify its “Property Index Number” (PIN). All of it fits within 04-31-300-003-0000 – the database pointed to “Catholic Cemeteries” as the property owner.

I looked up that PIN on the Cook County Recorder of Deeds to try and find more information. Lo and behold, the most recent document that was recorded against the PIN was for a “Memorandum of Option Agreement” that identified the Catholic Bishop of Chicago as the landlord and Patriot Acres, LLC, as the tenant.

Patriot Acres has a website that describes a new commercial composting facility being built at this location.

I re-tagged the feature on OSM with “amenity=waste_disposal” and “waste=organic” to properly describe it as a composting facility, and not a park. Case closed.

Chicago bonfire rules

I believe that more people are hosting more backyard bonfires in Chicago in order to spend time with friends outdoors, because it’s one of the lowest risk ways to still enjoy life during the COVID-19 pandemic that Trump is prolonging.

The City of Chicago has rules on what can be burned. Be sensitive to neighbors who rely in keeping their windows open to cool and ventilate their homes.

The summary version is this: You can burn manufactured firewood that you purchased, and you can burn wood logs from any trees someone manually cut. All logs have to be untreated and two feet long or shorter.

You can’t burn scrap wood, lumber, plywood, or particle board. You also cannot burn garbage. Avoid having bonfires when the wind speed is 15 miles per hour or faster.

Read all the rules in this PDF. The rules aren’t in the municipal code; the code in section 11-4-740 says that the environment commissioner shall make rules (Chicago doesn’t currently have a department of environment).

This is my process to improve the performance of my PostgreSQL database and queries

Every now and then I’ll run the queries below to help me figure out ways to optimize the queries my application is using, and to figure out if certain tables and views (relations) need to be optimized.

Updated January 18, 2020, to add a query to alter the autovacuum and autoanalyze factors on a per table basis because I have a “cache” table that is updated quite frequently and generates many dead tuples every minute. The table would balloon from 250 MB to 15 GB in a few hours. I hope that reducing these factors to just a threshold of 100 dead tuples will prevent the inflation. I also need to monitor this that it doesn’t make my RDS database work too hard.

/* Show the last time at which every table and view was vacuumed and analyzed */
SELECT relname, last_vacuum, last_autovacuum, last_analyze, last_autoanalyze  
FROM pg_stat_all_tables  d_d
WHERE schemaname = 'public';  

/* See which queries are most popular */
SELECT * FROM pg_stat_statements where calls >=1000; /* replace 1000 with any integer, depending on how active your database is queried */
select pg_stat_statements_reset();

/* Kill a specific query */
select pg_terminate_backend(INTEGER); /* replace INTEGER with the process ID of a specific query */

/* Get the size of the database */
select pg_size_pretty(pg_database_size('DATABASE_NAME')); /* replace DATABASE_NAME with the name of your database */

/* Kill queries that are taking too long; replace DATABASE_NAME with the name of your database */
SELECT query, pg_terminate_backend(pid) 
    FROM pg_stat_activity 
    WHERE datname = 'DATABASE_NAME' 
      AND pid <> pg_backend_pid() 
      AND (state = 'active' 
      OR state = 'idle') 
      AND state_change < current_timestamp - INTERVAL '15' MINUTE; /* change 15 to any integer that you think means a query is taking too long */

/* See which queries are running */
select * from pg_stat_activity;
SELECT * FROM pg_stat_activity WHERE wait_event IS NOT NULL AND backend_type = 'client backend';

/* Find very large indexes and indexes that aren't used much */
WITH table_scans as ( 
    SELECT relid, 
        tables.idx_scan + tables.seq_scan as all_scans, 
        ( tables.n_tup_ins + tables.n_tup_upd + tables.n_tup_del ) as writes, 
                pg_relation_size(relid) as table_size 
        FROM pg_stat_user_tables as tables 
),
all_writes as ( 
    SELECT sum(writes) as total_writes 
    FROM table_scans 
),
indexes as ( 
    SELECT idx_stat.relid, idx_stat.indexrelid, 
        idx_stat.schemaname, idx_stat.relname as tablename, 
        idx_stat.indexrelname as indexname, 
        idx_stat.idx_scan,
        pg_relation_size(idx_stat.indexrelid) as index_bytes, 
        indexdef ~* 'USING btree' AS idx_is_btree 
    FROM pg_stat_user_indexes as idx_stat 
        JOIN pg_index 
            USING (indexrelid) 
        JOIN pg_indexes as indexes 
            ON idx_stat.schemaname = indexes.schemaname 
                AND idx_stat.relname = indexes.tablename 
                AND idx_stat.indexrelname = indexes.indexname 
    WHERE pg_index.indisunique = FALSE 
),
index_ratios AS ( 
SELECT schemaname, tablename, indexname, 
    idx_scan, all_scans,
    round(( CASE WHEN all_scans = 0 THEN 0.0::NUMERIC 
        ELSE idx_scan::NUMERIC/all_scans * 100 END),2) as index_scan_pct, 
    writes,
    round((CASE WHEN writes = 0 THEN idx_scan::NUMERIC ELSE idx_scan::NUMERIC/writes END),2) 
        as scans_per_write, 
    pg_size_pretty(index_bytes) as index_size, 
    pg_size_pretty(table_size) as table_size, 
    idx_is_btree, index_bytes
    FROM indexes 
    JOIN table_scans 
    USING (relid) 
),
index_groups AS ( 
SELECT 'Never Used Indexes' as reason, *, 1 as grp 
FROM index_ratios 
WHERE
    idx_scan = 0
    and idx_is_btree 
UNION ALL 
SELECT 'Low Scans, High Writes' as reason, *, 2 as grp 
FROM index_ratios 
WHERE
    scans_per_write <= 1
    and index_scan_pct < 10 
    and idx_scan > 0 
    and writes > 100 
    and idx_is_btree 
UNION ALL 
SELECT 'Seldom Used Large Indexes' as reason, *, 3 as grp 
FROM index_ratios 
WHERE
    index_scan_pct < 5
    and scans_per_write > 1 
    and idx_scan > 0 
    and idx_is_btree 
    and index_bytes > 100000000 
UNION ALL 
SELECT 'High-Write Large Non-Btree' as reason, index_ratios.*, 4 as grp 
FROM index_ratios, all_writes 
WHERE
    ( writes::NUMERIC / ( total_writes + 1 ) ) > 0.02 
    AND NOT idx_is_btree 
    AND index_bytes > 100000000 
ORDER BY grp, index_bytes DESC ) 
SELECT reason, schemaname, tablename, indexname, 
    index_scan_pct, scans_per_write, index_size, table_size
FROM index_groups; 

/* Modify the autovacuum and autoanalyze factors for a single table 
https://klotzandrew.com/blog/posgres-per-table-autovacuum-management */
ALTER TABLE cache SET (autovacuum_analyze_scale_factor = 0, autovacuum_analyze_threshold = 100);

WITH raw_data AS (
  SELECT
    pg_namespace.nspname,
    pg_class.relname,
    pg_class.oid AS relid,
    pg_class.reltuples,
    pg_stat_all_tables.n_dead_tup,
    pg_stat_all_tables.n_mod_since_analyze,
    (SELECT split_part(x, '=', 2) FROM unnest(pg_class.reloptions) q (x) WHERE x ~ '^autovacuum_analyze_scale_factor=' ) as c_analyze_factor,
    (SELECT split_part(x, '=', 2) FROM unnest(pg_class.reloptions) q (x) WHERE x ~ '^autovacuum_analyze_threshold=' ) as c_analyze_threshold,
    (SELECT split_part(x, '=', 2) FROM unnest(pg_class.reloptions) q (x) WHERE x ~ '^autovacuum_vacuum_scale_factor=' ) as c_vacuum_factor,
    (SELECT split_part(x, '=', 2) FROM unnest(pg_class.reloptions) q (x) WHERE x ~ '^autovacuum_vacuum_threshold=' ) as c_vacuum_threshold,
    to_char(pg_stat_all_tables.last_vacuum, 'YYYY-MM-DD HH24:MI:SS') as last_vacuum,
    to_char(pg_stat_all_tables.last_autovacuum, 'YYYY-MM-DD HH24:MI:SS') as last_autovacuum
  FROM
    pg_class
  JOIN pg_namespace ON pg_class.relnamespace = pg_namespace.oid
    LEFT OUTER JOIN pg_stat_all_tables ON pg_class.oid = pg_stat_all_tables.relid
  WHERE
    n_dead_tup IS NOT NULL
    AND nspname NOT IN ('information_schema', 'pg_catalog')
    AND nspname NOT LIKE 'pg_toast%'
    AND pg_class.relkind = 'r'
), data AS (
  SELECT
    *,
    COALESCE(raw_data.c_analyze_factor, current_setting('autovacuum_analyze_scale_factor'))::float8 AS analyze_factor,
    COALESCE(raw_data.c_analyze_threshold, current_setting('autovacuum_analyze_threshold'))::float8 AS analyze_threshold,
    COALESCE(raw_data.c_vacuum_factor, current_setting('autovacuum_vacuum_scale_factor'))::float8 AS vacuum_factor,
    COALESCE(raw_data.c_vacuum_threshold, current_setting('autovacuum_vacuum_threshold'))::float8 AS vacuum_threshold
  FROM raw_data
)
SELECT
  relid,
  nspname,
  relname,
  reltuples,
  n_dead_tup,
  ROUND(reltuples * vacuum_factor + vacuum_threshold) AS v_threshold,
  n_mod_since_analyze,
  ROUND(reltuples * analyze_factor + analyze_threshold) AS a_threshold,
  c_analyze_factor as caf,
  c_analyze_threshold as cat,
  c_vacuum_factor as cvf,
  c_vacuum_threshold as cvt,
  analyze_factor as af,
  analyze_threshold as at,
  vacuum_factor as vf,
  vacuum_threshold as vt,
  last_vacuum,
  last_autovacuum
FROM
  data
ORDER BY n_dead_tup DESC;

My investment in crowdfunded projects hasn’t gone well

A second project that I invested in through the crowdfunding portal, Small Change, has “failed”. It’s a project for a worker-owned bed and breakfast. According to the project owner, the project needed more funds than what they were able to obtain from Small Change investors and other investors. The owner found a new investor, who required that all Small Change investors be removed from the project. They announced that to us earlier this year, and said we would receive our initial investment + 3%.

The Small Change leadership didn’t think this plan was good enough, starting with disagreeing the removal of Small Change investors.

Eventually, the project owner returned our initial investment + 8%. I don’t know if the project owner had the right to cancel and return our investments, but I’m happy to have my money back + 8%.

I’m disappointed in the ultimate outcome of not being able to participate in 1476 Magazine because this is the second project I invested in via Small Change that has not succeeded. The first was for a group of three single-family houses built to a high energy efficiency standard that have not yet sold.

I have an investment in a third project (in Philadelphia) that is converting a disused industrial building into mixed-use with residential and I hope that one succeeds, but I haven’t received an update since it was funded. Update: I inquired and the project owner sent me the update they sent in the last couple of months, which I seemed to have missed.

Golaski Labs in Philadelphia. A Small Change poster advertises the project. The photo was taken in 2018, and significant progress has been made since then.


I believe in the Small Change mission, and crowdfunding broadly, but once I made my third investment I paused so that I could wait to see what happens with all of the projects.