Seeing Is Believing: Understanding and Expanding Multifamily Building Retrofit Markets Through Large-Scale Computer Vision Analysis of Building Characteristics

Resulting dataset can help determine retrofit candidates, build capacity, and prioritize investments in technologies that address market gaps.

Under its contract with the New York State Energy Research and Development Authority (NYSERDA), Cadmus conducted a statewide multifamily baseline study, including an experimental aerial image analysis using machine learning/computer vision to identify externally visible characteristics in multifamily buildings. This paper describes Cadmus’ approach to the image analysis component of the study and how the technology can be implemented and scaled to support other large-scale studies, as well as understanding target markets for retrofits and other services. Cadmus also authored a second paper with NYSERDA that summarizes the more traditional building assessment component of the study. 

Read the full paper: Seeing Is Believing: Understanding and Expanding Multifamily Building Retrofit Markets Through Large-Scale Computer Vision Analysis of Building Characteristics

Authors: Conner Geery, Nathan Hinkle, and Allyson Dugan; Cadmus; James Geppner; New York State Energy Research and Development Authority

Decarbonizing millions of buildings equitably, effectively, and affordably demands urgent innovation and financial resources, but market data for retrofit solutions is largely unavailable. To address this gap, our team extracted details of over 130,000 multifamily buildings—nearly every one in New York State—by using machine learning to develop a novel process to analyze over 650,000 aerial images. By benchmarking our results against data collected in 391 in-person site visits, we were able to assess the model’s performance compared to human observation across a range of building styles, sizes, densities, and vintages.

The resulting dataset contains dozens of attributes for each multifamily building. Energy efficiency and clean energy programs, manufacturers, providers, and others can search and filter these data to determine the distribution and precise locations of retrofit candidates, build capacity based on accurate information about the size of the market, and prioritize investments in technologies that address market gaps.

This paper describes our challenges with and solutions for systematically acquiring multiple aerial images of each building; distinguishing the targeted buildings from others nearby; training and deploying instance segmentation models to identify building features; and analyzing the outputs to calculate building configurations and dimensions, window-to-wall ratios, rooftop equipment quantities, and other metrics. We also explore how the results of this study can be used to direct investment in building retrofit programs, the performance and limitations of our technique and how to apply it in other jurisdictions, and topics for future research.