Dr. Philip Agee, is an Assistant Professor of Building Construction and the Assistant Director of the Virginia Center for Housing Research (VCHR) at Virginia Tech. His research advances industrial and systems engineering design and evaluation methods across the lifecycle of the built environment (e.g., design, construction, operation).
Significant data is generated in the built environment. Design and construction documents, performance simulations, commissioning results, and building performance data (BPD) are generated for all modern buildings. Today, system advancements in energy efficiency, integration, monitoring, and connectivity are expanding our design and facility management options throughout the built environment. Internet-enabled monitoring, ubiquitous user interfaces, machine learning, and artificial intelligence are facilitating analytical approaches not previously afforded to our industry. As buildings become more complex, the need to measure, analyze, and share BPD has become salient.
Open datasets are critical for advancing building performance and analytical methods. Open BPD can be leveraged to calibrate design simulations and inform future design, construction, and operational decisions. Open BPD can facilitate benchmarking performance across building types, vintage, and geographic distributions1. Conversely, open BPD can be used to benchmark machine learning techniques for the built environment2. Simply put, open BPD are a critical component in the systems approach necessary to improve outcomes in the built environment including, but not limited to, reducing carbon emissions and operating costs and improving human experiences in the built environment.
The Architecture, Engineering, Construction, and Operation (AECO) industry rarely systematically collects or shares BPD today. Researchers increasingly collect BPD, yet rarely share BPD in open-source data repositories3. In recent years, however, there has been a trend to develop and share open-source BPD4â12. For example, there are recent open BPD contributions focused on occupant behavior impacts on energy use and indoor environmental quality (IEQ). These contributions span human-building interactions with appliances5, heat pumps7, and natural ventilation systems8. Other recent contributions span energy use across building typologies (e.g., commercial and residential) 9â12. This article contributes to the recent trend toward open BPD datasets.
The Why, What, How, and When of Building Performance Data
Why, What, and How?
Reliable data collection can be expensive and time consuming. Just because you can collect data does not mean that you should collect data. Even as a professional researcher, I am lazy and only collect data when I absolutely have to collect data. The collection of data, database development (defining variables, data types), sorting/cleaning, analyzing, validating, and synthesizing findings into useful information is hard. Dr. Joe Lstiburek says it best â âlife is hardâ and I would suggest you not collect data just because we now have internet-enabled sensing and logging capabilities. Instead, I suggest only collecting data to answer a specific question. The research question guides what data you need to collect (e.g., quantitative, qualitative) and how you need to measure/analyze the data (e.g., for diagnostic purposes or for long-term reporting) to answer the question.
It is important to remember that most building performance questions require both quantitative and qualitative data. As engineers, we cannot just rely on numbers, we have to talk to people. Quantitative data can tell me if something is (statistically) significant, but it cannot tell me if the phenomena I am studying is important. Conversely, qualitative data can tell me if something is important, but it cannot tell me if it is significant. When we are working on complex, systems-based problems that often relate to human factors, the mixing of quantitative and qualitative methods is necessary to unpack these interactions and answer our research question(s).
Think about solving a thermal comfort problem in a home. The home is not uncomfortable, the people are. To answer the thermal comfort question, you need to mix quantitative data measured empirically (e.g., operative temperature, airspeed, relative humidity) with qualitative data (e.g., occupant perceptions collected from surveys, interviews). So how do you do this? Table 1. provides an example of how you can align research questions with aims for the data collection, the method(s), and data. I do this at the start of every research project. This is a fundamental research skill that is taught during Ph.D. training.
| QUESTION | AIM | METHOD/INSTRUMENT | DATA | ||
| 1. | What is the median electricity end-use profile for each apartment type? | 1.1 | Characterize hourly end-uses/costs for: â˘heat pump domestic water heating â˘washer/dryer â˘range â˘miscellaneous electric loads (MELs) | â˘Circuit-level energy monitoring â˘Empirical analysis | â˘kWh/end-use â˘unit $/end-use â˘unit HDD65 / CDD65 â˘simulation outputs |
| 1.2 | Develop resident energy use profile to target energy literacy intervention | â˘Resident survey(s) â˘Resident interview(s) â˘Descriptive statistics | â˘attitudes â˘beliefs system interactions â˘kWh/end-use/unit | ||
When to Collect Data?
When should you collect building performance data? It depends. I do want to introduce an approach that is fundamental and ubiquitous in the design of human-centered systems, but has not found a home in the built environment (pun intended). Formative and summative evaluations help us align when and how we should collect data with our research question(s).
Formative and Summative Evaluation Approaches
Formative evaluation helps you âformâ the design and construction of a product or system13. If you are familiar with rapid prototyping â this is formative. In the built environment context, design reviews, energy simulations and load calculations, pre-drywall visual inspections and testing, and commissioning are examples of formative evaluations. Data from formative evaluations reduce risk by helping us align the system with our requirements. The more you get user feedback, the better you can adjust and reduce the risk of the product sucking â a non-technical way of saying ânot meeting system requirements.â This formative data does not (typically) inform policy decisions or contribute to the body of scientific knowledge of a problem area.
Conversely, summative evaluations help you âsumâ the outputs and outcomes of a product or system13. In the built environment context, summative evaluations could be performed with long-term monitoring of energy use and/or indoor environmental quality data (e.g., temperature, relative humidity, occupant surveys) after a building has been placed in service. Summative evaluations typically do inform policy and contribute to the body of scientific knowledge of a problem area. These evaluations require rigorous data collection procedures and aim for statistical significance.
The takeaway? Do not just collect data for dataâs sake. Align your data collection efforts to answer a research question, and identify how and when to evaluate the results (e.g., use of formative and/or summative evaluation) to improve outcomes in the built environment.
Example of Open Source Building Performance Data
If you have made it this far and want to see an example output of an open Building Performance Dataset and the supporting data descriptor, you can check out the example below or those linked in the references. Thank you to the OpenHVAC community for the platform to discuss this important topic; I am excited to see what this community can do together.
