President, Chief Appraiser at Argianas and Associates, specializing in the valuation of complex institutional quality properties.
getty
Automated valuation models (AVMs) debuted in the 1950s as computer-assisted mass appraisals (CAMAs) that were created to meet assessors’ demand during the post-depression/WWII housing boom. They are still used in today’s online real estate marketplace to assign values to neighborhood homes. Modern AVMs have improved from initial legacy offerings, yet nuanced problems remain. But there is a simple solution: The secret to any good AVM is integration and a competent appraiser.
The Integrity Of The Real Estate Markets
A significant portion of a financial institution’s portfolio/business plans includes lending secured by real estate. Lessons learned from the Great Depression, the Korean War housing crunch, the commercial market collapse in the 1980s and the financial crisis of the mid-2000s repeatedly demonstrate the need for caution and efficient and high-fidelity management of real estate lending programs. As recently as 2010, with the passage of the Dodd-Frank Wall Street Reform and Consumer Protection Act, the importance of accurate and valid real estate valuations continues to be reaffirmed.
All AVMs Are Not Created Equal
There is no gold standard when it comes to AVMs. They use different analytical methods or variables, serve disparate client bases, source their information from potentially incomplete and inaccurate public records, miss off-market listings and cannot confirm if a property even exists. The development of any AVM requires experience, machine learning and research and development — a time-consuming and costly endeavor, to be sure. The industry continues to move toward a set of standards; however, as any stats professor will tell you, garbage in equals garbage out. Past performance may not indicate future trends, spatial or geographic differences can be missed and some proprietary AVM models are not independently tested for accuracy.
Two types of AVMs exist: consumer-grade and lender-grade models. Consumer-grade models are available at no cost and target pedestrians and homeowners searching for property value information. You would not predicate a buy/sell decision on these models. Because consumer-grade models may or may not participate in coordinated AVM testing and validation, they are not accepted by the OCC, FDIC, NCUA or the FRB, regulatory agencies adopted by lenders and financial institutions.
In contrast, there are approximately 20 (and growing) lender-grade AVMs, which regularly undergo stress testing and are employed by lenders regularly as a tool to improve the cumbersome credit underwriting process.
Weeding Out Good AVMs
One way an AVM’s accuracy is tested is through the continual calculation of a model’s performance metrics. One such metric is the predicted percentage error (PPE). The PPE defines how often the AVM value is within a given range of the true or benchmarked value which is established externally. A PPE of 10 would denote how frequently the AVM-derived value falls within +/- 10% of the benchmark value. The benchmark used represents a concurrent derived market value taken from contract price during purchase, recently completed sales, refinances or an appraisal valuation. Additionally, the benchmark assumes the true market value assuming a willing transaction between a willing buyer and seller absent duress. The expected or anticipated value and AVM-derived value can then be used to calculate the percentage difference, providing up-to-date information on how a model performs.
If we accept a PPE of 10 (on average, a +/- 10% margin for error is regarded as acceptable in the U.S.), approximately 65% of AVM valuations will fall within the +/- 10% bracket. However, depending on the location, valuations within the +/- 10% bracket can range from 20% to 92%. Scoring is done by county, and should incorporate the entire performance of the AVM, not just point values like mean error, median error or standard deviation.
Where Is The Valuation Industry Going?
Data science is a driver of innovation in real estate; however, this is dependent on the quality of the data and its accessibility and interpretation. Successful implementation of AI is more than a plug-and-play model. Instead, it is a complete transformation that comes with the foundational development of organizational structures geared toward machine and human integration and complementation. Across arenas, integrating AI with humans often performs superior to humans or AI alone.
We’re living in a digital age, and the nexus between real estate and technology will continue to blend into our everyday life. An AVM is an additional tool for a financial professional’s tool belt. The AVM is not intended to replace the financial professional, but rather supplement their efforts to obtain fair and equitable valuations. An AVM isn’t going to replace the necessity of a physical inspection, somehow bring public records (which are often inaccurate and incomplete) up to date, synthesize the improvements or external obsolescence of a property or correct the inaccuracies a competent appraiser identifies during the appraisal process.
While an AVM may miss the mark and real estate appraisers may exhibit a skills gap in analytics, together, they are better. Appraisers place the world of CRE into context with next-gen data and analysis, leading to a better and more secured lending process. If you open an appraisal report, chances are you’re already seeing data culled from next-gen models, which allows appraisers to spend more time doing what they do as real estate economists. Real estate is often an investment of capital, emotion and sweat equity. An AVM is not a substitute for a human, and suggesting otherwise exceeds the humanity that is investment.
Forbes Real Estate Council is an invitation-only community for executives in the real estate industry. Do I qualify?
