Zillow thought they had a great idea, using their big data and analytical tools to not only provide valuations for properties but also flip properties by buying homes in hot markets and selling them a few months later after adding some needed repairs and updates. They could profit from the general market appreciation as well as their skilled rehab of the home.
Zillow had lots of data on the selling prices and selling times of homes which allowed them to build trend following and price prediction models for home prices in very specific geographies. However, the models did not consider how difficult it would be to find a contractor when everyone on lockdown spent the money destined for the vacation they missed during COVID on a kitchen or bathroom upgrade, or even a new home office setup. That hit Zillow in two ways: first, labor and materials costs for home renovations had skyrocketed, and second, the lack of available supplies and contractors increased the holding time on those properties. Not only is there a time value of money, but models may be less accurate as the forecast horizon lengthens. Check out this article for an explanation of the types of drift in machine learning.
Human intelligence and artificial intelligence combined will outperform either system individually, as human translators are necessary to oversee areas the models can’t see.