When Learners Escape Models of Their Behavior

Modeling and measurement are stable in many domains, such as ballistics. So why are new models continually needed in other domains, such as macro-economic policy and spam filtering? Models always include unmeasured dimensions, but people (or any learners) can escape into those unmeasured dimensions. People don't always try to escape, so we look for conditions where escape is likely, prompted by the insight that too much striving for measured goals hurts unmeasured goals. For example, science hiring's quantity demands led to a drop in quality, eventually visible in the reproducibility crisis. We can also identify macro-level indicators such as increasing prices, new products, and divergent profits. Under the pressure of high protein prices, manufacturers of pet food (normally a commodity) substituted melamine for protein, gaming the way protein was measured. What can modelers do? First, modelers can try to change the conditions that caused escape pressure. Failing that, we can either speed up the evolution of our own models or slow down the evolution of others' models. Techniques to speed up include better feedback, models that are simple or explainable or modular, and using multiple models (an ensemble). Techniques to slow down others include obscuring our own model so that it cannot be backward-engineered. In sum, domains that continually need new models typically have pressure for learners to escape the models. We should study conditions that make escape likely and better meta-modeling techniques to deal with it