Proxy Variables

A few pages into The Principles of Product Development Flow, Donald Reinertsen brings up the notion of proxy variables and provides this definition: “A proxy variable is a quantified measure that substitutes for the real economic objective: life-cycle profits.”

He goes on to say, “By focusing on proxy variables, product developers delude themselves into thinking they understand their economics. They do not.”

And finally, “It’s only when we understand the mapping between proxy variables and life-cycle profits that we can really see the economic consequences of our choices.”

Reinertsen is writing about the influence of variables on large, complex systems. But the concept of proxy metrics lends itself well to other problems, including web analytics, where we often have cheap access to an abundance of behavioral tracking—producing proxy metrics—but little visibility into the actions that contribute to the “real economic objective.”

When all you have is proxy data, you start to forget that you’re not looking at the real thing. Pageviews, unique visitors, referral source, time-on-site, clicks. Rarely do these things have any innate value. Instead they serve as indicators or “intermediate metrics”.

Reinertsen makes two points that deserve emphasis.

The first is the assertion that by overvaluing/misvaluing proxy variables product developers “delude themselves into thinking they understand their economics.”

Just as it’s easy to forget the proxy-ish nature of our data it’s easy to trick ourselves into thinking we understand the underlying motivations of our users and effects of our product decisions, when all we often know is the effect on certain proxy metrics.

The second is the observation that understanding the relationship between proxy variables and the ultimate economic objective is the only way to understand the “economic consequences of our choices.”

For all the effort spent chasing proxy metric goals, how much time is spent refining the mapping to the real economic objective?