Big data and big plans

The year-end issue of The Economist includes a nice essay on planning. “Beware of the Borg” (title in print edition). Everyone plans; plans can be coordinated by markets or usurped by top-down grand plans. The latter often fail, ending in calamity and often much worse. We have heard about Venezuela, USSR, North Korea and many more. Top-down plans fail because utopians ignore the fact that knowledge is complex and dispersed. Promises are inevitably broken, people go hungry, and autocrats resort to terror and corruption to stay in power. 

In the real world, specialization and exchange prompt resource allocation and innovation, but only if rights are credible and secure.

Just as no one person can make a pencil, and yet pencils are everywhere, supply chains are everywhere. Supply chains are emergent. They are formed and maintained by savvy entrepreneurs. Add that there are supply chains for ideas and information. Commodity supply chains begin with supply chains for ideas. We all search for useful information all the time.

Just as entrepreneurs decide what to make vs what to buy, they also choose what to make and buy where. Sometimes the “where” question evokes various shades of nearby as part of the answer. The cities we get are actually a complex mesh of supply chains, including supply chains for ideas.

Transactions all across any chain are most likely where there is trust; trust is established when and where there is communication. Communication can be electronic or face-to-face. All of us pick the blend of communications channels that works best for us. In so doing, we pick a location that is best for the chosen blend.

Hayek (and many others) have argued that top-down planners cannot accomplish any of this. They could not replace the market’s discovery process and would always be badly informed. They are also (by definition) highly politicized which makes a bad situation worse.

All this is clear enough but utopian romances are hard to dislodge. Class struggle and the romance of righting historical grievances are always a problem and always fanned by opportunist politicians.

The Economist’s survey also links an old debate about markets and planning with a new one (a new utopian romance) about the potential of "big data" to change the old debate. David Gelernter argues that big data will not replace human imagination or ingenuity. The Economist also quotes Alex Tabarrok: "the problem of perfectly organizing an economy does not become easier with greater computing power precisely because greater computing power makes the economy more complex." Do read The Economist's whole essay.   

It takes a horrific hurricane

Where to start? We have too much crony capitalism. Many of our young people graduate unprepared for productive work. Productivity gaps translate into income and wealth gaps. Some of our poorest children are condemned to the worst schools.

Terry Moe and Russ Roberts report that these are all wound up as one big problem: the ways in which the education establishment has succeeded in choking off reform and innovation and experimentation. And many of the people who fret most about all these things are dug in on the anti-reform side.

Moe's research centers on the pre- and post-Katrina New Orleans schools. The school board resisted reforms and the outcomes were predictable -- and awful. Graduation rates were in he mid-20% range. But the Katrina devastation was so severe that not only buildings but the administrative infrastructure that ran the schools had to be replaced. In desperation, charter schools were allowed. Parents became involved. A freer atmosphere that allowed experimentation followed. Graduation rates doubled,

Moe and Roberts emphasize that none of this was planned. Good news and bad news. A small measure of openness brings welcome change. But it takes a mighty hurricane. What can we expect for the rest of the country?