The Nobel Prize in Economics for the leading practitioners of Randomized Control Trials (RCTs) has not ended the debate about the status of this methodology in empirical economics. Although the Nobel lectures of the three recipients, Duflo, Banerjee and Kremer, recently published in the American Economic Review, contained a brilliant defence of the benefits to be obtained from carefully designed randomized field experiments, a previous Economic Nobel Prize winner, Angus Deaton, has also made public an update of his criticism of the status of RCTs. Deaton is careful to admit that this methodology (consisting of treating a randomly selected part of a sample with some policy or remedy in the manner of medical experiments) should occupy a place in the toolbox of empirical economists, but this place should not be higher in the hierarchy relative to other more traditional observational methods that do not rely on randomization. His criticism includes doubts that many RCTs are able to split samples in a way that the only difference between the treatment and control groups can be isolated to be the treatment policy, and also concerns that the difficulties in administering the experiment may outweigh any statistical advantages of a "clean" experiment. But the most troubling questions concern ethics, "especially when very poor people are experimented on" with experiments conducted by economists from rich countries, often financed by corporations or foundations of corporations from the first world. For example, an experiment in political economy may change the result of an election: why should economists from rich country universities interfere with the choices of poor voters? Additionally, the external validity of some RCTs, that is, the lessons to be drawn for contexts other than those where the experiment is conducted, should be treated with extreme caution. What "works" depends on for whom and for what purpose; what works involves values as well as facts. It is not unusual that populist (sometimes with autocratic temptations) or tribalist leaders, like Modi in India, clean their reputation by hiring experimenters that dress his policies as the result of technical solutions, intensifying the ethical dilemmas. Jean Drèze, in an article mentioned by Deaton, stresses the impossibility of trying to sideline politics in development policies.
A recent working paper of the National Bureau of Economic Research in the US, written by Paul Gertler of UC Berkeley and coauthors, is a good example of the ethical risks of randomized control trials. This article, in its working paper version, reports about the results of an RCT where citizens of poor slums in Nairobi (Kenya) were randomly threatened with discontinuity of water service for not paying their bills. The experiment was conducted apparently with funds and cooperation from the local utility company, and the result of the experiment is that threatening to cut electricity is more effective than a soft campaign trying to convince citizens of the negative effects of not paying the bills. From ex post surveys on citizen activism, the scholars conclude that the threat and subsequent payment of bills would not have negative political consequences, something that will hardly convince anyone familiar with the commitment problem of utility investment in countries with deep poverty and inequality problems. The protest against the use of experimentation on poor people in this article in Twitter was such, that the authors felt under pressure to publish a comment on the ethical issues of their specific RCT, promising to clarify these issues in a later version of the article. The question here is how to address the real and acute problem of underinvestment of utilities in poor countries with really ethical research, that is, research that does not rely on arbitrarily and randomly upsetting the life of vulnerable citizens. There is a very extensive literature on the time inconsistency problem in regulation (my Google Scholar profile contains a sample of modest contributions) that may provide a guide to the difficulties and possible complex second best solutions that do exist. More recently, there is work by Ashraf, Glaeser and co-authors, which explain the difficulties (and also the benefits) of expanding infrastructure efficiently in developing countries with a theoretical model, and then apply the model to explain the existing challenges in places that go from New York in the 19th Century to contemporary African cities. Possible ways to alleviate the underinvestment problem include setting up efficient institutions (not easy, and certainly not an apolitical process) that are adapted to the institutional endowment of local contexts.
No comments:
Post a Comment