Esther Duflo

INTERVIEWER: Today is January 25th, 2016. I'm Joe McMaster. And as part of MIT'S Infinite History Project, I'm speaking with Dr. Esther Duflo, the Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics in the Department of Economics at MIT, and a Co-founder and Co-director of the Abdul Latif Jameel Poverty Action Lab, or J-PAL.

A development economist who has been described as a practical visionary, Duflo seeks to understand the social and economic forces perpetuating cycles of poverty around the world, with the aim of helping design and evaluate effective social policies. By combining innovative field experiments with rigorous empirical analysis, she is pioneering a new type of economics to identify linkages and causal relationships between policy, poverty, behavior, and socioeconomic status.

Duflo has received numerous academic awards and prizes from around the world, including the John Bates Clark Medal and a MacArthur Fellowship. Professor Duflo holds degrees in History and Economics from the Ecole Normale Superieure in Paris, and a master's in economics from the Ecole Normale and the Ecole Polytechnique, and a PhD in economics from MIT. Thank you for speaking with us, Dr. Duflo.

DUFLO: Thank you for having me.

INTERVIEWER: So I was interested to read I think it's an excerpt from your book that I thought would be a good place maybe for us to start. And you had this quote saying, "To progress with regard to alleviating poverty, we have to abandon the habit of reducing the poor to cartoon characters, and take the time to really understand their lives in all their complexity and richness. And for the past 15 years we have tried to do just that." Which I suspect this is now the past 20 years or so.

Maybe that's a good place to start. Maybe you can sort of tell me about that, and what you and your colleagues are doing, what you mean by the cartoon character and what you're doing to address that.

DUFLO: Typically, when people think about poverty, about the poor, they try to reduce it to one dimension. So for example, you think of the poor as being entrepreneurial. Like this wealth of entrepreneurship that only needs unlocking, to say finance. And then that naturally leads you for example to ideas like microcredit.

Or you try to think of the poor as being lazy and slothful, say like in Victorian times. So that naturally leads you to policies towards the poor that are supposed to put them back to the straight and narrow path. Or you think of the poor as being starved. So all your anti-poverty policy is going to be about trying to find ways to put food in the mouths of people, et cetera.

And it's not to say that there is not something that's true in any of these views, but it's like in isolation they fail to capture the whole complexity of what it is to being poor. And therefore, they have the nice aspect that they tend to lead you toward a silver bullet or single solution. So that's very seductive, but it's very misleading as well. And therefore, these solutions have tended to be much less effective than people have hoped.

INTERVIEWER: Is this sort of how you would characterize the state of things currently, or what you're trying to address and change I guess?

DUFLO: Well, even before we started working, academic economists I think have always had a probably more sophisticated view of what it is to be poor. But it is the temptation I would say that exists in the policy domain, or in the media, or in the press, to fall back onto one of these kind of fad.

So I don't think this is necessarily just a problem of poverty. Other policy sensitive domains have the same kind of feature, that you tend to go towards one very simple characterization and run with it. But poverty is certainly like that. So it's kind of a permanent running against the tide type of project.

INTERVIEWER: So picking up on that, how do you address this? How do you try to go against that tide then?

DUFLO: Well, by bringing up a somewhat more sophisticated view of what it is to be poor that try to integrate these different elements, different constraints, under which the poor are operating. I think our working way of, our working hypotheses, our starting point, is that the poor per se are no different than you or I, except for the fact that they have less money. But having less money changes many, many things about how they function.

So first of all, it changes the opportunities that they have. For example, someone who is poor may not be able to have access to credit. Or someone who is poor might live in an area that doesn't have a very good school, so won't get as good an education.

Second of all, it changes the way that they have access to information, and therefore what they even know about their world, compared to what we have. So they might not know for example, the benefits of education. They might not know what's a good line of business, or buy a line of business, to continue on these two domains.

Also, perhaps more subtly, it introduces some changes on the way that people perceive life itself. For example, being poor you might always be distracted, because you think that there are some big issues at home that you can't fully take out of your mind. And that we wouldn't have if we were richer, because for example, if you're poor you don't have drinking water. So you need to always keep in mind that you have to have collected enough water and boiled it, for example. Where if you're rich, you would take the water from the tap.

So that is also a very important part of it is that the richer one is, the more one is protected against risk that comes from outside, as well as one's own imperfection. And the poor people are basically on their own to deal both with these outside factors and these inside factors. That's another very defining feature of poverty. So I've just scratched the surface of different consequences of being poor.

You can't look at them all at once. Obviously, when you try to work on a project, an intervention, or paper, you're going to focus on one aspect. For example, you're going to think well, let me think about how one would address imperfection information. And what are the benefits of education, and what kind of carriers are good carriers, et cetera.

And then you zero in on those, and you try to do a good job on those. But you never forget the perspective that you only have one part of the thing. And then even if you have a great information experiment, that's very unlikely to be the beginning and the end of everything you would want to do.

So that's sort of the way we try to do it. Pretty much go into the detail and looking at one thing in particular, without ever forgetting that at the end, it's going to need to be integrated in a more complicated picture.

INTERVIEWER: So what would be-- I mean, I'm sure there are probably many examples-- but what's a particularly good example of a case or a study that you have done that would illustrate this, do you think?

DUFLO: There are many. So the way, as you pointed out in the introduction, the way we work in J-PAL, and we is now a large collection of people, it's over 100 professors in the world, around the world, are also affiliated with the network. And what defines us is the use of randomized controlled trials to evaluate the impact of intervention on people's life.

So what's a randomized controlled trial? The idea is very much similar to the clinical trial for a drug. So if you wanted to test a new drug, you take a population of people, you randomly assign them to two different groups. And one group you give drug A, and one group you give drug B. And then you see what happens, both on the outcomes that you're targeting, and maybe on side effects and stuff like that.

So it turns out you can do something reasonably similar with intervention in the field. Suppose you were interested-- I'll give you one concrete example in the field of education that I'm working on at the moment-- you're interested in the impact of a new way to introduce mathematics to preschoolers.

So the reason why you want to do that is you feel that if preschoolers from slums and stuff like that are better prepared to enter into primary school, they will be able to build on this provision to do much better in life, than if there was a small lag, which is kind of compounded by the way the system works, and dumped into a huge lag by the time the kid is seven or eight. So that's kind of the rational for that project.

So as an economist I knew that there was this tendency for the system to exacerbate existing inequality. As someone who is interested in policy action, I'm therefore interested in making this inequality smaller at the beginning. So I went and talked to some psychologists, child development psychologists, who are specialists in how kids learn mathematics. And we decided together to try and develop a curriculum for preschoolers in slums, and to test it.

So the curriculum was developed here in the US, in the lab. And then we bring it to Delhi, in India, where we work in 210 preschools. The 210 preschools are divided into three groups randomly. In one group, they just continue what they would normally do. In one group the program is introduced, so someone specially trained to do it, but someone from the community is trained to do it. And it's an hour every day or something like that, in this new curriculum for mathematics.

And then you might worry that if we do find an effect of that, it could be that it's just the extra attention that they receive, and the fact that they play games that have rules and stuff like that, and not the mathematics content. So we introduced another condition, which is the same rules with the same games with similar type of rules, but no mathematic content. So all of the idea of game play, and rules, and take turns, and all that is constant, but it doesn't involve mathematics.

So these schemes get developed by the team of psychologists, and they are implemented by local NGOs, who put them in the schools. And then we collect data before, then right after the end of the program, then six months later, then one year later. And look at the impact that it had in the short and the medium term.

So that's an example where ideally we learn some basic science, because in this instance, not so much in economics but more in psychology, because the psychologists are very interested in one way in which kids learn mathematics, which is having nothing to do with numbers.

Kids from a very young age have a notion of mathematics that are not symbolic. So if you show them, for example, a big blob of dots, blue dots and red dots, they are able to say that there are more blue dots than red dots, even though they cannot count, because it's a large number

There is a notion, there is a range, an area of notions, that kids already have. We already know that. People have that in any culture, the Amazonians, where they're never exposed to any numbers.

And they have this presumption that if you train these capacities, then it will provide a firm basis for the learning of symbolic elementary school mathematics. So finding out whether it does or doesn't brings us something for basic science. At the same time, if it does, then it can form the basis for a curriculum that can be replicated and scaled up. And we know that it works, because we've done it on a large scale. It's not a lab experiment.

And then if it doesn't, then you also know that, OK, that's not the way. But we can go back and think of another way to introduce preschoolers to mathematics. So that's kind of an idea.

INTERVIEWER: So this is one you mentioned is sort of ongoing now. Where are you in the process?

DUFLO: We completed the first phase. What we found was very interesting. We found that we definitely can train these notions, like kids become much better at non-symbolic mathematics in the short term, immediately after the training. And it persists, very persistent.

On the other hand, there is some improvement in the ability to manipulate numbers and the like in the short term. But it doesn't persist when kids have left and gone to regular schools. So it seems that on its own, the bridging is not happening.

So we have learned something important. And now from a policy point of view, what we need to do, and that we are doing now, we developed a new range of games that expertly does the bridging. So start by the training in non-symbolic mathematics, then move from that to addition, subtraction with numbers. So that is in the field at the moment.

INTERVIEWER: It sounds like, I mean there's obviously an enormous amount that goes into this. I think you just talked about it. Sort of developing this curriculum, and then implementing it, and studying it. Can you give me a sense of the scope and the size of J-PAL, or the reach of it. it sounds huge.

DUFLO: J-PAL is very large. As I said, there are over 100 affiliated professors around the world. Together they have, I think the current number is almost 700 projects, that are either ongoing or completed.

But because their growth is exponential, many of them are ongoing as opposed to completed. And what I call a project is something like I was just describing. So some of them, I would say the one I was describing, which works in 200 schools and has a time span of a few years, is pretty typical in size.

There are much bigger projects. For example, I was involved in a project with the government of Bihar, one of poor Indian states in the northeast of India, which reached a population of several million people. And some of them are smaller in a sense that they are more experimental, tightly controlled, a few hundred subjects.

INTERVIEWER: That's huge. So how did you, to go back in time, how did you get involved in this? How did the center come to be, and how did your interest in this field develop, or where does it come from? Obviously of course, you grew up in France. Where did this interest come from?

DUFLO: So I grew up in France, always been interested in poverty. As far as I can remember, I've been a bit fascinated trying to understand poverty and make sense of it, maybe with this very cartoonish view of poverty that we talk about in the book. But definitely it was something that was with me.

And this is really why I decided to study economics eventually, after some detours. When I finally decided to do economics, which was roughly when I switched to-- When I started to do the master's degree, and then came here for my PhD, it was with the view of doing something tackling this issue. And from the beginning I was interested in a very, to do very organicical work, very applied work.

The idea of using randomized controlled trials in the evaluation of the impact of policies and intervention was there before. It's been done in the US for many, many years. In fact, it was always continuing, mostly in the evaluation of welfare reform and welfare related topics. Also a little bit in education and criminal justice. So it was there.

When I was, during my PhD, I was not doing that obviously. As a PhD student I had no way to get that going. But a few people, including Abhijit Banerjee, who is also from the Department of Economics, and Michael Kremer, who was in the Economics Department at MIT then, and since then moved to Harvard, were starting to introduce it to development economics.

And I was watching them and at the same time doing sort of more classical economic stuff, applied work, but with existing data, et cetera. And then quickly became convinced that that was the way to go, and that's what I should do. So as soon as I got my degree, I kind of shifted to doing it. Started my first few experiments. Tried to understand how it works, et cetera.

And a few years into that, a few of us realized that if we wanted to have policy influence as opposed to just research influence, it was a very important to have a shell around us. So Poverty Action Lab Is more a network than a center. It's not your traditional research center or lab that's used to take money in, and then kind of give it back to people, although it does some of that.

But it's mostly a platform where people who work with these kind of methods can share their results in a way that's very accessible to policy makers. And we have a staff that is devoted to spreading out, and to also spread the idea that can be useful for policy making, both the process of going through an experiment, and of course the results.

So that infrastructure is the reason why we said we needed Poverty Action Lab. So a few of us got together with the support from the MIT administration, from our department, and then we kind of launched the lab. At the time, it was I think eight affiliated professors already from the beginning, not just MIT, and then it grew from there.

INTERVIEWER: Were there moments, were there things, you said as a child you were interested in poverty. Were there things, experiences, I don't know, things you read as a child that interested you, made you interested in that topic?

DUFLO: So it was a very kind of present topic in my household, because my mother is a doctor, pediatrician. And she was involved in a small NGO that dealt with children victims of war. As a volunteer, though she kind of went back and forth.

And one of my uncles started, was a co-founder of one of the large French doctor's NGOs. And he came by often, and so we were kind of, we had discussions on that. So it was kind of always there. My mother I think particularly made sure that we had books around, to make us aware of what it could have been to grow up in Africa as opposed to in a suburb of Paris. So I think that kind of was in the air.

Then of course there are defining moments that probably everybody encounters, like Ethiopia and We are the World, and this type of stuff. But I think for me it might have been less defining, because it was already so present.

INTERVIEWER: When you came to MIT and you started working with Professor Banerjee, as I understand it, what pushed you in the direction of the randomized controlled trials, and things like that?

DUFLO: Well, I was always interested in doing applied work, and in particular, in looking at the impact of policies. Both because that's important in and of itself, and because that seemed to me, and seems to many people, it's not a particular original thought, that it's a very useful lever to try to understand fundamental mechanism. Because a change in policy changes the environment in a way that, then if you see how people react to this change in the environment, you can see what makes him tick in a sense, and understand better how people function and how the world functions.

So the idea of using policy change as a sort of identification of important relationships is something that has been in economics for a very long time, and that I was very steeped in. And a few years before I was in graduate school, there had been a big, big progress in the field of public finance, liberal economics, mainly in the US, trying to use these kinds of methods.

So I was trained in that, and kind of was applying them to development economics. And the way we think of this in liberal economics is very much like, try to think about how to approximate the ideal experiment which is a randomized controlled trial. So after writing several papers, where you explain how you get as close as possible to the ideal experiment, which is a randomized controlled trial. Seeing that people actually we're starting to do randomized controlled trials, I was thinking well, you know, why try to approximate the ideal experiment, if you can just do the ideal experiment?

So that's kind of how I shifted towards doing this type of work, both the influence of this way of training, which makes you think of the ideal experiment as a randomized controlled trial, of course thinking about causality, and then seeing people doing it.

Practice and realizing that there is a ton of flexibility, especially in development, because a lot of organizations are not that big, and they don't have infinite budget anyway. And they have to find a way to allocate it efficiently. They have a lot of questions. Many organizations are not so set actually in what they want to do. So they're willing to experiment. So that's kind of a great opportunity.

INTERVIEWER: Have you encountered, is there resistance in any areas, or when you approach a project, do governments, or NGOs, or whatever, are they very open to this? How was it received in general?

DUFLO: Well, there is a whole spectrum of people. Especially at the beginning I think, when we were trying to spread the methods, when it was not as well known as it is today. Many people, maybe the majority of people, were somewhat skeptical that this could be done. That people would accept it. That it was practical and feasible, et cetera.

But at this time there were only very few of us doing it anyway. So it's not that we needed that many partners. So we found a couple of adventurous NGOs at the beginning, rather than governments, and learned together with them. They are still very close friends.

And that establishes a few principles that yes, you can do it. This is how you can do it. And this is how you can do it when you control things very well. And this is what progressively, we also learn to deal with situations that are less than perfect. How do you analyze detail that is not as tightly controlled as you would have hoped, et cetera.

So all of these things progress in tandem to the situation that you have now, where there are still some people who don't want to have anything to do with it. But there are enough people who are willing to do this type of work, especially governments, now that we could keep any cheaper affiliates busy, and some more.

INTERVIEWER: Have you, it must be very gratifying to see the success of this. Did you have any inkling of this at the beginning?

DUFLO: We had no idea. In fact, at least I had no idea, but I think I can speak for Abhijit and Sendhil, who started the lab with me, and that they also had no idea. At the time, as I was telling you, the first few affiliates of the lab were eight people. And we had in mind a structure that would serve those eight people, and serve the world by communicating what those eight people do, and then to have some little goals and students and stuff like that.

Where I'm particularly grateful to MIT, and in particular to the chairman of our department then, who was Ben Armstrong, is that he saw the importance that we had that we didn't see. He, in addition to being an amazingly talented economist, he was also a businessman. He had an understanding of business, because he had been sitting on boards. And he basically told us you have something that is going to just catch fire. And one could say it has caught fire.

INTERVIEWER: When you were considering getting a PhD, what made you choose to come to MIT over anywhere else? What was there going on here that ultimately led you here?

DUFLO: So what I was told by someone who was teaching here at the time whom I knew is that the way that economics is practiced here is very applied. And very in tune with the problems of the world and I would learn the kind of methods that I needed to do the very applied work that I wanted to do. And that was very much true, so this is why I applied here.

INTERVIEWER: Because it was more applied?

DUFLO: Because it was very applied, more applied then France. And frankly, I didn't look much beyond. I was told MIT was the best place and that it was very applied, and so I just applied here.

INTERVIEWER: So how does it compare to education in France? MIT and similar institutions that you've been part of in France.

DUFLO: So it's very different now than it was when I was in France, when I was starting my dissertation. So the comparison is a little bit unfair, because I think things have much, much, much improved in France. I mean they have changed. Some people might think they're worse.

I would say they were improved from the point of view of where the type of work that is being done in France is now. Also I mean there is arrangement, they is also a lot of very applied, super-important work that is being done. And you can just take Thomas Piketty's book that after all originated from, where I studied.

But when I was there at the cusp of starting my dissertation, the work was still quite theoretical. So you could study micro-theory, or macro-theory, or econometrics, and not so much else. Not too many applications.

So that's one difference. Another difference which was then at the time, and I think it's less now as well, is that France is very hierarchical. And professors don't make too much time for students. And maybe you'd never really meet your advisors, or not frequently. You have one year of coursework, and then you're on your own to figure out what your big opus might be.

I didn't feel ready. I felt that I needed an environment, and MIT provided that in droves. So that was just, there is absolutely no question that I could have done anything, or even thought of anything like I ended up doing, if I had stayed in France at that time.

I think there is no question that this has changed. That someone could now stay in France and do good work. But there is also no question that MIT is the best place anywhere in the entire world to be a graduate student.

It's still true today, precisely because of the school pretty open, the relationship that exists both between field of economics, and between people across ages. And the only thing that matters is what you have to say. You're not going to be protected because you are a student. And you're not going to be protected because you're a tenured professor. You just have to [? say ?] thing. So that's a very exciting environment to be in, was and still is today.

INTERVIEWER: Do you have any thoughts about where that comes from? Of course, there are things about MIT that I guess one could point to way back. But just in your mind, what creates this kind of environment?

DUFLO: So I think we have a huge amount to thank Bob Solow for, and Paul Samuelson to some extent. I think they started the culture that we inherited. And in particular, Bob Solow. I just overlapped with him a little bit when I arrived as a student.

And he is just a wonderful individual. And he modeled the whole environment of the department around his wonderfulness. So he had some ideas for example, the lunch seminars. Now everybody in the world has copied them. But we had them from the very beginning. And we have tea discussions also in the field. Completely open.

The idea is that the free discussion of economics is what matters, not who you are, et cetera. And among the faculty there is also no hierarchy. So it's not that the power belongs to the older people. In fact, they purposefully withdrew when they were still very active as researchers.

But a decision worker was in the department, to let the younger generation. The younger generation, when they became old in turn decided that it was good enough for Solow and Samuelson, it's good enough for me. And they did the same thing, et cetera. So some of that culture is past is culture is past from time to time.

I think it's also a small department, and therefore it works as a family. It doesn't have a tendency to divide. And finally, it's a department which puts the training of graduate students at the very core of its mission.

That means we treat the graduate well, which means we get the best graduate student. And also it keeps us honest, because it forces you, the graduate student is where you get a mirror of what you teach. Like if you're not doing exciting stuff, they won't be doing exciting stuff. And you also don't get pushed, because that's where the ideas are.

So I think all of these reasons are why the department is a very unique place. And I don't think I could have done what I've done, in the sense of diving into these randomized controlled trials when I was just out of graduate school, without the support of the department, where I had a sense that I could do whatever crazy thing I wanted and they were going to support me.

INTERVIEWER: The department of economics of course, is within the School of Humanities and Arts and Social Sciences. Does being in that school play into this? Is that part of the culture there too, would you say?

DUFLO: Not all that much. I would say, I mean it's a little bit its own island. Maybe what matters, what creates the slight family thing to be perfectly honest, is to be a little bit isolated in what is otherwise largely an engineering school.

So you know, MIT as an institution is quirky. The economics department within MIT is quirky for being kind of not at all-- In most institutions economics is kind of where students go. For us, it's like they might never even know about us after four years in this place.

INTERVIEWER: A little off the beaten track, as it were?

DUFLO: Yeah.

INTERVIEWER: What kinds of things are you working on now? You mentioned one, but there clearly are many others. What kinds of things are you particularly sort of excited about right now?

DUFLO: So I'm very excited about this collaboration with the psychologists that we were discussing before, as I hoped you gathered from my description of it. I hope you finally decided to [? post, ?] because it's totally new.

I had to learn psychology. I went to a summer school in psychology last year. It was wonderful to learn new stuff. Great, great field. Also because I think at the methodological level, it has a potential of replicating for psychology what we did in economics. So make them much more field experiment based, which they are not today at all.

So it has a chance to have some, and to do some disruption over there. And of course, first and foremost, because I think it's important, and it can really to make a difference in the life of many kids. So that's something I highly love.

Another set of things I'm really excited about, in a very different part of the spectrum, is working with governments to run projects with them, where typically the objective is to help them do what they want to do anyway, better.

So governments are very large bureaucracies. And often, even if there is an idea at the top that they have decided that they wanted to implement, there's any number of issues that crops along the way. Transmission of information, transmission of incentives, and potential corruption and stuff like that. And I find it very interesting to work with government to do experiments on how to implement policies

So with the project that I was talking about that we did in Bihar, was a very large sample. It was working with them. There is a special welfare program in India, which is a workfare program. So you're entitled to go work for the government if you have nothing else to do, at the minimum wage. Each family is entitled to 100 days in the year.

So that's very nice and well, but if you want to make that possible, there have to be work site where the people can go. And there have to be money that has to be transferred to people, et cetera. And of course, all the money for that comes from the center, but all the implementation is done at the very local level.

So there is this kind of basic plumbing question if you will, which is how do you transmit the money from the center to the local level. And that might sound technical, and financial, and boring, but it's very important. All the programs in the world face this issue. That the money is there, and it needs to go there, and sometimes the pipes are leaky, and how do you make them not leaky.

Typically, the way these problems are solved is that the local governments get an advance, and then when they've spent it, they ask for, they kind of justify more or less well, more or less completely, and they ask for some more.

And the problem, of course, is that they will have a tendency to take some along they way. So they are monitored by the level above, and the level above is monitored by the level above. So you have a whole hierarchy. That hierarchy creates its own set of problems.

So that is two issues. One is the possibility that money disappears. And the other is that some places have lots of money and they don't really need it. And some places have very little money, when they're already done and they could do so much more.

So now it's possible to do much better than that in principle, because with the internet and stuff like that, you can transmit information on what money you need in pretty much real time. So with the government of Bihar, we experimented with a system that they wanted to, or some faction of the government wanted to push, of direct invoicing.

So instead of getting a whole advance that I spend and then ask again, is when I need money to pay someone, I say, hey, Joe just worked for me for two weeks. Will you please give me a thousand dollars to pay him for these two weeks. So the invoice can be transmitted electronically to the central bank who can send the money. So we tried that.

So you can see that if you can't do it in like one village, because it's like a whole financial system, but we managed to do it in some municipalities and not others. So we treated I think a thousand municipalities, and we had 2,000 controls.

And when we did that for a few months, it was implemented for a few months, it had all sorts of problems. But despite that, in the course of these few months, we estimate that the amount of funds that vanished disappeared by 20%. The actual ability of the program to hire people didn't seem to have changed.

And you have also less money just sitting idle somewhere when it was needed elsewhere. So during these few months, the government saved something like $6 million dollars. And the entire experiment must have cost about $200,000. Some computer and stuff like that. Not very much.

So I find this type of work really exciting. We have a partnership with the government of Tamil Nadu in the south of India, where researchers can come and work with different departments to propose programs, which are usually of this nature. And how can you do what you're doing already better.

And this is a completely different conversation. Of course it has impacts hopefully, immediately. But also I think more deeply, it contributes to create a culture of learning within the government that can really improve the process of policy making, I think much more fundamentally. So that's kind of the other thing that I'm very interested in.

INTERVIEWER: That's great. How generalizable are these things? And maybe it's just case by case, but I mean what works in one place, and one culture, and one country, do you find, can that be generalized to other places, or is it always very unique?

DUFLO: So to answer these questions, the only way is to try the same project in different countries. And this is starting to happen. So either it happens systematically, for example, I was involved in a project that was led by Dean Karlan, a professor from Yale, where as a team we evaluated the exact same program in seven countries, in different continents. So India, Honduras, Ethiopia, et cetera.

So this was conceived from the beginning to be done in all of the countries at once. So once you do that, you can really answer the question you're asking. Sometimes another way it can happen is that because there are so many people running projects, you would just have to kind of put it together.

So for microcredit, because I did a journal, it turns out so many studies pop up. And I could kind of grab them and put them together in one issue, something you can answer your questions, how, I'm then going to answer your question.

And finally, my hope is that we can do that a little bit more systematically, because the issue we have now is that it might be the best for many questions, it's possible to answer the questions you ask, but we don't know, because there is a study here and a study there, and a study there. And the results have not been necessarily published, because maybe it didn't work someplace. And things that don't work are less likely to be published.

So something that I have been working on is create a registry of experiments, where people can write down that they are working on such and such issue, in such and such country. So that when all the results come in, and hopefully they are putting report or whatever comes in, so that someone can then take all of the projects on a particular type of intervention, say remedial education across countries, and answer the questions you are asking.

Now for the few examples where we know, I would said the extent to which things generalize [INAUDIBLE] is surprisingly high. And I would have thought that there would be a lot of context dependence, and there might be many questions for which there is. But for example, for microcredit, it universally doesn't work.

For the program that we looked at, where we did everything together, which is an asset transfer. So if you wanted to microcredit, but you don't get the loan back, and you provide some support for people to help to take care of that asset. That seems to be across the board a huge success.

So both in the case of one failure, in the case of one success, we seem to have a fair amount of generalization. But I think it's pretty early days to say. And I think it's going to be application dependent. Some things will have more [? deniability ?] ability that others.

INTERVIEWER: You mentioned the microcredit example, and I remember a few years ago that was very much in the press, and it was held out as a great great hope for many situations.

Have there been things, that or others, well, I'm interested first of all, in what happened with that, because you say it wasn't a success. But also, have there been things that, outcomes that have really surprised you? Things that you sort of thought going into it, I really would have thought X would work, but boy, I never saw that coming, the way it turned out?

DUFLO: Oh, yeah. There are many surprises. I think I'm not the person who was has the most strong views on things. I mean I have strong views on how to find things out. But I don't have very strong views on what the results might be. I'm bit of a caricatural economist in this way, on the one hand, on the other hand.

So I can see, I can usually persuade myself either way. I remember when I was in high school in the philosophy class, the textbook had always, it was presented as for each topic, you had on one page, one text that argued A, and one text that argued the opposite of A. And I remember being very distressed by the fact that I had always been convinced by whatever I was reading.

So in that sense, I personally, I'm always capable, or too capable in the sense of seeing both sides of an argument. Therefore I don't get surprised very easy at least by signs. You do get surprised by magnitude sometimes, where you think that something will have an effect, but the effect is much larger than anything that you would have suspected from any reasonable model you can have applied.

For example, we did some work on immunization. And we worked with an NGO that was, it was in India in a local district where the immunization rates are really, really extremely low. And what the NGO did is two things.

Again, to imagine a randomized controlled trial. Three groups of villages. The first group, we just do nothing in particular. The people get immunized at the sub-center as usual. The second group, The NGO run camps, monthly camps, where they went and they advertised, and they put really a lot of effort to the camp very regularly, and to remind the people and all that.

And the third situation was the same camp, plus a kilo of lentils for each immunization. My presumption was that the kilo of lentils would make a difference, but what I was surprised by was how large the difference was. The effect was pretty big, which is when you go from doing nothing to doing the camp, you increase immunization rates from about 5% to about, I think it was 17%. So nothing to sneeze about. That's not bad.

But when you add the lentils, you increase it to 38%. That's full immunization rates, and it was at the time. They're still doing it, this NGO, and now they are like in the 90%.

So you would say well, yes, you know, I was expecting an effect. I wasn't expecting an effect that was quite as large. So it's nice, because when you find something like that, it leads you to think how come people don't get persuaded, or don't really get moved by having the camps and the reminder and stuff like that. But then you add a kilo of lentils into the mix, which is really not such valuable, and they come. It makes you think about how people are thinking about this problem.

INTERVIEWER: What do you think about that? How do you think they are thinking about it?

DUFLO: So I think there are two things. One is that the benefits of immunization are not particularly clear to people. They are being told that immunization is important, and they are willing to buy it, but maybe they don't, their belief on the effect of immunization is not that large.

They are generally in favor of, but maybe not by as much as they should given the actual benefits of being immunized, in a country like India in particular, where a lot of diseases that are immunized again are very prevalent. And there are actually private returns to being immunized.

So that's one problem. The other problem is, the benefits whatever such as they are, they are in the future and at some unknown dates and stuff like that. And there is both this uncertainty and the fact that it's lethal, whereas the cost is to be experienced now. If you want to go now, you have to take the half day, and the kid's going to be sick and all that. So I think there is the costs that are to be explained today are always a little bit more salient than the benefits that are to be experienced in some future.

So that's another reason that would lead people to procrastinate. I think if the perceived benefits were really as large as real benefits, people would be able to overcome procrastination on their own. But if you combine the two, maybe the underestimation of the benefits, plus this tendency that we all have to procrastinate when there are costs in the present for benefit in the future, that can lead to this type of stuff. Now what does the lentil do? It upsets the small cost with a small gain. Both of them are in the present. I think this is why it has this effect.

INTERVIEWER: But it's small, it's a small gain in the present it sounds like.

DUFLO: It's a small gain and a small cost. So they kind of cancel out. You say, well, fine. I have to pay the cost today, but I'm getting the lentils today. So I might as well do it.

INTERVIEWER: Interesting. One thing I wanted to ask you about too is digital learning, which MIT of course, and many places have gone into. What's been your experience with it? You've done some classes that have been taped and whatnot. But tell me about what's been your experience so far?

DUFLO: So I've invested a lot in it. We have invested a lot in it, with my colleagues in the development groups at MIT and in J-PAL. We have invested a lot in it, because we think that-- in these MOOC classes-- because we think that that's a way to reach many more people for what we do.

We feel that we need to reach people outside of MIT, and who cannot come to MIT. And so we see this in the course of what we do. We train a ton of people, because we have so much staff. And we have hundreds and hundreds and hundreds of staff around the world working on this project. And we spend a lot of effort teaching them.