Out of the Darkness and Into the Light? Development Effects of Rural Electrification In India

At long last, Louis and I have a working paper version of our paper, "Out of the Darkness and Into the Light? Development Effects of Rural Electrification in India" (Appendix here - warning: it's a pretty big file). The paper has, as you might expect, a lot of gory details in it, and the appendix even more so, so I'll try to provide a less technical overview here. 

The basic motivation behind the paper is the following: there are still over a billion people around the world without access to electricity, most of whom are poor, rural, and live in South Asia or Sub-Saharan Africa. Developing country governments and NGOs are pouring billions of dollars into rural electrification programs with the explicit goal of bringing people out of poverty - one of the new UN Sustainable Development Goals is to "Ensure Access to Affordable, Reliable, Sustainable and Modern Energy for All."

In light of the big pushes towards universal energy access (pun only half intended), it's important that we understand what these investment dollars are going towards - even more so when we're talking about developing country dollars, which have opportunity costs like schools and health clinics and roads. It turns out, though, that we know surprisingly little about the actual effects of electrification on economic development. There is a strong positive correlation between GDP per capita and electricity consumption, with rich countries like the USA and Japan consuming much more energy per capita than poorer places like Nigeria and Bangladesh:

Data source: World Bank

Data source: World Bank

As we all know, though, correlation does not equal causation (thanks, XKCD!). Figuring out the causality here is definitely not trivial: at the country level, there are lots of things that drive income differences between Japan and Bangladesh that have nothing to do with electricity. It turns out that these omitted variable bias problems plague sub-national-level studies as well. If we just regressed incomes (or better yet, welfare!) on electricity infrastructure at the village level in a developing country (say, India), we'd likely end up with a number that was way too high. Why? Energy infrastructure projects are large and expensive, and (correctly) not randomly placed: governments think long and hard about where to put the electricity grid. In practice, this usually means that places that are already doing well economically, or places that are expected to do well in the future, are the first to get access to electricity. In more technical terms, that regression I described above is subject to a ton of omitted variable bias. 

In our new paper, Louis and I try to shed light on the following question: What does electrification do to rural economies? To do this, we take advantage of a natural experiment built into India's national rural electrification program, RGGVY, which would eventually expand electricity access in over 400,000 villages across 27 states. In order to keep costs down, in the first phase of the program, only villages with neighborhoods of 300+ people were eligible for electrification. This means that we can compare villages with 299-person neighborhoods to those with 301-person neighborhoods to look at the effect of electrification. The idea is that, in the absence of RGGVY, these villages would be virtually indistinguishable - and indeed, in the paper, we show that prior to the program, villages just above and just below the 300-person cutoff look the same. Having a cutoff of this kind built into the program is really nice from an evaluation standpoint, because it allows us to mimic a randomized experiment couched in the natural (large-scale) rollout of RGGVY. 

Having identified this cutoff, we get to bring a bunch of cool data to bear on the problem. The first thing we need to do is to demonstrate that RGGVY actually led to increases in electricity access for eligible villages. We're concerned that a simple 1/0 electrification indicator doesn't actually capture power availability and use - it would mask variation in power quality or in the number of households that have access to the grid, for example. Instead, we turn to remote sensing. We got access to boundary shapefiles (outlines) of [almost] every village in India, and superimposed them on top of NOAA's DMSP-OLS nighttime lights dataset - this is a satellite-based measure of nighttime brightness. We combine these to datasets and "cookie-cutter" out the nighttime lights values for each village, which allows us to create statistics like the average or maximum brightness in each location.

Yum, village cookies!

Yum, village cookies!

We pair nighttime brightness data from 2011, 6 years after the announcement of the program, with population information from the 2001 Census (the official population of record for RGGVY), and look at the effects of eligibility for RGGVY on night lights. Another advantage of the cutoff-based ("regression discontinuity", for those in the know) analysis? It lends itself naturally to blogging, because you should be able to see the difference between ineligible villages and eligible villages very clearly in a figure. Et voila:

Effects of eligibility for RGGVY on nighttime brightness. Each dot contains ~1,800 villages.

Effects of eligibility for RGGVY on nighttime brightness. Each dot contains ~1,800 villages.

We find that RGGVY eligibility (aka crossing the 300-person threshold) led to a sharp increase in nighttime brightness, as visible from space - remember that we're looking at villages of around 300 people, so we're pretty impressed that we can detect this. It helps that we're studying India, where there are upwards of 30,000 villages in our main estimation sample. It's a little hard to interpret this effect directly, since it's in units of brightness points, but previous papers that have gone out and groundtruthed the relationship between nighttime lights and electrification suggest that this is consistent with about a 50-percent increase in the household electrification rate in our sample. We do a bunch of work in the paper and in the appendix to show that these changes are actually attributable to RGGVY - if you're curious, check it out!


Okay, great: it looks like RGGVY brought electricity to Indian villages - but what about the effects of electricity access on things we care about, like the workforce, asset ownership, housing stock characteristics, village-wide outcomes, and education? We gathered a bunch of village-level data from the Indian Census of 2011 and the District Information System for Education, and look, among other things, at the effects of electrification on outcomes like men and women working in agriculture vs. the more formal sector; ownership of assets like TVs and motorcycles; whether households are classified as "dilapidated" or have mud floors, and other housing stock outcomes; the presence of mobile phone coverage, agricultural credit societies, and other village-wide services; and the number of children enrolled in primary and upper-primary school. Here's a snapshot of (some of) our results:

For all of the outcomes I've shown here, we see very little evidence of sharp discontinuities at the 300-person threshold. We do see clear evidence of men shifting from agricultural to non-agricultural employment (see p. 20 in the paper), but little else. The graphical evidence is borne out by the regression estimates as well: in almost no cases can we reject the null hypothesis of zero effect, but more importantly, we can reject even modest effects in nearly all outcomes. (The exception to this is education: we again see no visual evidence of effects on education, but our sample size is much smaller for these outcomes, since not all villages actually contain schools, so we have less precision with which to rule out effects; the schools effects are also sensitive to specification choices, bandwidths, etc in a way that the other outcomes are not.) To reiterate: in the medium term, we find that eligibility for RGGVY caused a substantial increase in electricity use, but can reject small effects on labor markets, asset ownership, housing characteristics, and village-wide outcomes; we do not find robust evidence that RGGVY led to changes in education. We were pretty surprised by these results, so we threw a bunch of checks at them (see the Appendix) - but they seem to hold up. (Turns out that our results are also consistent with new evidence from our Berkeley colleagues Ken Lee, Ted Miguel, and Catherine Wolfram in Kenya - see the abstract here).

A couple of these tests in particular are worth highlighting:

You might be concerned that we're not finding much because the program wasn't implemented well, or because we're lumping a bunch of villages where electrification did a lot in with villages where electrification didn't do anything, so things are averaging out to zero. When we cherry-pick the states that saw the largest increase in nighttime brightness as a result of the program, however, we don't see evidence of this. Among this selected sample, the nighttime lights effect approximately doubles - but the effects on the other outcomes stay the same. 

You might also be concerned that villages with around 300 people are unlikely to see big effects - they might be too poor, too credit-constrained, etc, etc, etc. A couple of responses to this: first, if we care about electrification from a poverty-reduction standpoint, then we should be worried about people being too poor to take advantage of electrification. But that's more speculative than data-driven, so we do a more formal test to think about effects of electrification for the rest of the villages in India. Rather than relying on our nice cutoff, we instead do a difference-in-differences (DD) analysis: we compare villages electrified in the first wave of the program (like a "treatment" group), before and after electrification, to villages electrified in the second wave of the program (like a "control" group). When we do this, and calculate different effects by population groups, here's what we find:


There are a couple important takeaways from this figure: first, our cutoff-based estimate (navy dots) line up remarkably well with the DD point estimates for the appropriately sized villages; and second, the brightness effect is increasing in population, but the other outcomes (proportion of men working in agriculture shown here) are not, suggesting that our original estimates might actually generalize to the rest of the population. 

So what does it all mean? We present well-identified quasi-experimental evidence from the world's largest unelectrified population. Taken together, our results suggest that rural electrification may not be as beneficial as previously thought. Does our paper say that we shouldn't be implementing these kinds of electrification programs, or that electricity isn't making people better off? No - we explicitly don't make any statements about overall welfare in the paper, because we don't have the data to support these types of claims. In fact, we visited some villages in Karnataka in December, which made it pretty clear that people like having access to power (so do I!). But we can say that it's difficult to find evidence in the data that electrification is dramatically transforming rural India after 5 years or so. We think this is a case where highlighting a null result is really important - take that, file drawer! At the end of the day, in the medium term, rural electrification just doesn't appear to be a silver bullet for development. In typical Ivory Tower fashion, we think more research is needed to understand where and when power can transform economies - maybe we should be targeting electricity infrastructure upgrades to urban areas, for instance. We've got a couple projects in the works to look at some of these questions - so check back with us in a few years, and hopefully we'll have some more answers!


Officially SSMART!

I've been a bad blogger over the past month or so, something I'm hoping to remedy over the coming weeks. (Somewhere out there, a behavioral economist is grumbling about me being present-biased and naive about it. Whatever, grumbly behavioral economist.) I'm writing this from SFO, about to head off to Bangalore (via Seattle and Paris, where I'll meet my coauthor/adventure buddy Louis), thanks to USAID and Berkeley's Development Impact Lab. We're hoping to study the effects of the smartgrid in urban India, as well as to learn more about what energy consumption looks like in Bangalore in general. There is a small but quickly-growing body of evidence on energy use in developing countries (see Gertler, Shelef, Wolfram, and Fuchs -- forthcoming AER, and one of my favorite of Catherine's papers! -- and Jack and Smith -- AER P&P on pre-paid metering in South Africa -- for a couple of recent examples). Still, there's a lot that we don't know, and, of course, a lot more that we don't know that we don't know. Thanks a lot, Rumsfeld.

Feeling SSMARTer already!

Feeling SSMARTer already!

In other exciting news, the Berkeley Initiative for Transparency in the Social Sciences (BITSS) has released its SSMART grant awardees - and my new project (with Matt and Louis, and overseen by Catherine) on improving power calculations and making sure researchers get their standard errors right has been funded! Very exciting. Check out the official announcement here, and our page on the Open Science Framework here. Since this is a grant explicitly about transparency, we'll be making our results public as we go through the process. Our money is officially for this coming summer, so look for an update / more details in a few months.

Where we are currently: there are theoretical reasons to be handling standard errors differently than we currently are in a lot of empirical applications, and there are also theoretical reasons that existing formula-based power calculations might be ending up under powered. In progress: how badly wrong are we when we use current methods? 

My flight is boarding, so I'll leave you with that lovely teaser!

Occasional Workshop Recap 2015

The Occasional Workshop in Environmental and Resource Economics is one of my favorite conferences. No, not only because it's in beautiful Santa Barbara and we got to stick our feet in the warm Southern California ocean on our way back home! This year, there were a few "long" presentations (20' each, with a 10' discussant slot), but most of the papers were "short" presentations - 8' each, with a couple of minutes for questions afterwards. There are no parallel sessions - everyone sees everything - and 8' is a great way to get a quick introduction to a topic. We see only a very limited part of the environmental economics spectrum at Berkeley, for better or for worse, and so getting to see papers on land use and endangered species and fish is really fun. I was also lucky enough to get to present some of my ongoing work on rural electrification in India, and managed to cut Louis' 60-minute, 75-slide presentation of our paper from Berkeley's development lunch earlier in the week down to an 8-minute, 20-slide presentation (which actually fit into 8', in an homage to Meredith's speaking speed). 

A couple of my favorite presentations (the full list can be found here):

  • Robyn Meeks' "Economics of a Light Bulb: Experimental Evidence on CFLs and End-User Behavior" - a cool thought experiment (plus RCT!) on technology adoption in the face of learning and technical externalities
  • The always-excellent Sol Hsiang's "A Global Experiment in Black Market Dynamics: The Effect of Legal Ivory Sales on Illegal Ivory Production" - allowing legal sales of ivory is disastrous for elephant poaching
  • Amanda Faig's "The Economic Gains to Accounting for Fisheries Induced Evolution" - I'm not a fishery person at all, but this was a really cool look at how fishery managers can induce fish evolution - and how, if they don't account for this, they're much worse off
  • Jeff Shrader's "Estimating Adaptation Using Forecasts" - thinking about how to estimate adaptation in a context with skilled forecasters. Application to ENSO (a rash of cool papers using ENSO makes me want to write a paper using ENSO...does that make me a bandwagon researcher?)
  • Corbett Grainger's "Strategic Placement of Ambient Pollution Monitors: How Local Regulators Comply with Federal Rules" - turns out EPA air pollution monitors might not be optimally placed from a social planner's perspective. Cool.

I'm excited about the future of environmental and resource economics!

Obligatory beach photo courtesy of  Becca  (who also gave a great presentation at the Occasional  and  made amazing cookie road trip snacks).

Obligatory beach photo courtesy of Becca (who also gave a great presentation at the Occasional and made amazing cookie road trip snacks).

Finally, two more items of note:

1) T-one week until the wedding!

2) This might be my new favorite energy phenomenon: British grids need to supply additional power during commercial breaks in popular TV shows - because everybody uses them to plug in electric kettles, causing a demand spike. Offfffff course they do.