The problem with literature reviews in economics

One of the problems with economics articles is that they are essentially just reports of new findings with only minimal references to relevant literature. Not only do economics articles fail to draw connections between the new findings and foundational literature, but they don’t even provide a thorough survey of literature from the last forty years! Rather the typical economics article cites about a dozen articles, most of them closely related findings from the ten years preceding the article’s publication. Obviously this constrains the development of theory-building to a cohesive paradigm and the average economist has no sense whatsoever of how any particular finding speaks to a general theory of market activity or human action. As a consequence economics as a whole is hopelessly fuzzy and muddled.

Sociology has a far superior system in which every single article is prefaced by an exhaustive review of the relevant literature, plus as many tangentially related literatures as possible. Not surprisingly, sociologists therefore have a very strong sense of how any particular finding fits into sociology as a whole and have developed a robust understanding not only of a general theory of human interaction but of how each aspect of sociology fits into that theory and where the boundaries of the field are to be found. Whereas economists spend endless hours debating whether a piece of research “is economics” the parallel question is rarely heard about sociology. This is entirely attributable to sociology’s convention of demanding that work be thoroughly situated within the literature, rather than merely bouncing around untethered to anything. In addition to theory-building, extensive lit reviews have several advantages over the economic system: convenience to the reader, facilitating peer review, reducing the volume of research, and benefits to incumbent researchers.

The most important virtue of exhaustive lit reviews is the convenience it provides to the reader who may be coming to a piece of research completely ignorant of the theoretical context in which the research was done. If someone was raised by wolves and upon reaching human civilization were given a copy of American Sociological Review, that person would have a thorough understanding of how each article addresses a body of literature. This writing style acknowledges that scientific articles should not be so presumptuous as to assume that the readers are competent scholars already understanding the theoretical and substantive issues involved. At present an ignorant person wanting to understand an issue of Quarterly Journal of Economics would probably have to first read several issue of Journal of Economic Perspectives, which, needless to say, is an inconvenience. Why not effectively build JEP into every journal?

Exhaustive lit reviews not only benefit the reader, but also the writer. Since peer review is double-blind, one can not know in advance who will be selected as peer reviewers. The prudent course of action is therefore to work in citations to every person who has ever addressed your subject matter. This avoids both the embarrassment of failing to acknowledge reviewer X’s contribution to the area and the publication lag incumbent in having to work in citations to X in the R&R process.

Furthermore, the fact that econ articles essentially lack literature reviews means that the articles are so short that the journals can publish them by the bushel. The typical issue of QJE publishes eleven papers and AER has about fifteen papers and eight research notes. Pity the poor economist who has to keep up with the approximately hundred and fifteen articles published annually between these two journals. In contrast, because sociology articles are so long, ASR usually publishes only about eight papers per issue and AJS about six papers per issue, for a total of less than eighty article a year between them. While eighty long articles might seem more taxing to read than a hundred and fifteen short articles, in fact most sociologists find this quite manageable because the articles don’t take very long to read if you skip the lit reviews and go straight to the findings. Thus if only economists were to make extensive literature reviews like sociologists, economists would have to devote less time to the tedium of reviewing literature.

Finally, building up a robust understanding of an area of literature takes much more time commitment than creating an original empirical contribution. Furthermore whereas one can only milk a single empirical finding for, at most, four or five articles, one can build a career using the same literature review. To put it in terms economists can understand, empirical research is best conceived of as a “variable cost” and learning the literature as a “fixed cost.” In any industry, including academia, large fixed costs serve as a barrier to entry which in turn constrict supply and raise prices. Thus incumbent economists have a tremendous rent-seeking opportunity available to them by demanding more extensive literature reviews which will restrict the entry of younger economists (who may be able to analyze data but can scarcely learn hundreds of citations) and this restricted entry will make for a seller’s labor market in economics. The implications of such a rent-seeking strategy can be easily seen by contrasting the penury typical of economists to the opulent compensation enjoyed by sociologists.

Thus part of the explanation for the wretched state of economics as a discipline is the failure to acknowledge that they stand on the shoulders of giants — usually at least sixty or seventy giants.

8 Responses to “The problem with literature reviews in economics”

  1. […] How to tell an economist that their literature review sucks. […]

  2. stevphel Says:

    And an economist would say that their strong normal science paradigm obviates the need for an exhaustive literature review. Sociology, on the other hand, with no central paradigm must communicate their “position” in every article.

  3. “If someone was raised by wolves and upon reaching human civilization were given a copy of American Sociological Review, that person would have a thorough understanding of how each article addresses a body of literature.”

    I suppose that is a virtue if your modal reader is raised by wolves. It is a grossly inefficient means of communication among peers, however. In economics, as in most academic disciplines, the typical reader is already a specialist. (Whether hyper-specialization is good for the advancement of science is another question, but the problem is hardly unique to economics.) Why bore or distract the reader with page after page of arguments he already knows? A paragraph or two is usually sufficient to put the paper’s arguments in context. After all, lengthy literature reviews are costly to write, and costly to read. Marginal analysis suggests that they should be included only up the point where the marginal benefit equals the marginal cost. If there are no opportunity costs, why stop with five or ten pages? Why not fifty pages? Should every sociology article article begin with Parsons, or Weber, or Comte? Heck, why not Plato and Aristotle?

  4. mgsmith Says:

    “As a consequence economics as a whole is hopelessly fuzzy and muddled.”
    Strong words coming from a sociologist.

    I have a problem with the argument that long exhaustive lit reviews are beneficial to the layreader. Lit reviews generally give quick summaries of the relevant aspects of previous arguments. The author clearly expects the reader to have some knowledge of the previous lit. Otherwise the lit review is hardly comprehensible, let alone informative. You acknowledge that scholars don’t generally benefit from exhaustive lit reviews, they skip the lit review altogether and going straight to the findings.

    Your strongest arguments in favor of exhaustive lit reviews are 1.) It helps avoid hurting the feelings/ego of possible reviewers, and 2.) It allows hacks to keep pumping out the same garbage with minimal empirical contribution. Are these good things?

  5. mgsmith Says:

    Update: I’m an idiot.

  6. […] this has to be the funniest discussion of the structure of academic articles in economics and sociology […]

  7. […] For the law professors in the room, pay special attention to this one. […]

  8. […] myself occasionally do the “sociologists can be funny” genre (see here , here, and here) but these are basically elaborate deadpan in-jokes and I am under no illusions […]

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: