On June 24, 2012 the New York Times
Sunday Review ran my essay Political
Scientists Are Lousy Forecasters explaining the problems
with the National Science Foundation's current political science
grants. Not surprisingly, it drew strong responses. I intended to reply to these earlier but some unanticipated time-sensitive work intervened.
The question of how best to study
politics is an immense one that I will take up in more detail
elsewhere. For now I want to review a few specific responses to the
essay, and raise some big picture questions, the first one being about the word "science" in the name
of our professional association. I also want to address in
an abbreviated fashion some of criticisms of what I wrote. I'll do this in a few posts, of which this is the first.
Here are a couple criticisms: 1) political scientists do not claim to predict anything; and 2) Karl Popper's work has been relegated to intellectual history. Quick responses: first, many political scientists and New York Times blogger and statistical enthusiast Nate Silver disagree with both of these positions. I mention Silver and will go through the work of others later because the guys writing for the political science blogs offered no conclusive evidence for their claims that political scientists do not or should not attempt to make predictions nor that Karl Popper is wrong, but rather make these ad populi assertions in the bombastic fashion that is typical of their bluster and a major reason for, and symptom of, our discipline's intellectual impoverishment.
Substantively, here are five quick thoughts I'd like to share.
First, the assertion in my New York Times piece was that political scientists who were modeling nuclear war and deterrence scenarios, and experienced Sovietologists and even Kremlinologists, just blew it. And they knew it at the time--no idea why folks now are trying to muddy the waters on this point now. I was a grad student at Berkeley in 1991 and had taken classes with Ken Jowett, who repeatedly claimed that the Communist Party in the USSR was like the Catholic Church and would never willingly give up power. His was not just a majority position but encompassed the views of every expert in the field. Even those who didn't specify this position as a specific prediction were implicitly predicting it in every sentence they wrote. Likewise, Middle East experts also didn't see the Arab Spring coming. Period. To the extent they discussed the upending of regimes in places such as Tunisia, Libya, Egypt, and Syria, it was to explain why this was not happening nor going to happen.
Second, the most interesting response to my analysis was the claim that some to many political scientists are not even trying to orient us to the future but merely using statistical analysis to retrospectively understand causal mechanisms associated with changes in cases in their databases.
Assuming that some political scientists really are only quantitative historians (and perhaps should be applying for history grants, anyway) these folks, too, are making predictions. I was very surprised to read this criticism because it was coming from the folks who are supposedly in the more subtle and sophisticated wing of the quantitative field. Regardless, they know, or should know, that their claims about "expected values," either as a universal snapshot to be generalized or just retrospectively, are by definition predictions about the likelihood of a particular case having characteristics consistent with the categorical characteristics imputed to the mean: "expected value: The mean value of the theoretical sampling distribution of any statistic. Statistical reasoning is centrally concerned with the expected value, as opposed to any particular observed value. In statistical procedures that seek to predict the values of the dependent variable using one or more independent variables, the predictions are typically estimated expected values, conditional on the independent variables included in the analysis." Jason Seawright and David Collier, Rethinking Social Inquiry: Diverse Tools, Shared Standards. p. 326.
This is important because many cases in these sorts of studies have outcomes that are diametrically the opposite of that predicted by the so-called scientists' expected values; if you think Popper is right, and many of us do, including those who do work emphasizing necessary but not sufficient conditions, then these cases have falsified the predictions of the expected value.
Third, some claimed that when political scientists get it wrong, and are corrected later, say in the example of the civil war studies I referenced, then a) I am a hypocrit for quoting quantitative work; and b) the system works. Steven Saideman, in his post referring to me as a "Self-hating Political Scientist," made this point. (Professor Saideman: Really? You want to associate yourself with Zionist ideologues who attempt to avoid accountability for racist policies by name-calling tactics? And me with leftist Jewish critics of racist policies? I'm perfectly fine with this but thought it might merit further reflection.)
I have a few thoughts on the boosterism for quantitative work's putatively cumulative approach to knowledge. The first is that my juxtaposition of positions on the relevance of ethnic grievances to civil wars -- comparing the analyses of James Fearon and David Laitin with that of Lars-Erik Cederman, Nils Weidmann, and Kristian Skrede Gleditsch -- was to emphasize that the work of the former had been falsified and to perform how quantitative analysts are re-representing journalistic observations through equations and not producing new knowledge.
If you read the latter article's commendable qualifications and especially the last line, this becomes especially obvious: "It is very unlikely that such conflicts can ever be understood, let alone durably solved, without taking seriously the claims of marginalized populations." "Horizontal Inequalities and Ethnonationalist Civil War: A Global Comparison," American Political Science Review, p. 492. These guys, who, for what it's worth, did not receive any attributed funding from the U.S. NSF -- Saideman's claim to the contrary notwithstanding -- but did receive funding from the Air Force Office of Scientific Research, are saying, hey, we are going to tell you through our databases we spent tens of thousands of dollars putting together the same thing we hear from the people enduring violence from civil conflicts.
I understand that the authors and Saideman likely will claim that political scientists with lots of gigabytes still need to sweep in and use their magical variables and formulas to render the "claims of marginalized populations" as knowledge, but then we're just back in the old debate about what counts as knowledge. In other words, the article Saideman claims is evidence of the effectiveness of quantitative work also is evidence that this work at its best is merely an echo chamber for the claims of marginalized populations that are noted by journalists in newspapers such as the New York Times.
As far as the assertion that science-in-progress means lots of mistakes and corrections. Come on, guys! Sounds to me as though the great thing about this "probabilistic nonsense," as Popper called it, is that it helps you put together a protection racket for the guys who crunch. As long as you keep doing it, it doesn't matter if you are right or wrong. In other words, you can have a long and lucrative, government-funded career if you just keep reviewing each other's work, as Saideman himself claims occurred in the case above, and you never have to worry about getting it wrong. Also, as Nate Silver points out, echoing Popper, if your field is prone to mistakes and the work is probabilistic, then it's impossible to know when someone really has made a correction or when the supposed correction is just noise. For instance, Fearon and Laitin were supposedly correcting previous quantitative analyses of civil war studies. (Professor James Fearon did write a thoughtful response and I will be engaging this in more detail in another post.)
(Charles Lane's column in the Washington Post, also critical of NSF funding, by the way, was included, and also an opinion piece by a psychologist on his inferiority complex among natural scientists. I asked APSA President Bingham Powell, Jr. about this and he refused to share with me the names of individuals making decisions about the contents of the APSA web pages on the NSF funding and he gave me an explanation for the exclusion of my piece that would have required as well the exclusion of the opinion pieces by Lane and the psychologist Timothy Wilson.)
-----
UPDATE (7/31/2012): Since this was posted, Professor Powell and I have shared additional e-mails that have been mutually collegial, and for which I am very grateful. He has clarified that Michael Britnall, the APSA Executive Director, is supervising the staff making decisions on the APSA NSF funding web pages and that the decision to present information for the purposes of advocacy on this issue reflected a longstanding commitment to this objective, though Professor Powell did not specify any particular APSA directive or decision initiating this commitment. Especially heartening was Professor Powell's assurance that he would raise at the next Council meeting the question of whether the APSA should provide guidance to the NSF on the nature of what the APSA considers its "pure research" priorities, and he was receptive to my question about the relevance and accuracy of the current name for our professional association.
-----
UPDATE (7/31/2012): Since this was posted, Professor Powell and I have shared additional e-mails that have been mutually collegial, and for which I am very grateful. He has clarified that Michael Britnall, the APSA Executive Director, is supervising the staff making decisions on the APSA NSF funding web pages and that the decision to present information for the purposes of advocacy on this issue reflected a longstanding commitment to this objective, though Professor Powell did not specify any particular APSA directive or decision initiating this commitment. Especially heartening was Professor Powell's assurance that he would raise at the next Council meeting the question of whether the APSA should provide guidance to the NSF on the nature of what the APSA considers its "pure research" priorities, and he was receptive to my question about the relevance and accuracy of the current name for our professional association.
-----
I have a few other thoughts on the positive and negative responses I will make in another post.
but to conclude for now...
Why are we still calling our
professional organization the American Political Science Association?
Leaving aside, for now, the word “American,” why is “science”
in the title, a word that does not appear in the professional
association names for any other discipline engaged in empirical and
theoretical research on our macro- or micro-level institutions,
communities, practices, and ideas, to wit: the American
Anthropological Association, the American Economic Association, the
American Historical Association, the American Psychological
Association, and the American Sociological Association
Why not change our association's name
to the American Politics Association, or the American Political
Association, or even the American Politicological Association, which
is the least preferable among the three but still more accurate than
our current name?
(That “politicological” does not
roll easily off one's tongue is not because the syllables are
inherently more jargony or difficult to pronounce than, say,
“sociological” or “anthropological,” but because of habits
associated with our ordinary use today of “political science,” or
“politics,” and because “politicological” is not a word. )
Instead of debating whether we members
of the American Political Science Association and the authors of the
articles appearing in the American Political Science Review
are really scientists, thus forcing us to engage the question of what
counts as a science and whether we support or oppose working under
this rubric, why not eliminate this question altogether, and work
under a disciplinary heading that embraces scholarship about
politics?
Why are we not those who for the domain
of politics “craft knowledge,” a somewhat literal translation of
“Wissenschaft,” as opposed to aspiring to be politics'
scientists? Why not discuss a more interesting, fundamental, and
difficult question, of which the debate about science is one tangent:
what counts as knowledge about politics and how do we know this?
(“Wissen” is the perfectly
colloquial verb “to know” and does not have the white lab coat
connotations of the English “science.” It is possible to use a
broader understanding of knowledge than that pursued by some of my
colleagues through considering the etymology of the English “science”
as well, but the more general connotations of knowledge are less
idiomatic than for the German “wissen.”)
The word “science” in our
professional association title is an empty abstraction and
distraction. If my colleagues want to be scientists, let them defend
their definition of science and their fitness for this vocation
through their research, publications, and logical arguments, and let
them not occupy the throne of scholarly authority by bullying and
vague nomenclature.
Especially in light of the fact that
some, though not all, of my colleagues are claiming that the
“scientific method” they advocate has nothing to do with the
scientific method from the time of the APSA's founding, or what
science looked like in the 1930s, the word “science”-s
significance and meaning in the name of our professional association
today seems especially unclear and hence indefensible and ripe for
elimination.
Please note that I am not claiming here
that “scientific” research is this or that, or good or bad—this
discussion comes later. I am simply pointing out that in light of
the intellectual diversity of our actual membership and the
metaphysical uncertainty about methods most conducive to knowledge,
not to mention the contradictory defenses of the “science” in
political science—some today think political scientists should be
good at prediction, and are or are not, others that this is not
required of political science—there are no consistent and widely
accepted reasons to claim that we all are or should be working as
scientists; and hence the word “science” has no good grounds for
remaining in the name of our professional association.
2 comments:
For discussions of political science methods, only attributed comments will be posted.
Post a Comment