Are the APSA President and Executive Director Hypocrits? An Open Letter Responding to their Open Letter to Senators

The American Political Science Association is still issuing alarmist statements about Congress voting to defund political science research along the lines of the rhetoric that inspired me to write "Political Scientists are Lousy Forecasters," an opinion piece published in the New York Times Sunday Review on June 24, 2012.

I have never endorsed the Congressional proposals to defund political science research, but in the June opinion piece, and in other correspondence and posts on this blog, I have argued that the apoplexy about this is irrational; that our country would be better served by funding priorities other than those currently used by NSF grant adjudicators; and that the APSA needs to have an open conversation about the criteria we believe are reasonable for assessing grant proposals, including entertaining the possibility that distributing research funding at random to proposals that are coherent and have sound budgets might be a strategy most likely to yield robust research agendas and knowledge that isn't just a hot finding for a few years and then displaced by the next election or war or spurious correlation announced as a breakthrough in discovering a new causal mechanism.

A colleague this morning brought to my attention that Sen. Tom Coburn quoted from the Times article in his floor speech in support of defunding the discipline.  He is not a very good reader.  I wrote:

To shield research from disciplinary biases of the moment, the government should finance scholars through a lottery: anyone with a political science Ph.D. and a defensible budget could apply for grants at different financing levels. And of course government needs to finance graduate student studies and thorough demographic, political and economic data collection. I look forward to seeing what happens to my discipline and politics more generally once we stop mistaking probability studies and statistical significance for knowledge.

Still, I understood when I wrote the opinion piece that it might be misused in exactly the way Coburn misused it and decided that these consequences and any debate about this would be preferable to business-as-usual.

For that discussion to occur, however, requires that the APSA stop worrying about protecting democracy in Congress and start practicing it in our own organization.  What follows is a letter I sent via email this morning to Jane Mansbridge, President, APSA and Michael Brintnall, Executive Director, APSA.

Dear Professor Mansbridge and Mr. Brintall,
I hope this finds you well.  I am writing because a colleague just brought to my attention that Senator Tom Coburn cited in his floor speech last week an opinion piece I wrote last summer in which I questioned whether quantitative political science research of the sort prioritized by the NSF contributes to knowledge about politics. 

As a result of that piece not appearing on the APSA NSF page alongside other opinion pieces about the NSF controversy, including another piece critical of NSF funding, I was in correspondence over the summer with Mr. Brintnall and Professor Powell about how how they had ascertained that the position statements coming out of the APSA actually were representing the interests of its membership.

The response was that they had no specific authorization from either the Council nor the membership for these statements and were issuing them on the basis of previous commitments that also had no specific authorization from the APSA membership or Council.  Professor Powell indicated that there would be a discussion of the APSA relation to the NSF, including input on funding criteria, at the Council meeting in New Orleans.

That meeting was cancelled and I am wondering if the Council has actually discussed the statements issued in our name.  I am especially concerned about the language in your March 15, 2013 letter echoing previous statements offering our colleagues' services to so-called national security and defense interests.  Senator Coburn seems to have taken you up on this.  I am wondering on what basis you decided to emphasize our colleagues' availability for this research, and not, say, research on how vague nativist national security anxieties such as those invoked by your letter undermine the rule of law, not to mention rational research priorities.

Senator Coburn affirmed one part of my argument but then contradicted himself by creating an exception to the ban on NSF funding for political science if it is used to advance the national security or economic interests of the United States.  If political scientists are bad at producing useful knowledge, the part of my argument Coburn affirms, then it seems illogical to rely on these dart-throwing monkey equivalents in areas about which Coburn prioritizes. However, alas, this is exactly the exception emphasized in your open letter.

Your open letter to senators also states that our "discipline [is] devoted to learning how to make democracies work better" and you mention your concern that the passage of this measure would be an "embarrassment for the world's exemplary democracy."  In light of the fact that you are advancing positions without following a single recognized procedure of representative democracy these lofty sentiments seem hypocritical and absurd.  The Senate openly debated and voted on a measure and amendment ascertaining the relationship between the NSF and our discipline.  I am aware of no deliberative or decision-making procedures that have been followed pursuant to establishing a position on NSF funding for our own organization.  Instead, the President and Executive Director in 2012 and 2013 appear to have been simply issuing these statements by fiat.

We do not need NSF funding to bring democracy to our own organization. Why not a moderated digital conversation under the aegis of the APSA about the priorities we would seek of the NSF funding of political science?  What about actually being the democratic change and using tools for digital democracy among our own membership?

In short, the APSA is a membership organization but follows no procedures for ascertaining its best interests much less for representing these to the public.  Without this, the claims you make on behalf of political scientists as saviors of democratic values lack any integrity.  Instead of presenting your membership as eager to lick up the scraps alongside the trough of the country's militarist, homeland security funding, why not treat us as serious intellectuals who may be driven by intellectual curiosity and ideals of creativity, justice, freedom, peace, and, yes, democracy?

I am copying this to the Council Secretary John Green and am requesting that he distribute this to the current Council members (not all of them have their email addresses available online).  I understand it is possible that there has been some conversation among them and look forward to having these views more openly distributed and a range of positions on important questions such as the government funding for our discipline more systematically deliberated.

Yours,

Jackie

-- Jacqueline Stevens
Professor
Political Science and Legal Studies Board
Northwestern University

Director
Deportation Research Clinic
Buffett Center for International and Comparative Studies
http://www.cics.northwestern.edu/programs/deportationresearch/

phone 847-467-2093
fax  847-491-8985

Mail
601 University Place
Department of Political Science
Northwestern University
Evanston, IL  60208

http://www.jacquelinestevens.org
http://stateswithoutnations.blogspot.com

Some Thoughts on the American Political Science Association

On June 24, 2012 the New York Times Sunday Review ran my essay Political Scientists Are Lousy Forecasters explaining the problems with the National Science Foundation's current political science grants. Not surprisingly, it drew strong responses. I intended to reply to these earlier but some unanticipated time-sensitive work intervened.

The question of how best to study politics is an immense one that I will take up in more detail elsewhere. For now I want to review a few specific responses to the essay, and raise some big picture questions, the first one being about the word "science" in the name of our professional association.  I also want to address in an abbreviated fashion some of criticisms of what I wrote. I'll do this in a few posts, of which this is the first.

Here are a couple criticisms: 1)  political scientists do not claim to predict anything; and 2)  Karl Popper's work has been relegated to intellectual history.  Quick responses: first, many political scientists and New York Times blogger and statistical enthusiast Nate Silver disagree with both of these positions.  I mention Silver and will go through the work of others later because the guys writing for the political science blogs offered no conclusive evidence for their claims that political scientists do not or should not attempt to make predictions nor that Karl Popper is wrong, but rather make these ad populi assertions in the bombastic fashion that is typical of their bluster and a major reason for, and symptom of, our discipline's intellectual impoverishment.   

Substantively, here are five quick thoughts I'd like to share.

 First, the assertion in my New York Times piece was that political scientists who were modeling nuclear war and deterrence scenarios, and experienced Sovietologists and even Kremlinologists, just blew it.  And they knew it at the time--no idea why folks now are trying to muddy the waters on this point now.  I was a grad student at Berkeley in 1991 and had taken classes with Ken Jowett, who repeatedly claimed that the Communist Party in the USSR was like the Catholic Church and would never willingly give up power.  His was not just a majority position but encompassed the views of every expert in the field.  Even those who didn't specify this position as a specific prediction were implicitly predicting it in every sentence they wrote.  Likewise, Middle East experts also didn't see the Arab Spring coming.  Period.  To the extent they discussed the upending of regimes in places such as Tunisia, Libya, Egypt, and Syria, it was to explain why this was not happening nor going to happen. 

Second, the most interesting response to my analysis was the claim that some to many political scientists are not even trying to orient us to the future but merely using statistical analysis to retrospectively understand causal mechanisms associated with changes in cases in their databases.  

Assuming that some political scientists really are only quantitative historians (and perhaps should be applying for history grants, anyway) these folks, too, are making predictions.  I was very surprised to read this criticism because it was coming from the folks who are supposedly in the more subtle and sophisticated wing of the quantitative field.  Regardless, they know, or should know, that their claims about "expected values," either as a universal snapshot to be generalized or just retrospectively, are by definition predictions about the likelihood of a particular case having characteristics consistent with the categorical characteristics imputed to the mean: "expected value: The mean value of the theoretical sampling distribution of any statistic.  Statistical reasoning is centrally concerned with the expected value, as opposed to any particular observed value.  In statistical procedures that seek to predict the values of the dependent variable using one or more independent variables, the predictions are typically estimated expected values, conditional on the independent variables included in the analysis."  Jason Seawright and David Collier, Rethinking Social Inquiry: Diverse Tools, Shared Standards. p. 326.  

This is important because many cases in these sorts of studies have outcomes that are diametrically the opposite of that predicted by the so-called scientists' expected values; if you think Popper is right, and many of us do, including those who do work emphasizing necessary but not sufficient conditions, then these cases have falsified the predictions of the expected value.

Third, some claimed that when political scientists get it wrong, and are corrected later, say in the example of the civil war studies I referenced, then a) I am a hypocrit for quoting quantitative work; and b) the system works.  Steven Saideman, in his post referring to me as a "Self-hating Political Scientist," made this point.  (Professor Saideman: Really?  You want to associate yourself with Zionist ideologues who attempt to avoid accountability for racist policies by name-calling tactics?  And me with leftist Jewish critics of racist policies?  I'm perfectly fine with this but thought it might merit further reflection.)

I have a few thoughts on the boosterism for quantitative work's putatively cumulative approach to knowledge. The first is that my juxtaposition of positions on the relevance of ethnic grievances to civil wars -- comparing the analyses of James Fearon and David Laitin with that of Lars-Erik Cederman, Nils Weidmann, and Kristian Skrede Gleditsch -- was to emphasize that the work of the former had been falsified and to perform how quantitative analysts are re-representing journalistic observations through equations and not producing new knowledge. 

If you read the latter article's commendable qualifications and especially the last line, this becomes especially obvious: "It is very unlikely that such conflicts can ever be understood, let alone durably solved, without taking seriously the claims of marginalized populations."  "Horizontal Inequalities and Ethnonationalist Civil War: A Global Comparison," American Political Science Review, p. 492.  These guys, who, for what it's worth, did not receive any attributed funding from the U.S. NSF -- Saideman's claim to the contrary notwithstanding -- but did receive funding from the Air Force Office of Scientific Research,  are saying, hey, we are going to tell you through our databases we spent tens of thousands of dollars putting together the same thing we hear from the people enduring violence from civil conflicts.

I understand that the authors and Saideman likely will claim that political scientists with lots of gigabytes still need to sweep in and use their magical variables and formulas to render the "claims of marginalized populations" as knowledge, but then we're just back in the old debate about what counts as knowledge.  In other words, the article Saideman claims is evidence of the effectiveness of quantitative work also is evidence that this work at its best is merely an echo chamber for the claims of marginalized populations that are noted by journalists in newspapers such as the New York Times.

As far as the assertion that science-in-progress means lots of mistakes and corrections.  Come on, guys!  Sounds to me as though the great thing about this "probabilistic nonsense," as Popper called it, is that it helps you put together a protection racket for the guys who crunch.  As long as you keep doing it, it doesn't matter if you are right or wrong.  In other words, you can have a long and lucrative, government-funded career if you just keep reviewing each other's work, as Saideman himself claims occurred in the case above, and you never have to worry about getting it wrong.  Also, as Nate Silver points out, echoing Popper, if your field is prone to mistakes and the work is probabilistic, then it's impossible to know when someone really has made a correction or when the supposed correction is just noise.  For instance, Fearon and Laitin were supposedly correcting previous quantitative analyses of civil war studies.  (Professor James Fearon did write a thoughtful response and I will be engaging this in more detail in another post.)

As I mentioned at the beginning, I think this discussion is part of a larger debate and to that end want to end this post by asking why we bother inserting the word "science" in the name of our professional association, which, by the way, is excluding only my opinion piece from its web page advocating the position of the association.

(Charles Lane's column in the Washington Post, also critical of NSF funding, by the way, was included, and also an opinion piece by a psychologist on his inferiority complex among natural scientists.  I asked APSA President Bingham Powell, Jr. about this and he refused to share with me the names of individuals making decisions about the contents of the APSA web pages on the NSF funding and he gave me an explanation for the exclusion of my piece that would have required as well the exclusion of the opinion pieces by Lane and the psychologist Timothy Wilson.) 

-----
UPDATE (7/31/2012):  Since this was posted, Professor Powell and I have shared additional e-mails that have been mutually collegial, and for which I am very grateful.  He has clarified that Michael Britnall, the APSA Executive Director, is supervising the staff making decisions on the APSA NSF funding web pages and that the decision to present information for the purposes of advocacy on this issue reflected a longstanding commitment to this objective, though Professor Powell did not specify any particular APSA directive or decision initiating this commitment. Especially heartening was Professor Powell's assurance that he would raise at the next Council meeting the question of whether the APSA should provide guidance to the NSF on the nature of what the APSA considers its "pure research" priorities, and he was receptive to my question about the relevance and accuracy of the current name for our professional association.
-----

I have a few other thoughts on the positive and negative responses I will make in another post.  

but to conclude for now...

Why are we still calling our professional organization the American Political Science Association? 

 Leaving aside, for now, the word “American,” why is “science” in the title, a word that does not appear in the professional association names for any other discipline engaged in empirical and theoretical research on our macro- or micro-level institutions, communities, practices, and ideas, to wit: the American Anthropological Association, the American Economic Association, the American Historical Association, the American Psychological Association, and the American Sociological Association

Why not change our association's name to the American Politics Association, or the American Political Association, or even the American Politicological Association, which is the least preferable among the three but still more accurate than our current name?

(That “politicological” does not roll easily off one's tongue is not because the syllables are inherently more jargony or difficult to pronounce than, say, “sociological” or “anthropological,” but because of habits associated with our ordinary use today of “political science,” or “politics,” and because “politicological” is not a word. )

Instead of debating whether we members of the American Political Science Association and the authors of the articles appearing in the American Political Science Review are really scientists, thus forcing us to engage the question of what counts as a science and whether we support or oppose working under this rubric, why not eliminate this question altogether, and work under a disciplinary heading that embraces scholarship about politics?

Why are we not those who for the domain of politics “craft knowledge,” a somewhat literal translation of “Wissenschaft,” as opposed to aspiring to be politics' scientists? Why not discuss a more interesting, fundamental, and difficult question, of which the debate about science is one tangent: what counts as knowledge about politics and how do we know this?

(“Wissen” is the perfectly colloquial verb “to know” and does not have the white lab coat connotations of the English “science.” It is possible to use a broader understanding of knowledge than that pursued by some of my colleagues through considering the etymology of the English “science” as well, but the more general connotations of knowledge are less idiomatic than for the German “wissen.”)

The word “science” in our professional association title is an empty abstraction and distraction. If my colleagues want to be scientists, let them defend their definition of science and their fitness for this vocation through their research, publications, and logical arguments, and let them not occupy the throne of scholarly authority by bullying and vague nomenclature.

Especially in light of the fact that some, though not all, of my colleagues are claiming that the “scientific method” they advocate has nothing to do with the scientific method from the time of the APSA's founding, or what science looked like in the 1930s, the word “science”-s significance and meaning in the name of our professional association today seems especially unclear and hence indefensible and ripe for elimination.

Please note that I am not claiming here that “scientific” research is this or that, or good or bad—this discussion comes later. I am simply pointing out that in light of the intellectual diversity of our actual membership and the metaphysical uncertainty about methods most conducive to knowledge, not to mention the contradictory defenses of the “science” in political science—some today think political scientists should be good at prediction, and are or are not, others that this is not required of political science—there are no consistent and widely accepted reasons to claim that we all are or should be working as scientists; and hence the word “science” has no good grounds for remaining in the name of our professional association.

More on the Sunday Review Essay

 Ten Commandments Posted in Griffin Courthouse, Spaulding County, Georgia

shadows from right to left--Mark Lyttle, his wife, and me

An opinion piece is a great way to stir up a conversation and a lousy way to present arguments in scholarly detail or provide sources.  Here are a few other thoughts related to "Political Scientists are Lousy Forecasters," published in hard copy of the New York Times Sunday Review on June 24, 2012.  (I am not under the impression that a whole lot of people are interested, but for folks with questions, it seemed more efficient to do this here than on email.)

Auto- Q and A

Q.  Why did you write this?
A.  Because  a couple weeks ago I returned from a day in Judge Fletcher Sam's court in Griffin, Georgia and listened to an impassioned speech just outside his court room by Sernita Trice, a one-time political science major at Penn State, that began, "There is no justice here.  No one cares about us."   The details are not relevant here, but she was right and I was embarrassed for my discipline for ignoring a myriad of politically and intellectually gripping questions that are in cities like Griffin and counties like Spaulding and states like Georgia throughout our country.

I returned to my room at the Griffin Inn and that's when I saw e-mails from my chair and APSA urging me to defend political science against being cut from the National Science Foundation budget.  Of course, I've had my frustrations about political science research agendas long, long before this but it was the specific conjuncture of frustration and the timing of the appeals for NSF support that moved me to write this.

Q.  Geez!  I thought we'd gone beyond that old Wolin v. the behavioralist debate.  Why are you stirring up that old can of worms?
A.  I'm not.  I disagree quite strongly with both the claim of quantitative political scientists to be emulating the method of Karl Popper and also Sheldon Wolin's attack on them for doing so.  As I point out in the piece, Popper rejected behavioralism; he also rejects probabilistic research's claim to being scientific knowledge.  I've published chunks of this in the Methods Appendix to States Without Nations: Citizenship for Mortals (Columbia University Press, 2009).

Also, I've criticized political theorists for making non-falsifiable claims about the alleged decline of the nation-state, including Wendy Brown's Walled States, Waning Sovereignty (Verso, 2010).

Q.  Does this mean that there's a rift between the theorists and everyone else in your department?
A.  NO!  My colleagues are intellectually curious, well-read, and great interlocutors, as well as supportive of my less traditional research agendas.  It's true that I've been trying to encourage more reflection on the implications of Tetlock's research as well as more general questions about our scholarly objectives and I see this piece as part of our ongoing conversation.    

Q.  You really go after James Fearon and David Laitin.  What's that about?
A.  The quantitative civil wars studies industry reveals various troubling and even horrifying commitments, especially to reductionism, inductivism, and the axiomatic status of the nation and ethnic groups. These studies radically impair our society's ability to grapple with pressing problems generated by kinship groups in this country and elsewhere.

Unfortunately there wasn't room to go into the many problems with Fearon's and Laitin's work.  Here's a link to a work in progress:"A Popperian Reading of Civil War Studies," is what I think I'll call it.  It received comments from a peer reviewed journal with a recommendation that I revise and resubmit but this all happened during the time-frame when I was moving and starting new projects so I haven't yet followed up. 

Q.  Also, that whole Tetlock metaphor with the chimps throwing darts: isn't that a bit unfair?  It's such a provocative image. 
A.  Yes, it is a provocative image.  Thanks, Professor Tetlock, for providing this!  All representations of ideas from math and indeed all observations use metaphors. Words, ideas, and symbols that are vivid, accessible, and clarifying are better than others.  

Q.  What are the sources you're using for the claims in "Political Scientists Are Lousy Forecasters"?
A.  Here you go: a list based on the materials I put together for the intrepid New York Times fact-checker. 

Q.  Any reading recommendations?
A.  Yes.  There are lots of greats methods work.  Alas, I don't have the time for putting together a full list but I do want to plug this one:  Kristin Monroe put together an amazing collection of essays that should be required reading for every political scientist, and is available on Amazon for $12!:  Contemporary Empirical Political Theory (University of California Press, 1997).

Q.  Did you have to make any cuts?
A.  Alas, several.  The ones I feel the worst about are the places describing examples of terrific work my colleagues are doing. Here they are:

Intriguingly, another article cautions against taking at face value what people tell survey researchers, and not just on hot button issues such as racial attitudes.  In an ingenious study, funded by the NSF Digital Government Research Program, not part of the Political Science division, political scientist Michael Neblo and his colleagues found that people in their sample who were angry about politics were substantially more likely to express a disinterest in meeting their members of Congress when asked about this  as a survey question than when researchers arranged actual online meetings with their representatives.

.... Government can and should assist political scientists, especially those who use history and theory to yield insights beyond the daily headlines. [THE LAY READER DOESN’T NEED THIS such as Jeffrey Winters’ analysis of oligarchy in the United States, Charli Carpenter’s reflections on the "everyday politics," shaping political science research, Dorian Warren’s work on the intersection of race and labor politics, and other articles appearing in the APSA’s Perspectives on Politics, a journal offering readable, relevant expert analyses that orients us to shifting political contexts and challenges our intuitions.CUT]

Q.  I was reading other stuff on your blog.  Why do you have all these posts on U.S. citizens being detained and deported?
A.  This is a little experiment in be-the-change, politically and intellectually.  I'm drawing on these very specific people and their stories to engage theories of citizenship and membership and also to throw a few monkey wrenches into the system producing these outcomes.  (Hmmm, wonder how political scientists compare with monkeys throwing their wrenches.)  I focus on the plight of U.S. citizens being unlawfully detained and deported because they're the 900 pound gorilla in the mine: if even the rights of U.S. citizens are not being protected then that tells us a lot about the treatment of everyone else.

Also, anything the government does to protect due process rights for U.S. citizens will improve the rights of everyone else -- because the only way to establish citizenship is to provide people with assigned attorneys.  (People may not realize they are U.S. citizens or be in a position to file the right motions to stop the deportation machine, especially if they are 19 year-old guys just being released from jail for stealing a car and they are whisked straight from the jail lobby to an ICE detention center without anyone telling their families.)  If you're interested in more about this, please see "U.S. Government Unlawfully Detaining and Deporting U.S. Citizens as Aliens," Virginia Journal of Social Policy and Law, 18:3 (2011), 115 p.

In addition, I'm supervising a Deportation Research Clinic.  The major project is to conduct a survey of misconduct in deportation proceedings.   If any of this is of interest and you want to collaborate, please be in touch.

Q.  Anything else you are working on?
A.  My major work-in-progress is a long-term project, "200 Per Cent American."  This is the response Mark Lyttle gave to a deportation officer who asked if he was a U.S. citizen after he was detained in the Atlanta airport on his return from Guatemala City.  Mark, who is cognitively disabled and bipolar, had been deported to Mexico and then from there deported to Honduras.  It took him more than four months before a consular officer figured out his story and issued him a U.S. passport--and it was this passport that the government said was fraudulently obtained when he returned and had the interview.   (Mark's attorney was unable to obtain his release from ICE custody and emailed me.  I was at a spring workshop in Dartmouth and called an ICE agent in Washington D.C. with whom I was working on another story, and she called the Atlanta ICE office and procured Mark's release.)

The book I'm writing is a story about his experiences told through the historical and narrative lens of Miguel Cervantes, Don Quixote, a novel making fun of the Spanish conquistadors for acting on the basis of the crazy myths that led them to the Americas. 

Q.  Where are you?
A.  I'm dividing my time between Chicago and New York City this summer.

UPDATE (June 26, 2012)
Q.  What has been the response to your piece?
A.   A deluge of grateful emails, including from political science NSF panel advisers -- the plurality of which literally say, "the emperor has no clothes"; and ferocious, incoherent, illogical, and often ad hominem blogosphere attacks from the naked emperor's fashion consultants desperately trying to convince the elite guard that the finery really is there.

(Clearly an email that says "the emperor has no clothes" won't work; these statements only work when the spectators/chorus do this together and thus embarrass the emperor's henchmen into abandoning their latest round of propaganda and group-think activities.)

I will let the dust settle a bit and then respond sometime next week in a separate post.