Progress:What's Your PQ? Take the test and find out!
The roll-call votes, on which the quiz questions are based, are chosen by the Americans for Democratic Action (ADA). As Groseclose, Steve Levitt, and James Snyder have argued (see their article in the 1999 volume of the American Political Science Review), it is not proper to compare ADA scores across time or between chambers. The reason is similar to the reasons why teachers sometimes “curve” exams. That is, for instance, if you score a 70 on an exam in the early part of the term and score a 90 on an exam in the later part of the term, then it is tempting to conclude that you became smarter. However, another possibility is simply that the later exam contained easier questions. Consequently, the teacher might “curve” the second exam (i.e. add or subtract points from students’ scores) in order to make scores from the two exams comparable.
The same issues arise with ADA scores. Because the “questions” (i.e. roll call votes) that the ADA chooses differ across years, in some years it may be easier to vote on the liberal side than in other years. Groseclose, Levitt, and Snyder have created mathematical formulas that “curve” the ADA scores. Specifically, the formulas convert each year’s/chamber’s ADA score to a common scale. PQ scores are measured on that scale. (For more details, see chapter 4 of Left Turn: How Liberal Media Bias Distorts the American Mind.) Most important, because PQs use those conversion formulas, your PQ is directly comparable to the PQs that I list of politicians (even though some of them were not even alive in 2009, when Congress considered the above issues).
A number of people have asked why I list at the end of each question how the two parties voted. I do this for at least two reasons.
One involves issues of summarizing the roll call votes (on which the questions are based). Many of the roll calls involve legislation that is hundreds of pages long. It is nearly impossible to represent such legislation with a one-paragraph summary. Sometimes, when I would write the summaries, I’d think, “Given that summary, how could anyone vote on the liberal side?” Yet, several members of Congress did on the liberal side. The reason usually was because there was information in the legislation (say, an important amendment to a bill) that would make a person want to vote on the liberal side; however, that information was not included in the summary. The most direct solution to this problem is to include that information in the summary. However, at some point, once one keeps applying that solution, the summaries become very long. As it stands, the summaries are so long that visitors to the web site often become bored with the quiz and stop taking it – only about half actually make it to the end of the quiz. Another solution is simply to list how members of Congress vote. That is, for instance, if the reader can see that almost all Democrats voted on the liberal side of the question, then he or she can correctly infer, “Well, there must be something in the legislation, perhaps not included in the summary, that would make some reasonable people want to vote on the liberal side.” I think it is important that a reader have the necessary information to make such an inference – hence, he or she should be able to see how members of Congress voted.
Another reason is that I want readers to put themselves in the shoes of members of Congress when answering the questions. Related, I want readers to possess the same information that members of Congress possessed when they voted on legislation. When members of Congress vote, they can view a tote board that lists how all previously voting members voted. Of course, in the above questions, I list final vote totals–that is, the totals after all members have voted. Thus, one might argue that the early-voting members did not possess this information when they voted. However, members of Congress are allowed to change their votes. Further, just before time expires for voting, the presiding officer usually asks (and I am aware of only a handful of cases where this was not the case) “Is there anyone in the chamber who has not voted or wishes to change his or her vote?” At that point, when the presiding officer asks that question, members of Congress make their final decision whether they favor or oppose the legislation they are voting on. At that point, simply by looking at the tote board, they know how all other members of Congress have voted on the legislation. I think it is important for readers taking my quiz to have that same information. For this reason I list the vote totals of the two parties.