Words as Data? Estimating the Fiscal Conservativeness of Provincial Premiers Using the Wordscore Procedure of Content Analysis

André S. Gosciniak

Andre.Gosciniak@capp.ulaval.ca

Laval University


What is the usefulness of the new computer wordscoring method developed by Laver, Benoit and Garry (2003) for research in political science, and particularly for estimating the policy position of key political actors? This is the central question I address in this paper. Using the computer wordscoring methodology I analyze the speeches given in the legislative assemblies of Quebec and Ontario by provincial premiers to assess their budgetary policy position. I present the empirical results of the wordscoring content analysis and discuss the validity and the effectiveness of this new technique.

Manual content analysis, but also traditional computer content analysis (either using a dictionary approach or another one) has involved large amounts of highly skilled labor required for developing and testing the coding dictionaries that are central to these approaches. In addition, important human involvement brings the risk of potentially biased human coders. These two shortcomings can be avoided by using a new probabilistic word-scoring method developed recently by Laver, Benoit and Garry (2002 & 2003) which extracts policy positions from political texts by treating them as data in the form of words. 

The technique using the software Wordscore estimates policy positions by comparing a set of texts whose positions are known or can be either estimated with confidence from independent sources or assumed uncontroversially as representative for a policy position (reference texts) and a set of texts whose policy position one must search to uncover (virgin texts). 

Using the computer wordscoring methodology I analyze the policy position of provincial premiers in the budgeting process in Quebec and in Ontario in the years 1968-2004 using the guardian-spender budgetary framework (Wildavsky 1964 & 1984), which supposes the existence of two main roles attached to the institutional positions held by the participants in the budgetary negotiations: they are either guardians of the treasury (participants from central agencies controlling the budget: Minister of Finance, President of the Treasury board) or advocates of program spending coming from program ministries (minister of Health, minister of Education, etc). 

Since we can not predict the fiscal behaviour of the government on the base of a simple partisan left-right dichotomy (Imbeau), the traditional models of budgetary theories employing politicians' [here called actors] preferences do not give us any appropriate measure of the budgetary position of key political actors. Thus, a way to uncover policy position of a premier is to assess his or her preferences on an additional dimension, differentiating between budgetary actors with a total vision of budget (guardians) and those with a partial vision of it (spenders). The vision is total if the budget balance is more important than a party's taxing or spending preferences. On the other hand, the vision of the budget is partial if the party program (more or less tax and spending) is more important than the budget balance. 

Which texts are representative of the policy position of budgetary actors we are interested in? We certainly can assume that the Speech from the Throne delivered by the Lieutenant-Governor at the opening of each legislative session expresses the Premier's policy preferences. Its delivery constitutes one of the central moments in a session, thus it is carefully crafted by the Premier's Office and its content is reviewed many times before delivery. The Throne speeches are compared to two reference texts: the budget speech representing the guardians' expression of a total view of the budget, and the preliminary remarks by the Ministers of Health and of Education at budget hearings representing the spenders' view of a partial vision of the budget. Although all three texts are usually drafted so as to represent the position of the government, they should be differentiated because of the policy role that each assumes.

Based on these assumptions and using the wordscoring technique of content analysis, I develop a fiscal conservatism index of budgetary actors. I then assess the internal validity of my results using a variety of validity tests. 



Bibliography


Imbeau, L.M.
Deficits and Surpluses in Federated States: A Review of the Public Choice Empirical Literature
Paper presented at the annual conference of the Canadian Political Science Association, Winnipeg
2004

Laver, M.
Benoit, K.
Garry, J.
Placing Political Parties in Policy Spaces
Trinity College
Dublin
2002

Laver, M.
Benoit, K.
Garry, J.
Extracting Policy Positions from Political Texts Using Words as Data
American Political Science Review
97
311-331
2003

Wildavsky, A.B.
The Politics of the Budgetary Process
Little, Brown and Company
Boston
1964

Wildavsky, A.B.
The Politics of the Budgetary Process (4th Edition)
Little, Brown and Company
Boston
1984