Risky Predictions, Damn Strange Coincidences, and Theory Appraisal: A Multivariate Corroboration Index for Path Analytic Models

Author

Kristine Y. Hogarty, University of South Florida

Graduation Year

2003

Document Type

Dissertation

Degree

Ph.D.

Degree Granting Department

Measurement and Evaluation

Major Professor

Jeffrey D. Kromrey, Ph.D.

Committee Member

Robert Dedrick, Ph.D.

Committee Member

John Ferron, Ph.D.

Committee Member

Kathleen McNelis, Ph.D.

Keywords

theory testing, model fit, path analysis, verisimilitude, precision

Abstract

The empirical testing of theories is an important component of research in any field. Yet despite the long history of science, the extent to which theories are supported or contradicted by the results of empirical research remains ill defined. Quite commonly, support or contradiction is based solely on the "reject" or "fail to reject" decisions that result from tests of null hypotheses that are derived from aspects of theory. Decisions and recommendations based on this forced and often artificial dichotomy have been scrutinized in the past.

In recent years, such an overly simplified approach to theory testing has been vigorously challenged in the past.Theories differ in the extent to which they provide precise predictions about observations. The precision of predictions derived from theories is proportional to the strength of support that may be provided by empirical evidence congruent with the prediction. However, the notion of precision linked to strength of support is surprisingly absent from many discussions regarding the appraisal of theories.

Meehl (1990a) has presented a logically sound index of corroboration to summarize the extent to which empirical tests of theories provide support or contradiction of theories. The purpose of this study was to evaluate the utility of this index of corroboration and its behavior when employing path analytic methods in the context of social science research.

The performance of a multivariate extension of Meehl’s Corroboration Index (C_i) was evaluated using Monte Carlo methods. Correlational data were simulated to correspond to tests of theories via traditional path analysis. Five factors were included in the study: number of variables in the path model, level of intolerance of the theory, correspondence of the theory to the ‘true’ path model used for data generation, sample size and level of collinearity.

Results were evaluated in terms of the mean and standard error of the resulting multivariate C_i values. The level of intolerance was observed to be the strongest influence on mean C_i. Verisimilitude and model complexity were not observed to be strong determinants of the mean C_i. Sample size and collinearity evidenced small relationships with the mean value of C_i, but were more closely related to the sampling error.

Implications for theory and practice include alternatives and complements to tests of statistical significance, a shift from comparing findings to the null hypothesis, to the comparison of alternative theories and models, and the inclusion of additional logical components besides the theory itself. Lastly, an alternative conceptualization of the multivariate corroboration index is advanced to guide future research efforts.

Scholar Commons Citation

Hogarty, Kristine Y., "Risky Predictions, Damn Strange Coincidences, and Theory Appraisal: A Multivariate Corroboration Index for Path Analytic Models" (2003). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/1392