Faculty of Economics and Business Administration Publications Database

Pitfalls of Post-Model-Selection Testing: Experimental Quantification

Demetrescu, Matei
Kuzin, Vladimir
Volume: 40
Number: 2
Pages: 359 - 372
Month: April
ISSN-Print: 0377-7332
Link External Source: Online Version
Year: 2011
Keywords: Pre-test Estimator; Model Selection; Empirical Size
Abstract: Traditional specification testing does not always improve subsequent inference. We demonstrate by means of computer experiments under which circumstances, and how severely, data-driven model selection can destroy the size properties of subsequent parameter tests, if they are used without adjusting for the model-selection step. The investigated models are representative of macroeconometric and microeconometric workhorses. The model selection procedures include information criteria as well as sequences of significance tests (“general-to-specific”). We find that size distortions can be particularly large when competing models are close, with closeness being defined relatively to the sample size.