A Map of the Poor or a Poor Map?

This paper evaluates the performance of different small area estimation methods using model and design-based simulation experiments. Design-based simulation experiments are carried out using the Mexican Intra Censal survey as a census of roughly 3.9 million households from which 500 samples are drawn using a two-stage selection procedure similar to that of Living Standards Measurement Study surveys. Several unit-level methods are considered as well as a method that combines unit and area level information, which has been proposed as an alternative when the available census data is outdated. The findings show the importance of selecting a proper model and data transformation so that the model assumptions hold. A proper data transformation can lead to a considerable improvement in mean squared errors. The results from design-based validation show that all small area estimation methods represent an improvement, in terms of mean squared errors, over direct estimates. However, methods that model unit level welfare using only area level information suffer from considerable bias. Because the magnitude and direction of the bias are unknown ex ante, methods that rely only on aggregated covariates should be used with caution, but they may be an alternative to traditional area level models when these are not applicable