Information, data dimension and factor structure

This paper employs concepts from information theory to choosing the dimension of a data set. We propose a relative information measure connected to Kullback-Leibler numbers. By ordering the series of the data set according to the measure, we are able to obtain a subset of a data set that is most informative. The method can be used as a first step in the construction of a dynamic factor model or a leading index, as illustrated with a Monte Carlo study and with the U.S. macroeconomic data set of Stock and Watson [22].