We propose an estimation procedure for a semiparametric panel data censored regression model in which the error terms may be subject to general forms of nonstationarity. Specifically, we allow for heteroskedasticity over time and a time varying factor load on the individual specific effect. Empirically, estimation of this model would be of interest to explore how returns to unobserved skills change over time—see, e.g., Chay (1995, manuscript, Princeton University) and Chay and Honoré (1998, <italic>Journal of Human Resources</italic> 33, 4–38). We adopt a two-stage procedure based on nonparametric median regression, and the proposed estimator is shown to be <inline-graphic>null</inline-graphic>-consistent and asymptotically normal. The estimation procedure is also useful in the group effect setting, where estimation of the factor load would be empirically relevant in the study of the intergenerational correlation in income, explored in Solon (1992, <italic>American Economic Review</italic> 82, 393–408; 1999, <italic>Handbook of Labor Economics</italic>, vol. 3, 1761–1800) and Zimmerman (1992, <italic>American Economic Review</italic> 82, 409–429).
