Data-driven estimation of economic indicators with search big data in discontinuous situation

Economic indicators are essential for policymaking and strategic decisions in both the public and private sectors. However, due to delays in the release of government indicators based on macroeconomic factors, there is a high demand for timely estimates or “nowcasting”. Many attempts have been made to overcome this challenge using macro indicators and key variables such as keywords from social networks and search queries, but with a reliance on human selection. We present a fully data-driven methodology using non-prescribed search engine query data (Search Big Data) to approximate economic variables in real time. We evaluate this model by estimating representative Japanese economic indicators and confirm its success in nowcasting prior to official announcements, even during the COVID-19 pandemic, unlike human-selected variable models that struggled. Our model shows consistent performance in nowcasting indices both before and under the pandemic before government announcements, adapting to unexpected circumstances and rapid economic fluctuations. An exhaustive analysis of key queries reveals the pivotal role of libidinal drives and the pursuit of entertainment in influencing economic indicators within the temporal and geographic context examined. This research exemplifies a novel approach to economic forecasting that utilizes contemporary data sources and transcends the limitations of existing methodologies.