Optimising web‐based information retrieval methods for horizon scanning

Purpose – Web‐based information retrieval offers the potential to exploit a vast, continuously updated and widely available repository of emerging information to support horizon scanning and scenario development. However, the ability to continuously retrieve the most relevant documents from a large, dynamic source of information of varying quality, relevance and credibility is a significant challenge. The purpose of this paper is to describe the initial development of an automated web‐based information retrieval system and its application within horizon scanning for risk analysis support. Design/methodology/approach – Using an area of recent interest for the insurance industry, namely, space weather — the changing environmental conditions in near‐Earth space — and its potential risks to terrestrial and near‐Earth insurable assets, the authors benchmarked the system against current information retrieval practice within the emerging risks group of a leading global insurance company. Findings – The results highlight the potential of web‐based horizon scanning to support risk analysis, but also the challenges of undertaking this effectively. The authors addressed these challenges by introducing a process that offers a degree of automation — using an API‐based approach — and improvements in retrieval precision — using keyword combinations within automated queries. This appeared to significantly improve the number of highly relevant documents retrieved and presented to risk analysts when benchmarked against current practice in an insurance context. Originality/value – Despite the emergence and increasing use of web‐based horizon scanning in recent years as a systematic approach for decision support, the current literature lacks research studies where the approach is benchmarked against current practices in private and public sector organisations. This paper therefore makes an original contribution to this field, discussing the way in which web‐based horizon scanning may offer significant added value for the risk analysts, for what may be only a modest additional investment in time.