Can the Content of Public News be used to Forecast Abnormal Stock Market Behaviour?

A popular theory of markets is that they are efficient: all available information is deemed to provide an accurate valuation of an asset at any time. In this paper, we consider how the content of market-related news articles contributes to such information. Specifically, we mine news articles for terms of interest, and quantify this degree of interest. We then incorporate this measure into traditional models for market index volatility with a view to forecasting whether the incidence of interesting news is correlated with a shock in the index, and thus if the information can be captured to value the underlying asset. We illustrate the methodology on stock market indices for the USA, the UK, and Australia.