Discover Interesting Patterns for Investment Decision Making with GLOWER- A Genetic Learner Overlaid With Entropy Reduction

Prediction in financial domains is notoriously difficult for a number ofreasons. First, theories tend to be weak or non-existent, which makesproblem formulation open ended by forcing us to consider a large numberof independent variables and thereby increasing the dimensionality ofthe search space. Second, the weak relationships among variables tend tobe nonlinear, and may hold only in limited areas of the search space.Third, in financial practice, where analysts conduct extensive manualanalysis of historically well performing indicators, a key is to findthe hidden interactions among variables that perform well incombination. Unfortunately, these are exactly the patterns that thegreedy search biases incorporated by many standard rule learningalgorithms will miss. In this paper, we describe and evaluate severalvariations of a new genetic learning algorithm (GLOWER) on a variety ofdata sets. The design of GLOWER has been motivated by financialprediction problems, but incorporates successful ideas from treeinduction and rule learning. We examine the performance of severalGLOWER variants on two UCI data sets as well as on a standard financialprediction problem (S&P500 stock returns), using the results toidentify one of the better variants for further comparisons. Weintroduce a new (to KDD) financial prediction problem (predictingpositive and negative earnings surprises), and experiment with GLOWER,contrasting it with tree- and ruleinduction approaches. Our results areencouraging, showing that GLOWER has the ability to uncover effectivepatterns for difficult problems that have weak structure and significant nonlinearities.