Evaluation of methods for exploring ill-defined problems

Methods for exploring ill-defined problems are now being developed. Claims for these methods are frequently based on analogies with optimization processes. More persuasive backing for these claims can be obtained by conducting experiments that consider both the analyst or decision-maker and the method. The most fruitful experimental design is for each subject to apply one method to one problem. Solution quality is then assessed by external judges, using the solutions from all the subjects as a source of knowledge about the ill-defined problem.