A general methodology for evaluation of fuzzy rules extracted from data is presented. Though the primary goal of most data mining systems is high classification or prediction accuracy, the user may be interested in rules which are not necessarily the most accurate. Our approach provides an alternative measure of rule validity, based on methods of fuzzy set theory. When the rules to be tested come from a human expert, the method can be viewed as a verification-based data mining method. If the rules are generated by another (discovery-based) data mining method (such as a decision-tree algorithm), the method can be seen as a post-processing step in the KDD process, aimed at evaluating the extracted rules. The method involves four major steps: hypothesis formulation, data selection, hypothesis testing, and decision. In the hypothesis formulation step, a set of fuzzy rules are created using conjunctive antecedents and consequent functions. In the data selection step, a subset of all data in the database is chosen as a sample set. This sample should contain enough records to be representative of the data to a certain degree of user satisfaction. In hypothesis testing, a fuzzy implication is applied to the selected data for each extracted rule and the results are combined using some aggregation function. These results are used in the final step to evaluate the validity of each rule. The presented technique is applied to the rules generated by the C4.5 algorithm from two sample databases. The experimental results demonstrate potential benefits of using validity-based evaluation of rules.
