An accuracy-level method for robust evaluation in predictive analytics

Evaluation metrics are essential in selecting the most appropriate predictive model. Conventional metrics such as R-squared, root mean squared error (RMSE), and mean absolute percentage error (MAPE) rely on arithmetic averages, making them sensitive to outliers, undefined values, and scale differences. This study introduces a novel accuracy-level approach for evaluating continuous data models, using a formula derived from the combination of the "counted" concept in metrics for discrete data and the "error" concept in metrics for continuous data. This formula has not been proposed in other metrics for continuous data, which typically rely on arithmetic or average-based approaches. The second novelty is the introduction of an accuracy-level method, derived from error thresholds for each level. Unlike robust metrics, which trim some observations from the dataset, the proposed metrics incorporate all observations in the dataset, offering a more inclusive evaluation. Four novel metrics are introduced: counted squared error (CSE), counted absolute error (CAE), counted absolute percentage error (CAPE), and symmetric counted absolute percentage error (SCAPE). These metrics assess model performance based on the accuracy value for each level, providing a more robust and interpretable evaluation. We compare the proposed metrics not only with conventional metrics but also with robust alternatives, including median absolute error, trimmed mean squared error, Huber loss, and quantile loss. In regression models with ten outlier scenarios, the proposed metrics show accuracy patterns consistent with robust metrics and different from average-based metrics. In imputation models, both proposed and robust metrics consistently identify predictive mean matching (PMM) as the optimal model, while average-based metrics yield inconsistent results. Interestingly, in time series models, robust metrics misleadingly favor ARIMA, whereas both the proposed and conventional metrics select Holt-Winters as the most appropriate. Overall, this is the first study to extend counted accuracy from discrete to continuous predictive models, introducing a robust, outlier-resistant, and universally interpretable 0%-100% framework for model evaluation.