Fast Sharpness-Aware Minimization for Time Series Classification and Forecasting

Although various deep learning architectures have been developed to capture long-term dependency on time series data, they still have weaknesses, such as overfitting and computational time.The recently proposed training method called Sharpness-Aware Minimization (SAM) improves generalization in various domains by minimizing a perturbed loss, the maximum loss within the nearby parameter space. However, SAM requires double (2$\times$) training time to calculate the perturbed loss because of additional forward and backward steps. This computational overhead hinders the applications of time series models in practical scenarios such as real-time assessment.In this study, we first prove that sharpness-aware training helps generalization in time series data. Then, to reduce the computational burden, we demonstrate that the periodic characteristics of time series data can be used in approximating the perturbed loss. To this end, we propose a novel sharpness-aware training, Cost-Free Sharpness-Aware Minimization (CF-SAM), without additional forward and backward computations by calculating perturbed loss with the previous information.Empirically, the proposed method improves generalization in time series classification and forecasting without the heavy computational overhead