Energy Efficient Big Data Processing: A Comprehensive Survey on Techniques, Trends, and Future Directions

Abstract The exponential rise in big data has resulted in higher energy requirements in data processing frameworks, which present a major environmental and practical concern. As the amount of data being generated grows, cost effective and energy efficient big data processing has become critical. This paper reviews different techniques that improve energy efficiency in big data processing from hardware level optimization, software level adaptation and data level optimization. Proposed and implemented low power processors and energy aware storage; energy efficient scheduling; data compression; and data reduction strategies such as edge computing have been found to be effective in the energy management of big data processing. Other new paths include artificial intelligence based energy management and green data centers. The goal of this survey is to give an overview of the existing situation, show examples of the implementation of energy-efficient BD processing frameworks, and point out the possible directions for their further development.