Mixed-Input Residual Network for Air Temperature Forecasting by Low-Power Embedded Devices

  • Kayode P. Ayodele EEE, OAU
  • Farouk O. Adekola
  • Abiodun A. Ogunseye
  • Adedayo Abiona
  • Olawale Akinwale
  • Oladoyin Arewa
  • Funmilayo Offiong
  • Alex Olawole
  • Vincent Ajayi

Abstract

Accuracy and model compactness are essential requirements for weather forecasting models designed for operation on low-power embedded devices. This study developed Mixed-Input Residual Network (MIRNet), a compact temperature-forecasting deep neural network model. MIRNet integrates stacked bidirectional long short-term memory layers using concatenated 1-dimensional and 2-dimensional convolutional layers to improve model accuracy. MIRNet was trained and tested on two datasets: one, IfeData, comprising historical weather data from Ile-Ife, Nigeria and the other a standard weather forecasting dataset called the Jena dataset. Training was carried out using 100 epochs of data partitioned in the standard 80:20 ratio, with an adaptable learning rate strategy. The model was tested for Nth-hour-ahead prediction for 1N24; where N are natural numbers, and performance quantified using metrics such as mean absolute percentage error (MAPE) and mean square error (MSE). The model was also implemented on a Raspberry Pi 4 device with a 1.8 GHz 64-bit quad-core ARM Cortex-A72 processor. The model achieved a MSE of 1.00 x 10-3 on the IfeData dataset, and 1.23x10-4 on the Jena dataset for 1-hour ahead forecasting. This is currently the best verifiable result achieved on the Jena dataset by any prediction model globally. For Nth hour ahead forecasting, MIRNet achieved an MSE generally below 2.0x10-3 for all values of N on the standard Jena dataset. The MSE of MIRNet for N-sequential 1-hour ahead and single Nth hour predictions using the Jena dataset reveal quadratic and linear relationships with N respectively. The model compares favourably with existing models for multi-hour predictions. The developed model is compact and has good forecasting properties.   

References

Abdel-Aal, R. E. (2004). Hourly temperature forecasting using abductive networks. Engineering Applications of Artificial Intelligence,17(5):543-556. https://doi.org/10.1016/j.engappai.2004.04.002

Abubakar, A., Chiroma, H., Zeki, A., & Uddin, M. (2016). Utilising key climate element variability for the prediction of future climate change using a support vector machine model. International Journal of Global Warming, 9(20):129-151. https://doi.org/10.1504/IJGW.2016.074952

Al Sadeque, Z. & Bui, F. M. (2020). A Deep Learning Approach to Predict Weather Data Using Cascaded LSTM Network. In IEEE Canadian Conference on Electrical and Computer Engineering. 1-5. https://doi.org/10.1109/CCECE47787.2020.9255716

Chevalier, R. F., Hoogenboom, G., McClendon, R. W., & Paz, J. A. (2011). Support vector regression with reduced training sets for air temperature prediction: A comparison with artificial neural networks. Neural Computing and Applications, 20(1): 151-159. https://doi.org/10.1007/s00521-010-0363-y

Curceac, S., Ternynck, C., Ouarda, T. B. M. J., Chebana, F., & Niang, S. D. (2019). Short-term air temperature forecasting using Nonparametric Functional Data Analysis and SARMA models. Environmental Modelling and Software, 111: 394-408. https://doi.org/10.1016/j.envsoft.2018.09.017.

Debelee, T. G., & Ayano, Y. M. (2022). Deep Learning Models for Audio Processing Applications Under Resource-Constrained Devices: A Survey. In Pan African Conference on Artificial Intelligence, 209-232. Cham: Springer Nature Switzerland.

Dirmeyer, P. A. (2003). The role of the land surface background state in climate predictability. Journal of Hydrometeorology, 4(3): 599-610.

Dixon, M. (2022). Industrial forecasting with exponentially smoothed recurrent neural networks,Technometrics, 64(1): 114-124. https://doi.org/10.1080/00401706.2021.1921035

Fang, X., & Yuan, Z. (2019). Performance enhancing techniques for deep learning models in time series forecasting. Engineering Applications of Artificial Intelligence, 85: 533-542. https://doi.org/10.1016/j.engappai.2019.07.011

Graves, A. & Schmidhuber, J. (2005). Framewise phoneme classification with bidirectional LSTM networks. In Proceedings of IEEE International Joint Conference on Neural Networks, 4:2047-2052. /home/kermorvant/refbase_files/2005/graves/1055_graves_schmidhuber2005.pdf

Haque, E., Tabassum, S., & Hossain, E. (2021). A Comparative Analysis of Deep Neural Networks for Hourly Temperature Forecasting. IEEE Access. 160646-160660. https://doi.org/10.1109/ACCESS.2021.3131533

Hassani, H., Silva, E. S., Gupta, R., & Das, S. (2018). Predicting global temperature anomaly: A definitive investigation using an ensemble of twelve competing forecasting models. Physica A: Statistical Mechanics and Its Applications, 509: 121-139. https://doi.org/10.1016/j.physa.2018.05.147

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 770–778. https://doi.org/10.1109/CVPR.2016.90

Hewage, P., Trovati, M., Pereira, E., & Behera, A. (2021). Deep learning-based effective fine-grained weather forecasting model. Pattern Analysis and Applications, 24(1): 343-366. https://doi.org/10.1007/s10044-020-00898-1

Hou, J., Wang, Y., Zhou, J., & Tian, Q. (2022). Prediction of hourly air temperature based on CNN–LSTM, Geomatics. Natural Hazards and Risk, 13(1): 1962-1986. https://doi.org/10.1080/19475705.2022.2102942

Hossain, M., Rekabdar, B., Louis, S. J., & Dascalu, S. (2015). Forecasting the weather of Nevada: A deep learning approach. In IEEE international joint conference on neural networks (IJCNN). 1-6. https://doi.org/10.1109/IJCNN.2015.7280812

Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700-4708. https://doi.org/10.1109/CVPR.2017.243

Ko, M. S., Lee, K., Kim, J. K., Hong, C. W., Dong, Z. Y., & Hur, K. (2020). Deep concatenated residual network with bidirectional LSTM for one-hour-ahead wind power forecasting. IEEE Transactions on Sustainable Energy, 12(2):1321-1335. https://doi.org/10.1109/TSTE.2020.3043884

Li, C., Zhang, Y., & Zhao, G. (2019). Deep learning with long short-term memory networks for air temperature predictions. In International Conference on Artificial Intelligence and Advanced Manufacturing (AIAM). 243-249. https://doi.org/10.1109/AIAM48774.2019.00056

Li, Y., Lu, X., Xiong, H., Tang, J., Su, J., Jin, B., & Dou, D. (2023). Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution. arXiv preprint arXiv:2301.02068. https://doi.org/10.48550/arXiv.2301.02068

Li, H. C., Deng, Z. Y., Chiang, H. H. (2020). Lightweight and Resource-Constrained Learning Network for Face Recognition with Performance Optimization. Sensors (Basel). 20(21): 6114. doi: 10.3390/s20216114. PMID: 33121101; PMCID: PMC7662273.

Liang, S., Wang, D., Wu, J., Wang, R., & Wang, R. (2021). Method of bidirectional LSTM modelling for the atmospheric temperature. Intelligent Automation and Soft Computing, 30(20): 701-714. https://doi.org/10.32604/iasc.2021.020010

Kumar, R. (2023). Data Mining Techniques for Weather Forecasting. The Visual Computer, 15(5): 56-63.

Ortiz-García, E. G., Salcedo-Sanz, S., Casanova-Mateo, C., Paniagua-Tineo, A., & Portilla-Figueras, J. A. (2012). Accurate local very short-term temperature prediction based on synoptic situation Support Vector Regression banks. Atmospheric Research 107: 1- 8. https://doi.org/10.1016/j.atmosres.2011.10.013

Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11): 2673-2681. https://doi.org/10.1109/78.650093

Sekertekin, A., Bilgili, M., Arslan, N., Yildirim, A., Celebi, K., & Ozbek, A. (2021). Short-term air temperature prediction by adaptive neuro-fuzzy inference system (ANFIS) and long short-term memory (LSTM) network. Meteorology and Atmospheric Physics, 133:943-959. https://doi.org/10.1061/(ASCE)IR.1943-4774.0000298

Shen, Z., Zhang, Y., Lu, J., Xu, J., & Xiao, G. (2020). A novel time series forecasting model with deep learning. Neurocomputing, 396: 302–313. https://doi.org/10.1016/j.neucom.2018.12.084

Smith, B. A., McClendon, R. W., & Hoogenboom, G. (2006). Improving air temperature prediction with artificial neural networks. International Journal of Computational Intelligence, 3(3):179-186.

Smith, B. A., Hoogenboom, G., & McClendon, R. W. (2009). Artificial neural networks for automated year-round temperature prediction, Computers and Electronics in Agriculture, 68(1): 52-61. https://doi.org/10.1016/j.compag.2009.04.003

Utku, A., & Can, U. (2023). An efficient hybrid weather prediction model based on deep learning. International Journal of Environmental Science and Technology. 1-14. https://doi.org/10.1007/s13762-023-05092-4

Wang, J., Lin, L., Teng, Z., & Zhang, Y. (2022). Multitask Learning Based on Improved Uncertainty Weighted Loss for Multi-Parameter Meteorological Data Prediction. Atmosphere, 13(6): 989. https://doi.org/10.3390/atmos13060989

Zhang, K., Sun, M., Han, T. X., Yuan, X., Guo, L., & Liu, T. (2018). Residual Networks of Residual Networks: Multilevel Residual Networks. IEEE Transactions on Circuits and Systems for Video Technology, 28(6), 1303–1314. https://doi.org/10.1109/TCSVT.2017.2654543

Zhang, F., Gao, X., Zhang, S., Wang, Q., & Lin, L. (2021). Atmospheric Environment Data Generation Method Based on Stacked LSTM-GRU. 15th IEEE International Conference on Electronic Measurement & Instruments (ICEMI). 17-26. https://doi.org/10.1109/icemi52946.2021.9679551

Zhao, L., Wang, J., Li, X., Tu, Z., & Zeng, W. (2016). On the connection of deep fusion to ensembling. arXiv preprint arXiv:1611.07718.

Published
2024-03-29