Cilt 01, Sayı 1, Sayfalar 1 - 6 2017-08-22

Multi-layer Perceptron and Pruning

Cyril Voyant [1] , Christophe Paoli [2] , Marie-Laure Nivet [3] , Gilles Notton [4] , Alexis Fouilloy [5] , Fabrice Motte [6]

106 70

A Multi-Layer Perceptron (MLP) defines a family of artificial neural networks often used in TS modeling and forecasting. Because of its “black box” aspect, many researchers refuse to use it. Moreover, the optimization (often based on the exhaustive approach where “all” configurations are tested) and learning phases of this artificial intelligence tool (often based on the Levenberg-Marquardt algorithm; LMA) are weaknesses of this approach (exhaustively and local minima). These two tasks must be repeated depending on the knowledge of each new problem studied, making the process, long, laborious and not systematically robust. In this short communication, a pruning process is presented. This method allows, during the training phase, to carry out an inputs selecting method activating (or not) inter-nodes connections in order to verify if forecasting is improved. We propose to use iteratively the popular damped least-squares method to activate inputs and neurons. A first pass is applied to 10% of the learning sample to determine weights significantly different from 0 and delete other. Then a classical batch process based on LMA is used with the new MLP. The validation is done using 25 measured meteorological TS and cross-comparing the prediction results of the classical LMA and the 2-stage LMA.

ANN,Forecasting,Time series,Meteorology
  • C. Voyant, M. Muselli, C. Paoli and M.L. Nivet. "Numerical weather prediction (NWP) and hybrid ARMA/ANN model to predict global radiation", Energy, vol. 39, no. 1, (2012), pp. 341–55. doi:10.1016/j.energy.2012.01.006.
  • C. Voyant, G. Notton, C. Paoli, M.L. Nivet, M. Muselli and K. Dahmani, "Numerical weather prediction or stochastic modeling: an objective criterion of choice for the global radiation forecasting", International Journal of Energy Technology and Policy, vol. 12, no. 3, (2014), pp. 01-28.
  • A. Mellit, S.A. Kalogirou, L. Hontoria and S. Shaari, "Artificial intelligence techniques for sizing photovoltaic systems: A review", Renewable and Sustainable Energy Reviews, vol. 13, no. 2, (2009), pp. 406–419.
  • G. Cybenko, "Approximation by superpositions of a sigmoidal function", Mathematics of Control, Signals and Systems, vol. 2, no. 4, (1989), pp.303–314.
  • J. Fan and J. Pan, "A note on the Levenberg–Marquardt parameter", Applied Mathematics and Computation, vol. 207, no. 2, (2009), pp. 351-359.
  • C.K. Yoo, S.W. Sung and I-B. Lee, "Generalized damped least squares algorithm", Computers & Chemical Engineering, vol. 27, no. 3, (2003), pp. 423–431.
  • C. Voyant, W.W. Tamas, C. Paoli, A. Balu, M. Muselli, M.L. Nivet and G. Notton, "Time series modeling with pruned multi-layer perceptron and 2-stage damped least-squares method", 2nd International Conference on Mathematical Modeling in Physical Sciences 2013 (IC-MSQUARE 2013), Prague, Czech Republic, (2013), September 1–5.
  • C. Voyant, W.W. Tamas, M.L. Nivet, G. Notton, C. Paoli, A. Balu and M. Marc, "Meteorological time series forecasting with pruned multi-layer perceptron and two-stage Levenberg-Marquardt method", International Journal of Modelling, Identification and Control, vol. 23, no. 3, (2015), pp. 287-294.
  • H. Brusset, D. Depeyre, J-P. Petit and F. Haffner, "On the convergence of standard and damped least squares methods", Journal of Computational Physics, vol. 22, no. 4, (1976), pp. 534–542.
  • J-P. Kreiss and E. Paparoditis, "Bootstrap methods for dependent data: A review", Journal of the Korean Statistical Society, vol. 40, no.4, (2011), pp. 357–378.
  • F.M. Dias, A. Antunes, J. Vieira and A. Mota, "A sliding window solution for the on-line implementation of the Levenberg–Marquardt algorithm", Engineering Applications of Artificial Intelligence, vol. 19, no. 1, (2006), pp. 1–7.
Konular Matematik ve İstatistik
Dergi Bölümü Articles
Yazarlar

Orcid: orcid.org/0000-0003-0242-7377
Yazar: Cyril Voyant
E-posta: voyant@univ-corse.fr
Ülke: France


Orcid: orcid.org/0000-0002-3093-1119
Yazar: Christophe Paoli
E-posta: christophe.paoli@univ-corse.fr

Yazar: Marie-Laure Nivet
E-posta: marie-laure.nivet@univ-corse.fr

Yazar: Gilles Notton
E-posta: gilles.notton@univ-corse.fr

Yazar: Alexis Fouilloy
E-posta: fouilloy_a@univ-corse.fr

Yazar: Fabrice Motte
E-posta: motte@univ-corse.fr

Bibtex @araştırma makalesi { forecasting306808, journal = {Turkish Journal of Forecasting}, issn = {}, address = {Giresun University Forecast Research Laboratory}, year = {2017}, volume = {01}, pages = {1 - 6}, doi = {}, title = {Multi-layer Perceptron and Pruning}, language = {en}, key = {cite}, author = {Nivet, Marie-Laure and Notton, Gilles and Motte, Fabrice and Voyant, Cyril and Paoli, Christophe and Fouilloy, Alexis} }
APA Voyant, C , Paoli, C , Nivet, M , Notton, G , Fouilloy, A , Motte, F . (2017). Multi-layer Perceptron and Pruning. Turkish Journal of Forecasting, 01 (1), 1-6. Retrieved from http://dergipark.gov.tr/forecasting/issue/28688/306808
MLA Voyant, C , Paoli, C , Nivet, M , Notton, G , Fouilloy, A , Motte, F . "Multi-layer Perceptron and Pruning". Turkish Journal of Forecasting 01 (2017): 1-6 <http://dergipark.gov.tr/forecasting/issue/28688/306808>
Chicago Voyant, C , Paoli, C , Nivet, M , Notton, G , Fouilloy, A , Motte, F . "Multi-layer Perceptron and Pruning". Turkish Journal of Forecasting 01 (2017): 1-6
RIS TY - JOUR T1 - Multi-layer Perceptron and Pruning AU - Cyril Voyant , Christophe Paoli , Marie-Laure Nivet , Gilles Notton , Alexis Fouilloy , Fabrice Motte Y1 - 2017 PY - 2017 N1 - DO - T2 - Turkish Journal of Forecasting JF - Journal JO - JOR SP - 1 EP - 6 VL - 01 IS - 1 SN - - M3 - UR - Y2 - 2017 ER -
EndNote %0 Turkish Journal of Forecasting Multi-layer Perceptron and Pruning %A Cyril Voyant , Christophe Paoli , Marie-Laure Nivet , Gilles Notton , Alexis Fouilloy , Fabrice Motte %T Multi-layer Perceptron and Pruning %D 2017 %J Turkish Journal of Forecasting %P - %V 01 %N 1 %R %U