Training an artificial neural network using the PSOJaya hybrid optimization algorithm

E.M. Kazakova

---

Abstract: Particle swarm optimization (PSO) and Jaya are heuristic optimization algorithms that are used to find optimal solutions in optimization problems. Each of these methods has its own advantages and disadvantages, and the choice between them depends on the specific optimization problem and performance requirements. This paper proposes a hybrid of PSO and Jaya algorithms to improve optimization efficiency. In this paper PSO, Jaya, PSOJaya are used as artificial neural network (ANN) training methods for the classification task on the Balance Scale dataset. The results of the hybrid algorithm are compared with the results of the Backpropagation, PSO and Jaya algorithms. The test calculations compare the algorithms based on the mean, median, standard deviation, and “best” minimum error value after 30 simulations. The experiment results show that the ANN trained with PSOJaya has higher accuracy than the one trained with Backpropagation, PSO and Jaya.

Keywords: heuristic algorithm, optimization, particle swarm method (PSO), Jaya, Backpropagation, hybrid algorithm, pipeline hybridization, ANN, classification

For citation. Kazakova E.M. Training an artificial neural network using the PSOJaya hybrid optimization algorithm. News of the Kabardino-Balkarian Scientific Center of RAS. 2023. No. 6(116). Pp. 95–102. DOI: 10.35330/1991-6639-2023-6-116-95-102

References

1. Kennedy J., Eberhart R. Particle Swarm Optimization. IEE International Conference on Neural Networks. 1995. Pp. 1942–1948. DOI: 10.1109/ICNN.1995.488968
2. Rao R. Jaya: A simple and new optimization algorithm for solving constrained and unconstrained optimization problems. Int J Indus Eng Comput. 2016. Vol. 1. No. 7. Pp. 19–34. DOI: 10.5267/j.ijiec.2015.8.004
3. Karpenko A.P. Sovremennyye algoritmy poiskovoy optimizatsii. Algoritmy, vdokhnovlennyye prirodoy [Modern search engine optimization algorithms. Algorithms inspired by nature].Moscow: MGTU im. N. E. Baumana, 2017. 446 p. (In Russian)
4. Garg H. A hybrid PSO-GA algorithm for constrained optimization problems. Applied Mathematics and Computation. 2016. Vol. 274. Pp. 292–305. DOI: 10.1016/j.amc.2015.11.001
5. Mirjalili S., Hashim S.-Z.M. A new hybrid PSOGSA algorithm for function optimization. Proceedings of ICCIA 2010-2010 International Conference on Computer and Information Application. 2010. Pp. 374–377. DOI: 10.1109/ICCIA.2010.6141614
6. Şenel F.A., Gökçe F. Yüksel A.S. et al. A novel hybrid PSO–GWO algorithm for optimization problems. Engineering with Computers. 2019. Vol. 35. Pp. 1359–1373. DOI: 10.1007/s00366-018-0668-5
7. Zhou Y., Shengyu P.A. Hybrid Co-evolutionary particle swarm optimization algorithm for solving constrained engineering design problems. J. Comput. 2010. Vol. 6. No. 5. Pp. 965–972. DOI: 10.4304/jcp.5.6.965-972
8. Mirjalili S.A., Hashim S.-Z.M., Sardroudi H.M. Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Applied Mathematics and Computation. 2012. Vol. 22. No. 218. Pp. 11125–11137. DOI: 10.1016/j.amc.2012.04.069
9. Junior F.-E.F., Yen G.G. Particle swarm optimization of deep neural networks architectures for image classification. Swarm and Evolutionary Computation. Vol. 49. 2019. Pp. 62–74. DOI: 10.1016/j.swevo.2019.05.010
10. Garro B.A., Vázquez R.A. Designing artificial neural networks using particle swarm optimization algorithms. Computational intelligence and neuroscience. 2015. P. 61. DOI: 10.1155/2015/369298
11. Zhang J.R., Zhang J., Lok T.M., Lyu M.R. A hybrid particle swarm optimization–backpropagation algorithm for feedforward neural network training. Applied mathematics and computation. 2007. Vol. 2. No. 185. Pp. 1026–1037.
12. Siegler R. Balance Scale. UCI Machine Learning Repository. 1994. https://doi.org/10.24432/C5488X.

Information about the author

Kazakova Elena Musovna, Junior Researcher of the Department of Neuroinformatics and Machine Learning, Institute of Applied Mathematics and Automation – branch of Kabardino-Balkarian Scientific
Center of the Russian Academy of Sciences;
360000, Russia, Nalchik, 89 A Shortanov street;
shogenovae@inbox.ru, ORCID: https://orcid.org/0000-0002-5819-9396