Methods of artificial intelligence in tasks of managing the robotic and mechatronic systems: review

Development of robotics and mechatronics creates new problems for engineers and attracts the artificial intelligence methods to solve them. It is natural that the complex management problems require modern solutions. The paper presents an overview of the latest developments in managing the mechatronic systems. It demonstrates connection between methods of the classical theory of automatic control and the machine learning. The well-known methods of the classical management theory are briefly described. They include optimization, adaptation and fuzzy logic, and form the basis for artificial neural networks and reinforcement learning. The latest achievements in the intelligent management application for solving the current problems in various technology areas are presented. Literature analysis shows that future research is aimed at the increasing degree in automation and autonomy of the control objects, while their functional properties and nature should approach the human intelligence.

References

[1] Vepa R. A review of techniques for machine learning of real-time control strategies. Intell. Syst. Eng., 1993, vol. 2, no. 2, pp. 77–90, doi: https://doi.org/10.1049/ise.1993.0009

[2] Mitchell T. Machine learning. McGraw-Hill, 1997.

[3] Mosheiov G. Scheduling problems with a learning effect. Eur. J. Oper. Res., 2001, vol. 132, no. 3, pp. 687–693, doi: https://doi.org/10.1016/S0377-2217(00)00175-2

[4] Gurel S., Akturk M. Scheduling preventive maintenance on a single CNC machine. Int. J. Prod. Res., 2008, vol. 46, no. 24, pp. 6797–6821, https://doi.org/10.1080/00207540701487833

[5] Dittrich M.-A., Uhlich F., Denkena B. Self-optimizing tool path generation for 5-axis machining processes. CIRP J. Manuf. Sci. Technol., 2019, vol. 24, pp. 49–54, doi: https://doi.org/10.1016/j.cirpj.2018.11.005

[6] Fomin V.N., Fradkov A.L., Yakubovich V.A. Adaptivnoe upravlenie dinamicheskimi obektami [Adaptive control on dynamics objects]. Moscow, Nauka Publ., 1981. 447 p. (In Russ.).

[7] Annaswamy A.M., Fradkov A.L. A historical perspective of adaptive control and learning. Annu. Rev. Control, 2021, vol. 52, pp. 18–41, doi: https://doi.org/10.1016/j.arcontrol.2021.10.014

[8] Perelman I.I. Analysis of current adaptive control methods from the standpoint of the application to the automation of technological processes. Avtomatika i telemekhanika, 1991, no. 7, pp. 3–32. (In Russ.). (Eng. version: Autom. Remote Control, 1991, vol. 52, no. 7, pp. 891–911).

[9] Andrievskiĭ B.R., Fradkov A.L. Speed gradient method and its applications. Avtomatika i telemekhanika, 2021, no. 9, pp. 3–72, doi: https://doi.org/10.31857/S0005231021090014 (In Russ.). (Eng. version: Autom. Remote Control., 2021, vol. 82, no. 9, pp. 1463–1518, doi: https://doi.org/10.1134/S0005117921090010)

[10] Polyak B.T. Vvedenie v optimizatsiyu [Introduction into optimization]. Moscow, Nauka Publ., 1983. 384 p. (In Russ.).

[11] Karpenko A.P. Sovremennye algoritmy poiskovoy optimizatsii [Modern algorithms of search optimization]. Moscow, Bauman MSTU Publ., 2014. 446 p. (In Russ.).

[12] Nikolenko S., Kadurin A., Arkhangelskaya E. Glubokoe obuchenie. Pogruzhenie v mir neironnykh setei [Deep learning. Diving into the world of neural networks]. Sankt-Petersburg, Piter Publ., 2021. 476 p. (In Russ.).

[13] Zadeh L. Ponyatie lingvisticheskoi peremennoi i ego primenenie k prinyatiyu priblizhennykh reshenii [The concept of a lingustic variable and its application to approximate reasoning]. Moscow, Mir Publ., 1976. 165 p. (In Russ.).

[14] Novák V., Perfilieva I., Mockor J. Mathematical principles of fuzzy logic. Springer, 1999. 320 p. (Russ. ed.: Matematicheskie printsipy nechetkoi logiki. Moscow, Fizmatlit Publ., 2006. 347 p.)

[15] Cybenko G. Approximation by superpositions of a sigmoidal function. Math. Control Signal. Systems, 1989, vol. 2, no. 4, pp. 303–314, doi: https://doi.org/10.1007/BF02551274

[16] Lee J., Shin J., Realff M. Machine learning: overview of the recent progresses and implications for the process systems engineering field. Comput. Chem. Eng., 2018, vol. 114, pp. 111–121, doi: https://doi.org/10.1016/j.compchemeng.2017.10.008

[17] Sutton R.S., Barto A.G. Reinforcement learning. MIT Press, 2018. 552 p.

[18] Buoniu L., Babuska R., De Schutter B. et al. Reinforcement learning and dynamic programming using function approximators. CRC Press, 2010. 280 p.

[19] Kurrek P., Jocas M., Zoghlami F. et al. AI motion control — a generic approach to develop control policies for robotic manipulation tasks. Proc. Int. Conf. Engineering Design, 2019, vol. 1, no. 1, pp. 3561–3570, doi: https://doi.org/10.1017/dsi.2019.363

[20] Menghal P., Laxmi A. Real time simulation: a novel approach in engineering education. ICECT, 2011, vol. 1, pp. 215–219, doi: https://doi.org/10.1109/ICECTECH.2011.5941592

[21] Powell W. Approximate dynamic programming. Wiley, 2011. 656 p.

[22] Bertsekas D., Tsitsiklis J. Neuro-dynamic programming: an overview. Proc. 34th IEEE Conf. on Decision and Control, 1995, vol. 1, pp. 560–564, doi: https://doi.org/10.1109/CDC.1995.478953

[23] Prokhorov D., Wunsch D. Adaptive critic designs. IEEE Trans. Neural Netw., 1997, vol. 8, no. 5, pp. 997–1007, doi: https://doi.org/10.1109/72.623201

[24] Werbos P.J. Reinforcement learning and approximate dynamic programming (RLADP)—foundations, common misconceptions, and the challenges ahead. In: Reinforcement learning and approximate dynamic programming for feedback control. Wiley, 2012, pp. 1–30.

[25] Lewis F., Vrabie D. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag., 2009, vol. 9, no. 3, pp. 32–50, doi: https://doi.org/10.1109/MCAS.2009.933854

[26] Lopes G., Ferreira M., Da Silva Simoes A. et al. Intelligent control of a quadrotor with proximal policy optimization reinforcement learning. Proc. LARS/SBR/WRE, 2018, pp. 503–508, doi: https://doi.org/10.1109/LARS/SBR/WRE.2018.00094

[27] Kim J., Hong M., Lee K. et al. Learning to walk a tripod mobile robot using nonlinear soft vibration actuators with entropy adaptive reinforcement learning. IEEE Robot. Autom. Lett., 2020, vol. 5, no. 2, pp. 2317–2324, doi: https://doi.org/10.1109/LRA.2020.2970945

[28] Puccetti L., Kopf F., Rathgeber C. et al. Speed tracking control using online reinforcement learning in a real car. ICCAR, 2020, pp. 392–399, doi: https://doi.org/10.1109/ICCAR49639.2020.9108051

[29] Sun L., You F. Machine learning and data-driven techniques for the control of smart power generation systems: an uncertainty handling perspective. Engineering, 2021, vol. 7, no. 9, pp. 1239–1247, doi: https://doi.org/10.1016/j.eng.2021.04.020

[30] Tomin N., Kurbatsky V., Guliyev H. Intelligent control of a wind turbine based on reinforcement learning. ELMA, 2019, doi: https://doi.org/10.1109/ELMA.2019.8771645

[31] Sierra-Garcia J.E., Santos M., Pandit R. Wind turbine pitch reinforcement learning control improved by PID regulator and learning observer. Eng. Appl. Artif. Intell., 2022, vol. 111, art. 104769, doi: https://doi.org/10.1016/j.engappai.2022.104769

[32] Gunther J., Pilarski P., Helfrich G. et al. Intelligent laser welding through representation, prediction, and control learning: an architecture with deep neural networks and reinforcement learning. Mechatronics, 2016, vol. 34, pp. 1–11, doi: https://doi.org/10.1016/j.mechatronics.2015.09.004

[33] Lin W.-S., Yang P.-C. Adaptive critic motion control design of autonomous wheeled mobile robot by dual heuristic programming. Automatica, 2008, vol. 44, no. 11, pp. 2716–2723, doi: https://doi.org/10.1016/j.automatica.2008.03.029

[34] Calise A., Rysdyk R. Nonlinear adaptive flight control using neural networks. IEEE Control Syst. Mag., 1998, vol. 18, no. 6, pp. 14–25, doi: https://doi.org/10.1109/37.736008

[35] Kruger T., Schnetter P., Placzek R. et al. Fault-tolerant nonlinear adaptive flight control using sliding mode online learning. Neural Netw., 2012, vol. 32, pp. 267–274, doi: https://doi.org/10.1016/j.neunet.2012.02.025

[36] Lin F.-J., Hung Y.-C., Chen S.-Y. Field-programmable gate array-based intelligent dynamic sliding-mode control using recurrent wavelet neural network for linear ultrasonic motor. IET Control. Theory Appl., 2010, vol. 4, no. 9, pp. 1511–1532, doi: https://doi.org/10.1049/iet-cta.2009.0066

[37] Zhu D., Mei T., Luo M. Adaptive sliding mode control for robots based on fuzzy support vector machines. ICMA, 2009, pp. 3469–3474, doi: https://doi.org/10.1109/ICMA.2009.5246264

[38] Perez J., Cabrera J., Castillo J. et al. Bio-inspired spiking neural network for nonlinear systems control. Neural Netw., 2018, vol. 104, pp. 15–25, doi: https://doi.org/10.1016/j.neunet.2018.04.002

[39] Perez-Pena F., Lenero-Bardallo J., Linares-Barranco A. et al. Towards bioinspired close-loop local motor control: a simulated approach supporting neuromorphic implementations. IEEE ISCAS, 2017, doi: https://doi.org/10.1109/ISCAS.2017.8050808

[40] Keshta H., Ali A., Saied E. et al. Application of Static Var Compensator (SVC) with PI controller for grid integration of wind farm using harmony search. Int. J. Emerg. Electr. Power Syst., 2016, vol. 17, no. 5, pp. 555–566, doi: https://doi.org/10.1515/ijeeps-2016-0159

[41] Atashpaz Gargari E., Hashemzadeh F., Rajabioun R. et al. Colonial competitive algorithm: a novel approach for PID controller design in MIMO distillation column process. Int. J. Intell. Comput. Cybern., 2008, vol. 1, no. 3, pp. 337–355, doi: https://doi.org/10.1108/17563780810893446

[42] Schillinger M., Hartmann B., Skalecki P. et al. Safe active learning and safe Bayesian optimization for tuning a PI-controller. IFAC-PapersOnLine, 2017, vol. 50, no. 1, pp. 5967–5972, doi: https://doi.org/10.1016/j.ifacol.2017.08.1258

[43] Berkenkamp F., Schoellig A., Krause A. Safe controller optimization for quadrotors with Gaussian processes. IEEE ICRA, 2016, pp. 491–496, doi: https://doi.org/10.1109/ICRA.2016.7487170

[44] Malikopoulos A., Papalambros P., Assanis D. A learning algorithm for optimal internal combustion engine calibration in real time. Proc. DETC2007, 2008, vol. 6-A, pp. 91–100, doi: https://doi.org/10.1115/DETC2007-34718

[45] Liang D., Ding W. Modelling and predicting of a switched reluctance motor drive using radial basis function network-based adaptive fuzzy system. IET Electr. Power Appl., 2009, vol. 3, no. 3, pp. 218–230, doi: https://doi.org/10.1049/iet-epa.2008.0096

[46] Elsodany N., Rezeka S., Maharem N. Adaptive PID control of a stepper motor driving a flexible rotor. Alex. Eng. J., 2011, vol. 50, no. 2, pp. 127–136, doi: https://doi.org/10.1016/j.aej.2010.08.002

[47] Mitra P., Maulik S., Chowdhury S. et al. Anfis based automatic voltage regulator with hybrid learning algorithm. Proc. Universities Power Engineering Conf., 2007, pp. 397–401, doi: https://doi.org/10.1109/UPEC.2007.4468980

[48] Chen H.-C. Optimal fuzzy pid controller design of an active magnetic bearing system based on adaptive genetic algorithms. Proc. ICMLC, 2008, vol. 4, pp. 2054–2060, doi: https://doi.org/10.1109/ICMLC.2008.4620744

[49] Wen S., Hu X., Lv X. et al. Q-learning trajectory planning based on Takagi–Sugeno fuzzy parallel distributed compensation structure of humanoid manipulator. Int. J. Adv. Robot. Syst., 2019, vol. 16, no. 1, doi: https://doi.org/10.1177/1729881419830204

[50] Ercin O., Coban R. Identification of linear dynamic systems using the artificial bee colony algorithm. Turk. J. Elec. Eng. & Comp. Sci., 2012, vol. 20, no. 7, pp. 1175–1188, doi: https://doi.org/10.3906/elk-1012-956

[51] Agand P., Shoorehdeli M.A., Khaki-Sedigh A. Adaptive recurrent neural network with Lyapunov stability learning rules for robot dynamic terms identification. Eng. Appl. Artif. Intell., 2017, vol. 65, pp. 1–11, doi: https://doi.org/10.1016/j.engappai.2017.07.009

[52] Sobol I.M., Statnikov R.B. Vybor optimalnykh parametrov v zadachakh so mnogimi kriteriyami [Choice of optimum parameters in multicriteria problems]. Moscow, Drofa Publ., 2006. 175 p. (In Russ.).

[53] Chernykh V.V., Statnikov R.B., Khrapov D.S. et al. Method for investigating the parameter space in problems of passenger car design. Problemy mashinostroeniya i nadezhnosti mashin, 2009, no. 4, pp. 19–25. (In Russ.). (Eng. version: J. Mach. Manuf. Reliab., 2009, vol. 38, no. 4, pp. 329–334, doi: https://doi.org/10.3103/S1052618809040049)

[54] Balochian S., Baloochian H. Social mimic optimization algorithm and engineering applications. Expert Syst. Appl., 2019, vol. 134, pp. 178–191, doi: https://doi.org/10.1016/j.eswa.2019.05.035

[55] Kumar S., Kumar A., Shankar G. Crow search algorithm based optimal dynamic performance control of SVC assisted SMIB system. NPSC, 2018, doi: https://doi.org/10.1109/NPSC.2018.8771814

[56] Abualigah L. Multi-verse optimizer algorithm: a comprehensive survey of its results, variants, and applications. Neural Comput. & Applic., 2020, vol. 32, no. 16, pp. 12381–12401, doi: https://doi.org/10.1007/s00521-020-04839-1

[57] Wang L., Liu Z., Chen C. et al. Support vector machine based optimal control for minimizing energy consumption of biped walking motions. Int. J. Precis. Eng. Manuf., 2012, vol. 13, no. 11, pp. 1975–1981, doi: https://doi.org/10.1007/s12541-012-0260-7

[58] Xiao L., Zhu Q., Li C. et al. Application of modified teaching-learning algorithm in coordination optimization of TCSC and SVC. In: CCPR. Springer, 2014, pp. 44–53, doi: https://doi.org/10.1007/978-3-662-45646-0_5

[59] Shouran M., Habil M. Tuning of PID controller using different optimization algorithms for industrial DC motor. ICACITE, 2021, pp. 756–759, doi: https://doi.org/10.1109/ICACITE51222.2021.9404616

[60] Silvestri M., Pedrazzoli P., Boer C. et al. Compensating high precision positioning machine tools by a self learning capable controller. Proc. EUSPEN, 2011, vol. 2, pp. 121–124.

[61] Tomizuka M. Zero phase error tracking algorithm for digital control. J. Dyn. Sys., Meas., Control., 1987, vol. 109, no. 1, pp. 65–68, doi: https://doi.org/10.1115/1.3143822

[62] Tomizuka M. Dealing with periodic disturbances in controls of mechanical systems. Annu. Rev. Control, 2008, vol. 32, no. 2, pp. 193–199, doi: https://doi.org/10.1016/j.arcontrol.2008.07.002

[63] Uchiyama M. Formation of high-speed motion pattern of a mechanical arm by trial. Transactions of the Society of Instrument and Control Engineers, 1978, vol. 14, no. 6, pp. 706–712, doi: https://doi.org/10.9746/sicetr1965.14.706

[64] Li W., Maier P., Enge H. Self-learning control applied to vibration control of a rotating spindle by piezopusher bearings. Proc. Inst. Mech. Eng., I: J. Syst. Control Eng., 2004, vol. 218, no. 3, pp. 185–196, doi: https://doi.org/10.1243/095965104773913867

[65] Hladowski L., Galkowski K., Rogers E. Further results on dynamic iterative learning control law design using repetitive process stability theory. 10th nDS, 2017, doi: https://doi.org/10.1109/NDS.2017.8070621

[66] Kurek J., Zaremba M. Iterative learning control synthesis based on 2-D system theory. IEEE Trans. Automat. Contr., 1993, vol. 38, no. 1, pp. 121–125, doi: https://doi.org/10.1109/9.186321

[67] Pakshin P., Emelianova J., Emelianov M. Iterative learning control design for multiagent systems based on 2D models. Autom. Remote Control, 2018, vol. 79, no. 6, pp. 1040–1056, https://doi.org/10.1134/S000511791806005X

[68] Seel T., Werner C., Raisch J. et al. Iterative learning control of a drop foot neuroprosthesis — generating physiological foot motion in paretic gait by automatic feedback control. Control. Eng. Pract., 2016, vol. 48, pp. 87–97, doi: https://doi.org/10.1016/j.conengprac.2015.11.007

[69] Seel T., Werner C., Schauer T. The adaptive drop foot stimulator — multivariable learning control of foot pitch and roll motion in paretic gait. Med. Eng. Phys., 2016, vol. 38, no. 11, pp. 1205–1213, doi: https://doi.org/10.1016/j.medengphy.2016.06.009

[70] Mohammadpour A., Mishra S., Parsa L. Iterative learning control for fault-tolerance in multi-phase permanent-magnet machines. Proc. American Control Conf., 2013, pp. 5929–5934, doi: https://doi.org/10.1109/acc.2013.6580768

[71] Li J., Wang S., Wang J. et al. Iterative learning control for a distributed cloud robot with payload delivery. Assem. Autom., 2021, vol. 41, no. 3, pp. 263–273, doi: https://doi.org/10.1108/AA-11-2020-0179

[72] Warrier R., Devasia S. Iterative learning from novice human demonstrations for output tracking. Trans. Hum. Mach. Syst., 2016, vol. 46, no. 4, pp. 510–521, doi: https://doi.org/10.1109/THMS.2016.2545243

[73] Lansdell B., Kording K. Towards learning-to-learn. Curr. Opin. Behav. Sci., 2019, vol. 29, pp. 45–50, doi: https://doi.org/10.1016/j.cobeha.2019.04.005