Reinforcement learning-based target tracking for unmanned aerial vehicle with achievement rewarding and multistage traning

Ahmed Abo Mosali, Najm Addin Mohammed (2022) Reinforcement learning-based target tracking for unmanned aerial vehicle with achievement rewarding and multistage traning. Doctoral thesis, Universiti Tun Hussein Onn Malaysia.

Preview	Text 24p NAJM ADDIN MOHAMMED AHMED ABO MOSALI.pdf Download (1MB) | Preview
	Text (Copyright Declaration) NAJM ADDIN MOHAMMED AHMED ABO MOSALI COPYRIGHT DECLARATION.pdf Restricted to Repository staff only Download (182kB) | Request a copy
	Text (Full Text) NAJM ADDIN MOHAMMED AHMED ABO MOSALI WATERMARK.pdf Restricted to Registered users only Download (2MB) | Request a copy

Abstract

Target tracking using an unmanned aerial vehicle (UAV) is a challenging robotic problem. It requires handling a high level of nonlinearity and dynamics device. The aim is to enable accurate target tracking by UAV with responding to the dynamic generated by the target such as sudden trajectory change using reinforcement learning which is proved to learn dynamic effectively. In this thesis, the Twin Delayed Deep Deterministic Policy Gradient Algorithm (TD3), as one recent and composite architecture of reinforcement learning (RL), has been explored as a tracking agent for the problem of UAV-based target tracking. This involved several improvements on the original TD3. First, the proportional-differential controller was used to boost the exploration of the TD3 in training. Second, a novel reward formulation for the UAV-based target tracking was proposed to enable a careful combination of the various dynamic variables in the reward functions. This was accomplished by incorporating two exponential functions to limit the effect of velocity and acceleration to prevent the deformation in the policy function approximation. Third, the concept of multistage training based on the dynamic variables was proposed as an opposing concept to one-stage combinatory training. Fourth, an enhancement of the rewarding function by including piecewise decomposition was used to enable more stable learning behaviour of the policy and move out from the linear reward to the achievement formula. Fifth, a novel agent selection algorithm was developed to enable the selection of the best agent and avoid under-fitting and over-fitting. For the purpose of evaluating the performance of the control system, flight testing was conducted based on three types of target trajectories, namely fixed, square, and blinking. The evaluation was performed in both simulation and real-world experiments. The results showed that the multistage training achieved the best-accomplished performance with both exponential and achievement rewarding for a fixed trained agent with a fixed and square moving target and for a combinatorial agent with both exponential and achievement rewarding for a fixed trained agent in the case of a blinking target. With respect to the traditional proportional differential (PD) controller, the maximum error reduction rate is 86%. The developed achievement rewarding and the multistage training opens the door to various applications of RL in target tracking.

Item Type:	Thesis (Doctoral)
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK5101-6720 Telecommunication. Including telegraphy, telephone, radio, radar, television
Divisions:	Faculty of Mechanical and Manufacturing Engineering > Department of Mechanical Engineering
Depositing User:	Mrs. Sabarina Che Mat
Date Deposited:	27 Feb 2023 01:03
Last Modified:	27 Feb 2023 01:03
URI:	http://eprints.uthm.edu.my/id/eprint/8452

Actions (login required)

View Item