![]() Movements Since we’re going to implement a DQN, we need to consider the action space. # Connect to the AirSim Drone client = airsim.MultirotorClient() nfirmConnection() client.enableApiControl(True) client.armDisarm(True) client.takeoffAsync().join() This framework will connect to the drone inside of the UE environment with the AirSim library, performing movements, and performing the required steps required by the TF-agents. RL Environment The first thing to implement, is the RL environment. UE4 AI ACTIONS INSTALLpip install airsim pip install tensorflow-gpu=2.4.0 pip install tf-agents Training this agent and running the simulated environment is resource heavy, which is why I used the GPU version of Tensorflow, but you can use the CPU version as well. However, since it is built on Tensorflow, we also want to install that. Next to this, we will want to use TF-Agents, which is also a Python library. Being able to communicated with the drone inside of the environment is the Python AirSim library. Next to this, even though TF-Agents simplifies a lot of the RL problem, when we provide our own environment, we need to make sure that the required methods have been implemented. Even though I managed to create a simulation environment, I still need my agent, which I will implement in Python, to communicate with the drone inside of this environment. The Blocks Environment with a person walking around (sped up) RL Framework Finally, in order to use the quadcopter in this newly created environment, AirSim needs to be added to this environment as a plugin. Luckily, once the person is placed in the environment, and a standard animation is chosen, creating a path for the person to follow is trivial. Adding this to the Blocks environment, we can have a person walk around according to a predetermined path. This pack includes a bunch of different people accompanied with different animations allowing me to move them in different ways. For this, I found the following Unreal Pack in the marketplace: Scanned 3D People Pack. However, this environment has no people walking around in it, which, in order for a drone to train, is quite crucial. We’ll use this to test whether we are able to make a drone follow a person in an extremely simple environment. In this case, I have chosen for the accompanied Blocks environment that the AirSim repository includes. We can’t have a physical drone try to perform a bunch of random actions to see which one works and which doesn’t, for… obvious reasons…Īs I described earlier, using Unreal Engine allows us to use pre-made environments from the Marketplace where we can instantiate an AirSim quadcopter. UE4 AI ACTIONS TRIALFirst, RL requires a lot of training data and therefore: trial and error. However, if I want to implement this, there are a couple of topics to keep in mind. Having shown promising abilities to perform autonomous behavior in a variety of situations where certain tasks need to be performed, this method could apply very well to this specific task. The approach that I decided to investigate, is the possibility of using Reinforcement Learning (RL) as a means to allow an agent to train itself to perform the desired behavior. Again, since autonomy and the decision making process are important here, we need a generalizable algorithm that can learn independently.įor my internship at ML6, I was given the opportunity to study ways in which to implement a drone that can perform follow-me behavior. Such behavior can also lead to the ability for a drone to accurately follow a person through different types of environments. Many of such tasks require accurate object tracking, but also autonomous decision making in uncertain and unpredictable situations. Their implementation is helpful in examples such as package delivery, maintenance work but also entertainment. Microsoft AirSim in control of a quadcopter inside of an Unreal Engine environmentĭrones are becoming increasingly popular in a variety of applications. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |