APRÈN. Portal de la producció docent del professorat de la UPC

Este proyecto exploró la viabilidad de utilizar el Aprendizaje por Refuerzo Profundo (DRL, por sus siglas en inglés) para gestionar el tráfico de drones en un Vertipuerto dedicado a la entrega automatizada de paquetes. Un agente DRL, utilizando el algoritmo Proximal Policy Optimization (PPO), fue entrenado para coordinar múltiples drones simultáneamente, con el objetivo de lograr operaciones seguras y eficientes dentro de un entorno urbano simulado. Se desarrolló un entorno de simulación realista mediante la integración del simulador AirSim con un servidor personalizado, 'DroneManagerServer', construido utilizando el framework FastAPI. Este entorno permitió la creación de escenarios complejos y proporcionó una plataforma para entrenar y evaluar el rendimiento del agente DRL. El proceso de aprendizaje del agente fue guiado por una función de recompensa meticulosamente diseñada que penalizaba las colisiones y las acciones ineficientes, al tiempo que incentivaba la separación segura entre drones y la finalización eficiente de las misiones de entrega. Si bien se demostró el potencial del DRL para la gestión de Vertiports, el proyecto también reveló desafíos asociados con la aplicación del DRL a problemas de control del mundo real. Las limitaciones en el rendimiento del simulador, la complejidad del ajuste de hiperparámetros y las complejidades del diseño de la función de recompensa influyeron en el rendimiento general del agente y limitaron el alcance de la evaluación a un único escenario desafiante. Las principales contribuciones del proyecto incluyen: demostrar con éxito la capacidad del DRL para aprender estrategias complejas de coordinación de drones; identificar desafíos específicos en la aplicación del DRL a la gestión del tráfico de drones (UTM); y desarrollar un framework DRL flexible que puede servir como base para futuras investigaciones. Las futuras direcciones de investigación incluyen: mejorar el entorno de simulación para acomodar mayores densidades de drones y escenarios más realistas; explorar técnicas avanzadas de DRL para mejorar el rendimiento y la escalabilidad; e investigar métodos para mejorar la robustez y las capacidades de generalización del agente. Además, la integración de las regulaciones del U-space y la exploración de enfoques híbridos que combinan DRL con otras técnicas de IA se identifican como vías prometedoras para avanzar en la UTM basada en DRL. Al abordar estas futuras direcciones de investigación, este proyecto tiene como objetivo contribuir al desarrollo de sistemas autónomos robustos y confiables para la gestión del tráfico de drones, facilitando en última instancia la integración segura y eficiente de los drones en el espacio aéreo urbano y desbloqueando todo el potencial de la Movilidad Aérea Urbana.

In this Bachelor's thesis, the main objective is to design an optimal network for Unmanned Aerial Vehicles (UAVs) in complex environments, such as urban areas with obstacles. The goal is to develop a network that optimally connects UAV bases and hospitals in the city of Madrid, while considering various constraints like distance, redundancy, and obstacle avoidance. To achieve this, the project proposes an algorithm for designing optimal networks. This involves designing the graph using either the Rapidly-exploring Random Graph (RRG) or the Probabilistic Roadmap (PRM). Simulated Annealing (SA) is then responsible for the optimization process. The project starts by contextualizing the current situation of drones in the delivery field. Due to the increasing integration of drones into the daily operations of such services, especially in urban logistics, the need for efficient airspace management and network design has arisen. Next, it explains the algorithms used (PRM, RRG and SA) in the literature review. This theoretical background is useful for comprehending the formalization of the final algorithm, which involves defining the problem, setting objectives, identifying variables and constraints, and designing the cost functions. All this leads to the methodology, explaining how the results are obtained. It describes the space in which the network is designed, the combined use of all the algorithms, and specific important processes needed to achieve the results. After understanding the rationale, the objectives, and how the simulations will be carried out, the results are presented. This section is divided into two parts: The first simulations are conducted in predefined scenarios to test the algorithm's performance. Once these tests are completed, the algorithm is applied to the airspace of Madrid. The information about Madrid is extracted from the MUSE Project. The results include not only the UAV network design but also a comparison between PRM and RRG, along with an analysis of the environmental impact of each approach. Finally, before the conclusions, there is also an analysis of the social impact this delivery method could have. The conclusions summarize the key findings and the algorithm's performance. Overall, this project provides a valuable approach to UAV network design, offering a foundation for further research and development in this field.

Artificial intelligence. We have just read two words that are very widespread among all of us nowadays. To a greater or lesser extent, artificial intelligence is present in the day-to-day life of today's society; this has arrived on a large scale and is here to stay. Its advances progress exponentially in many areas. Aviation, then, could not be less. The aeronautical world is constantly evolving, from the earliest days when the first flights were made to today. It is an industry that requires the latest technology; therefore, when a tool as powerful and useful as artificial intelligence appears, engineers aim to implement it and get the most out of it. Recently, a new concept has emerged that is increasingly gaining strength and will be the focus of this work: generative artificial intelligence. It is possible that a considerable part of society has made use of this, whether generating texts, images... And this raises the question... What is the potential of this in the aeronautical framework? Well, this is the main topic that this work investigates, with the aim of creating a complete state-of-the-art. To carry out this work, an exhaustive search will be made for all the necessary information in order to be able to answer all the questions about the state of generative artificial intelligence in aviation (specifically focusing on the air traffic management framework); in addition to contrasting it with the real EASA planning in terms of objectives, dates and safety requirements. Therefore, as the chapters progress, we will be able to find how in this work all the features and topics are analyzed and studied, which will finally lead to the compilation of the information necessary to make this analysis of the state-of-the-art and the potential future.

The rapid growth of the drone industry aims to develop applications and implement them in a wide range of areas. This includes busy urban areas for services such as surveillance, deliveries and monitoring. In this context, it is essential to have an excellent design of the airspace. This thesis focuses on the analysis of a dataset from the Very Large Demonstration project of CORUSXUAM, which contributes to the U-Space mission of developing a safe, sustainable, efficient and fully digitalized airspace for integrated Urban Air Mobility which does not interfere with current ATM operations. The dataset includes flight plans, telemetry, and U-space predictions for 72 drone flights. The analysis involves comparing intended trajectories with actual flight paths to identify factors contributing to deviations from the flight plan and computing relevant performance parameters to assess the adherence of the drones to the flight plan. This is done with the use of dynamic time warping algorithms in order to establish a link between the telemetry points and the flight plan, which sets the basis for the next section of the project. Having processed the data, during this project we develop machine learning models to predict telemetry parameters based on the input flight plan. Several models are tested and evaluated to find the most suitable one for our objective. The project also involves visualizing and interpreting the data to gain insights of the drone performance and adherence to the flight plan. Position prediction opens up a new area of research and in this project the approach is to use an alternative method to define the spacing and size of the airways that compose the flight plans so as to dictate safety areas to prevent any possible conflict that could appear in future flights in a busy area if the spacing were to be below the thresholds. The results of this study demonstrate a successful progression from raw data to a comprehensive analysis, offering valuable insights for evaluating drone performance and predicting flight times. The development of various data visualization functions enabled efficient and effective interpretation of the data. While the obtained results with the available dataset are remarkable, the potential for further improvement lies mainly in acquiring a larger dataset with more features and samples, which would enhance the performance of the machine learning models and yield even more accurate predictions.

Stress and academic anxiety problems can negatively impact numerous aspects of students lives, resulting in degrading their academic achievement, quality of life, and social behaviour. Various research suggests that depression is associated with lower academic performance of students. The aim of this research is twofold. Firstly, in order to establish a correlation between students mental stress level and their academic performance, a dataset has been compiled through gathering the data by conducting a survey in a university located in Punjab, Pakistan. The questionnaires were based on measuring the stress level of students using Perceived Stress Scale (PSS) , Cognitive performance assessment scale, in addition to some other demographic questions. Afterwards, this dataset has been analysed utilizing various machine learning algorithms. The second objective was to develop an innovative, affordable and smart performance predicting Learning Management System that takes into account students mental stress while predicting the students performance using machine learning models. The technique that was used for the mental stress measurements of the students was based on a phenomenon known as the Heart Rate Variability (HRV). A smart watch was utilized to measure the Heart Rate Variability of the students that was used to assess the stress level of students in academics. A Machine Learning (ML) model was trained using various parameters that were derived from the Heart Rate Variability. The original dataset that was used to train the model is known as Swell dataset. The SWELL dataset consists of HRV indices computed from the multimodal SWELL knowledge work dataset for research on stress and user modelling. The ML model effectively made prediction about the stress levels of the students with an accuracy of 98.1%.

The aim of this project is to justify the necessity of a specific airspace dedicated to drone operations, in particular in the Urban Air Mobility (UAM) field, and to expose which aspects are going to be the most limiting in the design of this airspace for autonomous aircrafts. The commercial aviation case is presented to demonstrate that an effective implementation of the UAM requires the creation of a dedicated airspace as well as international legal harmonisation: a mode of air transportation that carries out thousands of flights every day while ensuring high levels of safety at all times thanks to its defined rules and airspace structure. The different reasons why the aviation airspace cannot be escalated to the UAM are also exposed. Once the necessity for the UAM sector to have its own drone-designed airspace has been justified, the main barriers and potential solutions that the experts and corresponding authorities working on the sector have identified are exposed. These challenges come from fields as diverse as operational security, infrastructure and ground area protection, adverse weather conditions, technological and vehicle limitations, and a factor that is often overlooked but is crucial: social acceptance. The establishment of the UAM airspace will require the development and design of a ground infrastructure capable of ensuring that aircraft can takeoff and land safely in each operation. The main European vertiport guidelines are explained. To conclude the project and use the knowledge acquired in its elaboration, a use case in the UAM sector is briefly designed: the transport of VIPs by drone from different Catalan airports to the ¿Circuit de Montmeló¿ in punctual cases, such as when a Grand Prix is held.

Air traffic has been increasing and with it the workload of air traffic controllers. Despite the pandemic, the latest figures show a rapid recovery and forecast exponential growth. This indicates the need to modernise air traffic control and the technology used, which is already being developed and implemented by organisations like SESAR, like applying AI to air traffic control (DART). A support tool with automatic conflict avoidance would be a great step to address the problem of possible overcapacity of air traffic controllers. This document describes two possible implementations of a conflict avoidance tool. The approach is to use Deep Reinforcement Learning to select actions that avoid conflict and help the air traffic controllers to take faster and better decisions. The basis for both approaches is a simple 2D airspace simulator and the same policy applied to all the aircraft. The first proposal is a stand-alone DQN algorithm (DRL) that has a 7.06% improvement in the number of simultaneous conflicts compared to the original environment without applying a policy. The second approach is a DQN algorithm that incorporates transfer learning of the rules of the air, and it is called by the acronym DRLT. It resulted in a degradation compared to the original environment, with a 6% increase in unremembered conflicts. Nevertheless, Deep Reinforcement Learning has shown a decrease in decision time and the idea of reusing the same strategy for all aircraft has solved the problem of unpredictability issue that some reinforcement learning solutions had. The proposal could be a good start for a self-separation tool for unmanned aircraft but still needs future improvements in results. It is not suitable for air traffic controllers or piloted vehicles due to the increased workload it would suppose.

Drones are nowadays an air vehicle with many possible applications and with a huge field of development and research. This work deals, more specifically, with the delivery drones used for the distribution of packages to citizens. That is, they could be considered as a new way of distributing packages to the city. Today, this is seen as a hypothetical, because in order to achieve this, many points must be studied and new tools for their control and security must be developed. This work studies the field of predicting the trajectory of drones through Machine Learning and thus, providing a new point of view in the research and development of this new branch of aviation. The aim of this work is to predict, through Machine Learning, the trajectory of a drone in 4D based on the data collected in the test flights made by the different air operators participating in the CORUS-XUAM. The objective is to achieve results with sufficient precision to be able to project them valid in terms of air safety. To carry out this work, it has been necessary the data previously collected from the VLDs operated and the processing of them by means of the code made in Google Coolab. For this reason, different libraries have been needed, such as Pandas, Matplotlib, Numpy or Geopandas, among others. Finally, a Machine Learning has been used to carry out the predictions. To do this, it has been necessary to decide which will be the best method used, in our case it has been seen that the most appropriate is the Regression model. Regarding the results, it has been confirmed that the chosen method has been correct, since the margins of error are quite low and acceptable and the accuracy of it could be considered good with a margin error of maximum 3 minutes.

Avoiding collisions is one of the main tasks of air traffic control. The rapid increase in air traffic and its continued growth raises the issue of assisting air traffic controllers in high complex traffic scenarios, where safety and efficiency cannot be compromised. Predictions estimated for the future of several decades¿ state that the current air traffic management system will not be able to withstand this increase in demand for air transport with the required levels of safety, efficiency and sustainability. Air traffic management is changing faster and faster towards more advanced and efficient technology, with the integration of automated support tools to assist air traffic controllers. This document presents a method to enhance the performance and efficiency of the air traffic controllers¿ tasks: monitor and manage all aircraft flying through the airspace sector, performed in a centralised way. The objective is design a separation algorithm based on the general Rules of the Air. To achieve the goal of the project, Remain Well Clear and Closest Point of Approach concepts have been used as other sources of information. The policy algorithm is simulated in a controlled simulation environment. Regarding the results obtained at the end of this project, a reduction of the conflicts of a 99,41% is achieved compared with a situation without applying any instructions. The results of this project also include a sensitive analysis of the tuning of parameters. The performance of the policy model could be upgraded in the future applying Artificial Intelligence, so conflicts can be addressed in a much more efficient and accurate way.

Law enforcement agents have to care about the number of people in public areas to ensure security. The problem they have is that they do not have tools to measure the number of people in a fast and precise way. This need has been especially important since 2020 COVID pandemic arrived to our society and the control of people is relevant to avoid spread of COVID. This Master Thesis is complementing other previous Master Thesis presented in 2021 where via an Android app connected to a drone the system was able to count people from the images captured in real time. This solution was only able to count individual people, as crowds of people are complex to measure following standard object detection algorithms as YOLO technology. In our Master Thesis we are adding a new functionality by being able not only to count individuals but also counting crowds of people. With this new functionality the app could provide to the police a more accurate tool to be able to count people in different scenarios as prides, sports events, demonstrations, concerts¿ where crowd is a normal situation. As main technology driver we are working with CNN (Convolutional Neural Networks). First, we have been implementing a CNN density map using the CSRNet technology that is able to count people by measuring the concentration of people. Therefore, an important part of this Master Thesis is to create a process to split the input images in 2 (segmentation process), one for YOLO (individual persons) and other for CSRNET (crowds of people). This process has been implemented using a second CNN called Region-based CNN (R-CNN), that we found it was the most suitable tool to train a model to detect a crowd. The solution has been developed in Google Colab platform and using Python as programming language. We have been working with images taken from drones from Castelldefels Police and UPC but also public datasets. The final solution has been able to detect crowds and calculate the number of people in that crowd with a maximum error of 20% considering Mean Average Percentage Error (MAPE) and 89 considering Mean Absolute Error (MAE).

The Dynamic Airspace Configuration (DAC) concept goal is to create more flexible airspace designs and configurations as a way to improve airspace use and absorb the increasing traffic demand. This modifies the conception of an airspace comprised of fixed volumes to one using a bigger quantity of building blocks and/or flexible boundaries. This translates into an increase in the number of possible configurations, which results in the creation of a large-scale optimization problem. To tackle this problem, CRIDA developed a sectorization optimizer ¿SECTORIA¿ which ranks all the possible configurations for a defined time and airspace in terms of occupancy and selects the optimal one. The aim of this thesis is to progress the tool in order to bring it closer to an operational environment. In particular, an improvement in its capacity constraint is added in order to obtain optimal solutions in all cases. A post-processing tool is also developed to refine the optimizer results and obtain more accurate configuration plans. The validation of these improvements is then carried out in two use cases: current sectorization in Madrid ACC Route 1 and a DAC sectorization of the same airspace, considering the addition of vertical cuts. The samples of traffic used are corresponding to the dates June 22nd, July 3rd, August 3rd and February 13th, 2019. The first feature is validated comparing the results obtained with and without the improvement, whereas the post-processing tool results are compared with the configuration plan opened with the current airspace design on the days of study and with a blind test performed on a Subject Matter Expert (SME). Results from the first validation show a better handling of inevitable overload, leading SECTORIA to provide more feasible solutions. On the other hand, the post-processing tool shows promising results, showing improvements in ATC hours which can reach up to 30\%. Conclusions and future work are then described following the analysis of the results.

Emerging technologies can receive negativity or positivity depending on the way they have been presented to the consumers. Sensitisation of a new technology is important for it to be properly understood. The disadvantages may overrun the advantages but accepted depending on prior information. There has been a difference between answers from those interviewed in questionnaires before information has been provided about an emerging technology for example drones, and after the information has been provided. Other technologies such as telephone phones and nuclear plants have devastating historical calamities mentioned in this report but due to their economical and societal importance, a lot of sensitisation has been carried out to convince the users of the positive and secure application of the technology towards its acceptance. After an extensive research process and analysing different surveys, this Thesis helps to understand the importance of indulging the society prior to any emerging technology for its existence. Several surveys could not be carried on face to face due to coronavirus protocol on interactions, therefore secondary data was collected from surveys and questionnaires. This quantitative data was later expressed numerically and categorised. Data entry was done in Excel and SPSS and results formulated accordingly. Most of the surveys were carried out overseas except one that was done in Kenya. The questions may not be necessarily the same in Kenya because the idea about drones is still new.

In view of the paramount importance of safety in the Air Traffic Management system, EUROCONTROL has been developing risk models for ATM to quantify and characterize how safe the different airspaces inside the ECAC region are. The risk models mainly characterize accidents by means of the quantification of barriers that avoid precursors becoming accidents due to their good functioning. While barriers are defined as probabilities of failure, and are quantified thanks to Fault Tree Analysis, where every element is a contributor to failure, precursors are quantified as frequencies of occurrence, and are based upon experience. This provides a static view of the models, and does not take into consideration more dynamic aspects such as the traffic flow characteristics, sector complexity, and many other features not used in the risk models because they would not allow generalization, and could only refer to specific airspaces as small as simple sectors. In view of the previous statements, this project is born with the objective to find a machine learning model capable of predicting air traffic control interventions for potential conflict avoidance inside an en-route sector from the FIR Península in Spain. The idea is to build a model explainable enough to provide sufficient information about complexity and traffic flow features in an en-route sector inducing tactical conflicts (precursors in the EUROCONTROL en-route Mid-Air Collision risk model), and offer a complementary vision of the given model. To develop the model, an analysis of operational data from the Spanish ANSP ENAIRE is made, accompanied by a study to find the losses of separation and potential tactical conflicts (mid-air collision accident precursors) during a 29-day period with a conflict detector tool. The last step is the training and testing of two tree-based supervised learning models. The outcome of the supervised learning model allows the identification of the flow and complexity features that induce air traffic control interventions to avoid potential tactical conflicts and LoS with a 99.79% of certainty in conflict prediction in a single en-route sector. Future evolutions of the model will consider the tactical ATCO support, conflict resolution with reinforcement learning, and finally the automation of ATCO tasks.

This project proposes the use of drones by law enforcement agents to fight against the COVID-19 pandemic. The main objective of the project is to allow the enforcement agents to control the capacity of potentially crowded public open places as beaches during summer or sport events. To do so, an Android application has been developed to work in conjunction with DJI drones so agents can easily deploy the system wherever necessary. Tasks to perform along the project are to analyse and select the best AI object detection algorithm. Then, using transfer learning, improve the model so it is able to detect people using top view images taken by the drone and also to detect a new object: crowd. Additionally, there are some restrictions as the need of hand-held device execution and perform the detection in real-time. Last but not least, in accordance with the General Data Protection Regulation (GDPR), persons¿ privacy will be ensured during the whole process by applying image blurring. The resulting system will facilitate the fast agents' action in preventing the spreading of the virus.