394 research outputs found
Role of exosomal micrornas in regulating the immune response to lung cancer
No full textLung cancer is one of the most frequent malignant neoplasms and the main cause of cancer death. Despite medical advances, survival remains low in non-small cell lung cancer (NSCLC), the commonest type. Macrophages play a substantial role in tumour progression because of their plasticity during activation and tumour infiltration. M1 polarised macrophages are associated with increase survival in lung cancer but tumour-associated macrophages (TAM) are distinct in their potential to promote or hinder tumour development. There has been increasing recognition of molecular drivers of cancer and amongst them, exosomes are thought to modulate the wider tumour micro-environment. Exosomes are extracellular vesicles secreted by all cell types and facilitate remodelling, immune escape promotion and tumour development in the tumour microenvironment. Exosomes can transport a diverse cargo of RNA, DNA, and protein. MicroRNAs form a potentially important exosomal cargo which may affect entire cellular pathways of recipient cells, such as macrophages. Identifying, which miRNAs are associated with these processes may be essential in management of lung cancer patients. I propose that TAMs are affected by exosomes generated by tumours, and this affects their ability to respond to the tumour. The hypothesis is that differentially expressed specific exosomal miRNAs derived from NSCLC tumour alters the phenotype of macrophages resulting in immune regulation of macrophages in the tumour microenvironment. In this project, I characterised tumour derived exosomes (TDE) and their paired normal lung tissue derived exosomes (NDE) and shown that I can sequence their microRNA cargo. The differential expression testing with DESeq2 of TDE and NDE identified 465 differentially expressed miRNAs. Eight miRNAs (miR-21-5p, miR-100-5p, miR-101-3p, miR-126a-5p, miR-133a-3p, miR-149-5p, miR193a-3p, and miR-205-5p) were statistically significantly differentially expressed and involved in signaling pathways related to polarisation of macrophages towards the anti-inflammatory phenotype M2. Those pathways were PI3K/Akt/mTOR, TLRs/NF-κB, JAK/STAT and JNK/MAPK. I subsequently compared and correlated the targeted mRNA of the TDE versus NDE differentially expressed miRNAs with the differentially expressed mRNA from tumour associated macrophages (TAM) and non-tumour associated macrophages (NTAM). Eleven miRNAs (miR-1-3p, miR-105-5p, miR-126a-3p, miR-16-5p, miR-193a-3p, miR-21-5p, miR2682-5p, miR-30a-3p, miR-34a-5p, miR-503-5p, miR-9-5p) were identified. The dysregulated mRNA and their signaling pathways were like the affected pathways from differentially expressed miRNAs in the TDE and NDE comparison group. These were PI3K/Akt/mTOR, TLRs/NF-κB and JAKSTAT signaling pathway. I subsequently exposed in vitro derived macrophages to TDE and their paired NDE. My results indicated that macrophages exposed to TDE had a milder inflammatory profile (dysregulation of IL6, SOCS-1, Serpin-B1 and CCL18) than those exposed to paired NDE. These macrophages were subsequently subjected to Next Generation Sequencing (NGS)/RNA to evaluate their transcriptome (mRNA and microRNA) expression. The aim was to establish if miRNA cargo can affect macrophage’s phenotype. The miRNA signature of those macrophages showed one significant differentially expressed miRNA (miR-451a) which is known to be dysregulated in cancer and has anti-inflammatory effect. My work has potential impact in identifying the effects of tumour derived exosomes and particularly their miRNAs in macrophage’s polarisation in the tumour microenvironment
Optimization-based Planning and Control for Autonomous Surface Vehicles
No full textWith autonomy offering a number of benefits in robotics applications, such as increased safety, better consistency and reliability, reduced environmental impact and higher efficiency, it is not surprising that the topic has seen an increase in interest from both the research community as well as commercial and defence industries. In the maritime sector, autonomy has mostly been limited to autonomous underwater vehicles (AUVs), where the operational conditions allow for only limited or delayed communication, making direct or remote control by humans difficult. In recent years however, the focus has shifted to include autonomous surface vehicles (ASVs), with applications such as surveying and mapping, surveillance, and transportation. In order to deliver on the promises of autonomy for ASVs, one of the challenges that needs to be overcome, is designing robust, efficient and safe control systems, enabling the ASVs to plan their mission, make decisions based on sensory feedback, and command the vehicle control surfaces.
This thesis presents topics related to optimization and control of ASVs. This includes low-level motion control, mid-level local trajectory planning and collision avoidance (COLAV), and high-level global trajectory planning. The main part of the thesis, is a collection of peer-reviewed articles, six journal articles and three conference papers. In addition to the article collection, the initial part of the thesis contains an introduction to the main topics of low-level motion control, mid-level local trajectory planning and COLAV and high-level global trajectory planning. This provides context to the publications, and explains the relationship between the different publications.
In the context of performing autonomous marine operations, one of the first tasks, is to plan a high-level path or trajectory in order to meet the mission objective. This should be done in a way that accounts for geographical data as well as the limitations of the ASV, in order to ensure that the vessel is able to follow the plan without having to worry about colliding into known static obstacles. As part of this thesis, we present three papers concerned with planning high-level global trajectories, which in addition to planning collision free trajectories for ASVs, also finds a trajectory which optimizes a performance measure, such as energy, time and distance. The proposed planning methods combine classical combinatorial planning algorithms and convex optimization into a new class of hybrid methods, which improves both the performance of the algorithms and the optimality of the planned trajectory.
Once an ASV is following the high-level global trajectory new obstacles such as other moving vessels and unmapped landmasses may be detected, leaving the initial global trajectory no longer feasible. To solve this problem, a mid-level local trajectory planner is needed, in order re-plan parts of the trajectory such that collisions with the obstacles is avoided. As part of this thesis, we present four papers concerned with planning mid-level local trajectories. Three of these papers focus on the problem of docking and berthing in confined waters, in a way that accounts for the vessel geometry, the harbor layout, and unmapped obstacles from exteroceptive sensors. The fourth paper discusses the problem of risk assessment and COLAV during transit, and proposes a novel approach for representing dynamic obstacles with both measurement and behavioural uncertainty.
Once a trajectory has been planned, we would like to execute the plan by maneuvering the ASV. This process, called motion control, involves controlling the actuators and control surfaces of the vessel in a way that follows a course, path or trajectory. For marine vessels, motion control is complicated by the unpredictable nature of the marine environment, and the complex hydrodynamic interactions, which can very significantly during operations. As part of this thesis, we present two papers on reinforcement learning (RL)-based motion control for marine vessels, which demonstrate how on-line learning can be used to optimize the performance of the motion control system.n reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of NTNU's products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html to learn how to obtain a License from RightsLink
Recurrent haemoptysis, left-sided chest pain and an evolving left lower lobe cavity in a 50-year old smoker with prior high ethanol intake
No full textThe authors describe a case of pancreatico-pleural fistula that presented as recurrent pyogenic chest disease in a patient with underlying ethanol related pancreatic disease. The diagnosis was suspected, given the context of non-resolving chest sepsis despite repeated antibiotics in a patient with known pancreatic disease. Although not revealed by initial tests the diagnosis was confirmed by repeating imaging investigations. The fistula was repaired surgically with consequent resolution of chest disease without need for extensive thoracic surgery
Automated Planning and Control for a Simulated Robot
No full textThe early work in robotics and Artificial Intelligence showed great promise, but because of the challenges and difficulties met in the early phases, the two fields drifted apart. Artificial Intelligence focused more on algorithms and the abstract method of approaching problems, while the robotics aspect focused more on electrical and mechanical engineering. Now with the recent developments in Machine Learning, big data, computing power, sensors, software etc., the two paths these to fields have been on are getting closer.
The goal of this thesis is to try to combine Artificial Intelligence and robotics. The author has the most experience with robotics, and will therefore try to focus on the Artificial Intelligence part by making a fully usable planner. A planner, in this case, means a program that will find a sequence of actions that will lead to the desired goal. All the code for the planner has been written from scratch including a parser that will read the problem description files which the planner will utilize to find a solution to the planning problem.
To test the planner on a robotics system, the robot named KUKA YouBot is used to solve different planning problems such as Tower of Hanoi, a stack/restack problem of blocks and moving around in a domain where the robot must interact with the environment to complete its goal.
As mentioned above, the AI community has focused much on algorithms and the abstract thinking around it. The problems that the planning algorithms have been based on have been in a deterministic matter where everything is known, which is not a realistic assumption of the real world where uncertainty plays a big part. The planner is based on a deterministic model where everything is known. This thesis will therefore make an attempt to adapt the planner such that it also can handle cases where not everything is known from before
Causal Episode Explanations for Reinforcement Learning Applications
No full textDenne master oppgaven presenterer en ny metode for omfattende forklaring av hvordan en forsterkningslærende agent handler i en episode. Metoden forklarer hvorfor agenten tar sine handlinger og hvordan disse handlingene påvirker dens fremtidige tilstander. Vi refererer til disse som årsakene og virkningene av agenten, henholdsvis, og er derfor vi kaller forklaringene for Cause and Effect Sequential (CES) forklaringer. CES forklaringer holdes enkle ved å gruppere lignende påfølgende handlinger og begrense tilstandene og handlingene nevnt i forklaringen til de mest innflytelsesrike. Resultatene fra metoden indikerer at metoden generelt fungerer bra. Evalueringer gjort av en godt trent agent virker logiske, mens evalueringene tatt av en dårlig trent agent er vanligvis er mer ulogiske (som de bør være). Vi validerer deler av forklaringen med SHapley Additive exPlanations (SHAP) og konkluderer med at metodene vanligvis er enige. Forfatteren foreslår å forbedre metoden ytterligere ved å justere parameterne for handlinggruppering automatisk, tillate lineær økning eller reduksjon av handlinggruppene, og å gjøre flere tiltak for å robust finne de innflytelsesrike delene av episodene.This thesis presents a new method for comprehensively explaining how a Reinforcement Learning agent acts in an episode. The method explains why the agent takes its actions and how these actions affect its future states. We refer to these as the causes and effects of the agent, respectively, which is why we call the explanations Cause and Effect Sequential (CES) explanations. CES explanations are kept simple by grouping similar subsequent actions and limiting the states and actions mentioned in the explanation to the most influential ones. The results of the method indicate that the method generally works well. Assessments taken by a well-trained agent seem logical, while the evaluations are more illogical for a poorly trained agent (as they should be). We validate parts of the explanation with SHapley Additive exPlanations (SHAP) and conclude that the methods usually agree. The author suggests improving the method further by adjusting the action grouping parameters automatically, allowing for linear increasing or decreasing action groups, and doing several measures to find the influential parts of the episodes more robustly
Automated Planning and Control for a Simulated Robot
No full textThe early work in robotics and Artificial Intelligence showed great promise, but because of the challenges and difficulties met in the early phases, the two fields drifted apart. Artificial Intelligence focused more on algorithms and the abstract method of approaching problems, while the robotics aspect focused more on electrical and mechanical engineering. Now with the recent developments in Machine Learning, big data, computing power, sensors, software etc., the two paths these to fields have been on are getting closer.
The goal of this thesis is to try to combine Artificial Intelligence and robotics. The author has the most experience with robotics, and will therefore try to focus on the Artificial Intelligence part by making a fully usable planner. A planner, in this case, means a program that will find a sequence of actions that will lead to the desired goal. All the code for the planner has been written from scratch including a parser that will read the problem description files which the planner will utilize to find a solution to the planning problem.
To test the planner on a robotics system, the robot named KUKA YouBot is used to solve different planning problems such as Tower of Hanoi, a stack/restack problem of blocks and moving around in a domain where the robot must interact with the environment to complete its goal.
As mentioned above, the AI community has focused much on algorithms and the abstract thinking around it. The problems that the planning algorithms have been based on have been in a deterministic matter where everything is known, which is not a realistic assumption of the real world where uncertainty plays a big part. The planner is based on a deterministic model where everything is known. This thesis will therefore make an attempt to adapt the planner such that it also can handle cases where not everything is known from before
Augmented reality based operator interface for increased depth perception and decision support during subsea IMR operations
No full textDroner eller kjøretøy styres ofte ved hjelp av kontrollere i områder som ikke er til syne for operatøren. Disser er derfor vanligvis utstyr med et kamera som filmer og presenterer denne filmen på en 2D skjerm, slik at operatøren kan se omgivelsene. I slike situasjoner står operatøren ovenfor oppgaven om å måtte styre noe som befinner seg i et 3D miljø med bare 2D informasjon tilgjengelig. Denne rapporten ser på hvordan augmentert virkelighet (AR) kan brukes for å gi operatører dybde syn, altså 3D informasjon, når de opererer under slike forhold. I tillegg er det også av interesse å se på mulighetene for å kunne lage beslutningstøttene funksjonalitet ved hjelp av AR, å hjelpe operatører til å ta bedre beslutninger.
Subsea felt er plassert i områder hvor det er ekstremt vanskelig og farlig for mennesker å befinne seg. Derfor brukes fjernstyrte kjøretøy når fysiske oppgaver skal utføres på subsea utsty. Slike oppgaver faller ofte innenfor kategorien IMR operasjoner som er inspeksjons-, vedlikeholds- og reparasjonsoperasjoner. Under slike operasjoner oppstår situasjonen beskrevet ovenfor hvor de som styrer undervannsfartøyene (ROVene) ofte bare har 2D informasjon tilgjengelig. Derfor ønsker denne rapporten å se nærmere på problemstillingen: Er det mulig ved hjelp av fremtredende teknologi innenfor AR, å utvikle brukervennlig dybde syns og beslutningsstøttene funksjonalitet i et 2D monitor basert grensesnittet mellom subsea ROVer og dem som styrer de?
For å undersøke denne problemstillingen har et AR operatør grensesnitt blitt utviklet ved hjelp av Unity og utviklerpakken Vuforia. I dette grensesnittet er det implementert ulike dybde verktøy, med hensikt om å presentere dybde informasjon til operatøren. I tillegg er det også utviklet andre AR komponenter som viser relevant operasjonsinformasjon, med ønsket om å tilby beslutningsstøttene funksjonalitet. Det ble møtt ulike utfordringer i prosessen med å utvikle dette AR-systemet, hvor kanskje den største var oppgaven med å legge AR informasjonen oppå den fysiske virkeligheten på en robust og god måte.
For å kunne evaluere om det utviklede grensesnittet var velfungerende, ble det satt opp et eksperiment for å teste AR-systemet. I dette eksperimentet deltok 8 frivillige personer. Til eksperimentet ble de det utviklet et verktøy. Dette kan styres med 5 DOF, og er utstyrt med et kamera. Ved å kontrollere dette verktøyet skulle deltakerne gjennomføre tre ulike operasjoner. Disse ble utført to ganger. Første gang bare med hjelp av video strømningen fra kameraet, som ble presentert på en 2D skjerm. Andre gangen skulle det samme gjøres, bare nå med AR-systemet presentert på skjermen. Etter disse operasjonene svarte deltakerne på en spørreundersøkelse for å evaluere ulike områder ved systemet og brukervennligheten av det.
Rapporten konkluderer med at det ved hjelp av AR er mulig å utvikle et system som inneholder dybde syns og beslutningsstøttene funksjonalitet, selv om det presenteres på en 2D skjerm. Gjennom test og evaluering kom det i tillegg fram at AR grensesnittet, med den ønskede funksjonaliteten, har god brukervennlighet. Det ble også i rapporten oppdaget og diskutert komponenter som gjerne ikke utnyttet sitt fulle potensiale. Disse kunne ha tjent på forbedringer og økt fokus under videre arbeid for å øke robustheten, påliteligheten, og den overordene brukervennligheten av systemet. Dette, i tillegg til fokus på å løse de begrensningen som rapporten presenterer, må arbeides videre med før et slikt AR grensesnitt kan tas i bruk for å assistere operatører som styrer subsea ROVer
Path Following and Collision Avoidance for Marine Vessels with Deep Reinforcement Learning
No full textInteressen for fullt autonome kjøretøy har økt raskt i løpet av de siste årene, motivert av løfter om økt effektivitet samt reduserte kostnader og miljøpåvirkning. Innenfor fartøystyring er kollisjonsunngåelse en viktig del av full autonomi, da slike oppgaver vanligvis innebærer å følge en sti i tillegg til deteksjon og unngåelse av uforutsette hindringer. Spesielt for maritim navigasjon er at fartøyet også må følge de internasjonale reglene for kollisjonsunngåelse på sjøen (COLREGS). Reglene ble utarbeidet for å passe til menneskelig resonnering og har ennå ikke blitt tilpasset maskiners fastsatte natur, noe som gjør det utfordrende å utvikle autonome marine fartøy.
Fremskritt gjort innen kunstig intelligens og dyp læring støtter påstanden om at intelligente autonome systemer er innen rekkevidde, og dyp forsterkende læring (eng. deep reinforcement learning, DRL) er et av feltene som er svært lovende. DRL-metoder optimerer oppførsel basert på et brukerdefinert ytelsesmål og krever ingen tidligere kunnskap om de kontrollerte fartøyenes dynamikk eller om verdenen som opereres i, og er derfor velegnet for komplekse oppgaver der miljøforstyrrelser og modelleringsunøyaktigheter er til stede. I denne oppgaven vil en DRL-algoritme egnet for kontinuerlige systemer anvendes på to ulike fartøy i et sti-følgesystem med hastighetskontroll. Resultatene viser at kontrollsystemet er vellykket i den forstand at det kan optimalisere pådraget for å oppnå sti-følging.
Sti-følgesystemet har blitt videreutviklet for inkludering av kollisjonsunngåelse, og eksperimenter med et containerskip i en typisk situasjon innen kollisjonsunngåelse viser lovende resultater. Det foregående illustrerer dyp forsterkende lærings potensiale innen kompliserte oppgaver, og tyder på at DRL kan anvendes til utvikling av fullstendig autonome systemer for kollisjonsunngåelse.Interest in fully autonomous vehicle control has increased rapidly in recent years, motivated by promises of higher efficiency as well as reduced cost and environmental impact. Within vessel control, collision avoidance is a vital component of full autonomy, as it usually entails the following of a path as well as detection and avoidance of unforeseen obstacles. In marine navigation in particular, the vessel is also required to follow the International Regulations for Preventing Collision at Sea (COLREGS). The regulations were created to suit human reasoning and have not yet been fully adapted to the fixed nature of computers, thus complicating the development of autonomous marine vessels.
Advances within artificial intelligence and deep learning have supported the claims that intelligent autonomous systems are achievable, and deep reinforcement learning (DRL) is one of the fields that have shown great promise. DRL methods optimise behaviour based on a user-specified performance measure and require no a priori knowledge of dynamics of the controlled vessels or the world they operate in, and are therefore well suited for complex tasks involving environmental disturbances and inaccuracy in modelling. In this thesis, a DRL algorithm suitable for continuous systems will be used in the implementation of a path following system with surge control, which is applied to two different vessels. Results show a successful control system that is able to optimise its control input to achieve approximate path convergence.
The path following system is further developed to include collision avoidance, and experimental results in a collision avoidance situation with a container vessel show promising results. This illustrates the potential DRL has in solving complicated control tasks and indicates that completely autonomous collision avoidance can be developed using DRL
COLREGs-aware Trajectory Planning and Collision Avoidance for Autonomous Surface Vessels
No full textThis thesis considers trajectory planning and collision avoidance for autonomous surface vessels (ASVs) operating in complex domains in the presence of other vessels. In particular, the task of maneuvering in compliance with the International Regulations for Preventing Collisions at Sea (COLREGs), which are the rules of the road on water, is considered. The contributions are directed towards COLREGs-aware trajectory planning and collision avoidance, where COLREGs rules 8 and 13-17 are addressed. These rules consider the conduct of vessels in encounters where risk of collision is present. The rules address how the maneuvering obligations are assigned to the involved vessels as a function of the encounter geometry and relative velocity. Rules 13-15 are encounter-type specific and consider overtaking encounters, head-on encounters, and crossing encounters, respectively. Rules 8, 16, and 17 address in more general terms how vessels that have either give-way or stand-on obligations are to maneuver to reduce the risk of collision. The main motivation behind the work is to enable electric autonomous passenger ferries as an efficient and environmentally friendly means of transporting pedestrians in urban environments. Still, the concepts and methods are applicable to most surface vessel operations.
The first step in maneuvering in compliance with the COLREGs is to determine which rules that apply to the ASV. In this work, a COLREGs classification algorithm has been developed, to determine the encounter type and hence the maneuvering obligations of the ASV in a vessel-to-vessel encounter between the ASV and each so called target ship, which is another vessel that the ASV must avoid collision with.
Determining the obligations of the ASV is, however, the easy part, whereas maneuvering in compliance with the obligations is a more challenging one. The COLREGs are written by humans and for humans, and its formulation is in some parts qualitative, to allow for humans to assess the situation based on experience and skills. This poses a challenge when it comes to evaluating and acting on these rules through machine code, where quantitative statements are preferred. This thesis presents a novel mechanism for enforcing maneuvering in compliance with the COLREGs.
It comprises a target ship domain with broad consideration to the regulations, where the encounter type, encounter geometry, relative velocity and available space to maneuver are considered. The domain is designed such that if the ASV maneuvers as to not violate the domain, the ASV is consequently maneuvering in compliance with the encounter-type specific COLREGs rules 13-15 and 17. By enforcing the target ship domains as strict constraints in the trajectory planning and collision avoidance algorithms, the proposed domain robustly enforces COLREGs compliance independently of other objectives such as trajectory tracking, energy efficiency and passenger comfort.
Several reactive collision avoidance methods are also proposed for ensuring safe operation of ASVs in dynamic and unstructured areas with other vessels and restricted space to maneuver. The methods include capacity for COLREGs-aware maneuvering when avoiding collision with target ships, and also collision avoidance with static obstacles with complex geometries. The methods have a varying degree of coupling with the ASV's guidance, navigation, and control (GNC) system, which makes the proposed mechanisms for COLREGs-aware and collision-free maneuvering easy to integrate in an arbitrary GNC architecture.
A trajectory planner for path following and collision avoidance with static and dynamic obstacles is also proposed. The trajectory planner is formulated as an optimal control problem, minimizing the tracking error to the path and the induced accelerations. In addition to the COLREGs rules considered by enforcing the novel target ship domain, the trajectory planner includes consideration to rules 8 and 16, regarding making maneuvers that are readily apparent and performed in ample time to stay well clear of target ships which the ASV has give-way obligations to. This is achieved by assigning windows of reduced cost for the tracking error and the induced accelerations in the control horizon. These windows facilitate any maneuver to avoid collision to be performed within them.
The windows are parameterized by a small set of intuitive parameters, and enable, if circumstances of the case admit, maneuvers to avoid collision to be conducted in ample time, in accordance with Rule 8 and Rule 16.
The work in this thesis has both a theoretical and practical focus, to develop and also test new methods. The proposed navigation algorithms have been tested through an extensive set of simulations in relevant operational domains, where it is demonstrated that the proposed target ship domain robustly enforces compliance with COLREGs rules 13-15 and 17, and that the windows of reduced cost increase compliance with rules 8 and 16. Furthermore, some algorithms have been tested in full-scale experiments with an electric prototype autonomous passenger ferry. In the experiments, a radar- and lidar-based target tracking system has been applied to close the autonomy loop, demonstrating that the proposed methods are suitable for real-time operation, and are robust to a realistic level of noise and uncertainties in the tracking data.In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of NTNU’s products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html to learn how to obtain a License from RightsLink
Camera-Based Perception System for Autonomous Drones in Search-and-Rescue Missions at Sea
No full textEn av de største utfordringene i søk og rednings-operasjoner til sjøs er å lokalisere mennesker i mulige store områder. Mens helikoptre tradisjonelt sett har blitt brukt for å få et fugleperspektiv under søk i sjøen, har luftbårne droner de siste årene blitt et godt alternativ ettersom de har blitt både billigere og mer anvendelige. Dronene bør helst være autonome for å kunne levere verdifull informasjon til redningsteamet uten å være en byrde.
Denne oppgaven har som mål å utvikle et kamerabasert situasjonsbevisst persepsjonssystem for en autonom drone. Persepsjonssystemet er delt inn i fire hoveddeler: en helipad sporingsmodul, en søkemodul, en modul for å generere sikre landingspunkter og en modul som utfører georeferering. helipad sporingsmodulen kombinerer to kamerabaserte posisjonsestimater i et modellbasert Kalman-filter sammen med GNSS-målinger, hastighetsmålinger, og høydemålinger for å estimere posisjonen til en helipad i løpet av et oppdrag. Søkemodulen oppdager og sporer mennesker og flytende objekter ved å kombinere en dyp læring-basert detektor med flerobjektsporing. I tilfelle der det blir bekreftet en menneskedeteksjon, georefererer systemet deteksjonen sammen med en alvorlighetsgrad. Til slutt kombinerer modulen for sikre landingspunkter sanntidsbildeinformasjon og kartdata for å finne trygge landingspunkter for dronen. De trygge landingspunktene blir også georeferert.
Systemet er testet i eksperimenter ute, og funnene i oppgaven indikerer at det fullstendige persepsjonssystemet er i stand til å levere verdifull informasjon. Helipadens posisjon estimeres med god nøyaktighet i løpet av dronens oppdrag. Videre er søkemodulen i stand til å oppdage og spore både mennesker og flytende objekter ved tilstrekkelig høy flygehøyde. For å øke søkemodulens ytelse bør detektoren trenes med nye data for å øke dens deteksjonsegenskaper i et større høydeintervall, og en flerobjekts-sporer med reidentifikasjonsegenskaper bør undersøkes videre. Sikre landingspunkter genereres på hensiktsmessige steder, men det foreslås forbedringer for å øke sanntidsegenskapene til modulen i et større høydeintervall. Nøyaktigheten til de georefererte posisjonene er tilfredsstillende gitt nøyaktigheten til dronens GNSS-målinger.One of the largest challenges in Search And Rescue (SAR) operations at sea is to locate people in possibly vast areas. While helicopters traditionally have been used to get an aerial view while searching the sea, aerial drones have in later years become a good alternative as they become both cheaper and more capable. The drones should advantageously be autonomous to deliver valuable information to the rescue team without being a burden.
This thesis aims to develop a camera-based situational awareness perception system for an autonomous drone. The perception system is split into four main submodules; a helipad tracker, a search module, a safe landing point generator module, and a module that performs georeferencing. The helipad tracker combines two camera-based position estimators in a model-based Kalman filter with GNSS, velocity, and altitude corrections in order to track the position of a helipad during the extent of a mission. The search module detects and tracks humans and floating objects by combining a deep learning-based detector with multi-object tracking. In the case of confirmed human detection, the system georeferences the detection together with a severity level. Finally, the safe landing point generator module combines real-time image data and offline map data to locate safe landing points for the drone. The safe points are also geo-referenced.
The system is tested in real-world experiments, and the thesis findings indicate that the full perception system is able to provide valuable information. The helipad is tracked with good accuracy for the extent of drone operations. Furthermore, the search module is able to detect and track both humans and floating objects given a large enough flight altitude. In order to increase the search modules' performance, the detector should be retrained with new data to increase its altitude range and a multi-object tracker with re-identification properties should be further investigated. Safe landing points are generated at reasonable locations, however, suggestions are proposed to further increase the real-time properties of the submodule in a larger altitude range. The accuracy of the geo-referenced locations is adequate given the accuracy of the drone's GNSS measurements
- …
