Multimedia ONline ARchiv CHemnitz
Not a member yet
4978 research outputs found
Sort by
Reinforcement Learning with Guarantees
No full textThis dissertation summarizes my scientific work originating from 2015, focusing on reinforcement learning and subjects that surrounded it. Three major research tracks are featured in this dissertation:
- Lyapunov-based reinforcement learning with guarantees with its apex approach called CALF ('Critic as Lyapunov Function').
- Predictive reinforcement learning with guarantees, so-called 'stacked reinforcement earning'.
- Novel approaches to formal control system analysis, whose tools were widely employed the above two in their evolution.
Whereas each track comprises multiple research papers produces throughout the years, for brevity and ease of technical exposition, only CALF is detailed in this dissertation, with only a brief account of stacked reinforcement learning. The reader should consult the referenced works, whose list is given in Section 7.1.
The outline of this dissertation is as follows:
Section 4 and Section 5 give a brief sketch of history of modern control and, respectively, reinforcement learning.
Section 7 summarizes the major contributions and lists the key publications related to the dissertation.
Section 8 overviews the contemporary approaches to reinforcement learning guarantees and briefly analyses them.
Section 9 presents a detailed account of CALF.
Section 10 is dedicated to stacked reinforcement learning.
Section 11 presents the studies in stabilization, constructive control and overviews some selected application works.
The main text is concluded by a summary and research outlook into future possible directions. The CALF analysis if presented in the appendix.
We begin with a short overview of modern automatic control history, with a focus on adaptive and optimal control, and then proceed to a sketch of history of artificial intelligence, emphasizing reinforcement learning and its interplay with automatic control. We believe that precisely the optimal and adaptive control fields feature the most notable overlap with reinforcement learning generally, and reinforcement learning with guarantees in particular.:1 Acknowledgment
2 Glossary
3 Introduction
4 Brief History of Modern Control, with Focus on Adaptive and Optimal Control
4.1 Introduction
4.2 Rise of Optimal Control
4.3 Dynamic Programming
4.4 Emergence of Adaptive Control
4.5 Historical Outlook
5 Brief History of Artificial Intelligence and Reinforcement Learning
5.1 On Physiological Research Roots of Reinforcement Learning
5.2 On More Technical Roots of Reinforcement Learning
6 Conclusion of Historical Overviews
7 Contribution and Related Publications
7.1 Related Publications
8 Related Work
8.1 Introduction
8.2 Popular Baselines
8.3 Reinforcement Learning with Guarantees
8.4 Concluding Analysis of Related Work
8.5 Popular Software
9 Critic as Lyapunov Function
9.1 Technical Description of Problem
9.2 Introduction to CALF
9.3 Lyapunov-like Constraints and Implementation Details
9.4 Transfer of Knowledge
9.5 Formal Analysis
9.6 Simulation Studies with CALF
9.7 Experimental Studies with CALF
9.8 Conclusion from Studies with CALF
10 Stacked Reinforcement Learning
11 Stabilization, Formal Control and Applications
12 Summary and Research Outlook
Appendices
A. Details of Formal Analysis of CALF
A.1 Recalls and Definitions
A.2 Proof of Main Theorem
A.3 On ω-Uniform Convergence Moduli
References
B. Main Publications in Full Text Supporting the Dissertation
B.1
L. Beckenbach, P. Osinenko, and S. Streif.
“A stabilizing reinforcement learning approach for sampled systems with partially unknown models”.
International Journal of Robust and Nonlinear Control (2024)
B.2
P. Osinenko, G. Yaremenko, G. Malaniia, and A. Bolychev.
“An actor-critic framework for online control with environment stability guarantee”.
IEEE Access, Aug. 17, 2023
B.3
P. Osinenko, D. Dobriborsci, G. Yaremenko, and G. Malaniya.
“A generalized stacked reinforcement learning method for sampled systems”.
IEEE Transactions on Automatic Control, Feb. 27, 2023
B.4
D. Dobriborsci, R. Zashchitin, M. Kakanov, W. Aumer, and P. Osinenko.
“Predictive reinforcement learning: map-less navigation method for mobile robot”.
Journal of Intelligent Manufacturing (2023)
B.5
P. Osinenko, G. Yaremenko, and G. Malaniia.
“On stochastic stabilization via non-smooth control Lyapunov functions”.
IEEE Transactions on Automatic Control, 68.8, Oct. 10, 2022
B.6
P. Osinenko and S. Streif.
“On constructive extractability of measurable selectors of set-valued maps”.
IEEE Transactions on Automatic Control, 66.8 (2021)
B.7
P. Schmidt, P. Osinenko, and S. Streif.
“On inf-convolution-based robust practical stabilization under computational uncertainty”.
IEEE Transactions on Automatic Control, 66 (11), Jan. 18, 2021
B.8
P. Osinenko.
“Towards a constructive framework for control theory”.
IEEE Control Systems Letters, 6, Apr. 30, 2021
B.9
P. Osinenko and D. Dobriborsci.
“Effects of sampling and horizon in predictive reinforcement learning”.
IEEE Access, 9, Sept. 13, 2021
B.10
L. Beckenbach, P. Osinenko, and S. Streif.
“A Q-learning predictive control scheme with guaranteed stability”.
European Journal of Control, 56 (2020)
B.11
T. Göhrt, P. Osinenko, and S. Streif.
“Adaptive dynamic programming using Lyapunov function constraints”.
IEEE Control Systems Letters, 3.4 (2019). Presented at CDC
B.12
P. Osinenko, L. Beckenbach, and S. Streif.
“Practical sample-and-hold stabilization of nonlinear systems under approximate optimizers”.
IEEE Control Systems Letters, 2.4 (2018). Presented at CDC
B.13
L. Beckenbach, P. Osinenko, and S. Streif.
“Model predictive control with stage cost shaping inspired by reinforcement learning”.
Conference on Decision and Control (CDC), 2019
B.14
P. Osinenko and G. Yaremenko.
“On stochastic stabilization of sampled systems”.
Conference on Decision and Control (CDC), Dec. 14, 202
Detaillierte Offshore-Windenergieanlagen-Simulation mit alaska/Wind
No full textDie Auslegung von Windenergieanlagen findet zu einem Großteil simulativ statt. Dabei ist eine Fülle an zu untersuchenden Umgebungsbedingungen und Betriebszuständen zu berücksichtigen, welche es nicht erlauben alle Komponenten in jeder Situation in höchster Detailstufe abzubilden. Im Vortrag werden verschiedene Detaillierungen von Modellkomponenten für Offshore Windenergieanlagen dargestellt, welche in einem Forschungsprojekt in die Mehrkörperdynamiksoftware alaska/Wind integriert werden.
Ziel ist die Verfügbarkeit von wählbaren Detailstufen der Anlagenkomponenten in einer Gesamtsimulation, so dass für jeden Anlagentyp und jedes Betriebsszenario die notwendige Detailstufe für eine korrekte Berücksichtigung der Gesamtdynamik ermittelt werden kann.The design of wind turbines heavily depends on simulations. In doing so, a wide range of environmental conditions and operating states must be considered, which do not allow all components to be modeled at the highest level of detail. The talk will present various details of model components for offshore wind turbines, which are currently integrated into the multibody dynamics software alaska/Wind in a research project.
The aim is the availability of selectable detail levels of the turbine components in a holistic simulation, so that the necessary level of detail for a correct consideration of the overall dynamics can be determined for each turbine type and each operating state
Hochauflösende Spektroskopie an (Al,In)GaN Laserdioden
No full textGrüne und blaue Laserdioden im (Al,In)GaN Materialsystem sind aus modernen Beleuchtungs- und Projektionsanwendungen nicht mehr wegzudenken. Die stete Bestrebung des besseren Verständnisses und der Verbesserung der Bauteile dienen als Antrieb für die weitere Forschung. Aus diesem Grund werden mittels hochaufgelöster Spektroskopie die spektralen Eigenschaften der Dioden gemessen und unter Zuhilfenahme verschiedener Auswertemethoden die Verluste der Dioden ermittelt. Außerdem geht es darum, einen Parametersatz für je eine blaue und grüne Laserdiode zu ermitteln, welcher als Grundlage für Simulationen dienen kann. Ein weiterer Aspekt war die Untersuchung der spektralen Eigenschaften von Breitstreifenlaserdioden mittels ultrahochaufgelöster Spektroskopie. Dabei ermöglicht die Kopplung eines Gitterspektrometers und eines Fabry-Pérot Interferometers eine spektrale Auflösung von unter einem Pikometer. Hiermit gelingt eine Unterscheidung
zahlreicher longitudinaler Modenkämme der Breitstreifenlaserdioden
Efficient Simulation of Biologically Realistic Neural Networks on Different Parallel Hardware Using Code Generation
No full textComputational neuroscience is a rapidly developing field exploring the principles of information encoding and decoding in neural systems and trying to understand the brain on a functional level. The ongoing research in this field leads to models increasing in size and complexity. Modern multi-core CPUs and graphic processing units (GPUs) offer increasing computational power on shared memory systems. Neural simulation tools should help to make use of this parallel computational power for the simulation of biologically inspired networks. As we will show in this thesis, developing such neural simulation tools demands a good understanding of both models of biologically inspired networks and current hardware architectures. The simulation of rate-coded and spiking models places different requirements on their efficient implementation. At this point, one rapidly notices the problem of specialization and generalization in simulation frameworks. Code generation approaches, already used in Brian, GeNN, or ANNarchy, seem to be a suitable solution for this dilemma. Code generation allows the adjustment of generated simulation code based on the used hardware platform and the structure of the target network. In this thesis, we will discuss the implementation of key operations within rate-coded and spiking neural networks and the impact of different data representations on those. Based on this acquired knowledge, we selected the code templates used in the code generation of our neural simulation framework ANNarchy. In summary, we could achieve a noticeable improvement on rate-coded neural models while we achieve comparable performance on spiking model benchmarks on shared memory systems
Powering the World of Microrobots with Micro Energy Systems
No full textThis doctoral thesis addresses the development of micro energy storage and generation systems for microelectronic and biomedical applications, focusing on compact, flexible, and high-performance solutions. By integrating Swiss-roll and origami technologies, modular micro-origami robots (“smartlets”) are developed, capable of incorporating energy harvesters, sensors, processors, micro-LEDs, and actuators.
A nano-biosupercapacitor made from fully biocompatible materials is introduced, occupying only 1 nanoliter while delivering up to 1.6 V in blood. Its tubular geometry provides self-protection and enhanced performance via redox enzymes and living cells, enabling it to power integrated pH sensors.
Additionally, a micro-organic solar cell using the Swiss-roll design demonstrates 7% power conversion efficiency and broad-angle light absorption, providing energy for surface electrochemical actuators.
The thesis further explores 3D self-assembly of thin membranes into modular architectures, enabling smartlets with autonomous, collective functionality, on-board energy harvesting, and inter-module communication. These innovations advance autonomous modular microrobotics and micro-scale energy systems for biomedical and electronic applications
Ab-initio Simulation of Silicon Nanowires
No full textThis thesis addresses the enhancement of semiconductor materials for modern electronic and energy conversion applications through two complementary investigations. The first focuses on silicon nanowires (SiNWs), promising candidates for next-generation nanoelectronic and nanoelectromechanical systems (NEMS), due to their unique electronic, mechanical, and quantum confinement properties. Density Functional Theory (DFT) calculations were carried out on ultra-thin SiNWs oriented along [001] and [111] crystallographic directions. Results reveal that [001] SiNWs exhibit metallic behavior, whereas [111] SiNWs display a negligible direct bandgap, making them suitable for semiconductor applications. Doping alters electronic properties: boron doping (p-type) induces lattice contraction and introduces hole states that enhance charge carrier mobility; phosphorus doping (n-type) injects electrons into the conduction band, leading to lattice expansion and, in some cases, metallicity. Site-selective doping enables fine-tuning of SiNW electronic properties for use in transistors, sensors, and quantum computing. The study also examines mechanical properties of pristine and doped SiNWs, including bulk modulus (B), shear modulus (G), Young’s modulus (Y), and Poisson’s ratio (v). Results indicate significant anisotropy, with [111]-oriented SiNWs demonstrating greater stiffness due to atomic bonding configurations. Quantum confinement and high surface-areato-volume ratios reduce stiffness in ultra-thin SiNWs compared to bulk silicon. Boron doping enhances stiffness via increased Young’s and bulk moduli, while phosphorus doping improves shear resistance by raising G and lowering Cauchy’s pressure. Dopant orientation influences mechanical behavior, underscoring the importance of optimizing dopant positioning for future device engineering. The second part explores InGaN-based solar cells using SCAPS-1D simulations. Increasing indium content in InGaN layers boosts power conversion efficiency, reaching 23.8% for 0.6Ga0.4N. A thick p-layer and a thin, wide-bandgap top layer in a p-p-n junction elevate efficiency to 34.07%. While defect density generally does not affect open-circuit voltage, high defect levels in the p-layer degrade performance. These studies highlight the critical role of materials optimization—via doping strategies in SiNWs and layer design in InGaN solar cells—for advancing nanoelectronic and photovoltaic technologies and offer foundational insights for designing next-generation semiconductor devices.Diese Dissertation befasst sich mit der Verbesserung von Halbleitermaterialien für moderne Anwendungen in der Elektronik und Energiewandlung durch zwei komplementäre Untersuchungen. Der erste Teil konzentriert sich auf Silizium-Nanodrähte (SiNWs), die aufgrund ihrer besonderen elektronischen, mechanischen und quantenmechanischen Eigenschaften als vielversprechende Kandidaten für zukünftige nanoelektronische und nanoelektromechanische Systeme (NEMS) gelten. Dichtefunktionaltheorie (DFT)-Berechnungen wurden an ultradünnen SiNWs mit Orientierung entlang der kristallographischen Richtungen [001] und [111] durchgeführt. Die Ergebnisse zeigen, dass [001]-SiNWs metallisches Verhalten aufweisen, während [111]-SiNWs eine vernachlässigbare direkte Bandlücke zeigen, was sie für Halbleiteranwendungen geeignet macht. Dotierung verändert die elektronischen Eigenschaften signifikant: Bor-Dotierung (p-Typ) verursacht Gitterkontraktion und erzeugt Lochzustände, die die Ladungsträgerbeweglichkeit erhöhen; Phosphor-Dotierung (n-Typ) führt zur Einspeisung von Elektronen in das Leitungsband, was Gitterexpansion und in manchen Fällen metallisches Verhalten verursacht. Eine ortsselektive Dotierung ermöglicht die gezielte Abstimmung der elektronischen Eigenschaften für Anwendungen in Transistoren, Sensoren und der Quanteninformatik. Die Studie untersucht auch die mechanischen Eigenschaften von unmodifizierten und dotierten SiNWs, einschließlich Volumenmodul (B), Schubmodul (G), Elastizitätsmodul (Y) und Poisson-Zahl (v). Die Ergebnisse zeigen eine deutliche Anisotropie, wobei [111]-SiNWs aufgrund ihrer atomaren Bindungskonfiguration höhere Steifigkeit aufweisen. Quantenkonfinierung und ein hohes Oberflächen-zu-Volumen-Verhältnis führen in ultradünnen SiNWs zu einer Reduktion der Steifigkeit im Vergleich zu Bulk-Silizium. Bor-Dotierung erhöht die Steifigkeit durch steigende Werte von Y und B, während Phosphor-Dotierung die Scherfestigkeit durch Erhöhung von G und Reduktion des Cauchy-Drucks verbessert. Die Dotierungsorientierung beeinflusst das mechanische Verhalten wesentlich und unterstreicht die Bedeutung der gezielten Platzierung von Dotierstoffen für die zukünftige Geräteentwicklung. Der zweite Teil der Arbeit widmet sich InGaN-basierten Solarzellen, untersucht mittels SCAPS-1D-Simulationen. Ein steigender Indiumgehalt in den InGaN-Schichten erhöht den Wirkungsgrad der Energieumwandlung auf bis zu 23,8% für 0.6 Ga0.4 N. Eine dicke p-Schicht sowie eine dünne, bandlückenreiche Oberschicht im p-p-n-Übergang steigern die Effizienz weiter auf 34,07%. Während die Defektdichte im Allgemeinen die Leerlaufspannung kaum beeinflusst, verschlechtert ein hoher Defektgehalt in der p-Schicht deutlich die Leistung. Insgesamt unterstreichen beide Studien die zentrale Rolle der Materialoptimierung – durch Dotierungsstrategien in SiNWs und Schichtdesign in InGaN-Solarzellen – für den Fortschritt in der Nanoelektronik und Photovoltaik. Die gewonnenen Erkenntnisse leisten einen grundlegenden Beitrag zur Entwicklung künftiger Halbleitertechnologien
Electrodeposition of Copper in Magnetic Field Gradients of Micrometer- to Nanometer-size Templates
No full textThe electrodeposition of 3D-structured copper in micrometer- to nanometer-size magnetic field gradients could offer significant advantages in cost-efficiency and scalability over conventional fabrication techniques for micro- and nanoscale devices. While earlier studies have demonstrated the feasibility of copper electrodeposition in micrometer-size gradients, the systematic downscaling of such magnetic field gradients remains largely unexplored. This raises key questions: To what extent can magnetoelectrodeposition be downscaled? What challenges arise, and how can they be addressed?
This work investigates the influence of magnetic field gradients on copper electrodeposition, beginning with process optimization in millimeter-size gradients and advancing toward systematic downscaling to micrometer size. To fabricate 3D copper structures, electrodeposition was carried out under varying potentials to enhance deposit quality and control morphology. Co/Pt thin film multilayers were subsequently examined as templates for copper electrodeposition in nanometer-size magnetic field gradients. Key outcomes of this research include the successful fabrication of 3D copper deposits using magnetic field gradients on the scale of hundreds of micrometers. The use of more negative deposition potentials was found to enhance the structural definition of these deposits. Moreover, the study demonstrated that both the shape and surface morphology of copper structures can be effectively tuned by adjusting the magnetic field gradient size and the potential. In the final stage, Co/Pt thin film multilayers were investigated. While only cyclic voltammetry revealed preliminary differences between magnetic states, the results represent an initial step toward copper electrodeposition in highly localized nanometer-size magnetic field gradients. Due to limited reproducibility and the small sample size, further experiments are needed to improve statistical significance and validate these early observations. Nevertheless, this work marks an important starting point for advancing magnetic field-assisted nanoscale fabrication.:INTRODUCTION 1
1 FUNDAMENTALS AND STATE-OF-THE-ART 4
1.1 Fundamentals of Metal Ion Electrodeposition 4
1.2 Electrodeposition of Copper from Acidic Solutions 8
1.3 The Magnetic Nature of Species in the Electrolyte Solution 12
1.4 Important Magnetic Forces in Magnetoelectrochemistry 12
1.5 Magnetic Templates for Electrochemistry 16
1.5.1 Electrodeposition in Millimeter-to-Micrometer-size Magnetic Field Gradients 16
1.5.2 Electrochemistry in Nanometer-size Magnetic Field Gradients 23
1.6 Summary and Objectives of the Thesis 29
2 EXPERIMENTAL METHODS 30
2.1 Preparation of the Magnetic Templates 30
2.1.1 Fe wire Templates for Millimeter-to-Micrometer-size Magnetic Field Gradients 30
2.1.2 Co/Pt Templates for Nanometer-size Magnetic Field Gradients 31
2.2 Electrochemical Methods 33
2.2.1 Cyclic Voltammetry 36
2.2.2 Potentiostatic Electrodeposition and Pulse-Reverse Plating 37
2.3 Characterization Methods 38
2.3.1 Structural and Microstructural Characterization Methods 38
2.3.2 Magnetic Structure Characterization Methods 40
3 RESULTS AND DISCUSSION 42
3.1 Electrodeposition of Copper in Millimeter-size Magnetic Field Gradients: Optimization of the Electrochemical Setup 43
3.1.1 Selection of the Electrolyte 43
3.1.2 Cyclic Voltammetry 44
3.1.3 Pulse-Reverse Plating 50
3.1.4 Structural and Microstructural Characterization of Copper Deposits 52
3.1.5 Summary 55
3.2 Electrodeposition of Copper in Millimeter-to-Micrometer-size Magnetic Field Gradients 56
3.2.1 Calculations of the Magnetic Field Gradient Product 56
3.2.2 Cyclic Voltammetry 59
3.2.3 Pulse-Reverse Plating 61
3.2.4 Structural and Microstructural Characterization of Cu Deposits 64
3.2.5 Summary 72
3.3 Electrodeposition of Copper in Nanometer-size Magnetic Field Gradients 73
3.3.1 Magnetic Structures in Various Co/Pt Thin Film Multilayers 73
3.3.2 Calculations of the Magnetic Field Gradient Product 79
3.3.3 Cyclic Voltammetry with in-situ Kerr Microscopy 81
3.3.4 Pulse-Reverse Plating 86
3.3.5 Microstructural Characterization of Cu Deposits 87
3.3.6 Outlook on Future Experiments 89
3.3.7 Summary 91
4. SUMMARY AND CONCLUSIONS 92
5. OUTLOOK 95
6. APPENDIX 97
6.1 Preparation of Co/Pt Thin Film Multilayers with Various Domain Patterns 97
6.2 Preparation of Co/Pt Thin Film Multilayers with Stripes Domains 98
6.3 Magnetic Structures Characterization of Co/Pt Thin Film Multilayers 101
6.4 Cyclic Voltammetry: Co/Pt Thin Film Multilayers in Saturated State 102
7. BIBLIOGRAPHY 103
8. LIST OF PUBLICATIONS AND CONFERENCE CONTRIBUTIONS 112
9. DECLARATION OF INDEPENDENCE 112
10. ACKNOWLEDGMENTS 11
Optimisation of the joining strategy and residual stress considerations for heated tool welding of polyethylene with high wall thicknesses
No full textHeated tool welded components made of polyethylene with high wall thicknesses are often used for the expansion of infrastructure, particularly regarding sustainability and renewable energies. For producing the weld seams, it is crucial to have a reliable process control system that ensures a long service life. However, current strategies lack a sound scientific background and can lead to low weld seam strengths, especially in long-term tests. In a research project, welding tests were conducted correlating the joining time with the cooling temperature of the weld. The results show that mechanical properties can be improved if the weld seam is not cooled under pressure over a long period of time but only for as long as it is in the molten state. The findings furthermore indicate that joining pressure needs to be adjusted accordingly. In addition to improved mechanical properties, this approach also reduces the required process times. By means of residual stress investigations, an attempt was made to increase the understanding of the fracture behaviour. The welding tests were primarily carried out with PE100 RC sheets with a wall thickness of 100 mm. Initial tests also showed that this strategy can be transferred to a wall thickness of 30 mm
21st International Symposium on Electrochemical Machining Technology INSECT 2025
No full textElectrochemical technologies are known for their ability for the precise shaping of high-strength
materials on a macro and micro geometric scale with high surface quality and defined chemical-
physical surface conditions. The effective application of electrochemical and EC-based
technologies such as ECM, Wire-ECM, PECM, Laser-ECM as well as PeP requires a thorough
understanding of the specific interactions between material, process and functional properties.
In addition to traditional materials used in mechanical engineering, the machining of materials
such as hard-to-machine metals, cemented carbides, superalloys and high entropy alloys are
increasingly coming into focus. High-resolution methods for the characterization of process
mechanisms and results, as well as advanced modelling and control strategies open up new
possibilities and extend the limits of EC-based processes. By combining high efficiency and
excellent surface quality, EC-based technologies offer an outstanding potential for sustainable
and precise material and surface processing. In addition to well-established sectors such as
aerospace, medical technology and tool manufacturing, an advanced knowledge of process
design and optimization enables access to new areas of application
Co-Creating Future Autonomous Vehicle HMIs: A Mixed-Methods Exploration of Passenger Information Needs
No full textThis paper investigates passenger information needs concerning the behavior of autonomous vehicles (AVs; SAE L4 and L5) in urban driving scenarios. Understanding these needs is essential for designing effective in-vehicle human-machine interfaces (HMIs) that foster trust and acceptance. A mixed-methods approach was employed to conduct co-creation interviews (N = 15), combining semi-structured interviews, quantitative questionnaires, real-world videos to contextualize critical scenarios, and a mix-and-match co-creation method where participants designed their own AV HMI concepts. The findings highlight key information needs related to AV decision-making, feedback, and safety, offering valuable insights for future HMI development