Offenburg University of Applied Sciences
Hochschulschriftenserver der Hochschule OffenburgNot a member yet
6641 research outputs found
Sort by
Alternative optimization methods for training of large deep neural networks
No full textDue to its performance, the field of deep learning has gained a lot of attention, with neural networks succeeding in areas like Computer Vision (CV), Neural Language Processing (NLP), and Reinforcement Learning (RL). However, high accuracy comes at a computational cost as larger networks require longer training time and no longer fit onto a single GPU. To reduce training costs, researchers are looking into the dynamics of different optimizers, in order to find ways to make training more efficient. Resource requirements can be limited by reducing model size during training or designing more efficient models that improve accuracy without increasing network size.
This thesis combines eigenvalue computation and high-dimensional loss surface visualization to study different optimizers and deep neural network models. Eigenvectors of different eigenvalues are computed, and the loss landscape and optimizer trajectory are projected onto the plane spanned by those eigenvectors. A new parallelization method for the stochastic Lanczos method is introduced, resulting in faster computation and thus enabling high-resolution videos of the trajectory and secondorder information during neural network training. Additionally, the thesis presents the loss landscape between two minima along with the eigenvalue density spectrum at intermediate points for the first time.
Secondly, this thesis presents a regularization method for Generative Adversarial Networks (GANs) that uses second-order information. The gradient during training is modified by subtracting the eigenvector direction of the biggest eigenvalue, preventing the network from falling into the steepest minima and avoiding mode collapse. The thesis also shows the full eigenvalue density spectra of GANs during training.
Thirdly, this thesis introduces ProxSGD, a proximal algorithm for neural network training that guarantees convergence to a stationary point and unifies multiple popular optimizers. Proximal gradients are used to find a closed-form solution to the problem of training neural networks with smooth and non-smooth regularizations, resulting in better sparsity and more efficient optimization. Experiments show that ProxSGD can find sparser networks while reaching the same accuracy as popular optimizers.
Lastly, this thesis unifies sparsity and neural architecture search (NAS) through the framework of group sparsity. Group sparsity is achieved through ℓ2,1-regularization during training, allowing for filter and operation pruning to reduce model size with minimal sacrifice in accuracy. By grouping multiple operations together, group sparsity can be used for NAS as well. This approach is shown to be more robust while still achieving competitive accuracies compared to state-of-the-art method
Unfolding Local Growth Rate Estimates for (Almost) Perfect Adversarial Detection
No full textConvolutional neural networks (CNN) define the state-of-the-art solution on many perceptual tasks. However, current CNN approaches largely remain vulnerable against adversarial perturbations of the input that have been crafted specifically to fool the system while being quasi-imperceptible to the human eye. In recent years, various approaches have been proposed to defend CNNs against such attacks, for example by model hardening or by adding explicit defence mechanisms. Thereby, a small “detector” is included in the network and trained on the binary classification task of distinguishing genuine data from data containing adversarial perturbations. In this work, we propose a simple and light-weight detector, which leverages recent findings on the relation between networks’ local intrinsic dimensionality (LID) and adversarial attacks. Based on a re-interpretation of the LID measure and several simple adaptations, we surpass the state-of-the-art on adversarial detection by a significant m argin and reach almost perfect results in terms of F1-score for several networks and datasets. Sources available at: https://github.com/adverML/multiLI
Increased shoe bending stiffness changes ankle kinematics during high degree cutting movements
No full textThe purpose of this study is to investigate the influence of shoe longitudinal bending stiffness on the ankle kinematics during indoor and court sport specific cutting movements
Verfahren zum Betrieb eines batterieelektrischen Fahrzeugs (DE102021003621A1)
No full textDie Erfindung betrifft ein Verfahren zum Betrieb eines batterieelektrischen Fahrzeugs mit einer elektrischen Maschine zum Antrieb des Fahrzeugs und einem Inverter (1) zum Ansteuern der elektrischen Maschine, wobei der Inverter (1) eine dreiphasige Brückenschaltung mit einer Anzahl von als Halbleiter ausgebildeten Schaltern (3) umfasst, wobei im Inverter (1) entstehende Verluste zum Heizen eines Innenraums des Fahrzeugs und/oder zum Temperieren einer Batterie und/oder zum Temperieren von Getriebeöl verwendet werden, wobei der Inverter (1) mittels Raumzeigermodulation gesteuert wird, wobei ein nicht-optimales Schaltverhalten des Inverters (1) herbeigeführt wird, indem nicht optimale Spannungs-Raumzeiger (e, eu, ev, ew, e1, e2, -e1, -e2) eingestellt werden, wobei eine Skalierung der Spannungs-Raumzeiger (e, e1, e2) über die Schaltung von Nullspannungsvektoren, die je nach zeitlichem Anteil die Spannung reduzieren, oder durch Zuhilfenahme eines jeweils gegenüberliegenden Spannungs-Raumzeigers (-e1, - e2) erfolgt, so dass eine Schaltfolge mit einer maximalen Anzahl von Schaltzyklen realisiert wird, wobei in der Mitte einer Schaltperiode (Tp) keine Symmetrie erzeugt wird
Wärmeversorgung im Geschosswohnungsbau mit Wärmepumpen
No full textWärmepumpen sind eine Schlüsseltechnologie der Wärmewende. Durch die Nutzbarmachung von Umweltwärme und den Antrieb mit Elektrizität, die zunehmend aus erneuerbaren Energien gewonnen wird, kann die CO2-Intensität der Wärmeversorgung gesenkt werden. Eine Herausforderung besteht in der Anwendung in größeren Mehrfamilienbestandsgebäuden. Lösungsansätze und beispielhafte Umsetzungen werden hierzu vorgestellt
Technisch-wissenschaftliche Analyse zur Energieeffizienz unterschiedlicher Trinkwasser-Erwärmungssysteme im Vergleich
No full textIn der Studie "Technisch-wissenschaftliche Analyse zur Energieeffizienz unterschiedlicher Trinkwasser-Erwärmungssysteme im Vergleich" im Auftrag der Viega GmbH & Co. KG werden verschiedene Trinkwasser-Erwärmungssysteme hinsichtlich ihrer Energieeffizienz in Wärmepumpensystemen vergleichend untersucht. Neben Aufbau und Parametrierung eines Simulationsmodells sowie Integration von Lastreihen nach Norm umfasst die Studie eine detaillierte Abbildung aller untersuchten Systeme. Dabei liegt ein Schwerpunkt auf der Einordnung des Energieeinsparpotenzials durch eine Warmwassertemperaturreduktion mit dem Viega AVS Trinkwasser Management System. Die untersuchten Varianten sind: Referenzsystem 1: Durchflusstrinkwassererwärmer DTE (1 stufig) mit Rücklaufeinschichtung. System 2: Viega DTE (2 stufig). System 3: Viega AVS Trinkwasser Management System mit DTE (2 stufig) und Ultrafiltrationsmodul im Zirkulationsrücklauf UFC. System 4: Wohnungsstation, 4-Leiter-System. System 5: Wohnungsstation, 2-Leiter-System. System 6: Elektrischer Durchlauferhitzer. Die Studie ergab, dass sich bei Einsatz einer Niedertemperatur-Wärmepumpe mit maximaler Vorlauftemperatur von 58 °C das Viega AVS System mit DTE und UFC, dezentrale elektrische Durchlauferhitzer sowie das 4-Leiter-System bei einer Trinkwassertemperatur von 45°C im Vergleich als energetisch am besten erweisen. Bei einer Wärmepumpe mit einer höheren maximalen Vorlauftemperatur von 64 °C kann auch das 4-Leiter-System bei einer Trinkwassertemperatur von 50°C sinnvoll eingesetzt werden. Die Ergebnisse zeigten auch, dass je höher die durch die Wärmepumpe bereitgestellte Temperatur (maximale Vorlauftemperatur), desto besser lassen sich auch die anderen Systeme einsetzen, da sich dadurch der Einsatz des Backup-Systems minimieren lässt. Das Viega Aqua VIP System mit Temperaturabsenkung schneidet im Vergleich sehr gut hinsichtlich des Einsatzes der Endenergie und der zu erreichenden Jahresarbeitszahl ab. Der Einsatz dieses Systems in Kombination mit einer Wärmepumpe bietet Potenzial für den Einsatz erneuerbarer Energien
Classification of Nature-Inspired Inventive Principles for Eco-innovation and Their Assignment to Environmental Problems in Chemical Industry
No full textEco-innovations in chemical processes should be designed to use raw materials, energy and water as efficiently and economically as possible to avoid the generation of hazardous waste and to conserve raw material reserves. Applying inventive principles identified in natural systems to chemical process design can help avoid secondary problems. However, the selection of nature-inspired principles to improve technological or environmental problems is very time-consuming. In addition, it is necessary to match the strongest principles with the problems to be solved. Therefore, the research paper proposes a classification and assignment of nature-inspired inventive principles to eco-parameters, eco-engineering contradictions and eco-innovation domains, taking into account environmental, technological and economic requirements. This classification will help to identify suitable principles quickly and also to realize rapid innovation. In addition, to validate the proposed classification approach, the study is illustrated with the application of nature-inspired invention principles for the development of a sustainable process design for the extraction of high-purity silicon dioxide from pyrophyllite ores. Finally, the paper defines a future research agenda in the field of nature-inspired eco-engineering in the context of AI-assisted invention and innovation
Methodology and Implementation for Monitoring Precise Time Synchronisation in TSN
No full textTSN, or Time Sensitive Networking, is becoming an essential technology for integrated networks, enabling deterministic and best effort traffic to coexist on the same infrastructure. In order to properly configure, run and secure such TSN, monitoring functionality is a must. The TSN standard already has some preparations to provide such functionality and there are different methods to choose from. We implemented different methods to measure the time synchronisation accuracy between devices as a C library and compared the measurement results. Furthermore, the library has been integrated into the ControlTSN engineering framework
Teaching visual programming: humanoid robot programming as a case study
No full textVisual programming languages (VPL) let users develop software programs by combining visual program elements, like lists of objects, loops or conditional statements rather than by specifying them textually.
Humanoid robots programming is a very attractive and motivating application domain for students, especially for programming beginners. Humanoid robots are constructed in such a way that they mimic the human body by using actuators that perform like muscles. Typically, a humanoid robot consists of sensors and actuators, i.e. torso, a head, two arms, and two legs, though some humanoid robots may replicate only part of the body, for example, from the waist up. In some cases, humanoid robots are equipped with heads designed to replicate additional human facial features such as eyes. Additional sensors are needed by a robot to gather information about the conditions of the environment to allow the robot to make necessary decisions about its position or certain actions that the situation requires, e.g. an arm movement or an open/close hand action. Other examples for sensor are reflective infrared sensors used to detect objects in proximity.
In this work, we introduce a use-case centered approach based on sensors and actors of a robot and a workflow model to visually describe the sequence of actions including conditional actions or concurrent actions. We provide an in-depth discussion of a new VPL based teaching method for programming humanoid robots based on VPLs. Open research challenges, limits and perspectives for further development of our teaching approach are discussed as well