1,721,137 research outputs found
Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics
No full textAn approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network's performance. The overarching aim of this paper is to improve the robustness in the selection of hyperparameters in Echo State Networks for the time-accurate prediction of chaotic solutions. We define the robustness of a validation strategy as its ability to select hyperparameters that perform consistently between validation and test sets. The goal is three-fold. First, we investigate routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional grid search for optimal hyperparameter selection. Numerical tests are performed on prototypical nonlinear systems that have chaotic and quasiperiodic solutions, such as the Lorenz and Lorenz-96 systems, and the Kuznetsov oscillator. Both model-free and model-informed Echo State Networks are analysed. By comparing the networks’ performance in learning chaotic (unpredictable) versus quasiperiodic (predictable) solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled – they are based on chaos theory such as the Lyapunov time – they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems
Stability analysis of chaotic systems from data
Full text linkThe prediction of the temporal dynamics of chaotic systems is challenging because infinitesimal perturbations grow exponentially. The analysis of the dynamics of infinitesimal perturbations is the subject of stability analysis. In stability analysis, we linearize the equations of the dynamical system around a reference point and compute the properties of the tangent space (i.e. the Jacobian). The main goal of this paper is to propose a method that infers the Jacobian, thus, the stability properties, from observables (data). First, we propose the echo state network (ESN) with the Recycle validation as a tool to accurately infer the chaotic dynamics from data. Second, we mathematically derive the Jacobian of the echo state network, which provides the evolution of infinitesimal perturbations. Third, we analyse the stability properties of the Jacobian inferred from the ESN and compare them with the benchmark results obtained by linearizing the equations. The ESN correctly infers the nonlinear solution and its tangent space with negligible numerical errors. In detail, we compute from data only (i) the long-term statistics of the chaotic state; (ii) the covariant Lyapunov vectors; (iii) the Lyapunov spectrum; (iv) the finite-time Lyapunov exponents; (v) and the angles between the stable, neutral, and unstable splittings of the tangent space (the degree of hyperbolicity of the attractor). This work opens up new opportunities for the computation of stability properties of nonlinear systems from data, instead of equations
Gradient-free optimization of chaotic acoustics with reservoir computing
Full text linkGradient-based optimization of chaotic acoustics is challenging for a threefold reason: (i) first-order perturbations grow exponentially in time; (ii) the statistics of the solution may have a slow convergence; and (iii) the time-averaged acoustic energy may physically have discontinuous variations, which means that the gradient does not exist for some design parameters. We develop a versatile optimization method, which finds the design parameters that minimize time-averaged acoustic cost functionals, and overcomes the three aforementioned challenges. The method is gradient-free, model-informed, and data-driven with reservoir computing based on echo state networks. First, we analyze the predictive capabilities of echo state networks in thermoacoustics both in the short- and long-time prediction of the dynamics. We find that both fully data-driven and model-informed architectures are able to learn the chaotic acoustic dynamics, both time-accurately and statistically. Informing the training with a physical reduced-order model with one acoustic mode markedly improves the accuracy and robustness of the echo state networks, while keeping the computational cost low. Echo state networks offer accurate predictions of the long-time dynamics, which would be otherwise expensive by integrating the governing equations to evaluate the time-averaged quantity to optimize. Second, we couple echo state networks with a Bayesian technique to explore the design thermoacoustic parameter space. The computational method is minimally intrusive because it requires only the initialization of the physical and hyperparameter optimizers. Third, we find the set of flame parameters that minimize the time-averaged acoustic energy of chaotic oscillations, which are caused by the positive feedback with a heat source, such as a flame in gas turbines or rocket motors. These oscillations are known as thermoacoustic oscillations. The optimal set of flame parameters is found with the same accuracy as brute-force grid search but with a convergence rate that is more than one order of magnitude faster. This work opens up new possibilities for nonintrusive (“hands-off”) optimization of chaotic systems, in which the cost of generating data, for example, from high-fidelity simulations and experiments, is high
A physical model for indirect noise in non-isentropic nozzles: Transfer functions and stability
Full text linkWe propose a mathematical model from physical principles to predict the sound generated in nozzles with dissipation. The focus is on the sound generated from the acceleration of temperature inhomogeneities (also known as entropy waves), which is referred to as indirect noise. First, we model the dissipation caused by flow recirculation and wall friction with a friction factor, which enables us to derive quasi-one-dimensional equations from conservation laws. The model is valid for both compact nozzles and nozzles with a spatial extent. Second, the predictions from the proposed model are compared against the experimental data available in the literature. Third, we compute the nozzle transfer functions for a range of Helmholtz numbers and friction factors. It is found that the friction and the Helmholtz number have a significant effect on the gain/phase of the reflected and transmitted waves. The analysis is performed from subsonic to supersonic regimes (with and without shock waves). The acoustic transfer functions vary significantly because of non-isentropic effects and the Helmholtz number, in particular, in the subsonic-choked regime. Finally, we calculate the effect that the friction of a nozzle guide vane has on thermoacoustic stability. It is found that the friction and the Helmholtz number can change thermoacoustic stability from a linearly stable regime to a linearly unstable regime. The study opens up new possibilities for the accurate prediction of indirect noise in realistic nozzles with implications on both noise emissions and thermoacoustic stability
Optimisation of chaotically perturbed acoustic limit cycles
No full textIn an acoustic cavity with a heat source, the thermal energy of the heat source can be converted into acoustic energy, which may generate a loud oscillation. If uncontrolled, these acoustic oscillations, also known as thermoacoustic instabilities, can cause mechanical vibrations, fatigue and structural failure. The objective of manufacturers is to design stable thermoacoustic configurations. In this paper, we propose a method to optimise a chaotically perturbed limit cycle in the bistable region of a subcritical bifurcation. In this situation, traditional stability and sensitivity methods, such as eigenvalue and Floquet analysis, break down. First, we propose covariant Lyapunov analysis and shadowing methods as tools to calculate the stability and sensitivity of chaotically perturbed acoustic limit cycles. Second, covariant Lyapunov vector analysis is applied to an acoustic system with a heat source. The acoustic velocity at the heat source is chaotically perturbed to qualitatively mimic the effect of the turbulent hydrodynamic field. It is shown that the tangent space of the acoustic attractor is hyperbolic, which has a practical implication: the sensitivities of time-averaged cost functionals exist and can be robustly calculated by a shadowing method. Third, we calculate the sensitivities of the time-averaged acoustic energy and Rayleigh index to small changes to the heat-source intensity and time delay. By embedding the sensitivities into a gradient-update routine, we suppress an existing chaotic acoustic oscillation by optimal design of the heat source. The analysis and methods proposed enable the reduction of chaotic oscillations in thermoacoustic systems by optimal passive control. Because the theoretical framework is general, the techniques presented can be used in other unsteady deterministic multi-physics problems with virtually no modification
Automatic-differentiated Physics-Informed Echo State Network (API-ESN)
No full textWe propose the Automatic-differentiated Physics-Informed Echo State Network (API-ESN). The network is constrained by the physical equations through the reservoir’s exact time-derivative, which is computed by automatic differentiation. As compared to the original Physics-Informed Echo State Network, the accuracy of the time-derivative is increased by up to seven orders of magnitude. This increased accuracy is key in chaotic dynamical systems, where errors grow exponentially in time. The network is showcased in the reconstruction of unmeasured (hidden) states of a chaotic system. The API-ESN eliminates a source of error, which is present in existing physics-informed echo state networks, in the computation of the time-derivative. This opens up new possibilities for an accurate reconstruction of chaotic dynamical states
Data-driven prediction and control of extreme events in a chaotic flow
No full textAn extreme event is a sudden and violent change in the state of a nonlinear system. In fluid dynamics, extreme events can have adverse effects on the system's optimal design and operability, which calls for accurate methods for their prediction and control. In this paper, we propose a data-driven methodology for the prediction and control of extreme events in a chaotic shear flow. The approach is based on echo state networks, which are a type of reservoir computing that learn temporal correlations within a time-dependent data set. The objective is fivefold. First, we exploit ad hoc metrics from binary classification to analyze (1) how many of the extreme events predicted by the network actually occur in the test set (precision) and (2) how many extreme events are missed by the network (recall). We apply a principled strategy for optimal hyperparameter selection, which is key to the networks' performance. Second, we focus on the time-accurate prediction of extreme events. We show that echo state networks are able to predict extreme events well beyond the predictability time, i.e., up to more than five Lyapunov times. Third, we focus on the long-term prediction of extreme events from a statistical point of view. By training the networks with data sets that contain nonconverged statistics, we show that the networks are able to learn and extrapolate the flow's long-term statistics. In other words, the networks are able to extrapolate in time from relatively short time series. Fourth, we design a simple and effective control strategy to prevent extreme events from occurring. The control strategy decreases the occurrence of extreme events up to one order of magnitude with respect to the uncontrolled system. Finally, we analyze the robustness of the results for a range of Reynolds numbers. We show that the networks perform well across a wide range of regimes. This work opens up new possibilities for the data-driven prediction and control of extreme events in chaotic systems
Real-time thermoacoustic data assimilation
Full text linkLow-order thermoacoustic models are qualitatively correct, but typically, they are quantitatively inaccurate. We propose a time-domain bias-aware method to make qualitatively low-order models quantitatively (more) accurate. First, we develop a Bayesian ensemble data assimilation method for a low-order model to self-adapt and self-correct any time that reference data become available. Second, we apply the methodology to infer the thermoacoustic states and heat-release parameters on the fly without storing data (real time). We perform twin experiments using synthetic acoustic pressure measurements to analyse the performance of data assimilation in all nonlinear thermoacoustic regimes, from limit cycles to chaos, and interpret the results physically. Third, we propose practical rules for thermoacoustic data assimilation. An increase, reject, inflate strategy is proposed to deal with the rich nonlinear behaviour; and physical time scales for assimilation are proposed in non-chaotic regimes (with the Nyquist-Shannon criterion) and in chaotic regimes (with the Lyapunov time). Fourth, we perform data assimilation using data from a higher-fidelity model. We introduce an echo state network to estimate in real time the forecast bias, which is the model error of the low-fidelity model. We show that: (i) the correct acoustic pressure, parameters, and model bias can be inferred accurately; (ii) the learning is robust as it can tackle large uncertainties in the observations (up to 50 % of the mean values); (iii) the uncertainty of the prediction and parameters is naturally part of the output; and (iv) both the time-accurate solution and statistics can be inferred successfully. Data assimilation opens up new possibility for real-time prediction of thermoacoustics by combining physical knowledge and experimental data synergistically
Hard-constrained neural networks for modeling nonlinear acoustics
Full text linkIn this computational paper, we model acoustic dynamics in space and time from synthetic sensor data. The tasks are (i) to predict and extrapolate the spatiotemporal dynamics and (ii) to reconstruct the acoustic state from partial observations. To achieve this, we develop acoustic neural networks. These are networks that learn from sensor data, while being constrained by prior knowledge on acoustic and wave physics. The prior knowledge is constrained as a soft constraint, which informs the training, and as a hard constraint (Galerkin neural networks), which constrains parts of the network's architecture as an inductive bias. First, we show that standard feedforward neural networks are unable to extrapolate in time, even in the simplest case of periodic oscillations. This motivates the constraints on the prior knowledge. Second, we constrain the prior knowledge on acoustics in increasingly effective ways by (i) employing periodic activations (periodically activated neural networks), (ii) informing the training of the networks with a penalty term that favors solutions that fulfill the governing equations (soft constrained), (iii) constraining the architecture in a physically motivated solution space (hard constrained), and (iv) a combination of these. Third, we apply the networks on two test cases for two tasks in nonlinear regimes, from periodic to chaotic oscillations. The first test case is a twin experiment, in which the data are produced by a prototypical time-delayed model. In the second test case, the data are generated by a higher-fidelity model with mean-flow effects and a kinematic model for the flame source. We find that (i) constraining the physics in the architecture improves interpolation while requiring smaller network sizes, (ii) extrapolation in time is achieved by periodic activations, and (iii) velocity can be reconstructed accurately from only pressure measurements with a combination of physics-based hard and soft constraints. In acoustics and thermoacoustics, this works opens possibilities for physics-constrained data-driven modeling. Beyond acoustics, this work opens strategies for constraining the physics in the architecture, rather than the training
A data-driven kinematic model of a ducted premixed flame
No full textReduced-order models of flame dynamics can be used to predict and mitigate the emergence of thermoacoustic oscillations in the design of gas turbine and rocket engines. This process is hindered by the fact that these models, although often qualitatively correct, are not usually quantitatively accurate. As automated experiments and numerical simulations produce ever-increasing quantities of data, the question arises as to how this data can be assimilated into physics-informed reduced-order models in order to render these models quantitatively accurate. In this study, we develop and test a physics-based reduced-order model of a ducted premixed flame in which the model parameters are learned from high-speed videos of the flame. The experimental data is assimilated into a level-set solver using an ensemble Kalman filter. This leads to an optimally calibrated reduced-order model with quantified uncertainties, which accurately reproduces elaborate nonlinear features such as cusp formation and pinch-off. The reduced-order model continues to match the experiments after assimilation has been switched off. Further, the parameters of the model, which are extracted automatically, are shown to match the first-order behavior expected on physical grounds. This study shows how reduced-order models can be updated rapidly whenever new experimental or numerical data becomes available, without the data itself having to be stored
- …
