21,018 research outputs found
Optimal control theoretic value function learning
No full textGenerating behaviours to complete complex tasks can be viewed under the paradigm of controlling dynamical systems. To solve such tasks, most approaches fall under two paradigms: Reinforcement Learning (RL) and Optimal Control (OC) theoretic approaches. OC theoretic solutions are mostly local and can only provide global controllers for special cases. As a result of this, local solutions become trajectories. Synthesising these trajectories in the deterministic setting is formalised under the calculus of variations. This paradigm imposes strict constraints on the objective landscape: differentiability and continuity. In the stochastic setting, OC theoretic solutions have been proposed to remove the burden of these constraints and infer the optimal trajectory through sampling. RL has very similar theoretical groundings but diverges significantly in its approach. For example, RL parametersises value and/or policy functions instead of trajectories, allowing generalisation to new initial conditions. Additionally, in its model-free setting, which is our focus in this thesis, there is no need for constraints such as differentiability on the objective. RL is capable of estimating the gradients via sampling. However, these gradient estimates come at the high price of noisy solutions and slow convergence. To this end, defining methods that can leverage the best of both approaches is desirable. Our thesis aims to derive methods that greatly remove the burden of cost function design on the user while enabling generalisation by efficiently learning approximate global controllers.
As our initial attempt at this formalisation, we introduce a local method that combines the efficiency of derivative-based OC-theoretic approaches with the flexibility of local solutions based on sampling. To this end, we propose a hybrid approach that aims for consensus between the derivative-based solution of iterative Linear Quadratic Regulator (iLQR) and the sampling-based solution of Path Integral (PI) control. We define an objective that enables us to sample when derivatives vanish and follow optimised trajectories when derivatives arise. We use the Kullback Leibler (KL) control interpretation of PI control to formulate an inference problem that computes the optimal controls constrained by an adaptive distribution defined by the solution of iLQR. Our results show better convergence on manipulation and obstacle avoidance tasks than sampling strategy, path integral control and gradient-based strategy iLQR.
In the second segment of this thesis, we evaluate the widely used RL algorithms and its core gradient estimation machinery, policy gradients, without the typical convergence strategies. Our results are obtained on simple nonlinear continuous control problems. We show that RL still requires extensive tuning, even on simple nonlinear problems and the flexibility gained by zeroth-order derivative estimation is paid for by hyperparameter tuning. In turn, we propose an OC-theoretic approach based on Bellman optimality that leverages differentiable dynamics and first-order gradients. Our approach can learn approximate time-varying value functions and robustly converge with minimal tuning. We further verify the ability of our method by relaxing the objective and obtaining first-order approximations of time-varying Lyapunov constraints. We further verify our approach by satisfying this first-order constraint over a compact set of initial conditions. When comparing our method to Soft Actor-Critic (SAC) and Proximal Policy Optimisation (PPO) we show faster convergence and outperform PPO and SAC in task cost by at least 2 and 4 orders of magnitude, respectively.
In the third part of the thesis, we combine our findings from the previous sections to create a method that can handle discontinuities using stochasticity, ensure convergence with differentiability, and generalise with function parameterisation. To achieve this, we approach the problem using stochastic optimal control and robustness. We use the stochastic Hamilton-Jacobi-Bellman equation, differentiable dynamics, and the natural smoothing induced by stochastic first-order gradients. Our results demonstrate that the policies based on learned value functions outperform SAC and PPO in task cost by factors of up to 1076.02 and 8, respectively. Moreover, we observe that adding noise to the dynamics smoothens the curvature of the value function. This effect is especially noticeable in our obstacle navigation task with discontinuous dynamics and costs, where the value functions learned under noisier dynamics follow wider paths around obstacles, making them more robust. Finally, we show that our learned value functions can also be integrated into local methods, reducing their effective search horizon by a factor of 15
Report on Meteorological Research March 1, 1935 (m-1)
No full textThe object of the report was to elucidate in detail the various features of the research program in meteorology being carried on at the Daniel Guggenheim Airship Institute in Akron, Ohio. Mr. L. J. Fangman, of the U.S. Weather Bureau, was collaborating with the author in carrying out work such as a study of autographic records of the various meteorological elements during frontal passages with a view to the possible prediction of the intensity of the accompanying disturbance as it may affect the operation of aircraft and a study of atmospheric gustiness with a view to finding the dependence between frequency end amplitude of velocity fluctuations and the vertical temperature and velocity gradients
(Fourth) Report on Meteorological Activities at the DGAI (8-1-36)(Weather Bureau Copy)
No full textThis report is on the investigations of frontal phenomena at the Daniel Guggenheim Airship Institute in Akron, Ohio from January 1, 1935 through August 1, 1936. The investigation was carried out with the cooperation of the U.S. Bureau of Aeronautics, the U.S. Weather Bureau, the California Institute of Technology, and the Guggenheim Airship Institute. Mr. R.C. Robinson of the Weather Bureau cooperated with the author in carrying out the investigation. The object of the investigation was to determine the intensity of the atmospheric disturbances (i.e. rapidity of wind shift and gustiness) accompanying the passage of cold fronts, along with a study of the characteristics of the air masses involved and other features which might affect the intensity of the disturbance. The report treated thirty cold fronts which passed the station during 1935 to 1936
Archives and Images as Repositories of Time, Language, and Forms from the Past: A Conversation with Daniel Eisenberg
No full textDaniel Akech
No full textabstract: Daniel was a little boy when the war came to his village. He witnessed people being shot and running for shelter. There was no food or water so he drank urine and ate tree leaves.
“Lost Boys Found” is an ongoing, interdisciplinary project that is collecting, recording and archiving the oral histories of the Lost Boys/Girls of Sudan. The collection is a work-in-progress, seeking to record the oral history of as many Lost Boys/Girls as are willing, and will be used in a future book.Age: 24Region: Upper NileThis picture and bio was donated to the "Lost Boys Found" oral history project from The Arizona Lost Boys Cente
Daniel Emmett postcard
No full textPostcard of Daniel Emmett and his home in Mount Vernon, Ohio. Emmett is considered to be the author of the antebellum song "Dixie," written in 1859, which became the unofficial song of the Confederate soldiers during the American Civil War. He was born in Mount Vernon in 1815 and taught himself the fiddle, and later became associated with minstrel shows and helped to define that genre. Minstrel shows traveled around the United States, presenting skits and musical performances. Emmett also composed many other songs, including "Old Dan Tucker," "Turkey in the Straw," and "The Blue Tail Fly." He died in 1904
Daniel Jau Maper
No full textabstract: Daniel Jau Maper was herding cattle when Arabs attacked his village.
“Lost Boys Found” is an ongoing, interdisciplinary project that is collecting, recording and archiving the oral histories of the Lost Boys/Girls of Sudan. The collection is a work-in-progress, seeking to record the oral history of as many Lost Boys/Girls as are willing, and will be used in a future book.Age: 27Region: Upper NileThis picture and bio was donated to the "Lost Boys Found" oral history project from The Arizona Lost Boys Cente
Daniel A. Ngor
No full textWhen Daniel was five years old Arab soldiers attacked his village.
“Lost Boys Found” is an ongoing, interdisciplinary project that is collecting, recording and archiving the oral histories of the Lost Boys/Girls of Sudan. The collection is a work-in-progress, seeking to record the oral history of as many Lost Boys/Girls as are willing, and will be used in a future book.Age : 23Region: Upper NileThis picture and bio was donated to the "Lost Boys Found" oral history project from The Arizona Lost Boys Cente
Personal Papers (MS 80-0002)
No full textLetter from Mary T. Steyn of The Readers Digest to Daniel W. Kempner providing some information on the author of an article he was asking about
Meet Daniel Melnick author of The Ash Tree
No full textMeet Daniel Melnick author of The Ash Tree. It tells a timeless story of the romance and marriage between an American Armenian girl and an immigrant who survived the 1915 Armenian Genocide in Turkey. In the aftermath of the Genocide from the 20s through the early 70s, the couple and their three children become vivid, quintessentially American characters, only for tragedy to find them again, echoing the staggering losses of 1915. The cover painting with its frayed, whitewashed frame is by the author’s wife, Jeanette Arax Melnick, and the novel is based partly on the lives of her family. Combining history and fictionalized memoir, The Ash Tree is an important, beautifully written novel of survival, new life, and heartbreak. Available from independent bookstores, Barnes and Noble, and Amazon.com. Further information at www.danielmelnick.com. Price: $25. ISBN: 9780981854762
- …
