213 research outputs found
On the anniversary of the breeder Grigory Fedorovich Monakhos
No full textOn March 20, 2024, an outstanding Russian breeder Grigory Fedorovich Monakhos, Head of a scientific school in the field of vegetable breeding, turned 70 years old. The labor, scientific and pedagogical activities of Grigory Fedorovich for more than forty years have been associated with «Timiryazevka” – the Russian State Agrarian University – Moscow Timiryazev Agricultural Academy. Grigory Fedorovich is the author/co-author of more than 70 hybrids of vegetable crops, of which more than 40 are of white cabbage. In his breeding work, G.F. Monakhos paid the greatest attention to the most complex aspects: the genetic resistance of plants to phytopathogens and pests. Under his leadership, 18 candidates of science defended their theses. G.F. Monakhos is a co-author of more than 130 publications, including a textbook and educational manuals. Grigory Fedorovich is a member of the editorial boards of scientific journals “Izvestiya of Timiryazev Agricultural Academy” and “Potato and Vegetables”
Another Loveless Father: Grigory in Dostoevsky???s The Brothers Karamazov
No full textOne of the major themes of The Brothers Karamazov is fathers and sons, whose bonds allow the author to explore the idea of active love. In the novel, positive father figures, such as Father Zosima, are presented alongside negative ones. Fyodor Karamazov is usually seen as the novel???s prime example of a loveless father, but another father is also worthy of critical attention in this regard: Grigory, Fyodor Karamazov???s loyal servant and Smerdyakov???s foster father. As a father, Grigory seems as incapable of love as Fyodor Karamazov. In fact, when discussing the evil nature of Smerdyakov, Golstein has argued that Grigory is ultimately to blame because his ???stubbornness, dogmatism, and constantly judgmental nature??? (98) have lead him to play a ???destructive role in the shaping of Smerdyakov??? (96). The text certainly provides ample evidence of Grigory???s disastrous conduct as a parent. In the account of Smerdyakov???s childhood, Grigory???s verbal and physical abuse of his foster son is repeatedly mentioned. But while Grigory???s negative role in Smerdyakov???s existence is unquestionable, what remains to be examined is the reason for Grigory???s hatred of him. Since Smerdyakov is a person nearly impossible to like, readers of the novel might well take Grigory???s harsh treatment of his foster son for granted. Yet I would argue that understanding Grigory???s motives requires more than judgments about his character. A more complete analysis of Grigory???s relationship with Smerdyakov will help explain what drives Grigory to become an unloving father. To this end, I will look into the motives, both on conscious and subconscious levels, for Grigory???s antagonism toward Smerdyakov
Federated Random Reshuffling with Compression and Variance Reduction
No full textRandom Reshuffling (RR), which is a variant of Stochastic Gradient Descent (SGD) employing sampling without replacement, is an immensely popular method for training supervised machine learning models via empirical risk minimization. Due to its superior practical performance, it is embedded and often set as default in standard machine learning software. Under the name FedRR, this method was recently shown to be applicable to federated learning (Mishchenko et al.,2021), with superior performance when compared to common baselines such as Local SGD. Inspired by this development, we design three new algorithms to improve FedRR further: compressed FedRR and two variance reduced extensions: one for taming the variance coming from shuffling and the other for taming the variance due to compression. The variance reduction mechanism for compression allows us to eliminate dependence on the compression parameter, and applying additional controlled linear perturbations for Random Reshuffling, introduced by Malinovsky et al.(2021) helps to eliminate variance at the optimum. We provide the first analysis of compressed local methods under standard assumptions without bounded gradient assumptions and for heterogeneous data, overcoming the limitations of the compression operator. We corroborate our theoretical results with experiments on synthetic and real data sets
Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning
No full textWe study distributed optimization methods based on the {\em local training (LT)} paradigm: achieving communication efficiency by performing richer local gradient-based training on the clients before parameter averaging. Looking back at the progress of the field, we {\em identify 5 generations of LT methods}: 1) heuristic, 2) homogeneous, 3) sublinear, 4) linear, and 5) accelerated. The 5th generation, initiated by the ProxSkip method of Mishchenko, Malinovsky, Stich and Richtárik (2022) and its analysis, is characterized by the first theoretical confirmation that LT is a communication acceleration mechanism. Inspired by this recent progress, we contribute to the 5th generation of LT methods by showing that it is possible to enhance them further using {\em variance reduction}. While all previous theoretical results for LT methods ignore the cost of local work altogether, and are framed purely in terms of the number of communication rounds, we show that our methods can be substantially faster in terms of the {\em total training cost} than the state-of-the-art method ProxSkip in theory and practice in the regime when local computation is sufficiently expensive. We characterize this threshold theoretically, and confirm our theoretical predictions with empirical results.We would like to thank Eduard Gorbunov for useful discussions related to some aspects of the theory
Can 5th Generation Local Training Methods Support Client Sampling? Yes!
No full textThe celebrated FedAvg algorithm of McMahan et al. (2017) is based on three components: client sampling (CS), data sampling (DS) and local training (LT). While the first two are reasonably well understood, the third component, whose role is to reduce the number of communication rounds needed to train the model, resisted all attempts at a satisfactory theoretical explanation. Malinovsky et al. (2022) identified four distinct generations of LT methods based on the quality of the provided theoretical communication complexity guarantees. Despite a lot of progress in this area, none of the existing works were able to show that it is theoretically better to employ multiple local gradient-type steps (i.e., to engage in LT) than to rely on a single local gradient-type step only in the important heterogeneous data regime. In a recent breakthrough embodied in their ProxSkip method and its theoretical analysis, Mishchenko et al. (2022) showed that LT indeed leads to provable communication acceleration for arbitrarily heterogeneous data, thus jump-starting the 5th generation of LT methods. However, while these latest generation LT methods are compatible with DS, none of them support CS. We resolve this open problem in the affirmative. In order to do so, we had to base our algorithmic development on new algorithmic and theoretical foundations.The work of M. Grudzien was performed during a Summer internship at KAUST in the Optimization & Machine Learning Lab led by P. Richtárik
Intuitions of future in “existential diaries” of 1920–1930s: Grigory Tseretely, Mikhail Prishvin, Gustav Shpet
No full textA key theme of this article is the relation of the eminent Russian intellectuals: scientist (Grigory Tseretely), writer (Mikhail Prishvin), philosopher (Gustav Shpet) to educational reforms of 1920–1930s in Soviet Russia expressed in their letters and diary notes. The author considersthese records to be an invaluable existential experience which bears evidence of historical dependency of the modern state of intellectual culture and education system in Russia
Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
No full textDistributed learning has emerged as a leading paradigm for training large machine learning models. However, in real-world scenarios, participants may be unreliable or malicious, posing a significant challenge to the integrity and accuracy of the trained models. Byzantine fault tolerance mechanisms have been proposed to address these issues, but they often assume full participation from all clients, which is not always practical due to the unavailability of some clients or communication constraints. In our work, we propose the first distributed method with client sampling and provable tolerance to Byzantine workers. The key idea behind the developed method is the use of gradient clipping to control stochastic gradient differences in recursive variance reduction. This allows us to bound the potential harm caused by Byzantine workers, even during iterations when all sampled clients are Byzantine. Furthermore, we incorporate communication compression into the method to enhance communication efficiency. Under general assumptions, we prove convergence rates for the proposed method that match the existing state-of-the-art (SOTA) theoretical results. We also propose a heuristic on adjusting any Byzantine-robust method to a partial participation scenario via clipping.The work of G. Malinovsky and P. Richtárik was supported by funding from King Abdullah University of Science and Technology (KAUST): i) KAUST Baseline Research Scheme, ii) Center of Excellence for Generative AI, under award number 5940, iii) SDAIA-KAUST Center of Excellence in Artificial Intelligence and Data Science
Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
No full textDistributed learning has emerged as a leading paradigm for training large machine learning models. However, in real-world scenarios, participants may be unreliable or malicious, posing a significant challenge to the integrity and accuracy of the trained models. Byzantine fault tolerance mechanisms have been proposed to address these issues, but they often assume full participation from all clients, which is not always practical due to the unavailability of some clients or communication constraints. In our work, we propose the first distributed method with client sampling and provable tolerance to Byzantine workers. The key idea behind the developed method is the use of gradient clipping to control stochastic gradient differences in recursive variance reduction. This allows us to bound the potential harm caused by Byzantine workers, even during iterations when all sampled clients are Byzantine. Furthermore, we incorporate communication compression into the method to enhance communication efficiency. Under general assumptions, we prove convergence rates for the proposed method that match the existing state-of-the-art (SOTA) theoretical results. We also propose a heuristic on adjusting any Byzantine-robust method to a partial participation scenario via clipping.The work of G. Malinovsky and P. Richtárik was supported by funding from King Abdullah University of Science and Technology (KAUST): i) KAUST Baseline Research Scheme, ii) Center of Excellence for Generative AI, under award number 5940, iii) SDAIA-KAUST Center of Excellence in Artificial Intelligence and Data Science
“THE JOURNAL OF TRAVELS TO RUSSIA’S EASTERN LAND IS SURE TO BE INTERESTING FOR THE PUBLIC...”: ON GRIGORY I. SPASSKY’S PARTICIPATION IN THE ACTIVITY OF THE FREE SOCIETY OF LOVERS OF LITERATURE, SCIENCE AND THE ARTS
No full textThe present research is dedicated to the initial period of the academic career of Grigory I. Spassky (the editor of the first magazine about Siberia in Russia). The main issue studied in the present article is Grigory I. Spassky’s participation in the activity of the Free Society of Lovers of Literature, Science and the Arts (VOLSNKh) in 1803-1823. This particular issue has never been studied; in the Russian historiography there are a few works where this fact was only mentioned. The author has been considered and analyzed the correspondence of Grigory I. Spassky with the members of VOLSNKh, as well as the contents of periodicals of the time, which were connected with VOLSNKh, such as “Severny Vestnik” [Northern Bulletin], “Tsvetnik” [Flower Garden], “Vestnik Evropy” [European Bulletin]. The research has revealed the facts of Grigory I. Spassky’s close collaboration with the Free Society; and in the beginning the Society played a leading role in the choice of the young scientist’s research areas. Through the VOLSNKh, on the pages of the Society’s periodicals there were published Spassky’s first articles, which brought him fame in the literary and research circles. Under the infl uence of the VOLSNKh banders’ recommendations, Grigory I. Spassky got the idea to publish his Siberian Sketchbook. This idea transformed into the plan of a periodical about Siberia and “some countries adjacent to it”. The publication made by Grigory I. Spassky on his return from Siberia in 1817 was called “Sibirsky Vestnik [Siberian Bulletin]”, and Spassky’s co-editor was one of the VOLSNKh members, Vasily V. Dmitriev. Thus, the analysis of the correspondence and the articles of some periodicals, connected with the VOLSNKh undoubtedly convinces that the members of the VOLSNKh made a great influence on the formation of the scientific worldview of the explorer of Siberia, Grigory I. Spassky
“The Eavesdropped Voice of a Risible Dream”: Neo-Primitivism in Grigory Musatov’s Oil Painting of the 1920s
No full textСтатья поступила в редакцию 30.04.2015 г.Статья посвящена одному из периодов творчества русского художника-эмигранта Г. А. Мусатова (1889–1941), жившего и работавшего в Чехословакии, преимущественно в Праге в 1920–1941 гг. В тексте отражены результаты исследования живописи мастера 1920-х гг., встраивающейся в стилистику неопримитивизма. В статье описывается и анализируется специфика творческого метода Г. Мусатова и особенности использования им неопримитивистских художественных приемов. На примере конкретных живописных произведений художника разбираются черты его индивидуального подхода к переработке и интерпретации ряда явлений народной культуры и городского фольклора (иконопись, народная роспись, лубок, провинциальная портретная фотография). Творчество Г. А. Мусатова рассматривается в контексте развития неопримитивизма не только в русском, но и в чешском искусстве первой трети ХХ в.The article is devoted to one of the periods of creative work of Grigory Musatov (1889–1941), a Russian émigré artist, who lived and worked in Czechoslovakia, mostly in Prague between 1920 and 1941. The text is based on the research of Musatov’s paintings of the 1920s that fit in the stylistics of neo-primitivism. The author describes and analyzes the specificity of his creative method and the peculiarity of his neo-primitivist artistic methods. The features of Musatov’s individual way of revision and interpretation of folk culture and urban folklore (icon-painting, folk painting, popular print, provincial portrait photography) are examined referring to a number of the artist’s paintings. Grigory Musatov’s creative work is analyzed in the context of neoprimitivism development not only in Russian but also in Czech art of the first third of the 20th century
- …
