127 research outputs found
Replication Data for: LAWS IN CONFLICT: Legacies of War, Gender, and Legal Pluralism in Chechnya
No full textHow do legacies of conflict affect choices between state and nonstate legal institutions? This article studies this question in Chechnya, where state law coexists with Sharia and customary law. The author focuses on the effect of conflict-induced disruption of gender hierarchies, because nonstate legal orders are explicitly discriminatory against women. The author finds that women in Chechnya are more likely to rely on state law than men and that this gender gap in legal preferences and behavior is especially large in more-victimized communities. The author infers from this that the conflict created the conditions for women in Chechnya to pursue their interests through state law—albeit not without resistance. Women’s legal mobilization has generated a backlash from the Chechen government, which has attempted to reinstate a patriarchal order. The article concludes that such conflict may induce legal mobilization among the weak and that gender might become a central cleavage during state-building processes in postconflict environments
Replication Data for: LAWS IN CONFLICT: Legacies of War, Gender, and Legal Pluralism in Chechnya
No full textHow do legacies of conflict affect choices between state and nonstate legal institutions? This article studies this question in Chechnya, where state law coexists with Sharia and customary law. The author focuses on the effect of conflict-induced disruption of gender hierarchies, because nonstate legal orders are explicitly discriminatory against women. The author finds that women in Chechnya are more likely to rely on state law than men and that this gender gap in legal preferences and behavior is especially large in more-victimized communities. The author infers from this that the conflict created the conditions for women in Chechnya to pursue their interests through state law—albeit not without resistance. Women’s legal mobilization has generated a backlash from the Chechen government, which has attempted to reinstate a patriarchal order. The article concludes that such conflict may induce legal mobilization among the weak and that gender might become a central cleavage during state-building processes in postconflict environments
ADOM: Accelerated Decentralized Optimization Method for Time-Varying Networks
No full textWe propose ADOM - an accelerated method for smooth and strongly convex decentralized optimization over time-varying networks. ADOM uses a dual oracle, i.e., we assume access to the gradient of the Fenchel conjugate of the individual loss functions. Up to a constant factor, which depends on the network structure only, its communication complexity is the same as that of accelerated Nesterov gradient method (Nesterov, 2003). To the best of our knowledge, only the algorithm of Rogozin et al. (2019) has a convergence rate with similar properties. However, their algorithm converges under the very restrictive assumption that the number of network changes can not be greater than a tiny percentage of the number of iterations. This assumption is hard to satisfy in practice, as the network topology changes usually can not be controlled. In contrast, ADOM merely requires the network to stay connected throughout time.The work of D. Kovalev, E. Shulgin and P. Richtarik was supported by the KAUST Baseline Research Funding Scheme. The work of A. Rogozin and A. Gasnikov was supported by the Russian Science Foundation (project 21-71-30005)
Integrated File Compression and Encryption : Optimizing Security and Efficiency in Data Handling
No full textIn the era of big data, the demands for both storage efficiency and security require integrated approaches to compression and encryption. Traditional methods often compromise either the compression ratio or cryptographic strength, and many existing tools are not well adapted for various data types. This research overcomes these shortcomings by developing a modular system that simultaneously optimizes both processes while balancing performance and security.
The study proposes an integrated compression-encryption system, a Python-based framework combining DEFLATE compression and AES-256 encryption. Theoretical foundations are based on hybrid algorithms and secure key management, and a unidirectional workflow ensures data integrity. The methodology includes benchmarking on metrics such as compression ratio, throughput and memory utilization, tested on high-performance hardware under controlled conditions.
The results demonstrate DEFLATE’s superiority in speed and AES-256’s cryptographic efficiency, achieving 99.9% compression ratios. LZMA excels in compression depth but demands excessive memory, limiting edge-device applicability. The key conclusions advocate DEFLATE + AES-256 for time-sensitive tasks and highlight metadata inflation as a critical bottleneck. Future work should explore hybrid pipelines and metadata-efficient formats to enhance usability and resource allocation. This research provides actionable recommendations for industries looking for secure and scalable data handling solutions
Model Agnostic Sparsified Training
No full textWe introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function. Unlike traditional formulations, the proposed approach explicitly incorporates an initially pre-trained model and random sketch operators, allowing for sparsification of both the model and gradient during training. We establish the insightful properties of the proposed objective function and highlight its connections to the standard formulation. Furthermore, we present several variants of the Stochastic Gradient Descent (SGD) method adapted to the new problem formulation, including SGD with general sampling, a distributed version, and SGD with variance reduction techniques. We achieve tighter convergence rates and relax assumptions, bridging the gap between theoretical principles and practical applications, covering several important techniques such as Dropout and Sparse training. This work presents promising opportunities to enhance the theoretical understanding of model training through a sparsification-aware optimization approach
Model Agnostic Sparsified Training
No full textWe introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function. Unlike traditional formulations, the proposed approach explicitly incorporates an initially pre-trained model and random sketch operators, allowing for sparsification of both the model and gradient during training. We establish the insightful properties of the proposed objective function and highlight its connections to the standard formulation. Furthermore, we present several variants of the Stochastic Gradient Descent (SGD) method adapted to the new problem formulation, including SGD with general sampling, a distributed version, and SGD with variance reduction techniques. We achieve tighter convergence rates and relax assumptions, bridging the gap between theoretical principles and practical applications, covering several important techniques such as Dropout and Sparse training. This work presents promising opportunities to enhance the theoretical understanding of model training through a sparsification-aware optimization approach
Towards a Better Theoretical Understanding of Independent Subnetwork Training
No full textModern advancements in large-scale machine learning would be impossible
without the paradigm of data-parallel distributed computing. Since distributed
computing with large-scale models imparts excessive pressure on communication
channels, significant recent research has been directed toward co-designing
communication compression strategies and training algorithms with the goal of
reducing communication costs. While pure data parallelism allows better data
scaling, it suffers from poor model scaling properties. Indeed, compute nodes
are severely limited by memory constraints, preventing further increases in
model size. For this reason, the latest achievements in training giant neural
network models also rely on some form of model parallelism. In this work, we
take a closer theoretical look at Independent Subnetwork Training (IST), which
is a recently proposed and highly effective technique for solving the
aforementioned problems. We identify fundamental differences between IST and
alternative approaches, such as distributed methods with compressed
communication, and provide a precise analysis of its optimization performance
on a quadratic model.Comment: Accepted to International Conference on Machine Learning (ICML), 202
Towards a Better Theoretical Understanding of Independent Subnetwork Training
No full textModern advancements in large-scale machine learning would be impossible without the paradigm of data-parallel distributed computing. Since distributed computing with large-scale models imparts excessive pressure on communication channels, significant recent research has been directed toward co-designing communication compression strategies and training algorithms with the goal of reducing communication costs. While pure data parallelism allows better data scaling, it suffers from poor model scaling properties. Indeed, compute nodes are severely limited by memory constraints, preventing further increases in model size. For this reason, the latest achievements in training giant neural network models also rely on some form of model parallelism. In this work, we take a closer theoretical look at Independent Subnetwork Training (IST), which is a recently proposed and highly effective technique for solving the aforementioned problems. We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication, and provide a precise analysis of its optimization performance on a quadratic model.We would like to thank Avetik Karagulyan (KAUST) and Andrea Devlin (KAUST) for their helpful comments and suggestions to improve the manuscript
On the Convergence of DP-SGD with Adaptive Clipping
No full textStochastic Gradient Descent (SGD) with gradient clipping is a powerful technique for enabling differentially private optimization. Although prior works extensively investigated clipping with a constant threshold, private training remains highly sensitive to threshold selection, which can be expensive or even infeasible to tune. This sensitivity motivates the development of adaptive approaches, such as quantile clipping, which have demonstrated empirical success but lack a solid theoretical understanding. This paper provides the first comprehensive convergence analysis of SGD with quantile clipping (QC-SGD). We demonstrate that QC-SGD suffers from a bias problem similar to constant-threshold clipped SGD but show how this can be mitigated through a carefully designed quantile and step size schedule. Our analysis reveals crucial relationships between quantile selection, step size, and convergence behavior, providing practical guidelines for parameter selection. We extend these results to differentially private optimization, establishing the first theoretical guarantees for DP-QC-SGD. Our findings provide theoretical foundations for widely used adaptive clipping heuristic and highlight open avenues for future research.We would like to thank anonymous reviewers for their valuable comments, which improved the manuscript, and Abdurakhmon Sadiev for helpful technical discussions. The research reported in this publication was supported by funding from King Abdullah University
of Science and Technology (KAUST): i) KAUST Baseline Research Scheme, ii) Center of Excellence for Generative AI, under award number 5940, iii) SDAIA-KAUST Center of Excellence in Artificial Intelligence and Data Science
Towards a Better Theoretical Understanding of Independent Subnetwork Training
No full textModern advancements in large-scale machine learning would be impossible without the paradigm of data-parallel distributed computing. Since distributed computing with large-scale models imparts excessive pressure on communication channels, significant recent research has been directed toward co-designing communication compression strategies and training algorithms with the goal of reducing communication costs. While pure data parallelism allows better data scaling, it suffers from poor model scaling properties. Indeed, compute nodes are severely limited by memory constraints, preventing further increases in model size. For this reason, the latest achievements in training giant neural network models also rely on some form of model parallelism. In this work, we take a closer theoretical look at Independent Subnetwork Training (IST), which is a recently proposed and highly effective technique for solving the aforementioned problems. We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication, and provide a precise analysis of its optimization performance on a quadratic model.We would like to thank anonymous reviewers, Avetik Karagulyan (KAUST) and Andrea Devlin (KAUST) for their helpful comments and suggestions to improve the manuscript
- …
