1,720,975 research outputs found
A Decentralized Matching Theory Framework to Match Data and Algorithms Providers
Get PDFAbstract
This paper presents a novel decentralized matching algorithm (DEMA) for pairing data and algorithm providers in AI ecosystems. DEMA addresses scalability, stability, and matching utility challenges in large-scale environments. Formulated as a two-sided matching game, our decentralized solution enables autonomous decision-making based on local information. Simulations demonstrate DEMA’s near-optimal matching quality and almost perfect stability. Furthermore, DEMA exhibits excellent scalability with execution times and memory usage growing much more slowly than centralized matching as the number of providers increases.Abstract
This paper presents a novel decentralized matching algorithm (DEMA) for pairing data and algorithm providers in AI ecosystems. DEMA addresses scalability, stability, and matching utility challenges in large-scale environments. Formulated as a two-sided matching game, our decentralized solution enables autonomous decision-making based on local information. Simulations demonstrate DEMA’s near-optimal matching quality and almost perfect stability. Furthermore, DEMA exhibits excellent scalability with execution times and memory usage growing much more slowly than centralized matching as the number of providers increases
Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation
No full textAbstract
Second-order federated learning (FL) algorithms offer faster convergence than their first-order counterparts by leveraging curvature information. However, they are hindered by high computational and storage costs, particularly for large-scale models. Furthermore, the communication overhead associated with large models and digital transmission exacerbates these challenges, causing communication bottlenecks. In this work, we propose a scalable second-order FL algorithm using a sparse Hessian estimate and leveraging over-the-air aggregation, making it feasible for larger models. Our simulation results demonstrate more than 67% of communication resources and energy savings compared to other first and second-order baselines.Abstract
Second-order federated learning (FL) algorithms offer faster convergence than their first-order counterparts by leveraging curvature information. However, they are hindered by high computational and storage costs, particularly for large-scale models. Furthermore, the communication overhead associated with large models and digital transmission exacerbates these challenges, causing communication bottlenecks. In this work, we propose a scalable second-order FL algorithm using a sparse Hessian estimate and leveraging over-the-air aggregation, making it feasible for larger models. Our simulation results demonstrate more than 67% of communication resources and energy savings compared to other first and second-order baselines
A Web-Based Solution for Federated Learning with LLM-Based Automation
Get PDFFederated Learning (FL) offers a promising approach for collaborative machine learning across distributed devices. However, its adoption is hindered by the complexity of building reliable communication architectures and the need for expertise in both machine learning and network programming. This paper presents a comprehensive solution that simplifies the orchestration of FL tasks while integrating intent-based automation. We develop a user-friendly web application supporting the federated averaging (FedAvg) algorithm, enabling users to configure parameters through an intuitive interface. The backend solution efficiently manages communication between the parameter server and edge nodes. We also implement model compression and scheduling algorithms to optimize FL performance. Furthermore, we explore intent-based automation in FL using a fine-tuned Language Model (LLM) trained on a tailored dataset, allowing users to conduct FL tasks using high-level prompts. We observe that the LLM-based automated solution achieves comparable test accuracy to the standard web-based solution while reducing transferred bytes by up to 64% and CPU time by up to 46% for FL tasks. Also, we leverage the neural architecture search (NAS) and hyperparameter optimization (HPO) using LLM to improve the performance. We observe that by using this approach test accuracy can be improved by 10-20% for the carried out FL tasks
A Web-Based Solution for Federated Learning With LLM-Based Automation
Get PDFAbstract
Federated Learning (FL) offers a promising approach for collaborative machine learning across distributed devices. However, its adoption is hindered by the complexity of building reliable communication architectures and the need for expertise in both machine learning and network programming. This paper presents a comprehensive solution that simplifies the orchestration of FL tasks while integrating intent-based automation. A user-friendly web application is developed supporting the federated averaging (FedAvg) algorithm, enabling users to configure parameters through an intuitive interface. The backend solution efficiently manages communication between the parameter server and edge nodes. Model compression and scheduling algorithms are implemented to optimize FL performance. Additionally, intent-based automation in FL is explored using a fine-tuned Language Model (LLM) trained on a tailored dataset, enabling users to perform FL tasks through high-level prompts. It is shown that the LLM-based automated solution achieves comparable test accuracy to the standard web-based solution while reducing transferred bytes by up to 64% and CPU time by up to 46% for FL tasks. Furthermore, neural architecture search (NAS) and hyperparameter optimization (HPO) are leveraged using the LLM to enhance performance, resulting in a 10-20% improvement in test accuracy for the conducted FL tasks.Abstract
Federated Learning (FL) offers a promising approach for collaborative machine learning across distributed devices. However, its adoption is hindered by the complexity of building reliable communication architectures and the need for expertise in both machine learning and network programming. This paper presents a comprehensive solution that simplifies the orchestration of FL tasks while integrating intent-based automation. A user-friendly web application is developed supporting the federated averaging (FedAvg) algorithm, enabling users to configure parameters through an intuitive interface. The backend solution efficiently manages communication between the parameter server and edge nodes. Model compression and scheduling algorithms are implemented to optimize FL performance. Additionally, intent-based automation in FL is explored using a fine-tuned Language Model (LLM) trained on a tailored dataset, enabling users to perform FL tasks through high-level prompts. It is shown that the LLM-based automated solution achieves comparable test accuracy to the standard web-based solution while reducing transferred bytes by up to 64% and CPU time by up to 46% for FL tasks. Furthermore, neural architecture search (NAS) and hyperparameter optimization (HPO) are leveraged using the LLM to enhance performance, resulting in a 10-20% improvement in test accuracy for the conducted FL tasks
Distributionally Robust Federated Learning with Client Drift Minimization
No full textAbstract
Federated learning (FL) faces critical challenges, particularly in heterogeneous environments where non-independent and identically distributed (non-IID) data across clients can lead to unfair and inefficient model performance. We introduce DRDM, a novel algorithm that integrates distributionally robust optimization (DRO) with dynamic regularization to explicitly mitigate client drift. Compared to previous approaches that address robustness or drift separately, DRDM combines both aspects within a unified framework, dynamically aligning local updates with the global robust objective to improve convergence toward a worst-case optimal model while maintaining fairness across clients. The robust objective is optimized through efficient local updates, which significantly reduce the number of communication rounds. We provide a theoretical convergence analysis for convex smooth objectives under partial client participation and multiple local update steps. Experiments on three benchmark datasets, covering various model architectures and levels of data heterogeneity, show that DRDM consistently improves worst-case test accuracy while requiring fewer communication rounds than state-of-the-art baselines. Furthermore, we analyze the impact of signal-to-noise ratio (SNR) and bandwidth on energy consumption, demonstrating that adaptive selection of local updates can achieve a target worst-case accuracy with minimal total energy cost across diverse communication environments.Abstract
Federated learning (FL) faces critical challenges, particularly in heterogeneous environments where non-independent and identically distributed (non-IID) data across clients can lead to unfair and inefficient model performance. We introduce DRDM, a novel algorithm that integrates distributionally robust optimization (DRO) with dynamic regularization to explicitly mitigate client drift. Compared to previous approaches that address robustness or drift separately, DRDM combines both aspects within a unified framework, dynamically aligning local updates with the global robust objective to improve convergence toward a worst-case optimal model while maintaining fairness across clients. The robust objective is optimized through efficient local updates, which significantly reduce the number of communication rounds. We provide a theoretical convergence analysis for convex smooth objectives under partial client participation and multiple local update steps. Experiments on three benchmark datasets, covering various model architectures and levels of data heterogeneity, show that DRDM consistently improves worst-case test accuracy while requiring fewer communication rounds than state-of-the-art baselines. Furthermore, we analyze the impact of signal-to-noise ratio (SNR) and bandwidth on energy consumption, demonstrating that adaptive selection of local updates can achieve a target worst-case accuracy with minimal total energy cost across diverse communication environments
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Communication-Efficient Second-Order Newton-Type Approach for Decentralized Learning
Get PDFAbstractIn this paper, we propose a decentralized Newton-type approach to solve the problem of decentralized federated learning (FL). Notably, our proposed algorithm leverages the fast convergence of the second-order methods while avoid sending the hessian matrix at each iteration. Therefore, the proposed approach significantly reduces the communication cost and preserves the privacy. Specifically, we alternate between two problems. The inner problem approximates the inverse Hessian-gradient product which is formulated as a quadratic optimization problem and approximately solved in a decentralized manner using one step of the group alternating direction method of multipliers (GADMM) method. The outer problem learns the model, which is solved by performing one decentralized Newton step at every iteration. Moreover, to reduce the communication-overhead per iteration, a quantized version (leveraging stochastic quantization) is also proposed. Simulation results illustrate that our algorithm outperforms the baselines of GADMM, Q-GADMM, Newton tracking, and Decentralized SGD, and provides energy and communication-efficient solutions for bandwidth-limited systems under different SNR regimes.Abstract
In this paper, we propose a decentralized Newton-type approach to solve the problem of decentralized federated learning (FL). Notably, our proposed algorithm leverages the fast convergence of the second-order methods while avoid sending the hessian matrix at each iteration. Therefore, the proposed approach significantly reduces the communication cost and preserves the privacy. Specifically, we alternate between two problems. The inner problem approximates the inverse Hessian-gradient product which is formulated as a quadratic optimization problem and approximately solved in a decentralized manner using one step of the group alternating direction method of multipliers (GADMM) method. The outer problem learns the model, which is solved by performing one decentralized Newton step at every iteration. Moreover, to reduce the communication-overhead per iteration, a quantized version (leveraging stochastic quantization) is also proposed. Simulation results illustrate that our algorithm outperforms the baselines of GADMM, Q-GADMM, Newton tracking, and Decentralized SGD, and provides energy and communication-efficient solutions for bandwidth-limited systems under different SNR regimes
Appropriate Similarity Measures for Author Cocitation Analysis
Get PDFWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
- …
