52 research outputs found
Clustered balanced minimum spanning tree for routing and energy reduction in wireless sensor networks
No full text
S1 Data -
No full textRecently big data and its applications had sharp growth in various fields such as IoT, bioinformatics, eCommerce, and social media. The huge volume of data incurred enormous challenges to the architecture, infrastructure, and computing capacity of IT systems. Therefore, the compelling need of the scientific and industrial community is large-scale and robust computing systems. Since one of the characteristics of big data is value, data should be published for analysts to extract useful patterns from them. However, data publishing may lead to the disclosure of individuals’ private information. Among the modern parallel computing platforms, Apache Spark is a fast and in-memory computing framework for large-scale data processing that provides high scalability by introducing the resilient distributed dataset (RDDs). In terms of performance, Due to in-memory computations, it is 100 times faster than Hadoop. Therefore, Apache Spark is one of the essential frameworks to implement distributed methods for privacy-preserving in big data publishing (PPBDP). This paper uses the RDD programming of Apache Spark to propose an efficient parallel implementation of a new computing model for big data anonymization. This computing model has three-phase of in-memory computations to address the runtime, scalability, and performance of large-scale data anonymization. The model supports partition-based data clustering algorithms to preserve the λ-diversity privacy model by using transformation and actions on RDDs. Therefore, the authors have investigated Spark-based implementation for preserving the λ-diversity privacy model by two designed City block and Pearson distance functions. The results of the paper provide a comprehensive guideline allowing the researchers to apply Apache Spark in their own researches.</div
A distributed computing model for big data anonymization in the networks.
No full textRecently big data and its applications had sharp growth in various fields such as IoT, bioinformatics, eCommerce, and social media. The huge volume of data incurred enormous challenges to the architecture, infrastructure, and computing capacity of IT systems. Therefore, the compelling need of the scientific and industrial community is large-scale and robust computing systems. Since one of the characteristics of big data is value, data should be published for analysts to extract useful patterns from them. However, data publishing may lead to the disclosure of individuals' private information. Among the modern parallel computing platforms, Apache Spark is a fast and in-memory computing framework for large-scale data processing that provides high scalability by introducing the resilient distributed dataset (RDDs). In terms of performance, Due to in-memory computations, it is 100 times faster than Hadoop. Therefore, Apache Spark is one of the essential frameworks to implement distributed methods for privacy-preserving in big data publishing (PPBDP). This paper uses the RDD programming of Apache Spark to propose an efficient parallel implementation of a new computing model for big data anonymization. This computing model has three-phase of in-memory computations to address the runtime, scalability, and performance of large-scale data anonymization. The model supports partition-based data clustering algorithms to preserve the λ-diversity privacy model by using transformation and actions on RDDs. Therefore, the authors have investigated Spark-based implementation for preserving the λ-diversity privacy model by two designed City block and Pearson distance functions. The results of the paper provide a comprehensive guideline allowing the researchers to apply Apache Spark in their own researches
Fig 4 -
No full textSchema of hierarchical data clustering in Spark framework: (a) reading dataset from HDFS into worker nodes. (b) Assigning a unique key to all recrods. (c) The first round of data clustering and forming sub-clusters. (d) The second round of data clustering and forming smaller sub-clusters.</p
Spark cluster architecture [19,21].
No full textRecently big data and its applications had sharp growth in various fields such as IoT, bioinformatics, eCommerce, and social media. The huge volume of data incurred enormous challenges to the architecture, infrastructure, and computing capacity of IT systems. Therefore, the compelling need of the scientific and industrial community is large-scale and robust computing systems. Since one of the characteristics of big data is value, data should be published for analysts to extract useful patterns from them. However, data publishing may lead to the disclosure of individuals’ private information. Among the modern parallel computing platforms, Apache Spark is a fast and in-memory computing framework for large-scale data processing that provides high scalability by introducing the resilient distributed dataset (RDDs). In terms of performance, Due to in-memory computations, it is 100 times faster than Hadoop. Therefore, Apache Spark is one of the essential frameworks to implement distributed methods for privacy-preserving in big data publishing (PPBDP). This paper uses the RDD programming of Apache Spark to propose an efficient parallel implementation of a new computing model for big data anonymization. This computing model has three-phase of in-memory computations to address the runtime, scalability, and performance of large-scale data anonymization. The model supports partition-based data clustering algorithms to preserve the λ-diversity privacy model by using transformation and actions on RDDs. Therefore, the authors have investigated Spark-based implementation for preserving the λ-diversity privacy model by two designed City block and Pearson distance functions. The results of the paper provide a comprehensive guideline allowing the researchers to apply Apache Spark in their own researches.</div
Balancing Speed and Accuracy in Influence Maximization: A Reinforcement Learning Solution
No full textInfluence maximization involves selecting an optimal subset of nodes within a graph to activate as many nodes as possible in a network. This approach is categorized as non-polynomial time, and no specific algorithm is currently available to run efficiently within a reasonable time frame, especially for large-scale networks. Numerous methods have been introduced to resolve this challenge, including greedy algorithms, structural heuristics, and metaheuristic approaches. Although greedy algorithms and their improved versions achieve high accuracy, they often suffer from poor scalability and slow execution times on large graphs. In contrast, structural methods offer faster computation but at the cost of reduced accuracy. Metaheuristic algorithms, while promising, face difficulties in balancing speed and accuracy due to the expansive search space inherent in complex social networks. This study introduces a novel method that leverages Q-learning, a reinforcement learning technique, to optimize influence maximization. The proposed method narrows down the search space by focusing on high-degree influential nodes. It dynamically updates the Q-table by assigning rewards and penalties based on the nodes’ impact during influence propagation, modeled using the Independent Cascade framework. This approach effectively balances exploration and exploitation, enabling the identification of a highly influential seed set with improved efficiency. Experiments conducted on various real-world datasets show that the Q-learning-based method significantly reduces execution time compared to genetic, particle swarm optimization, random, degree centrality, and K-shell algorithms while achieving higher influence spread in most cases. These results underscore the promise of reinforcement learning techniques in addressing complex network optimization problems such as influence maximization.
pone.0285212.t005 - A distributed computing model for big data anonymization in the networks
No full textpone.0285212.t005 - A distributed computing model for big data anonymization in the networks</p
Main steps of the proposed three-phase computing model.
No full textMain steps of the proposed three-phase computing model.</p
- …
