456 research outputs found
Preserving Node-level Privacy in Graph Neural Networks
No full textDifferential privacy (DP) has seen immense applications in learning on tabular, image, and sequential data where
instance-level privacy is concerned. In learning on graphs,
contrastingly, works on node-level privacy are highly sparse.
Challenges arise as existing DP protocols hardly apply to
the message-passing mechanism in Graph Neural Networks
(GNNs).We thank all anonymous reviewer construction feedback. Di Wang and Zihang Xiang were supported by BAS/1/1689-01-01, URF/1/4663-01-01, FCC/1/1976-49-01, RGC/3/4816-01-01, and REI/1/4811-10-01 of King Abdullah University of Science and Technology (KAUST) and KAUST-SDAIA Center of Excellence in Data Science and Artificial Intelligence. Tianhao Wang is supported by CNS-2220433
sj-docx-1-spp-10.1177_19485506221089804 – Supplemental material for Does Economic Growth Raise Happiness in China? A Comprehensive Reexamination
No full textSupplemental material, sj-docx-1-spp-10.1177_19485506221089804 for Does Economic Growth Raise Happiness in China? A Comprehensive Reexamination by Huajian Cai, Jingqi Yuan, Zhan Su, Xiaoou Wang, Zihang Huang, Yiming Jing and Ziyan Yang in Social Psychological and Personality Science</p
Privacy-preserving Sparse Generalized Eigenvalue Problem
No full textIn this paper we study the (sparse) Generalized Eigenvalue Problem (GEP), which arises in a number of modern statistical learning models, such as principal component analysis (PCA), canonical correlation analysis (CCA), Fisher's discriminant analysis (FDA) and sliced inverse regression (SIR). We provide the first study on GEP in the differential privacy (DP) model under both deterministic and stochastic settings. In the low dimensional case, we provide a ρ- Concentrated DP (CDP) method namely DP-Rayleigh Flow and show if the initial vector is close enough to the optimal vector, its output has an ℓ2-norm estimation error of Õ(n/d + d/n2ρ) (under some mild assumptions), where d is the dimension and n is the sample size. Next, we discuss how to find such a initial parameter privately. In the high dimensional sparse case where d ≫ n, we propose the DP-Truncated Rayleigh Flow method whose output could achieve an error of Õ(s log d/n + s log d/n2ρ) for various statistical models, where s is the sparsity of the underlying parameter. Moreover, we show that these errors in the stochastic setting are optimal up to a factor of Poly(log n) by providing the lower bounds of PCA and SIR under statistical setting and in the CDP model. Finally, to give a separation between ∊-DP and ρ-CDP for GEP, we also provide the lower bound Ω(d/n + d2/n2 ∊2) and Ω(s log d/n + s2 log2d/n2∊2) of private minimax risk for PCA, under the statistical setting and ∊-DP model, in low and high dimensional sparse case respectively.Lijie Hu, Zihang Xiang and Di Wang are supported in part by the baseline funding BAS/1/1689-01-01, funding from the CRG grand URF/1/4663-01-01, FCC/1/1976-49-01 from CBRC and funding from the AI Initiative REI/1/4811-10-01 of King Abdullah University of Science and Technology (KAUST). Di Wang was also supported by the funding of the SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI)
Nearly Optimal Rates of Privacy-preserving Sparse Generalized Eigenvalue Problem
No full textIn this paper, we study the (sparse) Generalized Eigenvalue Problem (GEP), which arises in a number of modern statistical learning models, such as principal component analysis (PCA), canonical correlation analysis (CCA), Fisher's discriminant analysis (FDA) and sliced inverse regression (SIR). We provide the first study on GEP in the differential privacy (DP) model under both deterministic and stochastic settings. In the low dimensional case, we provide a ρ -Concentrated DP (CDP) method namely DP-Rayleigh Flow and show if the initial vector is close enough to the optimal vector, its output has an ℓ2 -norm estimation error of O~(dn+dn2ρ) (under some mild assumptions), where d is the dimension and n is the sample size. Next, we discuss how to find such an initial parameter privately. In the high dimensional sparse case where d≫n , we propose the DP-Truncated Rayleigh Flow method whose output could achieve an error of O~(slogdn+slogdn2ρ) for various statistical models, where s is the sparsity of the underlying parameter. Moreover, we show that these errors in the stochastic setting are optimal up to a factor of Poly(logn) by providing the lower bounds of PCA and SIR under the statistical setting and in the CDP model. Finally, to give a separation between ϵ -DP and ρ -CDP for GEP, we also provide the lower bound Ω(dn+d2n2ϵ2) and Ω(slogdn+s2log2dn2ϵ2) of private minimax risk for PCA, under the statistical setting and ϵ -DP model, in low and high dimensional sparse case respectively. Finally, extensive experiments on both synthetic and real-world data support our previous theoretical analysis.Lijie Hu, Zihang Xiang and Di Wang are supported in part by the baseline funding BAS/1/1689-01-01, funding from the CRG grand URF/1/4663-01-01, FCC/1/1976-49-01 from CBRC and funding from the AI Initiative REI/1/4811-10-01 of King Abdullah University of Science and Technology (KAUST). Di Wang was also supported by the funding of the SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI)
rGO/CNTs Supported Pyrolysis Derivatives of [Mo<sub>3</sub>S<sub>13</sub>]<sup>2–</sup> Clusters as Promising Electrocatalysts for Enhancing Hydrogen Evolution Performances
No full textReduced graphene oxide/carbon nanotube (rGO/CNTs) supported [Mo3S13]2– clusters and [Mo3S13]2– pyrolysis derivatives were synthesized as electrocatalysts for hydrogen production. We investigated the physio-chemical characteristics and electrocatalytic abilities of the [Mo3S13]2– clusters and their pyrolysis derivatives. TEM images of pyrolysis derivatives of [Mo3S13]2– clusters indicated that some crystalline derivatives were surrounded by the amorphous derivatives at an annealing temperature of 200–270 °C, and some well-crystallized MoS2 with diameters of 50–100 nm were observed in the pyrolysis derivatives at 500 °C. Both the structure transition and the HER performance of [Mo3S13]2– pyrolysis derivatives were mapped in terms of temperature. The atomic ratio of S:Mo significantly decreased from 3.48 to 1.89 as the annealing temperature increased, which indicated the multiple transition forms in pyrolysis derivatives. XPS, XRD, and Raman spectra also indicated the decreased density of edge sites and a poor extent of ordering in the layers of pyrolysis derivatives as the annealing temperature increased. These results corresponded well to the HER activities of the rGO/CNTs macrostructures anchored with different pyrolysis derivatives. The rGO/CNTs anchored with pyrolysis derivatives (annealed at 270 °C) of [Mo3S13]2– exhibited an overpotential of ∼178 mV (10 mA cm–2) with Tafel slope value located at 64.2 mV/dec, which showed relatively higher HER performances than most analogous single-metal molybdenum sulfide nanocomposites. They also exhibited a performance close to those of multimetal nanocomposites.Yanan Shang, Xing Xu, Zihang Wang, Bo Jin, Rui Wang, Zhongfei Ren, Baoyu Gao, Qinyan Yu
A Theory to Instruct Differentially-Private Learning via Clipping Bias Reduction
No full textWe study the bias introduced in Differentially-Private Stochastic Gradient Descent (DP-SGD) with clipped or normalized per-sample gradient. As one of the most popular but artificial operations to ensure bounded sensitivity, gradient clipping enables composite privacy analysis of many iterative optimization methods without additional assumptions on either learning models or input data. Despite its wide applicability, gradient clipping also presents theoretical challenges in systematically instructing improvement of privacy or utility. In general, without an assumption on globally-bounded gradient, classic convergence analyses do not apply to clipped gradient descent. Further, given limited understanding of the utility loss, many existing improvements to DP-SGD are heuristic, especially in the applications of private deep learning.In this paper, we provide meaningful theoretical analysis validated by thorough empirical results of DP-SGD. We point out that the bias caused by gradient clipping is underestimated in previous works. For generic non-convex optimization via DP-SGD, we show one key factor contributing to the bias is the sampling noise of stochastic gradient to be clipped. Accordingly, we use the developed theory to build a series of improvements for sampling noise reduction from various perspectives. From an optimization angle, we study variance reduction techniques and propose inner-outer momentum. At the learning model (neural network) level, we propose several tricks to enhance network internal normalization and BatchClipping to carefully clip the gradient of a batch of samples. For data preprocessing, we provide theoretical justification of recently proposed improvements via data normalization and (self-)augmentation.Putting these systematic improvements together, private deep learning via DP-SGD can be significantly strengthened in many tasks. For example, in computer vision applications, with an (ϵ = 8, δ = 10 −5 ) DP guarantee, we successfully train ResNet20 on CIFAR10 and SVHN with test accuracy 76.0% and 90.1%, respectively; for natural language processing, with (ϵ = 4, δ = 10 −5 ), we successfully train a recurrent neural network on IMDb data with test accuracy 77.5%.We would like to thank Jun Wan for very helpful discussion and for reading several drafts of this paper. We also thank the anonymous reviewers for their constructive feedback. Hanshen Xiao was supported in part by DSTA, Singapore, and a MathWorks fellowship. Di Wang and Zihang Xiang were supported by BAS/1/1689-01-01, URF/1/4663-01-01, FCC/1/1976-49-01, RGC/3/4816-01-01, and REI/1/4811-10-01 of King Abdullah University of Science and Technology (KAUST) and KAUSTSDAIA Center of Excellence in Data Science and Artificial Intelligence
Four‐Peak Structure on Equatorial Ionization Anomaly Crests During the May 2024 Geomagnetic Storms
No full textThis study investigates the four-peak electron density structure in the Equatorial Ionization Anomaly (EIA) during the intense geomagnetic storms of 10–20 May 2024, utilizing dayside data at ∼500 km from the China Seismo-Electromagnetic Satellite (CSES-01). Observations revealed distinct four peaks in latitudinal profiles of electron density, symmetrically distributed across both hemispheres. Particularly, the poleward crests can expand beyond ±30° quasi-dipole latitude during the 10 May superstorm. To reveal the potential driver of such unique EIA structure, statistical analysis is conducted on 35 CSES-01 orbits which identified the four-peak phenomenon. The enhanced equatorial electrojet activity, inferred from the scalar magnetic field residuals, indicates that the four-peak morphology is likely related to the prompt penetration electric field (PPEF). Superposed epoch analysis also highlighted that intensification of the mean SuperMAG |SML| index preceded the onset of four-peak structures by ∼20 min, whereas the mean eastward equatorial electric field, derived from an empirical model of the interplanetary electric field driven penetration, exhibits no statistically significant enhancement, implying that substorm-associated PPEF may be the primary driver of this type of EIA structure. This study advances understanding of EIA variability under extreme space weather conditions and the findings underscore the critical role of magnetosphere-ionosphere coupling via substorm-driven electric fields in reshaping storm-time ionospheric plasma distribution
IMPROVING ATTENTION-BASED DEEP LEARNING MODELS WITH LOCALITY
No full textPh.DDOCTOR OF PHILOSOPHY (CDE-ENG
<b>10m winter wheat harvested area and planted area distribution map of China for five years (2018-2022)</b>
No full text------ 1. Introductory information ------Title: Mapping 10-m harvested area in the major winter wheat-producing regions of China from 2018 to 2022Format: TIFNaming convention: " ChinaWheatMap10_P_2018.tif" means winter wheat planted area map of 2018, and “ChinaWheatMap10_H_2018.tif" means winter wheat harvested area map of 2018.Authors: Jinkang Hu, Bing Zhang, Dailiang Peng, Jianxi Huang, Wenjuan Zhang, Bin Zhao, Enhui Cheng, Zihang Lou, Shengwei Liu, Songlin Yang, Yunlong Tan, and Yulong LvKey Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of SciencesCorresponding author: Bing ZhangContact Information: [email protected] (JK.H.); [email protected] (B.Z.)------ 2. Data specific information ------The dataset contains winter wheat maps of harvested area and planted area with 10m spatial resolution for five years (2018-2022) over eight provinces. In the datasets, the values equal to one means winter wheat.Software: ArcGIS, QGIS or ENVI are needed to read the dataset.</p
INVESTIGATING THE IMPACT OF ROADWAY GEOMETRY, SPEED DISTRIBUTION, AND WEATHER CONDITION ON ROADWAY DAILY CRASH OCCURRENCE AND SEVERITY BY USING MACHINE LEARNING METHODS
No full textConventional traffic crash analysis methods often use highly aggregated data, making it difficult to understand the effects of many time-varying factors on crash occurrence. Although studies have used data with small aggregation intervals, they typically analyze the effect of a single factor on crash occurrence. In this study, the collaborative effect of roadway geometry, speed distribution, and weather conditions on crash occurrence and severity is investigated using an interpretable or explainable machine learning method XGBoost (eXtreme Gradient Boosting) on daily level crash data. The data are collected from four different sources on roadways in Texas. Three roadway facility types are considered in this study: (1) Rural Interstate; (2) Rural Two-Lane; (3) Rural Multilane. In the feature selection process, the Pearson correlation coefficient is applied to remove highly correlated variables. The study then uses the synthetic minority over-sampling technique (SMOTE) method to mitigate the data imbalance issue. The XGBoost model is trained twice: first on data with all crash severity levels, and then only on data with fatal and severe injury crash levels. Finally, the SHAP (SHapley Additive exPlanation) method is applied to investigate the contribution of all variables on the model���s output. The results show that on different roadways facility types the contributions of variables tend to be different, and moreover, the variables also contribute differently on crashes with different severity levels
- …
