7,771 research outputs found
G-Rank: Unsupervised Continuous Learn-to-Rank for Edge Devices in a P2P Network
No full textRanking algorithms in traditional search engines are powered by enormous training data sets that are meticulously engineered and curated by a centralized entity. Decentralized peer-to-peer (p2p) networks such as torrenting applications and Web3 protocols deliberately eschew centralized databases and computational architectures when designing services and features. As such, robust search-and-rank algorithms designed for such domains must be engineered specifically for decentralized networks, and must be lightweight enough to operate on consumer-grade personal devices such as a smartphone or laptop computer. We introduce G-Rank, an unsupervised ranking algorithm designed exclusively for decentralized networks. We demonstrate that accurate, relevant ranking results can be achieved in fully decentralized networks without any centralized data aggregation, feature engineering, or model training. Furthermore, we show that such results are obtainable with minimal data preprocessing and computational overhead, and can still return highly relevant results even when a user’s device is disconnected from the network. G-Rank is highly modular in design, is not limited to categorical data, and can be implemented in a variety of domains with minimal modification. The results herein show that unsupervised ranking models designed for decentralized p2p networks are not only viable, but worthy of further research.https://github.com/awrgold/G-RankComputer Scienc
A solver for clustered low-rank SDPs arising from multivariate polynomial (matrix) programs
No full textIn this thesis, we give a primal-dual interior point method specialized to clustered low-rank semidefinite programs. We introduce multivariate polynomial matrix programs, and we reduce these to clustered low-rank semidefinite programs. This extends the work of Simmons-Duffin [J. High Energ. Phys. 1506, no. 174 (2015)] from univariate to multivariate polynomial matrix programs, and to more general clustered low-rank semidefinite programs. Clustered low-rank semidefinite programs are programs with low-rank constraint matrices where the positive semidefinite variables are only used within clusters of constraints. The free variables can be used in any constraint, and can be used to connect clusters. The solver uses this structure to speed up the computations in two ways. First, the low rank structure is used to reduce matrix products to products of the form uT M v, where M is a matrix and u and v are vectors, as already suggested by Löfberg and Parrilo in [43rd IEEE CDC (2004)]. Second, an additional block-diagonal structure is introduced due to the clusters. This gives the possibility to do operations such as the Cholesky decomposition block-wise. We apply this to sphere packing and spherical cap packing. For sphere packing, the speed of the solver is compared to the often used arbitrary precision solver SDPA-GMP, showing the theoretical speedup in time complexity. We give a new three-point bound for the maximum density when packing spherical caps of sizes on the unit sphere. https://github.com/nanleij/Clustered-Low-Rank-SDP-solver Repository link Github repository with the Julia code of the solverApplied Mathematics | Optimizatio
Reduced-rank adaptive least bit-error-rate detection in hybrid direct-sequence time-hopping ultrawide bandwidth systems
No full textDesign of high-efficiency low-complexity detection schemes for ultrawide bandwidth (UWB) systems is highly challenging. This contribution proposes a reduced-rank adaptive multiuser detection (MUD) scheme operated in least bit-errorrate (LBER) principles for the hybrid direct-sequence timehopping UWB (DS-TH UWB) systems. The principal component analysis (PCA)-assisted rank-reduction technique is employed to obtain a detection subspace, where the reduced-rank adaptive LBER-MUD is carried out. The reduced-rank adaptive LBERMUD is free from channel estimation and does not require the knowledge about the number of resolvable multipaths as well as the knowledge about the multipaths’ strength. In this contribution, the BER performance of the hybrid DS-TH UWB systems using the proposed detection scheme is investigated, when assuming communications over UWB channels modeled by the Saleh-Valenzuela (S-V) channel model. Our studies and performance results show that, given a reasonable rank of the detection subspace, the reduced-rank adaptive LBER-MUD is capable of efficiently mitigating the multiuser interference (MUI) and inter-symbol interference (ISI), and achieving the diversity gain promised by the UWB systems
Aggregation and Other Biases in the Calculation of Consumer Elasticities for Models of Arbitrary Rank
No full textConsumer-related policy decisions often require analysis of aggregate responses or mean elasticities. However, in practice these mean elasticities are seldom used. Mean elasticities can be approximated using aggregate data, but that introduces aggregation bias for full and compensated price elasticities, though interestingly not for expenditure elasticities. The biases corresponding to incorrect approximations of mean elasticities depend on the type of data (micro or aggregate), the type and rank of the model, and generalized measures of income inequality. These biases are distinct from the biases (already noted in the literature) when using aggregate data to estimate micro elasticites at mean income.Aggregate price and expenditure elasticities, aggregation bias, consumer demand, generalized measures of income inequality, income distribution
MeSH term explosion and author rank improve expert recommendations
No full textInformation overload is an often-cited phenomenon that reduces the productivity, efficiency and efficacy of scientists. One challenge for scientists is to find appropriate collaborators in their research. The literature describes various solutions to the problem of expertise location, but most current approaches do not appear to be very suitable for expert recommendations in biomedical research. In this study, we present the development and initial evaluation of a vector space model-based algorithm to calculate researcher similarity using four inputs: 1) MeSH terms of publications; 2) MeSH terms and author rank; 3) exploded MeSH terms; and 4) exploded MeSH terms and author rank. We developed and evaluated the algorithm using a data set of 17,525 authors and their 22,542 papers. On average, our algorithms correctly predicted 2.5 of the top 5/10 coauthors of individual scientists. Exploded MeSH and author rank outperformed all other algorithms in accuracy, followed closely by MeSH and author rank. Our results show that the accuracy of MeSH term-based matching can be enhanced with other metadata such as author rank
Matrix semigroups with commutable rank
No full textWe focus on matrix semigroups (and algebras) on which rank is commutable [rank(AB) = rank(BA)]. It is shown that in a number of cases (for example, in dimensions less than 6), but not always, commutativity of rank entails permutability of rank [rank(A(1)A(2)...A(n)) = rank(A(sigma(1))A(sigma(2))... A(sigma(n)))]. It is shown that a commutable-rank semigroup has a natural decomposition as a semi-lattice of semigroups that have a simpler structure. While it is still unknown whether commutativity of rank entails permutability of rank for algebras, the question is reduced to the case of algebras of nilpotents.PT: J; CR: ANDERSON FW, 1992, GRADUATE TEXTS MATH ANDO T, 1987, LINEAR ALGEBRA APPL, V90, P165 GANTMACHER FR, 1937, COMPOS MATH, P445 HORN RA, 1990, MATRIX ANAL LEVITZKI J, 1931, MATH ANN, V105, P620 LIVSHITS L, 1998, J OPERAT THEOR, V40, P35 OKNINSKI J, 1998, SERIES ALGEBRA, V6 PRASOLOV VV, 1994, PROBLEMS THEOREMS LI RADJAVI H, 2000, SIMULTANEOUS TRIANGU WHITNEY AM, 1952, J ANAL MATH, V2, P88; NR: 10; TC: 1; J9: SEMIGROUP FORUM; PG: 29; GA: 698NQSource type: Electronic(1
Low-rank-based residual statics estimation and correction
No full textSurface consistency forms the basis for short-wavelength statics estimation. When raypaths in the near surface diverge from a normal incidence or when the normal moveout (NMO) velocity is inaccurate, surface-consistent methods may fail to estimate accurate statics. Existing nonsurface-consistent techniques can be prone to errors due to the need to construct pilot traces or pick horizons while imposing additional computational costs. To overcome these limitations and correct for the surface- and nonsurface-consistent statics, we have developed a low-rank-based residual statics (LR-ReS) estimation and correction framework. The method makes use of the redundant nature of seismic data by using its low-rank structure in the midpoint-offset-frequency domain. Due to the near-surface effect, the low-rank structure is destroyed. Therefore, we estimate the statics by means of low-rank approximation and crosscorrelation. To alleviate the need for accurate rank selection for low-rank approximation and improved statics estimation, we implement the method in an iterative and multiscale fashion. Because the low-rank approximation deteriorates at high frequencies, we use its better performance at low frequencies and exploit the common statics among the different frequency bands. The LR-ReS estimation and correction can be applied to data without an NMO correction, which makes statics estimation independent of the NMO velocity errors. Consequently, it can reduce the multiple iterations of the NMO velocity estimation and short-wavelength statics correction commonly needed for conventional methods to improve their performance. Moreover, the LR-ReS estimation does not require windowing of a noise-free area containing aligned primaries or mute to avoid the NMO stretch effect, which enables statics correction of the wavefield of all offsets. To evaluate the performance of our method, we apply it to simulated data and a challenging field data set affected by complex weathering layers and noise, which indicate a substantial improvement compared with conventional short-wavelength statics correction.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.ImPhys/Medical ImagingImPhys/Verschuur grou
2022 Social Vulnerability by US Census Block Group
No full textblockgroupvulnerability
OPPORTUNITY
The US Centers for Disease Control (CDC) publishes a set of percentiles that compare US geographies by vulnerability across household, socioeconomic, racial/ethnic and housing themes. These Social Vulnerability Indexes (SVI) were originally intended to to help public health officials and emergency response planners identify communities that will need support around an event. They are generally valuable for any public interest that wants to relate themselves to needy communities by geography.
The SVI publication and its basis variables are provided at the Census tract level of geographic detail. The Census' American Community Survey is available down the to the block group level, however. Recasting the SVI methods at this lower level of geography allows it to be tied to thousands of other demographic variables available.
Because the SVI relies on ACS variables only available at the tract level, a projection model needs to applied to approximate its results using blockgroup level ACS variables. The blockgroupvulnerability dataset casts a prediction for the CDCs logic for a new contribution to the Open Environments blockgroup series available on Harvard's dataverse platform.
DATA
The CDC's annual SVI publication starts with 23 simple derivations using 50 ACS Census variables. Next the SVI process ranks census geographies to calculate a rank for each, where Percentile Rank = (Rank-1) / (N-1). The SVI themes are then calculated at the tract level as a percentile rank of a sum of the percentile ranks of the first level ACS derived variables. Finally, the overall ranking is taken as the sum of the theme percentile rankings.
The SVI data publication
is keyed by geography (7 cols) where ultimately the Census Tract FIPS code is 2 State + 3 County + 4 Tract + 2 Tract Decimals eg, 56043000301 is 56 Wyoming, 043 Washakie County, Tract 3.01
republishes Census demographics called 'adjunct variables' including area, population, households and housing units from the ACS daytime population taken from LandScan 2020 estimates
derives 23 SVI variables from 50 ACS 5 Year variables
with each having an estimate (E_), estimate precentage (EP_), margin of error (M_), margin percentage (MP_) and flag variable (F_) for those greater than 90% or less than 10%
provides the final 4 themes and a composite SVI percentile
annually vars = ['ST', 'STATE', 'ST_ABBR', 'STCNTY', 'COUNTY', 'FIPS', 'LOCATION'] +\ ['SNGPNT','LIMENG','DISABL','AGE65','AGE17','NOVEH','MUNIT','MOBILE','GROUPQ','CROWD','UNINSUR','UNEMP','POV150','NOHSDP','HBURD','TWOMORE','OTHERRACE','NHPI','MINRTY','HISP','ASIAN','AIAN','AFAM','NOINT'] +\ ['TOTAL','THEME1','THEME2','THEME3','THEME4'] + \ ['AREA_SQMI', 'TOTPOP', 'DAYPOP', 'HU', 'HH'] knowns = vars + \ # Estimates, the result of calc against ACS vars [('E_'+v) for v in vars] + \ # Flag 0,1 whether this geog is in 90 percentile rank (its vulnerable) [('F_'+v) for v in vars] +\ # Margine of error for ACS calcs [('M_'+v) for v in vars] + \ # Margine of error for ACS calcs, as percentage [('MP_'+v) for v in vars] +\ # Estimates of ACS calcs, as percentage [('EP_'+v) for v in vars] + \ # Estimated percentile ranks [('EPL_'+v) for v in vars] + \ # Sum across var percentile ranks [('SPL_'+v) for v in vars]+ \ # Percentile rank of the sum of percentile ranks [('RPL_'+v) for v in vars] [c for c in svitract.columns if c not in knowns]
The SVI themes range over [0,1] but the CDC uses -999 as an NA value; this is set for ~800 or 1% of tracts which have no total poulation.
The themes are numbered: Socioeconomic Status – RPL_THEME1 Household Characteristics – RPL_THEME2 Racial & Ethnic Minority Status – RPL_THEME3 Housing Type & Transportation – RPL_THEME4
The themes with their variables and ACS sources are as follows:
Unlike Census data, the CDC ranks Puerto Rico and Tribal tracts separately from the US otherwise.
Theme SVI Variable ACS Table ACS Variables
Socioeconomic E_UNINSUR S2701 S2701_C04_001E
Socioeconomic E_UNEMP DP03 DP03_0005E
Socioeconomic E_POV150 S1701 S1701_C01_040E
Socioeconomic E_NOHSDP B06009 B06009_002E
Socioeconomic E_HBURD S2503 S2503_C01_028E + S2503_C01_032E + S2503_C01_036E + S2503_C01_040E
Household E_SNGPNT B11012 B11012_010E + B11012_015E
Household E_LIMENG B16005 B16005_007E + B16005_008E + B16005_012E + B16005_013E + B16005_017E + B16005_018E + B16005_022E + B16005_023E + B16005_029E + B16005_030E + B16005_034E + B16005_035E + B16005_039E + B16005_040E + B16005_044E + B16005_045E
Household E_DISABL DP02 DP02_0072E
Household E_AGE65 S0101 S0101_C01_030E
Household E_AGE17 B09001 B09001_001E
Racial & Ethnic E_TWOMORE DP05 DP05_0083E
Racial & Ethnic E_OTHERRACE DP05 DP05_0082E
Racial & Ethnic E_NHPI DP05 DP05_0081E
Racial & Ethnic E_MINRTY DP05 DP05_0071E + DP05_0078E + DP05_0079E + DP05_0080E + DP05_0081E + DP05_0082E + DP05_0083E
Racial & Ethnic E_HISP DP05 DP05_0071E
Racial & Ethnic E_ASIAN DP05 DP05_0080E
Racial & Ethnic E_AIAN DP05 DP05_0079E
Racial & Ethnic E_AFAM DP05 DP05_0078E
Housing E_NOVEH DP04 DP04_0058E
Housing E_MUNIT DP04 DP04_0012E + DP04_0013E
Housing E_MOBILE DP04 DP04_0014E
Housing E_GROUPQ B26001 B26001_001E
Housing E_CROWD DP04 DP04_0078E + DP04_0079E
The Census American Community Survey is updated annually and accessible by API. For this effort, variables used commonly at the block group level were retrieved at the tract level so that a predictive method could be applied to detail. The specific variables used are shown as lists in the data retrieval functions below. The Census' TIGER\Line publication provides the geographic shapes and properties. The TIGER\Line dataset includes:
Geography, position
['STATEFP', 'COUNTYFP', 'TRACTCE', 'GEOID', 'INTPTLAT', 'INTPTLON']
Name with legal/statistical area description
'[NAME', 'NAMELSAD', 'MTFCC', 'FUNCSTAT']
Area of land and water in square meters
['ALAND', 'AWATER']
Geographic shape
['geometry']
See https://www2.census.gov/geo/pdfs/maps-data/data/tiger/tgrshp2020/TGRSHP2020_TechDoc.pdf
The supporting code is maintained on https://github.com/OpenEnvironments/blockgroupvulnerability In generally, variable names within the process are taken from the original SVI and ACS documentation. The variable names in the dataverse publication have the E_ prefix removed, maintaining the published variables relation to the SVI original.
MODEL
The models that generates this data publication uses block group level ACS variables aggregated by the Census to the tract level. The Census TIGER\Line data adds a variable, the land area of each geography, to calculate population density.
For context, there are about 85K tracts in the United States, while there are about 200K block groups. Each tract has between 1,200 and 8,000 people in it while each block group has between 600 and 3,000. Block groups are subdivisions of Census tracts. This level of detail is available for most of the SVI's Census sources, except for variables in the ACS Data Profiles and Subject Tables. These are only available at the tract level.
A model is trained, for each of the SVI's four themes as well as its composite. Each is a regressor, converted to its own percentile rank, and applied at a block group level version of the ACS and TIGER\Line features. The models performance compares the original targets to the block group estimates, aggregated by mean for each tract.
The root mean squared error (RMSE) for each theme are:
|Theme|RMSE| |---------------| |THEME1|0.148565| |THEME2|0.218488| |THEME3|0.086466| |THEME4|0.241419| |THEMES|0.154495|
CITATIONS
Centers for Disease Control and Prevention/ Agency for Toxic Substances and Disease Registry/ Geospatial Research, Analysis, and Services Program. CDC/ATSDR Social Vulnerability Index [Insert 2020, 2018, 2016, 2014, 2010, or 2000] Database [Insert US or State]. https://www.atsdr.cdc.gov/placeandhealth/svi/data_documentation_download.html. Accessed on November 8, 2022.
U.S. Census Bureau. (2020). 2020 American Community Survey 5-year Estimates. Retrieved from API calls to https://api.census.gov/data/2017/acs/acs5?get=NAME,B25077_001M&for=state:*
“TIGER\Line Tract Level Geographies.” Index of /Geo/Tiger/TIGER2020/Tract, US Census Bureau, 1 Feb. 2021, https://www2.census.gov/geo/tiger/TIGER2020/TRACT/.
Flanagan, Barry E.; Gregory, Edward W.; Hallisey, Elaine J.; Heitgerd, Janet L.; and Lewis, Brian (2011) "A Social Vulnerability Index for Disaster Management," Journal of Homeland Security and Emergency Management: Vol. 8: Iss. 1, Article 3. DOI: 10.2202/1547-7355.1792 Available at: http://www.bepress.com/jhsem/vol8/iss1/3
XGBoost, Xgboost.ai, https://xgboost.ai/.
Bryan, Michael B. “Block Group Datasets.” Open Environments Dataverse, Feb. 2022, https://dataverse.harvard.edu/dataverse/openenvironments. https://github.com/OpenEnvironments/blockgroupvulnerabilit
A Note on the Optimal Selection and Weighting of Comparable Properties
No full textThis paper reexamines the recommendation by Vandell (1991), Gau, Lai and Wang (1992, 1994) and Green (1994) for the use of the minimum variance and coefficient of variation criteria in the optimum selection of comparables. These authors under-emphasize the typical valuation scenario that involves extremely small samples. The analyst must rank the few available comparable properties and select the "best" to carry the most weight in the final estimate of value. Rank transformation regression is suggested as one approach that can be used to extract the buying trend. The commonly taught paired-sale analysis will remain as the industry tool until more accurate estimates of value are developed with small samples.
Elliptic curves with high rank
No full textIn this thesis we will look at methods for constructing elliptic curves over Q with high ranks. Using these methods we find an elliptic curve with rank at least 13, an infinite family of elliptic curves with rank at least 9, an elliptic curve with rank at least 10 and torsion-subgroup Z/2Z and an infinite family of elliptic curves with rank at least 8 and with torsion point of order 2.Industrial and Applied MathematicsApplied MathematicsElectrical Engineering, Mathematics and Computer Scienc
- …
