76 research outputs found
DDoS detection and mitigation using machine learning
No full textDistributed Denial of Service (DDoS) attacks are very common nowadays. It is evident that the current industry solutions, such as completely relying on the In- ternet Service Provider (ISP) or setting up a DDoS defense infrastructure, are not sufficient in detecting and mitigating DDoS attacks, hence consistent research is needed. In this thesis we first tried to understand how DDoS attacks happen, then we discussed a way to detect DDoS attacks using machine learning tools at the routers, instead of setting up a centralized analysis system. We have proposed a standard communication architecture which can be used across all the networking devices for mitigating DDoS attacks. We have also created a simulation program to demonstrate our detection technique.M.S.Includes bibliographical referencesby Arpit Ramesh Gawand
Computational pathology identifies immune-mediated collagen disruption to predict clinical outcomes in gynecologic malignancies
No full text
PoPE: Legendre Orthogonal Polynomials Based Position Encoding for Large Language Models
Get PDFThere are several improvements proposed over the baseline Absolute Positional
Encoding (APE) method used in original transformer. In this study, we aim to
investigate the implications of inadequately representing positional encoding
in higher dimensions on crucial aspects of the attention mechanism, the model's
capacity to learn relative positional information, and the convergence of
models, all stemming from the choice of sinusoidal basis functions. Through a
combination of theoretical insights and empirical analyses, we elucidate how
these challenges extend beyond APEs and may adversely affect the performance of
Relative Positional Encoding (RPE) methods, such as Rotatory Positional
Encoding (RoPE).
Subsequently, we introduce an innovative solution termed Orthogonal
Polynomial Based Positional Encoding (PoPE) to address some of the limitations
associated with existing methods. The PoPE method encodes positional
information by leveraging Orthogonal Legendre polynomials. Legendre polynomials
as basis functions offers several desirable properties for positional encoding,
including improved correlation structure, non-periodicity, orthogonality, and
distinct functional forms among polynomials of varying orders. Our experimental
findings demonstrate that transformer models incorporating PoPE outperform
baseline transformer models on the English-to-German translation
task, thus establishing a new performance benchmark. Furthermore, PoPE-based
transformers exhibit significantly accelerated convergence rates.
Additionally, we will present novel theoretical perspectives on position
encoding based on the superior performance of PoPE
Exposure, hazard and risk mapping during a flood event using open source geospatial technology
No full textAfter a flood event there is a need to delineate the hazard footprint as quickly as possible in order to assess the magnitude of losses and to plan for the relief operations. Delineation of such hazard footprint is generally hindered by the lack of geospatial data, technology and related software packages. This paper demonstrates the use of open source data and software packages which can be used to implement most recent technology available for flood hazard footprint delineation. This study utilizes open source software packages and web applications like Geographic Resource Analysis Support System, Quantum geographic information system and Google Earth to implement a complete process of hazard mapping using remotely sensed data which include pre-processing, mapping (both hazard and exposure) and accuracy assessment. In this study, Brisbane flood event of 2011 has been taken as a case study. For built-up extraction, the Landsat 7-band image has been transformed to a stack of 3-band image using vegetation, water and built-up indices. It has been observed by scattergram analysis that these transformations make vegetation, water and built-up classes more separable. Built-up area has been delineated using supervised maximum likelihood classification on the new 3-band image. For flood hazard mapping, thresholding of near-infrared band has been utilized along with the assistance of mid-infrared band to discriminate water from built-up classes. After delineating both exposure and hazard map, final risk map due to flood event has been generated to assess the urban exposure under the flood hazard impact
Statistical field estimation and scale estimation for complex coastal regions and archipelagos
Get PDFThesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2009.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 153-158).A fundamental requirement in realistic computational geophysical fluid dynamics is the optimal estimation of gridded fields and of spatial-temporal scales directly from the spatially irregular and multivariate data sets that are collected by varied instruments and sampling schemes. In this work, we derive and utilize new schemes for the mapping and dynamical inference of ocean fields in complex multiply-connected domains, study the computational properties of our new mapping schemes, and derive and investigate new schemes for adaptive estimation of spatial and temporal scales. Objective Analysis (OA) is the statistical estimation of fields using the Bayesian-based Gauss-Markov theorem, i.e. the update step of the Kalman Filter. The existing multi-scale OA approach of the Multidisciplinary Simulation, Estimation and Assimilation System consists of the successive utilization of Kalman update steps, one for each scale and for each correlation across scales. In the present work, the approach is extended to field mapping in complex, multiply-connected, coastal regions and archipelagos. A reasonably accurate correlation function often requires an estimate of the distance between data and model points, without going across complex land-forms. New methods for OA based on estimating the length of optimal shortest sea paths using the Level Set Method (LSM) and Fast Marching Method (FMM) are derived, implemented and utilized in general idealized and realistic ocean cases.(cont.) Our new methodologies could improve widely-used gridded databases such as the climatological gridded fields of the World Ocean Atlas (WOA) since these oceanic maps were computed without accounting for coastline constraints. A new FMM-based methodology for the estimation of absolute velocity under geostrophic balance in complicated domains is also outlined. Our new schemes are compared with other approaches, including the use of stochastically forced differential equations (SDE). We find that our FMM-based scheme for complex, multiply-connected, coastal regions is more efficient and accurate than the SDE approach. We also show that the field maps obtained using our FMM-based scheme do not require postprocessing (smoothing) of fields. The computational properties of the new mapping schemes are studied in detail. We find that higher-order schemes improve the accuracy of distance estimates. We also show that the covariance matrices we estimate are not necessarily positive definite because the Weiner Khinchin and Bochner relationships for positive deniteness are only valid for convex simply-connected domains. Several approaches to overcome this issue are discussed and qualitatively evaluated. The solutions we propose include introducing a small process noise or reducing the covariance matrix based on the dominant singular value decomposition.(cont.) We have also developed and utilized novel methodologies for the adaptive estimation of spatial-temporal scales from irregularly spaced ocean data. The three novel methodologies are based on the use of structure functions, short term Fourier transform and second generation wavelets. To our knowledge, this is the first time that adaptive methodologies for the spatial-temporal scale estimation are proposed. The ultimate goal of all these methods would be to create maps of spatial and temporal scales that evolve as new ocean data are fed to the scheme. This would potentially be a significant advance to the ocean community for better understanding and sampling of ocean processes.by Arpit Agarwal.S.M
Improving a Reinforcement Learning Negotiating Agent’s Performance by Extracting Information from the Opponent’s Sequence of Offers
No full textWith the prospects of decentralized multi-agent systems becoming more prevalent in daily life, automated negotiation agents have made their place in these collaborative settings. They are an approach to promote communication between the agents in reaching solutions that are better for all involved.Recent literature has shown great potential in using machine learning, particularly model-free deep reinforcement learning like Proximal Policy Optimization (PPO), to develop more performant automated negotiation strategies. This work focuses on using information from the opponent's sequence of offers in a bilateral negotiation to further improve a baseline PPO agent. This involves extracting and representing information from the opponent's sequence of offers into a state vector with a fixed dimension to modify the input to the agent's policy, and then comparing the utilities this modified agent achieves to the baseline PPO agent. Since there is a large variety of numerical measures to represent a sequence of offers, an ablation study is conducted to investigate the effectiveness of each.The modified agents consistently reached solutions that had higher social welfare, although the agent's own utility did not improve or diminish significantly in comparison to the base PPO agent.https://github.com/brenting/negotiation_PPO The repository containing all the code this paper used. The code for this specific paper was done in the 'sequence-of-offers-single-thread' branch.CSE3000 Research ProjectComputer Science and Engineerin
- …
