12 research outputs found
Optimization of Actor Critic Policy in Continuous Action Space
No full textThe implementation of Reinforcement learning algorithms has made a huge impact on various problems where no existing methodologies has succeeded in control task and make decision. In this paper we are implementing a hybrid algorithm to virtual selfdriving car through collating the Actor-Critic and Proximal Policy Optimization (PPO) methods to introduce a continuous control tasks for locomotion of cars. Successful locomotion of a self-driving car can be achieved through angular movements of the steering by understanding the changes in environment where the actions like to take turns smoothly or throttle maps to continuous action space. The policy which maps input received from the sensors which causes change of action in cars is upgraded to achieve rewards. Due to these upgraded techniques the general policy-based methods have been improvised by the Actor-Critic method. The primary purpose of the research is to study the performance of the modified policy optimization techniques which enhances the interaction of the agent with the environment resulting in improved rewards in comparison with other policy-based methods. The testbeds used for the implementation of the modified algorithm are Cartpole and MountainCarContinuous. The modified actor-critic algorithm has yielded consistent policy update reducing the risk of learning a sudden irreversible bad policy.
Keywords: Reinforcement learning, Machine learning, Policy Gradient, Actor-Critic, PP
COZMO—A New Lightweight Stream Cipher
No full textThis paper deals with the merger of the two lightweight stream ciphers&mdash;A5/1 and Trivium. The idea is to make the key stream generation more secure and to remove the attacks of the individual algorithms. The bits generated by the Trivium cipher (output) will act as the input of the A5/1 cipher. The registers used in the A5/1 cipher will be filled by the output bits of the Trivium cipher. The three registers will then be connected to generate an output which will be our required key stream.</jats:p
<i>In situ</i> development of bio-based polyurethane-<i>blend</i> -epoxy hybrid materials and their nanocomposites with modified graphene oxide via non-isocyanate route
No full textCOZMO - A new lightweight stream cipher
No full textThis paper deals with the merger of the two lightweight stream ciphers – A5/1 and Trivium. The idea is to make the key stream generation more secure and to remove the attacks of the individual algorithms. The bits generated by the Trivium cipher (output) will act as the input of the A5/1 cipher. The registers used in the A5/1 cipher will be filled by the output bits of the Trivium cipher. The three registers will then be connected to generate an output which will be our required key stream. we are using Trivium and A5/1 algorithm and making changes to suit our needs.</jats:p
A small molecule chemical chaperone optimizes its unfolded state contraction and denaturant like properties
No full textProtein aggregation is believed to occur through the formation of misfolded conformations. It is expected
that, in order to minimize aggregation, an effective small molecule chaperone would destabilize these
intermediates. To study the mechanism of a chemical chaperone, we have designed a series of mutant
proteins in which a tryptophan residue experiences different local environments and solvent exposures. We
show that these mutants correspond to a series of conformationally altered proteins with varying degree of
misfolding stress and aggregation propensities. Using arginine as a model small molecule, we show that a
combination of unfolded state contraction and denaturant like properties results in selective targeting and
destabilization of the partially folded proteins. In comparison, the effect of arginine towards the folded like
control mutant, which is not aggregation prone, is significantly less. Other small molecules, lacking either of
the above two properties, do not offer any specificity towards the misfolded proteins
HybridContextQA: A hybrid approach for complex question answering using knowledge graph construction and context retrieval with LLMs
No full textAugmenting domain-specific knowledge with Large Language Models (LLMs) to answer complex conditional questions is an important area of research. LLMs are good at answering general domain questions, however, their performance decreases when applied to a specific domain with complex conditional questions. We hypothesize that extracting context from relevant documents and Knowledge Graphs (KGs), and then feeding this combined knowledge to the LLM prompts, can provide better context to answer the complex conditional questions.
To test our hypothesis, we propose a hybrid approach called Hybrid Context for Complex Question-Answering (HybridContextQA) that can extract relevant context from documents as well as from a KG. To implement this, we create a Retrieval-Augmented Generation (RAG)-based hybrid context retrieval pipeline. This pipeline creates a KG from the provided documents and stores it in a Neo4j graph store. An LLM is used to automatically create a KG from the provided documents. The pipeline also stores the context extracted from the documents in vector form in a vector database. This combined context from KG and vector store can then be used for answering the complex conditional questions of that domain using an LLM.
We perform our experiments on a complex question-answering (QA) dataset called ConditionalQA. This dataset contains complex questions with conditional answers. We also compare the proposed approach with other approaches such as Code Prompt, Text Prompt, and Think-on-Graph. We find that the HybridContextQA approach performs better than the existing approaches for multiple LLMs, including Mistral and Mixtral.
We also conduct comprehensive experiments to analyze the contribution of the context from KG and vector form. We release the code implementing the HybridContextQA approach and the end-to-end pipeline with LLM promptsThis publication has emanated from research supported in part by a grant from Science Foundation Ireland under Grant number SFI/12/RC/2289_P2 {Insight} and a grant from Fidelity Investments. For the purpose of Open Access, the authors have applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission.peer-reviewe
Electrokinetics of Concentrated Suspension of Soft Particles with pH-Regulated Volumetric Charges
No full textThe non-native helical intermediate state may accumulate at low ph in the folding and aggregation landscape of the intestinal fatty acid binding protein
No full textThere has been widespread interest in studying early intermediate states and their roles in protein folding. The interest in intermediate states has been further emphasized in the recent literature because of their implications for protein aggregation. Unfortunately, direct kinetic characterization of intermediates has been difficult because of the limited time resolutions offered by the kinetic techniques and the heterogeneity of the folding and aggregation landscape. Even in equilibrium experiments, the characterization of intermediate states could be difficult because (a) their populations in equilibrium could be low and/or (b) they lack any specific biochemical or biophysical signatures for their identification. In this paper, we have used fluorescence correlation spectroscopy to study the nature of a low-pH intermediate state of the intestinal fatty acid binding protein, a small protein with predominantly β-sheet structure. Our results have shown that the pH 3 intermediate diffuses faster than the folded protein and has strong helix forming propensity. These behaviors support Lim’s hypothesis according to which even an entirely β-sheet protein would form helical bundles at the early stage. Using dynamic light scattering and thioflavin T binding measurements, we have observed that the pH 3 intermediate is prone to aggregation. We believe that early helix formation is the result of a local effect, which originates from the interaction of the neighboring amino acids around the hydrophobic core residues. This early intermediate reorganizes subsequently, and this structural reorganization is initiated by the destabilizing interactions induced by the distant residues, unfavorable entropic costs, and steric constraints of the hydrophobic side chains. Mutational analyses show further that the increase in the hydrophobicity in the hydrophobic core region increases the population of the α-helical intermediate, enhancing the aggregation propensity of the protein, while an identical change, distant from the hydrophobic core, does not show any effect. This study re-emphasizes an overlap between the folding and aggregation landscape of a protein, where the fine-tuning between the local and global effects may be important for the protein to fold efficiently or to aggregate
