62 research outputs found
Recommended from our members
Modeling Source Code For Developers
No full textSoftware Engineering practices are changing in an age of artificial intelligence. While the core activities of design, develop, maintain, test and evaluate remain, the methods used in these activities are evolving. The prevalence of generative programming models has the potential to reconstitute the duties of a software engineer. Widely adopted models like Copilot and Bard are IDE-based pair-programming assistants that create code from virtually any input: contextual code, natural language, specifications, input output pairs, etc. The way developers interact with these models will redefine some core ideas of software engineering. These models empower virtually anyone, of varying coding proficiency, to create software. Models with the capacity to code will surely manage to inherit software design and analysis capabilities [186], albeit for now, with specific training or prompting.Naturally, one wonders how language modeling, or more specifically the modeling of source code and its features, will impact developers. Researchers often conjecture on the varying degree of influence these methods will have, but certainly, these tools will support developers in new and existing tasks: code completion, bug and vulnerability detection, code summarization, type annotation, and more are already prominent use cases. One can envision a world where software developers delegate portions of their work to machine learning pipelines, such as unit testing and vulnerability testing of their code; how much of that code they actually write is up for debate as well. Developers will likely automate portions of their work flow but simultaneously gain new tasks and responsibilities. These tasks might include passing automatic code reviews that detect code smells, place code comments automatically, and detect refactorings; maybe using models from [56], [133], [59]. These capabilities come from modeling source code and its features directly by distilling down meaningful representations for the task at hand.This thesis explores learning meaningful representations from code through a variety of applications for developer supporting tools. The first application is a type-prediction model using representations learned with masked-language-modeling. While effective, we find that the off-the-shelf model fails at an aspect of modeling source code, namely the use of local user-defined types. The next application modifies the model learned representations with one characterized by an objective function capturing how developers actually use types. Along this body of work, the next two chapters present a type inference dataset for the community and a framework for new machine learning models with a Visual Studio plugin. This thesis concludes with a study of large language models on single statement bug introduction and proposes avoidance strategies. Finally I present some future work to improve these models. By reading this thesis, I hope the reader has a few takeaways:(1) Machine learning is an essential tool for capturing code and its meta data. Models trained on code and its features are capable of generalizing and improving old and new processes.(2) The data that models train on is not perfect, and the resulting models often inherit biases towards vulnerable and buggy code; researchers must evaluate the risk vs. reward with broadly trained models.(3) The objectives optimized for software models may not align with our goals; models that incorporate human feedback may ultimately align better to our values and understanding of code.(4) Large language models are powerful tools for software engineering, but they’re only part of the picture. Models that learn data and control flow, project and file meta data, local and global scope semantics, and information associated with code traces, are better informed on the source code it consumes and produces.This thesis attempts to quantify the utility of off-the-shelf LLMs like BERT, the misalignment of LLM representations to human derived representations of coding constructs, and the present risks of using LLM predictions at face value. Hopefully, in each case, the chapters leave you optimistic that many of the aforementioned concerns can be minimized, or mitigated with just a bit of ingenuity
GENOA—a customizable, front-end-retargetable source code analysis framework
No full text
Code analysis
tools provide support for such software engineering tasks as program understanding, software metrics, testing, and reengineering. In this article we describe GENOA, the framework underlying application generators such as Aria and GEN++ which have been used to generate a wide range of practical code analysis tools. This experience illustrates
front-end retargetability
of GENOA; we describe the features of the GENOA framework that allow it to be used with different front ends. While permitting arbitrary parse tree computations, the GENOA specification language has special, compact iteration operators that are tuned for expressing simple, polynomial-time analysis programs; in fact, there is a useful sublanguage of the GENOA language that can express precisely all (and only)
polynomial-time
(PTIME) analysis programs on parse trees. Thus, we argue that the GENOA language is a simple and convenient vehicle for implementing a range of analysis tools. We also argue that the “front-and reuse” approach of GENOA offers an important advantage for tools aimed at large software projects: the reuse of complex, expensive build procedures to run generated tools over large source bases. In this article, we describe the GENOA framework and our experiences with it.
</jats:p
Reverse Engineering the Bazaar: Collaboration and Communication in Open Source Development
No full text
Converging Work-Talk Patterns in Online Task-Oriented Communities.
No full textMuch of what we do is accomplished by working collaboratively with others, and a large portion of our lives are spent working and talking; the patterns embodied in the alternation of working and talking can provide much useful insight into task-oriented social behaviors. The available electronic traces of the different kinds of human activities in online communities are an empirical goldmine that can enable the holistic study and understanding of these social systems. Open Source Software (OSS) projects are prototypical examples of collaborative, task-oriented communities, depending on volunteers for high-quality work. Here, we use sequence analysis methods to identify the work-talk patterns of software developers in online communities of Open Source Software projects. We find that software developers prefer to persist in same kinds of activities, i.e., a string of work activities followed by a string of talk activities and so forth, rather than switch them frequently; this tendency strengthens with time, suggesting that developers become more efficient, and can work longer with fewer interruptions. This process is accompanied by the formation of community culture: developers' patterns in the same communities get closer with time while different communities get relatively more different. The emergence of community culture is apparently driven by both "talk" and "work". Finally, we also find that workers with good balance between "work" and "talk" tend to produce just as much work as those that focus strongly on "work"; however, the former appear to be more likely to continue to be active contributors in the communities
ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference
No full textIn this paper, we present ManyTypes4TypeScript, a very large corpus for training and evaluating machine-learning models for sequence-based type inference in TypeScript. The dataset includes over 9 million type annotations, across 13,953 projects and 539,571 files. The dataset is approximately 10x larger than analogous type inference datasets for Python, and is the largest available for TypeScript. We also provide API access to the dataset, which can be integrated into any tokenizer and used with any state-of-the-art sequence-based model. Finally, we provide analysis and performance results for state-of-the-art code-specific models, for baselining. ManyTypes4TypeScript is available on Huggingface and Zenodo.
This dataset was collected on January 22, 2022 and deduplicated with Allamanis code deduplication tool
An HMM with two states, i.e., “work” and “talk”, denoted by “W” and “T”, respectively.
No full textThe model is used to explain the W-T patterns of developers in different communities.</p
First International Workshop on Quantitative Stochastic Models in the Verification and Design of Software Systems (QUOVADIS 2010)
No full text- …
