1,721,095 research outputs found
Java Unit Testing Tool Competition - Seventh Round
No full textWe report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.Software Engineerin
Towards Automated Test Case Generation Maturity
No full textThis short paper reports our observations after six editions of the JUnitContest that benchmarks automated unit test generation tools for Java programs. We discuss our experience and depict the current state-of-the-art and identify potential future research directions. We advocate the use of benchmark as a standard practice to enhance maturity and foster adoption by the industry of automated test case generation tools.Software Engineerin
Message from the Chairs of the 23rd Belgium-Netherlands Software Evolution Workshop
No full textThe 23rd Belgium-Netherlands Software Evolution Workshop (BENEVOL 2024) was held in Namur, Netherlands, on November 21–22, 2024.BENEVOL 2024 offered various contribution formats: Technical papers (TECH), which present novel and validated research results; New Ideas and Controversial Perspectives (NICE), which are position papers presenting new and potentially controversial software maintenance and evolution perspectives; Presentation Abstracts (ABS) are summaries of research published elsewhere, and presented as posters during the workshop; Finally, Replicated or Invalidated Papers (RIP), concerned the replication efforts of existing studies.BENEVOL 2024 received 35 submissions, which is the second highest for the workshop, in particular: 15 TECH papers, 6 NICE papers and 14 presentation abstracts/posters. We accepted 14 TECH papers, 3 NICE papers, and 17 presentations abstracts (including 3 NICE papers accepted as ABS ones). Unfortunately, 3 ABS papers could not be presented during event, resulting in 31 contributions discussed during the workshop. These proceedings include only the 17 TECH and NICE papers.The accepted submissions represent a wide range of topics related to software evolution, including; software ecosystems, programming languages, static and dynamic analysis, software repositories, the growing use of artificial intelligence for testing and code comprehension, and insights from practice.BENEVOL 2024 also featured two amazing keynotes. Sonia Haiduc (Florida State University, USA) gave her keynote on “Studying and Supporting Developers' Online Information Seeking,” exploring the sources of information that developers privilege. Andy Zaidman (Delft University of Technology, The Netherlands) presented “Show Your True Testing Color,” investigating the impact of continuous development and testing practices on sustainability.With so many contributions and more than 80 participants, we believe BENEVOL 2024 was a successful edition! A special thanks to the authors for submitting and presenting their work, the keynote speakers for sharing their knowledge and experiences during the workshop, the program committee members for their availability and care in reviewing a large number of submissions, and the sponsors for providing resources that made the workshop possible and enjoyable
Message from the Chairs of the 23rd Belgium-Netherlands Software Evolution Workshop
No full textThe 23rd Belgium-Netherlands Software Evolution Workshop (BENEVOL 2024) was held in Namur, Netherlands, on November 21–22, 2024.BENEVOL 2024 offered various contribution formats: Technical papers (TECH), which present novel and validated research results; New Ideas and Controversial Perspectives (NICE), which are position papers presenting new and potentially controversial software maintenance and evolution perspectives; Presentation Abstracts (ABS) are summaries of research published elsewhere, and presented as posters during the workshop; Finally, Replicated or Invalidated Papers (RIP), concerned the replication efforts of existing studies.BENEVOL 2024 received 35 submissions, which is the second highest for the workshop, in particular: 15 TECH papers, 6 NICE papers and 14 presentation abstracts/posters. We accepted 14 TECH papers, 3 NICE papers, and 17 presentations abstracts (including 3 NICE papers accepted as ABS ones). Unfortunately, 3 ABS papers could not be presented during event, resulting in 31 contributions discussed during the workshop. These proceedings include only the 17 TECH and NICE papers.The accepted submissions represent a wide range of topics related to software evolution, including; software ecosystems, programming languages, static and dynamic analysis, software repositories, the growing use of artificial intelligence for testing and code comprehension, and insights from practice.BENEVOL 2024 also featured two amazing keynotes. Sonia Haiduc (Florida State University, USA) gave her keynote on “Studying and Supporting Developers' Online Information Seeking,” exploring the sources of information that developers privilege. Andy Zaidman (Delft University of Technology, The Netherlands) presented “Show Your True Testing Color,” investigating the impact of continuous development and testing practices on sustainability.With so many contributions and more than 80 participants, we believe BENEVOL 2024 was a successful edition! A special thanks to the authors for submitting and presenting their work, the keynote speakers for sharing their knowledge and experiences during the workshop, the program committee members for their availability and care in reviewing a large number of submissions, and the sponsors for providing resources that made the workshop possible and enjoyable
Hybrid Multi-level Crossover for Unit Test Case Generation
No full textState-of-the-art search-based approaches for test case generation work at test case level, where tests are represented as sequences of statements. These approaches make use of genetic operators (i.e., mutation and crossover) that create test variants by adding, altering, and removing statements from existing tests. While this encoding schema has been shown to be very effective for many-objective test case generation, the standard crossover operator (single-point) only alters the structure of the test cases but not the input data. In this paper, we argue that changing both the test case structure and the input data is necessary to increase the genetic variation and improve the search process. Hence, we propose a hybrid multi-level crossover (HMX) operator that combines the traditional test-level crossover with data-level recombination. The former evolves and alters the test case structures, while the latter evolves the input data using numeric and string-based recombinational operators. We evaluate our new crossover operator by performing an empirical study on more than 100 classes selected from open-source Java libraries for numerical operations and string manipulation. We compare HMX with the single-point crossover that is used in EvoSuite w.r.t structural coverage and fault detection capability. Our results show that HMX achieves a statistically significant increase in 30% of the classes up to 19% in structural coverage compared to the single-point crossover. Moreover, the fault detection capability improved up to 12% measured using strong mutation score.Software Engineerin
Multi-objective Test Case Selection Through Linkage Learning-Based Crossover
No full textTest Case Selection (TCS) aims to select a subset of the test suite to run for regression testing. The selection is typically based on past coverage and execution cost data. Researchers have successfully used multi-objective evolutionary algorithms (MOEAs), such as NSGA-II and its variants, to solve this problem. These MOEAs use traditional crossover operators to create new candidate solutions through genetic recombination. Recent studies in numerical optimization have shown that better recombinations can be made using machine learning, in particular linkage learning. Inspired by these recent advances in this field, we propose a new variant of NSGA-II, called L2-NSGA, that uses linkage learning to optimize test case selection. In particular, we use an unsupervised clustering algorithm to infer promising patterns among the solutions (subset of test suites). Then, these patterns are used in the next iterations of L2-NSGA to create solutions that preserve these inferred patterns. Our results show that our customizations make NSGA-II more effective for test case selection. The test suite sub-sets generated by L2-NSGA are less expensive and detect more faults than those generated by MOEAs used in the literature for regression testing.Software Engineerin
JUGE: An Infrastructure for Benchmarking Java Unit Test Generators
Full text linkResearchers and practitioners have designed and implemented various automated
test case generators to support effective software testing. Such generators
exist for various languages (e.g., Java, C#, or Python) and for various
platforms (e.g., desktop, web, or mobile applications). Such generators exhibit
varying effectiveness and efficiency, depending on the testing goals they aim
to satisfy (e.g., unit-testing of libraries vs. system-testing of entire
applications) and the underlying techniques they implement. In this context,
practitioners need to be able to compare different generators to identify the
most suited one for their requirements, while researchers seek to identify
future research directions. This can be achieved through the systematic
execution of large-scale evaluations of different generators. However, the
execution of such empirical evaluations is not trivial and requires a
substantial effort to collect benchmarks, setup the evaluation infrastructure,
and collect and analyse the results. In this paper, we present our JUnit
Generation benchmarking infrastructure (JUGE) supporting generators (e.g.,
search-based, random-based, symbolic execution, etc.) seeking to automate the
production of unit tests for various purposes (e.g., validation, regression
testing, fault localization, etc.). The primary goal is to reduce the overall
effort, ease the comparison of several generators, and enhance the knowledge
transfer between academia and industry by standardizing the evaluation and
comparison process. Since 2013, eight editions of a unit testing tool
competition, co-located with the Search-Based Software Testing Workshop, have
taken place and used and updated JUGE. As a result, an increasing amount of
tools (over ten) from both academia and industry have been evaluated on JUGE,
matured over the years, and allowed the identification of future research
directions
Basic Block Coverage for Unit Test Generation at the SBST 2022 Tool Competition
Full text linkBasic Block Coverage (BBC) is a secondary objective for search- based unit test generation techniques relying on the approach level and branch distance to drive the search process. Unlike the approach level and branch distance, which considers only information related to the coverage of explicit branches coming from conditional and loop statements, BBC also takes into account implicit branchings (e.g., a runtime exception thrown in a branchless method) denoted by the coverage level of relevant basic blocks in a control flow graph to drive the search process. Our implementation of BBC for unit test generation relies on the DynaMOSA algorithm and EvoSuite. This paper summarizes the results achieved by EvoSuite’s DynaMOSA implementation with BBC as a secondary objective at the SBST 2022 unit testing tool competition.<br/
- …
