1,720,998 research outputs found
On the Energy Consumption of Test Generation
No full textResearch in the area of automated test generation has seen remarkable progress in recent years, resulting in several approaches and tools for effective and efficient generation of test cases. In particular, the EvoSuite tool has been at the forefront of this progress embodying various algorithms for automated test generation of Java programs. EvoSuite has been used to generate test cases for a wide variety of programs as well. While there are a number of empirical studies that report results on the effectiveness, in terms of code coverage and other related metrics, of the various test generation strategies and algorithms implemented in EvoSuite, there are no studies, to the best of our knowledge, on the energy consumption associated to the automated test generation. In this paper, we set out to investigate this aspect by measuring the energy consumed by EvoSuite when generating tests. We also measure the energy consumed in the execution of the test cases generated, comparing them with those manually written by developers. The results show that the different test generation algorithms consumed different amounts of energy, in particular on classes with high cyclomatic complexity. Furthermore, we also observe that manual tests tend to consume more energy as compared to automatically generated tests, without necessarily achieving higher code coverage. Our results also give insight into the methods that consume significantly higher levels of energy, indicating potential points of improvement both for EvoSuite as well as the different programs under test
AugmenTest: Enhancing Tests with LLM-Driven Oracles
No full textAutomated test generation is crucial for ensuring the reliability and robustness of software applications while at the same time reducing the effort needed. While significant progress
has been made in test generation research, generating valid test oracles still remains an open problem. To address this challenge, we present AugmenTest, an approach leveraging Large Language Models (LLMs) to infer correct test oracles based on available documentation of the software under test. Unlike most existing methods that rely on code, AugmenTest utilizes the semantic capabilities of LLMs to infer the intended behavior of a method from documentation and developer comments, without looking at the code. AugmenTest includes four variants: Simple Prompt, Extended Prompt, RAG with a generic prompt (without the context of class or method
under test), and RAG with Simple Prompt, each offering different levels of contextual information to the LLMs. To evaluate our work, we selected 142 Java classes and generated multiple mutants for each. We then generated tests from these mutants, focusing only on tests that passed on the mutant but failed on the original class, to ensure that the tests effectively captured bugs. This resulted in 203 unique tests with distinct bugs, which were then used to evaluate AugmenTest. Results show that in the most conservative scenario, AugmenTest’s Extended Prompt consistently outperformed the Simple Prompt, achieving a success rate of 30% for generating correct assertions. In comparison, the state-of-the-art TOGA approach achieved 8.2%. Contrary to our expectations, the RAG-based approaches did not lead to improvements, with performance of 18.2% success rate for the most conservative scenario. Our study demonstrates the potential of LLMs in improving the reliability of automated test generation tools, while als
Evolv-1 at the ICST 2025 Tool Competition – UAV Testing Track
No full textEvolv-1 is a test case generation tool designed using Evolutionary Algorithms (EAs) to optimize UAV testing scenarios. This short paper presents Evolv-1's implementation as part of the ICST 2025 UAV Testing Tool Competition
Curiosity Driven Multi-agent Reinforcement Learning for 3D Game Testing
No full textRecently testing of games via autonomous agents has shown great promise in tackling challenges faced by the game industry, which mainly relied on either manual testing or record/replay. In particular Reinforcement Learning (RL) solutions have shown potential by learning directly from playing the game without the need for human intervention.In this paper, we present cMarlTest, an approach for testing 3D games through curiosity driven Multi-Agent Reinforcement Learning (MARL). cMarlTest deploys multiple agents that work collaboratively to achieve the testing objective. The use of multiple agents helps resolve issues faced by a single agent approach.We carried out experiments on different levels of a 3D game comparing the performance of cMarlTest with a single agent RL variant. Results are promising where, considering three different types of coverage criteria, cMarlTest achieved higher coverage. cMarlTest was also more efficient in terms of the time taken, with respect to the single agent based variant
EvoMBT at the SBFT 2023 Tool Competition
No full textEvoMBT is a model-based test generator that principally uses search algorithms to generate tests from a given extended finite state machine (EFSM). In the context of Cyber-physical systems (CPS) testing, and in particular self-driving cars, we use a model of road configurations from which EvoMBT generates different roads for testing the car. This report introduces EvoMBT and reports the results it achieved in the Cyber-physical systems testing competition at SBFT 2023. EvoMBT was able to trigger several out-of-bound events and stood fourth in the overall ranking consisting of a total of six tools
Communicating by compatibility
No full textAbstractA bio-inspired language is presented. Its terms are processes enclosed into boxes with typed interaction sites. The main feature of the formalism lays in the fact that the key-lock communication mechanism typically adopted by process calculi is partially relaxed in favour of a paradigm driven by a (parametric) notion of compatibility of interaction types.Two simple modelling examples are reported: one inspired by the immune system, and the other by web services. These examples show that embedding compatibility into the communication paradigm may be helpful for the specification of both biological and information technology scenarios
Developing An Hierarchical Simulator for Beta-binders
No full textBETA-BINDERS form a recently developed extension of stochastic pi CALCULUS to describe micro-biological systems. It introduces special binders to wrap processes just as membranes enclose some living matter and hence to mimic biological interfaces. One means to define the operational semantics of a modeling formalism is by an abstract simulator description. In developing an abstract simulator for BETA-BINDERS concepts are adopted that have been developed in the context of JAMES II. Processors of the simulator are structured into a hierarchy and each of them is splitted into different methods. This design reflects the structure of BETA-BINDERS models and facilitates experimenting with different operational semantics. Two discrete event simulation schemes, the First-Gillespie method and Gibson-Bruck method, are combined to calculate the reactions that occur within and between the modeled bioprocesses, respectively. The functioning of the simulator is illustrated by processing step-wise the reaction of an immune cell to the occurrence of a virus
Shape Spaces in Formal Interactions
No full textIn recent years formal methods from concurrency theory and process calculi have gained increasing importance in modelling complex biological systems. In this paper propensity to biological interaction, as seen by the shape spaces theory,
is given a linguistic interpretation. Entities from the living matter are viewed as terms of a formal concurrent language of processes with typed interaction sites.
The types are strings, and interaction depends on their distance. Further, the language is associated with syntax-driven rules that permit the inference of the possible computational behaviours of the specifi ed biological system. This approach
leads to the use of all the methods and techniques developed in the context of formal languages (e.g., language translation, model checking), opening new ways of studying complex biological systems
- …
