Masoumeh Taromirad

Thesis Title: Incorporation of Model-Based Testing into Modern Lightweight Development Processes                  

Abstract:

Studies show that testing and debugging account for more than half of the software development costs and are becoming serious bottlenecks in the software development process [6]. Nevertheless, in the modern software era, testing plays an invaluable role in software development as a quality assurance measure. In particular, lightweight methodologies recommend automated testing as the main method for quality assurance [1]. However, they are plagued with several deficiencies in this regard, including complex and difficult-to-maintain test scripts. A promising solution to testing issues lies in automated Model-Based Testing (MBT) processes [8], which provide a structured approach to testing from high-level behavioural models. The benefit of MBT is primarily in automated test case generation and automated analysis of the test results, and thus they have received significant attention in testing complex software systems. However, many of MBT methods are not directly suitable for application to lightweight methodologies [2].
Many of the existing studies have discussed that a major obstacle in the application of MBT in lightweight processes is that the models that are typically required by MBT methods, usually involve details that are not considered in normal lightweight processes, or are completely of different types from the models developed during software analysis and design [3, 5, 4]. Accordingly, this challenge has been mainly investigated with respect to the types of models (or the abstraction levels) involved in MBT techniques that could be adapted to (or used in) lightweight processes.
Another challenge in using MBT techniques in lightweight processes is to effectively deal with changes. Firstly, the evolution of development artefacts, including models, is an inherent characteristic of lightweight processes. Therefore, keeping the test models updated is a substantial problem. There are few studies addressing this issue in the literature (e.g., [7]). Nevertheless, this issue is not restricted to the use of MBT in lightweight processes and it is an important problem in development approaches that are based on models. Secondly, the (test) models become larger and more complex, and hence, the number of test cases is growing continuously and rapidly, which lead to high testing effort, if this growth is not properly managed. This makes it impossible to get frequent and quick feedback about the whole system, which is a mandatory requirement in lightweight processes. Although in MBT processes test cases are generated and executed automatically, they typically require re-generation of all test cases from the updated (test) models, which is however a time-consuming process. There are very few studies on incremental model-based test generation. Furthermore, MBT approaches also lack effective definitions of test coverage for deciding which subsets of tests to run under which conditions in a limited time frame.
This research focuses on the application of MBT methods in modern lightweight processes, addressing the aforementioned problems. It particularly investigates MBT techniques with respect to dealing with change. The research is carried out in three phases in one year:
Phase 1: Literature Survey (4 months) In this phase, the related literature on the integration on MBT and lightweight processes is surveyed.
1- (1 months) We first investigate the integration of MBT and lightweight processes with respect to the types of models in order to identify promising MBT techniques that can be adapted by modern lightweight processes. Then, based on the different types of change that are common in lightweight processes (e.g., changes considered in well-known agile methods), we provide a classification of MBT techniques, i.e., which technique could be in used in which process.
2- (2 months) In the next step, existing incremental MBT techniques (e.g., [9]), which provide more efficient test case generation, are surveyed and their possible application in lightweight processes is investigated, with respect to the previously identified types of change (e.g., we need to map or relate the changes in agile processes to the level of changes considered in incremental MBT approaches). The provided classification may introduce additional criteria or requirements for beneficial integration of lightweight processes and MBT techniques.
3- (1 month) We also investigate the test coverage metrics in the context of MBT techniques.
Phase 2: Engineering a Lightweight Process (4 months) The goal of this phase is to engineer a lightweight methodology with an (extended) incremental MBT technique in order to demonstrate how MBT techniques can be applied in lightweight processes which address the testing challenges while maintaining the agility of the process.
1. (2 weeks) We first choose a lightweight process (or a class of processes) as the base methodology, and accordingly find the set of possible MBT techniques (identified in the previous phase).
2. (2.5 months) We then select one of the techniques and might extend it regarding the additional requirements identified in the previous phase.
3. (1 month) Finally, a comprehensive and concrete integration of the selected lightweight process with the selected MBT technique is provided.
Phase 3: Case Studies and Evaluation (4 months) In this phase, the new engineered lightweight process is used in a number of case studies (empirical experiments) and the effects of using the new method and the original method on, for example, the number of test cases, the time required for test generation and test execution, simplicity of testing activities, are collected, compared, and evaluated.
References
1. K. Beck, Test Driven Development: By Example. Addison-Wesley Professional. (2002)
2. D. Farago: Model-based testing in agile software development. In Treffen der GI-Fachgruppe Test, Analyse & Verifikation von Software (TAV), Testing meets Agility. (2010)
3. D. Farago: Improved Underspecification for Model-based Testing in Agile Development. In Workshop on Formal Methods and Agile Methods. (2010)
4. D. Jalalinasab and R. Ramsin: Towards Model-Based Testing Patterns for Enhancing Agile Methodologies. In International Conference on Intelligent Software Methodologies, Tools and Techniques, pp. 57-72. (2012)
5. M. Katara and A. Kervinen: Making model-based testing more agile: A use case driven approach. In Haifa Verification Conference. vol. 4383 of LNCS. Springer, pp. 219-234. (2006)
6. G.J. Myers et al. The Art of Software Testing, Wiley. (2011)
7. B. Rumpe: Agile test-based modeling. In Software Engineering Research and Practice, CSREA Press, pp. 10-15. (2006)
8. J. Tretmans: Model Based Testing with Labelled Transition Systems. In Formal Methods and Testing 2008, vol. 4949 of LNCS, pp. 1-38, Springer. (2008)
9. M. Varshosaz, H. Beohar, and M.R. Mousavi: Delta-Oriented FSM-Based Testing. In International Conference on Formal Engineering Methods, vol. 9407 of LNCS, pp. 366-381, Springer. (2015)

                        

Contact Information

   Email: m[dot]taromirad[at] gmail [dot] com