Research Article
BibTex RIS Cite

An End to End Model-Based Approach for Graphical User Interface Testing

Year 2022, Volume: 12 Issue: 1, 7 - 19, 21.06.2022

Alper Silistre Onur Kılınççeker Fevzi Belli Moharram Challenger Geylani Kardaş

Abstract

Model-based Graphical User Interface (GUI) testing keeps its importance in software GUI testing. Manual testing is time-consuming and highly error prone. There are several test models in general use that the software testing community has been working on for many years. This article examines the different models used in model-based GUI testing. To create test cases and combine them into a single model, a method for how to convert several accepted models into Event Sequence Graphs (ESG) has been proposed, and a case study illustrating its use is presented. In addition, this article introduces an approach involving generating and running test sets over an ESG model obtained by the transformation from other models. Experimental studies have shown that this proposed approach is appropriate and effective. In this context, the application of the proposed approach enabled to receive the highest mutation score for 20 mutants obtained from the ESG.

Keywords

GUI Testing Model-Based Testing Finite State Machine Event Sequence Graph Event Flow Graph Regular Expression

References

  • [1] I. Banerjee, B. Nguyen, V. Garousi, and A. Memon, "Graphical user interface (GUI) testing: Systematic mapping and repository," Information and Software Technology 55, no. 10, pp. 1679-1694, 2013.
  • [2] Harrison, R., Flood, D. & Duce, D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 1, 1 (2013). https://doi.org/10.1186/2194-0827-1-1
  • [3] R. K. Shehady, D. P. Siewiorek, “A method to automate user interface testing using variable finite state machines,” Proceedings of IEEE 27th International Symposium on Fault Tolerant Computing, Seattle, WA, USA, pp. 80-88, 1997.
  • [4] T. S. Chow, “Testing software design modeled by finite-state machines,” in IEEE Transactions on Software Engineering, vol. SE-4, no. 3, pp. 178-187, May 1978.
  • [5] L. White, H. Almezen, “Generating test cases for GUI responsibilities using complete interaction sequences,” Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000, San Jose, CA, USA, pp. 110-121, 2000.
  • [6] F. Belli, “Finite state testing and analysis of graphical user interfaces,” Proceedings 12th International Symposium on Software Reliability Engineering, Hong Kong, China, pp. 34-43, 2001.
  • [7] A. M. Memon, M. E. Pollack, M. L. Soffa, “Hierarchical GUI test case generation using automated planning,” in IEEE Transactions on Software Engineering, vol. 27, no. 2, pp. 144-155, February 2001.
  • [8] A. Memon. "An event‐flow model of GUI‐based applications for testing," Software testing, verification and reliability 17.3 pp. 137-157, September 2007.
  • [9] Q. Xie, A. M. Memon, “Using a pilot study to derive a GUI model for automated testing,” ACM Trans. Software Eng. Methodol. 18, 2, pp. 1–35, November 2008.
  • [10] S. Huang, M. B. Cohen and A. M. Memon, “Repairing GUI test suites using a genetic algorithm,” 2010 Third International Conference on Software Testing, Verification and Validation, Paris, pp. 245-254, 2010.
  • [11] F. Belli, M. Beyazit, N. Güler, “Event-Oriented, model-based GUI testing and reliability assessment-approach and case study,” Advances in Computers, 85, pp. 277-326, 2012.
  • [12] F. Belli, M. Beyazıt, C. J. Budnik, T. Tuglular, "Advances in model-based testing of graphical user interfaces," In Advances in Computers, vol. 107, pp. 219-280. Elsevier, 2017.
  • [13] F. Belli, N. Güler, and M. Linschulte, “Layer-centric testing,” FERS-Mitteilungen: Vol. 30, No. 1, pp. 55-62, 2012.
  • [14] A. M. Memon, M. L. Soffa, and M. E. Pollack, "Coverage criteria for GUI testing," Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering, pp. 256-267, September 2001.
  • [15] A. M. Memon, "GUI testing: pitfalls and process," in Computer, vol. 35, no. 8, pp. 87-88, August 2002.
  • [16] D. Lee and M. Yannakakis, "Principles and methods of testing finite state machines-a survey," in Proceedings of the IEEE, vol. 84, no. 8, pp. 1090-1123, August 1996.
  • [17] S. Fujiwara, G. v. Bochmann, F. Khendek, M. Amalou and A. Ghedamsi, "Test selection based on finite state models," in IEEE Transactions on Software Engineering, vol. 17, no. 6, pp. 591-603, June 1991.
  • [18] M. Utting, A. Pretschner, and B. Legeard, "A taxonomy of model‐based testing approaches," Software testing, verification and reliability 22.5, pp. 297-312, 2012.
  • [19] F. Belli, M. Beyazit and A. Memon, "Testing is an Event-Centric Activity," 2012 IEEE Sixth International Conference on Software Security and Reliability Companion, Gaithersburg, MD, pp. 198-206, 2012.
  • [20] J. E. Hopcroft, R. Motwani, and J. D. Ullman, Automata theory, languages, and computation." International Edition 24.2.2, 2006.
  • [21] ISELTA websitesi, Çevrimiçi olarak mevcut: http://iselta.ivknet.de, Son erişim: 03.03.2021
  • [22] O. Kilincceker, A. Silistre, M. Challenger and F. Belli, "Random Test Generation from Regular Expressions for Graphical User Interface (GUI) Testing," 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C), Sofia, Bulgaria, pp. 170-176, 2019.
  • [23] G. Mercan, E. Akgündüz, O. Kilincceker, M. Challenger, and F. Belli, “Android uygulaması testi için ideal test ön çalışması,” CEUR Workshop Proceedings, 2018.
  • [24] O. Kilincceker, and F. Belli, “Grafiksel kullanıcı arayüzleri için düzenli ifade bazlı test kapsama kriterleri,” CEUR Workshop Proceedings, 2017.
  • [25] O. Kilinccceker, E. Turk, M. Challenger and F. Belli, “Regular expression based test sequence generation for HDL program validation,” 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), Lisbon, pp. 585-592, 2018.
  • [26] O. Kilincceker, E. Turk, M. Challenger and F. Belli, "Applying the ideal testing framework to HDL programs," ARCS Workshop 2018; 31th International Conference on Architecture of Computing Systems, Braunschweig, Germany, pp. 1-6, 2018.
  • [27] O. Kilincceker and F. Belli, "Towards uniform modeling and holistic testing of hardware and software," 2019 1st International Informatics and Software Engineering Conference (UBMYK), Ankara, Turkey, pp. 1-6, 2019.
  • [28] Brüggemann-Klein, A. (1993). Regular expressions into finite automata. Theoretical Computer Science, 120(2), 197-213.
  • [29] M. Linschulte, On the Role of Test Sequence Length, Model Refinement, and Test Coverage for Reliability (PhD Thesis, Univ. Paderborn), 2014.
  • [30] F. Belli, A.T. Endo, M. Linschulte, and A. Simao, A Holistic Approach to Model-Based Testing of Web Service Compositions, Software: Practice and Experience, vol.44, no.2, pp. 201–23, 2014.
  • [31] A. Silistre, O. Kilincceker, F. Belli, M. Challenger and G. Kardas, "Models in Graphical User Interface Testing: Study Design," 2020 Turkish National Software Engineering Symposium (UYMS), Istanbul, Turkey, 2020, pp. 1-6, doi: 10.1109/UYMS50627.2020.9247072.
  • [32] A. Silistre, O. Kilincceker, F. Belli, M. Challenger, and G. Kardas, “Community Detection in Model-based Testing to Address Scalability: Study Design”, 15th Conference on Computer Science and Information Systems (FedCSIS 2020), Track on Software and Systems Engineering, Advances in Software and Systems Engineering (ASSE 2020), Sofia, Bulgaria, 2020, IEEE, pp. 657-660, DOI: 10.15439/2020F163.

Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım

Year 2022, Volume: 12 Issue: 1, 7 - 19, 21.06.2022

Alper Silistre Onur Kılınççeker Fevzi Belli Moharram Challenger Geylani Kardaş

Abstract

Model tabanlı Grafiksel Kullanıcı Arayüz (GUI) testi, yazılım GUI testi içerisinde önemli bir yer tutmaktadır. Manuel test, zaman alıcı bir iştir ve büyük ölçüde hataya açıktır. Yazılım test topluluğunun uzun yıllardır üzerinde çalıştığı ve genel kullanımda olan birkaç test modeli vardır. Bu makale, model tabanlı GUI testinde kullanılan farklı modelleri incelemektedir. Test senaryoları oluşturmak ve bunları tek bir modelde birleştirmek amacıyla, kabul gören birkaç modelin Olay Sıra Çizgelerine (ESG) nasıl dönüştürüleceğine ilişkin bir yöntem önerilmiştir ve bunun kullanımını örnekleyen bir vaka çalışması sunulmuştur. Ayrıca bu makalede diğer modellerden dönüşümle elde edilen ESG modelinden test kümelerinin üretilmesi ve çalıştırılmasını içeren bir yaklaşım tanıtılmıştır. Deneysel çalışmalar öne sürülen bu yaklaşımın uygun ve etkili olduğunu göstermiştir. Bu kapsamda ESG’den elde edilen 20 mutant için öne sürülen yaklaşım en yüksek seviyede mutasyon skoru sonucunu vermiştir.

Keywords

GUI Testi Model-Tabanlı Test Sonlu Durum Makinesi Olay Sıra Çizgesi Olay Akış Çizgesi Düzenli İfade

References

  • [1] I. Banerjee, B. Nguyen, V. Garousi, and A. Memon, "Graphical user interface (GUI) testing: Systematic mapping and repository," Information and Software Technology 55, no. 10, pp. 1679-1694, 2013.
  • [2] Harrison, R., Flood, D. & Duce, D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 1, 1 (2013). https://doi.org/10.1186/2194-0827-1-1
  • [3] R. K. Shehady, D. P. Siewiorek, “A method to automate user interface testing using variable finite state machines,” Proceedings of IEEE 27th International Symposium on Fault Tolerant Computing, Seattle, WA, USA, pp. 80-88, 1997.
  • [4] T. S. Chow, “Testing software design modeled by finite-state machines,” in IEEE Transactions on Software Engineering, vol. SE-4, no. 3, pp. 178-187, May 1978.
  • [5] L. White, H. Almezen, “Generating test cases for GUI responsibilities using complete interaction sequences,” Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000, San Jose, CA, USA, pp. 110-121, 2000.
  • [6] F. Belli, “Finite state testing and analysis of graphical user interfaces,” Proceedings 12th International Symposium on Software Reliability Engineering, Hong Kong, China, pp. 34-43, 2001.
  • [7] A. M. Memon, M. E. Pollack, M. L. Soffa, “Hierarchical GUI test case generation using automated planning,” in IEEE Transactions on Software Engineering, vol. 27, no. 2, pp. 144-155, February 2001.
  • [8] A. Memon. "An event‐flow model of GUI‐based applications for testing," Software testing, verification and reliability 17.3 pp. 137-157, September 2007.
  • [9] Q. Xie, A. M. Memon, “Using a pilot study to derive a GUI model for automated testing,” ACM Trans. Software Eng. Methodol. 18, 2, pp. 1–35, November 2008.
  • [10] S. Huang, M. B. Cohen and A. M. Memon, “Repairing GUI test suites using a genetic algorithm,” 2010 Third International Conference on Software Testing, Verification and Validation, Paris, pp. 245-254, 2010.
  • [11] F. Belli, M. Beyazit, N. Güler, “Event-Oriented, model-based GUI testing and reliability assessment-approach and case study,” Advances in Computers, 85, pp. 277-326, 2012.
  • [12] F. Belli, M. Beyazıt, C. J. Budnik, T. Tuglular, "Advances in model-based testing of graphical user interfaces," In Advances in Computers, vol. 107, pp. 219-280. Elsevier, 2017.
  • [13] F. Belli, N. Güler, and M. Linschulte, “Layer-centric testing,” FERS-Mitteilungen: Vol. 30, No. 1, pp. 55-62, 2012.
  • [14] A. M. Memon, M. L. Soffa, and M. E. Pollack, "Coverage criteria for GUI testing," Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering, pp. 256-267, September 2001.
  • [15] A. M. Memon, "GUI testing: pitfalls and process," in Computer, vol. 35, no. 8, pp. 87-88, August 2002.
  • [16] D. Lee and M. Yannakakis, "Principles and methods of testing finite state machines-a survey," in Proceedings of the IEEE, vol. 84, no. 8, pp. 1090-1123, August 1996.
  • [17] S. Fujiwara, G. v. Bochmann, F. Khendek, M. Amalou and A. Ghedamsi, "Test selection based on finite state models," in IEEE Transactions on Software Engineering, vol. 17, no. 6, pp. 591-603, June 1991.
  • [18] M. Utting, A. Pretschner, and B. Legeard, "A taxonomy of model‐based testing approaches," Software testing, verification and reliability 22.5, pp. 297-312, 2012.
  • [19] F. Belli, M. Beyazit and A. Memon, "Testing is an Event-Centric Activity," 2012 IEEE Sixth International Conference on Software Security and Reliability Companion, Gaithersburg, MD, pp. 198-206, 2012.
  • [20] J. E. Hopcroft, R. Motwani, and J. D. Ullman, Automata theory, languages, and computation." International Edition 24.2.2, 2006.
  • [21] ISELTA websitesi, Çevrimiçi olarak mevcut: http://iselta.ivknet.de, Son erişim: 03.03.2021
  • [22] O. Kilincceker, A. Silistre, M. Challenger and F. Belli, "Random Test Generation from Regular Expressions for Graphical User Interface (GUI) Testing," 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C), Sofia, Bulgaria, pp. 170-176, 2019.
  • [23] G. Mercan, E. Akgündüz, O. Kilincceker, M. Challenger, and F. Belli, “Android uygulaması testi için ideal test ön çalışması,” CEUR Workshop Proceedings, 2018.
  • [24] O. Kilincceker, and F. Belli, “Grafiksel kullanıcı arayüzleri için düzenli ifade bazlı test kapsama kriterleri,” CEUR Workshop Proceedings, 2017.
  • [25] O. Kilinccceker, E. Turk, M. Challenger and F. Belli, “Regular expression based test sequence generation for HDL program validation,” 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), Lisbon, pp. 585-592, 2018.
  • [26] O. Kilincceker, E. Turk, M. Challenger and F. Belli, "Applying the ideal testing framework to HDL programs," ARCS Workshop 2018; 31th International Conference on Architecture of Computing Systems, Braunschweig, Germany, pp. 1-6, 2018.
  • [27] O. Kilincceker and F. Belli, "Towards uniform modeling and holistic testing of hardware and software," 2019 1st International Informatics and Software Engineering Conference (UBMYK), Ankara, Turkey, pp. 1-6, 2019.
  • [28] Brüggemann-Klein, A. (1993). Regular expressions into finite automata. Theoretical Computer Science, 120(2), 197-213.
  • [29] M. Linschulte, On the Role of Test Sequence Length, Model Refinement, and Test Coverage for Reliability (PhD Thesis, Univ. Paderborn), 2014.
  • [30] F. Belli, A.T. Endo, M. Linschulte, and A. Simao, A Holistic Approach to Model-Based Testing of Web Service Compositions, Software: Practice and Experience, vol.44, no.2, pp. 201–23, 2014.
  • [31] A. Silistre, O. Kilincceker, F. Belli, M. Challenger and G. Kardas, "Models in Graphical User Interface Testing: Study Design," 2020 Turkish National Software Engineering Symposium (UYMS), Istanbul, Turkey, 2020, pp. 1-6, doi: 10.1109/UYMS50627.2020.9247072.
  • [32] A. Silistre, O. Kilincceker, F. Belli, M. Challenger, and G. Kardas, “Community Detection in Model-based Testing to Address Scalability: Study Design”, 15th Conference on Computer Science and Information Systems (FedCSIS 2020), Track on Software and Systems Engineering, Advances in Software and Systems Engineering (ASSE 2020), Sofia, Bulgaria, 2020, IEEE, pp. 657-660, DOI: 10.15439/2020F163.
There are 32 citations in total.

Details

Primary Language Turkish
Subjects Engineering
Journal Section Akademik ve/veya teknolojik bilimsel makale
Authors

Alper Silistre

Onur Kılınççeker

Fevzi Belli

Moharram Challenger 0000-0002-5436-6070

Geylani Kardaş 0000-0001-6975-305X

Publication Date June 21, 2022
Submission Date March 8, 2021
Published in Issue Year 2022 Volume: 12 Issue: 1

Cite

APA Silistre, A., Kılınççeker, O., Belli, F., Challenger, M., et al. (2022). Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım. EMO Bilimsel Dergi, 12(1), 7-19.
AMA Silistre A, Kılınççeker O, Belli F, Challenger M, Kardaş G. Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım. EMO Bilimsel Dergi. June 2022;12(1):7-19.
Chicago Silistre, Alper, Onur Kılınççeker, Fevzi Belli, Moharram Challenger, and Geylani Kardaş. “Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım”. EMO Bilimsel Dergi 12, no. 1 (June 2022): 7-19.
EndNote Silistre A, Kılınççeker O, Belli F, Challenger M, Kardaş G (June 1, 2022) Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım. EMO Bilimsel Dergi 12 1 7–19.
IEEE A. Silistre, O. Kılınççeker, F. Belli, M. Challenger, and G. Kardaş, “Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım”, EMO Bilimsel Dergi, vol. 12, no. 1, pp. 7–19, 2022.
ISNAD Silistre, Alper et al. “Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım”. EMO Bilimsel Dergi 12/1 (June 2022), 7-19.
JAMA Silistre A, Kılınççeker O, Belli F, Challenger M, Kardaş G. Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım. EMO Bilimsel Dergi. 2022;12:7–19.
MLA Silistre, Alper et al. “Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım”. EMO Bilimsel Dergi, vol. 12, no. 1, 2022, pp. 7-19.
Vancouver Silistre A, Kılınççeker O, Belli F, Challenger M, Kardaş G. Grafiksel Kullanıcı Arayüzü Testi İçin Bir Uçtan Uca Model Tabanlı Yaklaşım. EMO Bilimsel Dergi. 2022;12(1):7-19.
 Download Cover Image

EMO BİLİMSEL DERGİ
Elektrik, Elektronik, Bilgisayar, Biyomedikal, Kontrol Mühendisliği Bilimsel Hakemli Dergisi
TMMOB ELEKTRİK MÜHENDİSLERİ ODASI 
IHLAMUR SOKAK NO:10 KIZILAY/ANKARA
TEL: +90 (312) 425 32 72 (PBX) - FAKS: +90 (312) 417 38 18
bilimseldergi@emo.org.tr