Test automation from natural language with reusable capture & replay and consistency analysis

Detalhes bibliográficos
Ano de defesa: 2017
Autor(a) principal: ARRUDA, Filipe Marques Chaves de
Orientador(a): SAMPAIO, Augusto Cezar Alves
Banca de defesa: Não Informado pela instituição
Tipo de documento: Dissertação
Tipo de acesso: Acesso aberto
Idioma: eng
Instituição de defesa: Universidade Federal de Pernambuco
Programa de Pós-Graduação: Programa de Pos Graduacao em Ciencia da Computacao
Departamento: Não Informado pela instituição
País: Brasil
Palavras-chave em Português:
Link de acesso: https://repositorio.ufpe.br/handle/123456789/25354
Resumo: A common scenario when automating tests begins with a test engineer writing test cases with no formal specification or software supervision. Then, there is a developer that tries to understand what an ambiguous test means in order to transform it into code. Finally, there is an experienced tester that is responsible to supervise the execution and to verify whether the failures are indeed bugs, or a mistake from the developer or test engineer, or just a matter of change in the requirements or user interface. To reduce these recurring problems in test automation, we propose a unified process, to write test cases using a controlled natural language to check consistency and dependencies automatically or search for similar test descriptions written in free natural language. The proposed process is applied to a reusable capture & replay strategy, so anyone can automate tests even without previous knowledge about coding, besides mitigating scalability/maintainability issues by reusing actions with granularity ranging from simple commands, such as open an application, to entire test cases or even test suites. The actions are represented by an abstract, framework-free notation. Besides, the implementation covers plugins, including voice recording and image processing; and a proactive traceability for elements in the user interface. The strategy was consolidated through its implementation in a tool, in the context of a partnership with Motorola Mobility. It has been adopted in practice for different test suites, achieving a reuse ratio up to 71% for test cases written with no standard whatsoever, yet presenting time gains similar to traditional C&R approaches when compared to coding. Furthermore, through standardization, it was possible to define a consistency notion and to capture an association graph (whose edges represent dependency and cancellation relations) among test actions. From these associations, it was possible to establish a mechanism to ensure that each test action is preceded by actions on which it depends to be granted to execute. An editor (Kaki) was implemented to mechanize both adherence to the CNL and the dependencies between tests. From the associations informed within the user interface, Kaki generates an Alloy model automatically and uses Alloy Analyzer to verify consistency, besides possibly suggesting the insertion of test actions to satisfy the dependencies, automatically.