VCAT: an automatic assessment model for visual programming languages

• Ano de defesa: 2023
• Autor(a) principal: Souza, Lucas Mendonça de
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Automatic assessment; Avaliação automática; Ensino de programação; Programação visual; Programming teaching; Visual programming
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/45/45134/tde-17012024-190825/

Learning how to code is becoming a crucial aspect of modern society. The presence of digital technologies in everyday life requires some basic understanding on how software works in order to be aware of how this affects everyones life. Moreover, concerns on privacy and digital security also raises the relevance of understanding computer science concepts. These aspects are not only related to programming and can be referred in the literature as Computational Thinking. Computational Thinking is understood as the application of computer science concepts in different contexts of day-to-day life. As an answer to this, governments worldwide are implementing new school curricula that incorporate programming as a key skill. However, the literature shows that learning how to program is a complex and difficult task. In some cases, the dropout and failure rates can be very alarming. So, in order to mitigate these learning problems some researchers employed the visual programming paradigm. This paradigm consists in using visual elements to code algorithms. In this context, the experiments reported in the literature claim improvements in the learning process, such as better grades and motivation to learn. Another technology employed to support teaching and learning programming is the automatic assessment of programs. These tools can automatically evaluate a program using different methods. They allow teachers to assess a large number of exercises and also provide quick feedback to students. Nonetheless, at the best of our knowledge, only two visual programming systems that provide automatic assessment were found: iVProg and Chentry. Both systems offer a limited method of assessment. Therefore, this research proposes VCAT, an automatic assessment model to support Visual Programming Systems on providing this functionality to diverse visual programming languages. In addition to providing such a functionality, it also introduces improvements to its output matching algorithms, in order to provide a better experience with the generated feedback. The model was successfully instantiated considering iVProg and Blockly, which indicates that it is independent of the underlying visual programming language of the system. An experiment with students of the summer program for introductory programming was designed to assess how the students perceived the feedback improvements and the use of visual programming systems. Data from the experiment shows that the changes implemented in the output matching feedback was well received by the students when compared to VPL, a similar tool using the same type of automatic assessment without improvements. Moreover, data also suggests benefits of using visual programming as a learning step toward text based programming.