Question Classification with Constrained Resources: A Study with Coding Exercises

• Main Author: Rodrigues L.
• Publication Date: 2023
• Other Authors: Pereira F., Santos J., Oliveira E., Mello R., Marques L., Dermeval D., Gasparini, Isabela, Bittencourt I.I., Isotani S.
• Format: Conference object
• Language: eng
• Source: Repositório Institucional da Udesc
• Download full: https://repositorio.udesc.br/handle/UDESC/2610

© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.Evidence-based learning strategies, such as the testing effect, might help address the achievement gap. However, exploiting the testing effect depends on having a set of instructional activities with fine-grained tagging. While instructors might find questions in textbooks, they often lack fine-grained tagging, and data labeling is laborious. Despite much research on text classification, to our best knowledge, state-of-the-art question classifiers are mostly based on extensive models (i.e., BERT) and English text. Respectively, those are incompatible with the resource-constrained devices (e.g., mobile) and languages (e.g., Portuguese) of many underprivileged countries in the global south. Therefore, we developed a question classifier on top of DistilBERT, a version of BERT compatible with resource-constrained applications, using grid search and hold-out. Based on a corpus of 1045 coding questions written in Brazilian Portuguese, we found a model that achieved a near-perfect performance on unseen data, similar to last-generation results using BERT for English text. Thus, we present a step towards equitable education by i) providing underprivileged Portuguese-speaking countries with the support that enables opportunities already available for first-world countries and ii) demonstrating the feasibility of creating resource-constrained applications compatible with state-of-the-art AIED systems.