Using Item Response Theory to evaluate feature relevance in missing data scenarios

Detalhes bibliográficos
Ano de defesa: 2022
Autor(a) principal: REINALDO, Jessica Tais de Souza
Orientador(a): PRUDÊNCIO, Ricardo Bastos Cavalcante
Banca de defesa: Não Informado pela instituição
Tipo de documento: Dissertação
Tipo de acesso: Acesso aberto
Idioma: eng
Instituição de defesa: Universidade Federal de Pernambuco
Programa de Pós-Graduação: Programa de Pos Graduacao em Ciencia da Computacao
Departamento: Não Informado pela instituição
País: Brasil
Palavras-chave em Português:
Link de acesso: https://repositorio.ufpe.br/handle/123456789/46381
Resumo: Item Response Theory (IRT) has been historically used to evaluate the latent abilities of human respondents to a set of items. Recently, e orts have been made to propose solutions that use IRT to solve classification problems, where the respondents are classifiers and the items are the instances of a dataset. Most of the initial works that tried to tackle this problem used a dichotomous IRT model, which is capable of modelling the classification problem only in terms of correct and wrong predictions. B3-IRT o ers a powerful tool to analyze datasets and classifiers, as the response is continuous, so instead of representing the predictions just as right or wrong answers, the response can be represented by the probability of a correct prediction. Although the IRT formulation can provide rich information about the behavior of the models towards the instances of a dataset, no previous work has investigated the application of IRT to rank features in an instance-based approach, or even to evaluate how missing data can impact the IRT parameters for instances (diculty and discrimination) and classifiers (ability). We propose a workflow that uses B3-IRT in missing data scenarios to evaluate the relevance of features both locally for each instance of a dataset, and globally for the whole dataset. In this workflow, data is missing at test time, and missing values are filled out with imputed values, in order to evaluate how much the missing data can a ect the ability of classifiers and di culty and discrimination of instances. This novel application represents an alternative to feature selection and feature ranking techniques that is capable to provide an overview of feature relevance both at global and instance level.