Using meta-learning to predict performance metrics in machine learning problems

Bibliographic Details
Main Author: Carneiro, Davide
Publication Date: 2023
Other Authors: Guimaraes, Miguel, Carvalho, Mariana, Novais, Paulo
Format: Article
Language: eng
Source: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
Download full: https://hdl.handle.net/1822/78002
Summary: First published: 29 November 2021
id RCAP_a20cc42e545ffb0f24a885b88f0c73e7
oai_identifier_str oai:repositorium.sdum.uminho.pt:1822/78002
network_acronym_str RCAP
network_name_str Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
repository_id_str https://opendoar.ac.uk/repository/7160
spelling Using meta-learning to predict performance metrics in machine learning problemserror predictioninteractive machine learningmeta-learningScience & TechnologyFirst published: 29 November 2021Machine learning has been facing significant challenges over the last years, much of which stem from the new characteristics of machine learning problems, such as learning from streaming data or incorporating human feedback into existing datasets and models. In these dynamic scenarios, data change over time and models must adapt. However, new data do not necessarily mean new patterns. The main goal of this paper is to devise a method to predict a model's performance metrics before it is trained, in order to decide whether it is worth it to train it or not. That is, will the model hold significantly better results than the current one? To address this issue, we propose the use of meta-learning. Specifically, we evaluate two different meta-models, one built for a specific machine learning problem, and another built based on many different problems, meant to be a generic meta-model, applicable to virtually any problem. In this paper, we focus only on the prediction of the root mean square error (RMSE). Results show that it is possible to accurately predict the RMSE of future models, event in streaming scenarios. Moreover, results also show that it is possible to reduce the need for re-training models between 60% and 98%, depending on the problem and on the threshold used.This work was supported by the Northern Regional Operational Program, Portugal 2020 and European Union, trough European Regional Development Fund (ERDF) in the scope of project number 39900 - 31/SI/2017, and by FCT – Fundação para a Ciência e Tecnologia within projects UIDB/04728/2020 and UIDB/00319/2020.WileyUniversidade do MinhoCarneiro, DavideGuimaraes, MiguelCarvalho, MarianaNovais, Paulo20232023-01-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/1822/78002engCarneiro, D., Guimarães, M., Carvalho, M., Novais, P. (2023). Using meta-learning to predict performance metrics in machine learning problems. Expert Systems, 40(1), e12900. https://doi.org/10.1111/exsy.129000266-472010.1111/exsy.12900https://onlinelibrary.wiley.com/doi/10.1111/exsy.12900info:eu-repo/semantics/openAccessreponame:Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)instname:FCCN, serviços digitais da FCT – Fundação para a Ciência e a Tecnologiainstacron:RCAAP2024-05-11T06:56:03Zoai:repositorium.sdum.uminho.pt:1822/78002Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireinfo@rcaap.ptopendoar:https://opendoar.ac.uk/repository/71602025-05-28T16:09:29.031038Repositórios Científicos de Acesso Aberto de Portugal (RCAAP) - FCCN, serviços digitais da FCT – Fundação para a Ciência e a Tecnologiafalse
dc.title.none.fl_str_mv Using meta-learning to predict performance metrics in machine learning problems
title Using meta-learning to predict performance metrics in machine learning problems
spellingShingle Using meta-learning to predict performance metrics in machine learning problems
Carneiro, Davide
error prediction
interactive machine learning
meta-learning
Science & Technology
title_short Using meta-learning to predict performance metrics in machine learning problems
title_full Using meta-learning to predict performance metrics in machine learning problems
title_fullStr Using meta-learning to predict performance metrics in machine learning problems
title_full_unstemmed Using meta-learning to predict performance metrics in machine learning problems
title_sort Using meta-learning to predict performance metrics in machine learning problems
author Carneiro, Davide
author_facet Carneiro, Davide
Guimaraes, Miguel
Carvalho, Mariana
Novais, Paulo
author_role author
author2 Guimaraes, Miguel
Carvalho, Mariana
Novais, Paulo
author2_role author
author
author
dc.contributor.none.fl_str_mv Universidade do Minho
dc.contributor.author.fl_str_mv Carneiro, Davide
Guimaraes, Miguel
Carvalho, Mariana
Novais, Paulo
dc.subject.por.fl_str_mv error prediction
interactive machine learning
meta-learning
Science & Technology
topic error prediction
interactive machine learning
meta-learning
Science & Technology
description First published: 29 November 2021
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-01-01T00:00:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv https://hdl.handle.net/1822/78002
url https://hdl.handle.net/1822/78002
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv Carneiro, D., Guimarães, M., Carvalho, M., Novais, P. (2023). Using meta-learning to predict performance metrics in machine learning problems. Expert Systems, 40(1), e12900. https://doi.org/10.1111/exsy.12900
0266-4720
10.1111/exsy.12900
https://onlinelibrary.wiley.com/doi/10.1111/exsy.12900
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Wiley
publisher.none.fl_str_mv Wiley
dc.source.none.fl_str_mv reponame:Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
instname:FCCN, serviços digitais da FCT – Fundação para a Ciência e a Tecnologia
instacron:RCAAP
instname_str FCCN, serviços digitais da FCT – Fundação para a Ciência e a Tecnologia
instacron_str RCAAP
institution RCAAP
reponame_str Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
collection Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
repository.name.fl_str_mv Repositórios Científicos de Acesso Aberto de Portugal (RCAAP) - FCCN, serviços digitais da FCT – Fundação para a Ciência e a Tecnologia
repository.mail.fl_str_mv info@rcaap.pt
_version_ 1833595768835932160

Similar Items