Research Problems in Data Quality: Addressing Imbalanced and Missing Data - Santos, Miriam Raquel Seoane Pereira Seguro (2022)
Nowadays, data is deeply entangled in nearly all aspects of our daily lives, from social, business, transportation, energy, and even medical applications. Data is among us, it’s continuously growing, and its potential is immensely powerful. Nevertheless, its only value relies on our ability to understand it and transform it into meaningful insights. This task currently falls upon the shoulders of machine learning algorithms, that due to their ability to establish connections, patterns, and trends we humans cannot see, have become the cornerstone in analyzing, interpreting, and extracting knowledge from data. Traditional machine learning algorithms expect their input data to be well-behaved regarding several factors, such as balanced class distributions, well-represented concepts, and decision boundaries, an adequate training set size, consistent and correctly labeled instances, and a complete set of observed values in all features, among others. However, when applied “in the wild”, machine learning algorithms are inevitably faced with data imperfection, as many of these assumptions are broken, giving rise to several data problems such as imbalanced data, small disjuncts, class overlap, lack of data, noisy data, dataset shift, and missing data. These imperfections may arise either due to errors in the data acquisition, transmission, and collection processes, or due to the intrinsic nature of the domains, and they are responsible for the degradation of classification performance, and the generation of biased predictions. What ultimately determines the success of machine learning applications is therefore their ability to transform imperfect data into smart data, i.e., data of sufficient quality to allow classifiers to draw accurate and reliable inferences on the domain. In order to move from imperfect to smart data, it is critical to develop a thorough data understanding, which comprehends a well-grounded perception of a multitude of aspects regarding the domain and the data at hand. This involves a strong understanding of the bias generated by each data imperfection and how it aligns with the learning bias of classification or preprocessing algorithms, how data imperfections relate to other characteristics of the domains, how they exacerbate each other when appearing in combination, and why certain circumstances are especially harmful to classification tasks. Following this line of thought, this thesis dedicates time and effort to the characterization and understanding of data imperfections. We focus particularly on the problems of imbalanced data and missing data, which currently constitute two major lines of research, and further discuss the issues of small disjuncts and class overlap within the scope of imbalanced data. Accordingly, our main goal is to transfer some thoughts, discuss observations, and produce perceptive insights on working with complex scenarios where these data imperfections occur. This comprises the characterization of the data domains and the bias they may entail; the identification, characterization, and quantification of data imperfections in real-world domains; the identification of proper conditions for the efficient use of classifiers and preprocessing techniques; and the analysis of the bias associated with certain experimental setup hazards – all of which fall onto our notion of data understanding.