Improving the accuracy of multivariate models: a study of sample dehydration and data preprocessing optimization - Cardoso, Wilson Júnior (2023)
The aim of this thesis is to study approaches to improve the accuracy of multivariate models. Two approaches were considered, one relating to sample preparation and the other related to data preprocessing. The first chapter aimed to study sample dehydration to improve the prediction of sucrose, glucose, and fructose in sugarcane juice using near-infrared (NIR) spectroscopy and partial least squares (PLS) regression models. Models using the NIR spectra acquired using the liquid (LSJ) and dehydrated sugarcane juice (DSJ) were compared. In addition, the NIR spectra were acquired using a benchtop and a portable instrument. Ordered predictors selection (OPS) was applied to select the most informative variable. The results indicated better predictions for all sugars using the DSJ for both instruments, being the benchtop statistically better than the portable instrument. To sum up, the dehydration approach showed to be a great technique to improve the predictability of PLS-OPS models for sugars in sugarcane juice using NIR spectra by removing the water and concentrating the analytes. The second chapter presented an algorithm that automatically searches for the best preprocessing strategy without fixing their order based on the artifact they fix, i.e., baseline correction, scatter correction, noise removal, and scaling. The number of preprocessing methods in each strategy and their hyperparameters were evaluated. The algorithm was compared with methods presented in the literature by Gerretzen et al. (2015) and Jiao et al. (2020). A fair, extensive, and comprehensive study was carried out, evaluating 67 different calibration datasets. This work demonstrated that not fixing the order in which the preprocessing is applied was essential to find the best models with a significant reduction in the RMSEP values when compared with the other methods, therefore presenting a comprehensive insight into data preprocessing. These results showed that a proper sample preparation and a proper optimization of the data preprocessing strategy are fundamental to build the best models. Keywords: Chemometrics. Sample Preparation. Water Removal. Data Preprocessing.