International Journal of Machine Learning (IJOML)

Evaluation of Undersampling and Oversampling Techniques in Term Deposit Prediction: A Gradient Boosting Approach

2026-01-27T07:56:21+07:00

Time deposits play a pivotal role in maintaining banking liquidity, yet telemarketing campaigns designed to secure them are often inefficient due to low response rates and untargeted outreach. The primary challenge in predictive marketing modeling lies in extreme data class imbalance, which renders standard algorithms prone to bias and leads to a failure in detecting potential customers. This study aims to validate the effectiveness of Gradient Boosting models and empirically evaluate the impact of various resampling techniques in mitigating class distribution disparities. The applied methodology encompasses the utilization of XGBoost, LightGBM, and CatBoost algorithms on the UCI Bank Marketing dataset, integrated with Random Under-Sampling, Random Over-Sampling, SMOTENC, and Tomek Links strategies. Experimental results reveal a significant trade-off between sensitivity and precision, wherein LightGBM paired with Random Under-Sampling achieved the highest detection capability with a Recall of 88.28%. Concurrently, the combination of CatBoost with Random Over-Sampling demonstrated the optimal balance, attaining an F1-Score of 0.6040, a Recall of 81.95%, and an AUC-ROC value reaching 0.9326. These findings offer a strategic contribution to bank management in selecting analytic approaches aligned with business priorities, whether the focus is on operational cost efficiency or aggressive market penetration to optimize customer acquisition.

Benchmarking Modern Optimizers for IndoBERT-Based Sentiment Analysis on Indonesian Gojek Reviews

2026-01-27T08:09:57+07:00

User reviews on platforms like Gojek serve as critical data for business intelligence, necessitating robust automated sentiment analysis models. While IndoBERT is the standard architecture for Indonesian natural language processing, the comparative impact of emerging optimizers on its performance remains underexplored, as most existing studies default to AdamW without investigating modern alternatives. This research comprehensively benchmarks five optimizers—AdamW, Muon, AdaMuon, Lion, and Sophia—by fine-tuning IndoBERT on 29,851 Indonesian Gojek reviews to identify the most effective training strategy. The study evaluates classification metrics alongside computational efficiency indicators, including training duration and peak memory usage. Empirical results demonstrate that AdamW, AdaMuon, and Lion achieve statistically equivalent superior performance, attaining an average accuracy of 91.6% and an F1-macro of 91.5%. Conversely, Muon and Sophia exhibit slightly lower predictive capability with higher resource demands. Regarding computational cost, AdamW and Lion provide the optimal balance of rapid convergence and memory efficiency, whereas Sophia demands significantly higher VRAM and matrix-based optimizers like Muon extend training duration. These findings confirm that AdamW remains the most robust and efficient choice for analyzing informal Indonesian text, indicating that the complex update mechanisms of newer optimizers do not yield necessary marginal gains for this specific classification task.

RoBERTa with Sample Reweighting and Temperature Scaling for Imbalanced Toxicity Detection: A Performance–Fairness–Calibration Study

2026-01-27T07:54:09+07:00

Detecting toxic language at scale requires models that are not only accurate but also robust to demographic subgroup bias and reliable in their probability estimates; however, these objectives can conflict, especially under severe class imbalance. This study investigates the performance–fairness–calibration interplay in toxicity detection using the Jigsaw Unintended Bias dataset (124,858 comments; 5.99% toxic; identity annotations in 9.39% of samples). We aim to quantify how sample reweighting and imbalance-aware training affect global discrimination, worst-subgroup behavior, and probabilistic calibration, and to assess post-hoc temperature scaling on predicted probabilities. We compare a TF-IDF + logistic regression baseline against RoBERTa variants trained without mitigation, with sample reweighting, and with an imbalance-oriented loss, using multi-metric evaluation (AUC, Min/Worst-Subgroup AUC, ECE, and NLL). RoBERTa consistently improves global AUC over the baseline (≈0.96 vs 0.9155) while worst-subgroup AUC remains substantially lower and varies modestly across RoBERTa variants (≈0.7726–0.7813). Calibration results indicate a marked gap between models: the baseline achieves the lowest ECE (0.0052), whereas RoBERTa exhibits higher ECE (≈0.0257) that increases further under reweighting and imbalance-oriented training (≈0.0490–0.0866), with NLL not improving consistently. These findings contribute empirical evidence that fairness-oriented interventions can shift error and calibration profiles, motivating holistic evaluation and methods that jointly constrain subgroup fairness and probabilistic reliability.

Cold-Start Generalization in Educational Interaction Data: Comparing Student-Wise and Question-Wise Splits with Probabilistic Calibration

2026-01-26T21:23:41+07:00

Predictive models in Intelligent Tutoring Systems often face performance degradation due to sparse data and the cold-start problem, further compounded by a lack of probability calibration in standard evaluations. This study bridges this gap by systematically evaluating the trade-off between discriminative accuracy and probabilistic reliability through student-wise and question-wise splits, utilizing interaction data from the MathE platform across eight countries. By comparing identifier-based and metadata-based Logistic Regression models under a Leave-One-Country-Out protocol, we assessed generalization capabilities against distribution shifts. The results reveal a fundamental dichotomy: while identifier-based models achieve superior precision (AUC 0.687) and calibration in scenarios with historical context, they suffer from significant performance drops in student cold-start settings and exhibit negative transfer during cross-country deployment. Conversely, metadata-based models demonstrate higher robustness and invariance across varying demographics. We conclude that relying solely on accuracy metrics masks model uncertainty in new domains and recommend a "safe-start" strategy that prioritizes metadata-based features for system initialization to ensure reliable pedagogical decision-making before personalizing based on accumulated user history.

Stability-Aware Hierarchical Forecasting: Synergizing Conformal Prediction with Decomposition Ensembles

2026-01-27T08:07:34+07:00

Accurate retail demand forecasting is frequently impeded by high-dimensional hierarchies and intermittent sales patterns, which destabilize traditional models and compromise operational decision-making. To address these challenges, this study develops a stability-aware forecasting framework that unifies global machine learning ensembles with hierarchical reconciliation and conformal uncertainty calibration. Utilizing the large-scale M5 dataset, the methodology synergizes decomposition-based feature engineering with a global Light Gradient Boosting Machine (LightGBM), reinforced by a robust Bottom-Up reconciliation strategy and Centered Conformalized Quantile Regression (CQR). Empirical results based on rolling-origin cross-validation demonstrate that the proposed framework achieves a superior Weighted Root Mean Squared Scaled Error (WRMSSE) of 8.7723, significantly outperforming both the standalone LightGBM (9.4846) and the Seasonal Naïve baseline (10.1740). Furthermore, the Centered CQR mechanism effectively balances predictive sharpness with coverage, attaining a Scaled Pinball Loss (SPL) of 0.2347, thereby mitigating error degradation often observed in sparse data regimes. These findings confirm that integrating structural decomposition with rigorous reconciliation acts as a potent regularizer, offering a scientifically robust solution for managing non-stationarity and signal sparsity in complex retail supply chains.