A Comprehensive Comparative Study of Classification and Regression Architectures: Empirical Performance Benchmarking on Standardized Datasets

Abstract

Supervised learning remains the backbone of predictive analytics. However, the decision to treat a target variable as continuous (Regression) or categorical (Classification) significantly alters model behavior and utility. This paper provides an exhaustive comparison of five classification and five regression techniques. Using the Wine Quality Dataset, we apply identical feature engineering to both paradigms. We measure performance through Mean Squared Error ($MSE$), $R^2$, Accuracy, and F1-Score. The results demonstrate that ensemble methods, specifically Random Forest and XGBoost, consistently outperform linear and kernel-based models, though classification provides a more robust framework for noisy data environments.