PSI - Issue 75

Philippe AMUZUGA et al. / Procedia Structural Integrity 75 (2025) 53–64 Author name / Structural Integrity Procedia 00 (2025) 000–000

57

5

(a) Simple linear regression without transformations or feature selection.

(b) GLM with polynomial transformations and optimized variable selection.

Fig. 4: Comparison of model performance before and after optimization of variable selection.

2.2. Theoretical Foundations of Robustness and Stability Analysis of Linear Estimators Under Artificial Gaussian Noise

To better understand the stability of estimators under perturbations, it is essential to recall two fundamental concepts in linear regression: parameters ( β ), fixed but unknown theoretical coe ffi cients defining the ideal relationship y = X β + ε , and estimators ( ˆ β ), computed from observed data as:

ˆ β = ( X ⊤ X ) −

1 X ⊤ y .

These estimators are random variables sensitive to noise in the response y and are evaluated based on their proximity to the theoretical parameters (bias and variance). The Gauss–Markov theorem ensures that the ordinary least squares (OLS) estimator has minimum variance among all unbiased linear estimators, assuming centered Gaussian errors. This study empirically investigates whether these theoretical properties hold in industrial settings, particularly when the model includes polynomial transformations and automated variable selection. From a theoretical perspective, linear regression retains its optimal properties in the presence of additive Gaussian noise. For an extended model incorporating injected noise:

2 I ) , η ∼N (0 ,σ 2

y = X β + ε + η,

ε ∼N (0 ,σ

η I ) ,

where ε denotes intrinsic error and η the artificial noise. The ordinary least squares (OLS) estimator ˆ β = ( X ⊤ X ) − 1 X ⊤ y retains the following properties:

1. Unbiasedness: E [ ˆ β ] = β , assuming E [ η ] = 0 [11]

2. Minimum variance among all unbiased linear estimators (Gauss–Markov theorem) [8, 9]

3. Maximum likelihood estimator under Gaussian noise [10]