MACHINE LEARNING

APPLICATION OF SUPERVISED LEARNING

LINEAR REGRESSION

Question [CLICK ON ANY CHOICE TO KNOW THE RIGHT ANSWER]
Suppose you have fitted a complex regression model on a dataset. Now, you are using Ridge regression with penalty () aka (
A
In case of very large (); bias is low
B
In case of very large (); bias is high
C
We can’t say about bias
D
None of these
Explanation: 

Detailed explanation-1: -Ridge regression shrinks the regression coefficients, so that variables, with minor contribution to the outcome, have their coefficients close to zero. The shrinkage of the coefficients is achieved by penalizing the regression model with a penalty term called L2-norm, which is the sum of the squared coefficients.

Detailed explanation-2: -So, if the alpha value is 0, it means that it is just an Ordinary Least Squares Regression model. So, the larger is the alpha, the higher is the smoothness constraint. So, the smaller the value of alpha, the higher would be the magnitude of the coefficients. So, the alpha parameter need not be small.

Detailed explanation-3: -In ridge regression, however, the formula for the hat matrix should include the regularization penalty: Hridge = X(X′X + I)−1X, which gives dfridge = trHridge, which is no longer equal to m. Some ridge regression software produce information criteria based on the OLS formula.

There is 1 question to complete.