Running a Lasso Regression

Running a Lasso Regression

Rashid Iqbal's photo
Rashid Iqbal
·

3 min read

Table of contents

What is Lasso Regression

Lasso regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. Lasso regression is very similar to ridge regression, but there are some key differences between the two that you will have to understand if you want to use them effectively. In this article, you will learn everything you need to know about lasso regression, the differences between lasso and ridge, as well as how you can start using lasso regression in your own machine learning projects.

Lasso Meaning

The word “LASSO” stands for Least Absolute Shrinkage and Selection Operator. It is a statistical formula for the regularization of data models and feature selection.

Lasso Regularization Technique

There are two main regularization techniques, namely Ridge Regression and Lasso Regression. They both differ in the way they assign a penalty to the coefficients. In this blog, we will try to understand more about Lasso Regularization technique.

L1 Regularization

If a regression model uses the L1 Regularization technique, then it is called Lasso Regression. If it used the L2 regularization technique, it’s called Ridge Regression. We will study more about these in the later sections. L1 regularization adds a penalty that is equal to the absolute value of the magnitude of the coefficient. This regularization type can result in sparse models with few coefficients. Some coefficients might become zero and get eliminated from the model. Larger penalties result in coefficient values that are closer to zero (ideal for producing simpler models). On the other hand, L2 regularization does not result in any elimination of sparse models or coefficients. Thus, Lasso Regression is easier to interpret as compared to the Ridge.

Mathematical equation of Lasso Regression

Residual Sum of Squares + λ * (Sum of the absolute value of the magnitude of coefficients)

Where,

  • λ denotes the amount of shrinkage.
  • λ = 0 implies all features are considered and it is equivalent to the linear regression where only the residual sum of squares is considered to build a predictive model
  • λ = ∞ implies no feature is considered i.e, as λ closes to infinity it eliminates more and more features
  • The bias increases with an increase in λ
  • variance increases with a decrease in λ

Lasso Regression Example

import numpy as np

Creating a New Train and Validation Datasets

from sklearn.model_selection import train_test_split
data_train, data_val = train_test_split(new_data_train, test_size = 0.2, random_state = 2)

Classifying Predictors and Target

#Classifying Independent and Dependent Features
#_______________________________________________
#Dependent Variable
Y_train = data_train.iloc[:, -1].values
#Independent Variables
X_train = data_train.iloc[:,0 : -1].values
#Independent Variables for Test Set
X_test = data_val.iloc[:,0 : -1].values

Evaluating The Model With RMLSE

def score(y_pred, y_true):
error = np.square(np.log10(y_pred +1) - np.log10(y_true +1)).mean() ** 0.5
score = 1 - error
return score
actual_cost = list(data_val['COST'])
actual_cost = np.asarray(actual_cost)

Building the Lasso Regressor

#Lasso Regression

 from sklearn.linear_model import Lasso
 #Initializing the Lasso Regressor with Normalization Factor as True
 lasso_reg = Lasso(normalize=True)
 #Fitting the Training data to the Lasso regressor
 lasso_reg.fit(X_train,Y_train)
 #Predicting for X_test
 y_pred_lass =lasso_reg.predict(X_test)
 #Printing the Score with RMLSE
 print("\n\nLasso SCORE : ", score(y_pred_lass, actual_cost))