LOFO (Leave one feature out) is an open-source Python library that helps in finding out the feature importance. It calculates the importance of features by iteratively removing a feature from the dataset and find the importance in each iteration.

(Source Code)

https://youtu.be/9Vv9UcK-vag?t=1499

• LOFO is not fast, because it trains a model!
• Even if I'm not using LightGBM, I can use LOFO with any model. XGBoost , CatBoost , Deep Neural Networks , etc.
• LOFO actually assigns negative importance to those features.
• Since LOFO is calculated over your cross-validation splits, it provides strong estimates of true importance. Plus, you get a variance for each importance score to see if it ever drops into the negatives.

Advantage

Most data scientists use linear/logistic regression to figure out which features are important in a dataset. But linear models struggle to provide useful insights when dealing with correlated features. If you're not going to use a linear model in production, why are you using it to generate insights for your production model? The less disconnected your insights, the better you'll understand your actual production model.

This is the advantage of LOFO, because you can use it with your particular model architecture! Mark Tenenholtz, uses leave-one-out feature importance (LOFO) + LightGBM.

One really helpful technique: compare the feature importances for different models to see how robust they are.

https://twitter.com/marktenenholtz/status/1577992133079756801

Disadvantage