Ismael Lemhadri | Feng Ruan | Louis Abraham | Rob Tibshirani |

[Paper] | [Code] | [Docs] | [Poster] | [Slides] |

**LassoNet** is a method for feature selection in neural networks, to enhance interpretability of the final
network.

- It uses a novel objective function and learning algorithm, that encourage the network to use only a subset of
the available input features, that is the resulting network is
**"feature sparse"** - This is achieved not by post-hoc analysis of a standard neural network but is
**built into objective function itself**:- Input to output (skip layer) connections are added to the network with an L1 penalty on its weights
- The weight for each feature in this layer acts as an upper bound for all hidden layer weights involving that feature

- The result is an
**entire path of network solutions**, with varying amounts of feature sparsity. This is analogous to the lasso solution path for linear regression

pip install lassonet

- standardizing the inputs
- making sure that the initial dense model (with $\lambda = 0$) has trained well, before starting the LassoNet regularization path. This may involve hyper-parameter tuning, choosing the right optimizer, and so on. If the dense model is underperforming, it is likely that the sparser models will as well.
- making sure the stepsize over the $\lambda$ path is not too large. By default, the stepsize runs in geometric increments until there is no feature left.

@article{lemhadri2019neural, title={LassoNet: Neural Networks with Feature Sparsity}, author={Lemhadri, Ismael and Ruan, Feng and Abraham, Louis and Tibshirani, Robert}, journal={arXiv preprint arXiv:1907.12207}, year={2019} }