[scikit-learn] RFE with logistic regression

Dear scikit-learn users, I am using the recursive feature elimination (RFE) tool from sklearn to rank my features: from sklearn.linear_model import LogisticRegression classifier_RFE = LogisticRegression(C=1e9, verbose=1, max_iter=1) from sklearn.feature_selection import RFE rfe = RFE(estim

Re: [scikit-learn] RFE with logistic regression

liblinear regularizes the intercept (which is a questionable thing to do and a poor choice of default in sklearn). The other solvers do not. On Tue, Jul 24, 2018 at 4:07 AM, Benoît Presles wrote: > Dear scikit-learn users, > > I am using the recursive feature elimination (RFE) tool from sklearn t

Re: [scikit-learn] RFE with logistic regression

Agreed. But then the setting is c=1e9 in this context (where C is the inverse regularization strength), so the regularization effect should be very small. Probably shouldn't matter much for convex optimization, but I would still try to a) set the random_state to some fixed value b) make sure

Re: [scikit-learn] RFE with logistic regression

I did the same tests as before adding fit_intercept=False and: 1. I have got the same problem as before, i.e. when I execute the RFE multiple times I don't get the same ranking each time. 2. When I change the solver to 'sag' (classifier_RFE=LogisticRegression(C=1e9, verbose=1, max_iter=1,

Re: [scikit-learn] RFE with logistic regression

On 07/24/2018 02:07 PM, Benoît Presles wrote: I did the same tests as before adding fit_intercept=False and: 1. I have got the same problem as before, i.e. when I execute the RFE multiple times I don't get the same ranking each time. 2. When I change the solver to 'sag' (classifier_RFE=Log

Re: [scikit-learn] RFE with logistic regression

I did the same tests as before adding random_state=0 and: 1. I have got the same problem as before, i.e. when I execute the RFE multiple times I don't get the same ranking each time. 2. When I change the solver to 'sag' or 'saga' (LogisticRegression(C=1e9, verbose=1, max_iter=1, fit_inte

Re: [scikit-learn] RFE with logistic regression

Can you share your data or reproduce with synthetic data? On 07/24/2018 02:43 PM, Benoît Presles wrote: I did the same tests as before adding random_state=0 and: 1. I have got the same problem as before, i.e. when I execute the RFE multiple times I don't get the same ranking each time. 2. Wh

Re: [scikit-learn] RFE with logistic regression

I addition to checking _n_iter and fixing the random seed as I suggested maybe also try normalizing the features (eg z scores via the standard scale we) to see if that stabilizes the training Sent from my iPhone > On Jul 24, 2018, at 1:07 PM, Benoît Presles > wrote: > > I did the same tests

Re: [scikit-learn] RFE with logistic regression

On Tue, Jul 24, 2018 at 08:43:27PM +0200, Benoît Presles wrote: > 3. With C=1, it seems that I have the same results at each run for all > solvers (liblinear, sag and saga), however the ranking is not the same > between the solvers. Your problem is probably ill-conditioned, hence the specific weig

Re: [scikit-learn] RFE with logistic regression

So you think that I cannot get reproducible and consistent results with this method ? If you would avoid RFE, which method do you suggest to find the best features ? Ben Le 24/07/2018 à 21:34, Gael Varoquaux a écrit : On Tue, Jul 24, 2018 at 08:43:27PM +0200, Benoît Presles wrote: 3. With C

Re: [scikit-learn] RFE with logistic regression

Univariate screening is somewhat hackish too, but much more stable -- and cheap. Best, Bertrand On 24/07/2018 23:33, Benoît Presles wrote: So you think that I cannot get reproducible and consistent results with this method ? If you would avoid RFE, which method do you suggest to find the best

[scikit-learn] Help with Pull Request( Checks failing)

Hi everyone, I submitted my first PR few hours back and I see that two tests failed. Would really appreciate if anyone can help me with how to fix these/ what I am doing wrong. Thank you ! ___ scikit-learn mailing list [email protected] https

Re: [scikit-learn] Help with Pull Request( Checks failing)

This is the link to the PR - https://github.com/scikit-learn/scikit-learn/pull/11670 On Tue, Jul 24, 2018 at 8:33 PM, Prathusha Jonnagaddla Subramanyam Naidu < [email protected]> wrote: > Hi everyone, > I submitted my first PR few hours back and I see that two tests > failed. Would really appr

Re: [scikit-learn] Help with Pull Request( Checks failing)

I am not a core dev, but I think I can see what's wrong there (mostly Flake8 issues). Let me comment about that over there. > On Jul 24, 2018, at 7:34 PM, Prathusha Jonnagaddla Subramanyam Naidu > wrote: > > This is the link to the PR - > https://github.com/scikit-learn/scikit-learn/pull/1167

Re: [scikit-learn] Would love to contribute to this library that I fell in love with. I have a question! FIRST TIMER

Hi Abishek, In case you can't tell from the response, this is not a straightforward question to answer. I hope you have looked at our contributor guidelines: http://scikit-learn.org/dev/developers/contributing.html. We encourage contributors to start with changes that focus on things like documen