Re: Ticket 2273: django.contrib.auth.models.User: username is case-sensitive

[Florian Apolloner]

On Saturday, August 29, 2015 at 9:59:30 PM UTC+2, Tim Graham wrote:
>
> The suggested change is to not allow a new user if there's a 
> username__iexact match.
>

Yes, but as change to what? To the admin interface? this will only cover 
admin usage. To a full_clean of the user model [This is often not used 
anyways]? I am somewhat against that… 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/650405a3-c860-4d84-be94-5b6a6c779aa2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding more __repr__() methods

[Ned Batchelder]

The eval criterion rarely seems like a useful rule of thumb to me. Does 
anyone actually use this to make objects?  I think the useful rule is, a 
repr should be an unambiguous representation of the object that is 
useful in debugging.  Certainly the "eval" rule is sufficient for the 
"unambiguous" rule, but it isn't necessary.  I would rather have 
something readable than something evalable.


--Ned.

On 8/29/15 1:05 PM, Tim Graham wrote:
On the pull request, Fábio has raised the argument, "the |__repr__| 
method should be unambiguous and |eval(repr(obj)) == obj|".


Is this something we should strive for?

On Friday, August 28, 2015 at 9:18:04 PM UTC-4, Josh Smeaton wrote:

Right, I agree with you in the case then. But I think the general
idea of more/improved repr methods is a good one.

On Friday, 28 August 2015 23:02:25 UTC+10, Tim Graham wrote:

The repr for MessageMiddleware includes settings.MESSAGE_STORAGE.

On Thursday, August 27, 2015 at 8:59:37 PM UTC-4, Josh Smeaton
wrote:

Generally, I'm a fan of adding repr methods. I don't think
we should run around finding all the places they might be
useful and adding them, but if someone is to put forward
patches where they've noticed a benefit, we should
encourage that.

I'm not sure I fully understand your concerns though. It
seems to me that the repr methods in that PR only include
things relating to the class being observed. Did I miss
something?

Cheers

On Friday, 28 August 2015 01:16:02 UTC+10, Tim Graham wrote:

I'd like to ask for other opinions on this pull
request which adds more __repr__() methods:
https://github.com/django/django/pull/5059


Some concerns:

* Adding things that aren't part of the class (e.g.
settings) seems questionable to me.

* There is a chance to leak sensitive data in
|__repr__| as done in #22990
 (doesn't
seems to be a concern with the changes here, but is
something to be mindful of).


Thanks!

--
You received this message because you are subscribed to the Google 
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com 
.
To post to this group, send email to 
django-developers@googlegroups.com 
.

Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3f26bb3a-5c3d-4057-a788-92cde6768afb%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55E30AB0.2050305%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ticket 2273: django.contrib.auth.models.User: username is case-sensitive

[Collin Anderson]

I propose adding a check on UserCreationForm (used by the admin). The
current implementation relies on the model fields unique=True check which
is case-sensitive (except on mysql).

Whenever I've made a login form, I've always used or made a copy of or at
least studied UserCreationForm. I think if we had a nice walk through of
how to create your own form which included the iexact hint, I think that
would also help solve this problem.

On Sun, Aug 30, 2015 at 6:43 AM, Florian Apolloner 
wrote:

>
>
> On Saturday, August 29, 2015 at 9:59:30 PM UTC+2, Tim Graham wrote:
>>
>> The suggested change is to not allow a new user if there's a
>> username__iexact match.
>>
>
> Yes, but as change to what? To the admin interface? this will only cover
> admin usage. To a full_clean of the user model [This is often not used
> anyways]? I am somewhat against that…
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/650405a3-c860-4d84-be94-5b6a6c779aa2%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S6yF4dyY8ZiLwDu5D_0ffT6Qe%3D1NoRMwD1GccsNtPc4qw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making the test suite run faster

[Aymeric Augustin]

Hello,

I polished my test parallelization patch a bit. I'd like to merge it before
1.9 alpha. I rewrote history heavily to make the changes easier to review:
https://github.com/django/django/pull/4761

Eventually I settled for the approach I dismissed in a previous email: run
tests in workers, pass "events" back to the master process, feed them to the
master test runner. I depend on the `tblib` package to make tracebacks
pickleable.

The patch is still missing support for Oracle, despite significant help I
received from Shai and others. I'm proposing to merge it anyway because I
don't want to delay it indefinitely. It's possible to implement the missing
methods on Oracle at any point in the future.

I used various techniques to make tests safe for concurrent execution on a
case by case basis. When possible, I moved all filesystem writes to a
temporary directory. When not possible, I serialized execution of conflicting
test cases by locking a file.

If you have concerns about adding this feature to Django or about the
implementation -- which isn't the most beautiful code I've written -- now is a
good time to bring them up. I’d like to merge the patch in one or two weeks.

Also it would be interesting to try parallelization on test suites other than
Django's own. Just apply my PR and run `django-admin test --parallel` or
`django-admin test --parallel-num=N`. (Unfortunately I suspect few projects
are compatible with Django's development version and have a large test suite.)

Thanks!

-- 
Aymeric.



> On 22 févr. 2015, at 16:29, Aymeric Augustin 
>  wrote:
> 
> **tl;dr** I can run the full test suite in 85 seconds on SQLite, a 4.8x 
> speedup.
> 
> 
> Hello,
> 
> Since I last wrote about this project, I improved parallelization by:
> 
> - reworking the IPC to avoid exchanging tracebacks
> - implementing database duplication for SQLite, PostgreSQL and MySQL
> 
> The code is still rough. Several options of runtests.py don't work with
> parallelization.
> 
> Initially performance was disappointing. I couldn’t max out my cores during
> the whole run. With some basic monitoring I noticed that the CPU load
> plummeted when there were spikes of disk writes. This led me to believe I was
> disk I/O bound even with an in-memory database and a SSD.
> 
> So I started optimizing disk I/O, which means doing less I/O and doing it only
> in a RAM-mounted temporary directory. Writing to RAM instead of writing to
> disk helps a lot. It’s as simple as creating a RAMdisk (that depends on your
> OS) and pointing the TMPDIR environment variable to the RAMdisk.
> 
> Unfortunately, i18n and migrations management commands write in the
> application directories. I haven't found (yet) a way to point them to a
> temporary directory instead.
> 
> My 2012 MacBook Pro with a 2.3 GHz Intel Core i7 (4 cores, 8 threads) takes:
> 
> - 30 seconds for creating the two databases
> 
> - 240 seconds to run the actual tests in a single process
> - 72 seconds in 4 processes (3.3x faster)
> - 60 seconds in 6 processes (4x faster)
> - 55 seconds in 8 processes (4.4x faster)
> 
> That looks quite close to what my hardware can do. Hyperthreading doesn't help
> as much as multiple cores when it comes to running multiple processes and the
> synchronization costs increase with the number of processes.
> 
> Creating the database accounts for more than a third of the total runtime. I'm
> not sure how much time is spent in the migrations framework and how much doing
> the table creations. --keepdb helps but it requires an on-disk database. An
> on-RAMdisk database would most likely be the best option. I haven't tried it
> yet. It should work with at least SQLite and PostgreSQL (using tablespaces).
> 
> If you want to help, I'd be interested in:
> 
> - reports of whether parallelization works for test suites other than Django's
>   own -- apply my pull request and run `django-admin test --parallel` or
>   `django-admin test --parallel-num=N`
> - a patch implementing database duplication on Oracle
> 
> Let me know if you have questions or concerns.
> 
> -- 
> Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/F7BA7C0F-3214-42E2-AAD2-B90B62EC07E2%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Making the test suite run faster

[Andrew Godwin]

I'm on board merging this without Oracle support - if I understand right,
it means we just have to run those non-parallel for now, so we're not
losing anything, right?

Andrew

On Sun, Aug 30, 2015 at 1:19 PM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Hello,
>
> I polished my test parallelization patch a bit. I'd like to merge it before
> 1.9 alpha. I rewrote history heavily to make the changes easier to review:
> https://github.com/django/django/pull/4761
>
> Eventually I settled for the approach I dismissed in a previous email: run
> tests in workers, pass "events" back to the master process, feed them to
> the
> master test runner. I depend on the `tblib` package to make tracebacks
> pickleable.
>
> The patch is still missing support for Oracle, despite significant help I
> received from Shai and others. I'm proposing to merge it anyway because I
> don't want to delay it indefinitely. It's possible to implement the missing
> methods on Oracle at any point in the future.
>
> I used various techniques to make tests safe for concurrent execution on a
> case by case basis. When possible, I moved all filesystem writes to a
> temporary directory. When not possible, I serialized execution of
> conflicting
> test cases by locking a file.
>
> If you have concerns about adding this feature to Django or about the
> implementation -- which isn't the most beautiful code I've written -- now
> is a
> good time to bring them up. I’d like to merge the patch in one or two
> weeks.
>
> Also it would be interesting to try parallelization on test suites other
> than
> Django's own. Just apply my PR and run `django-admin test --parallel` or
> `django-admin test --parallel-num=N`. (Unfortunately I suspect few projects
> are compatible with Django's development version and have a large test
> suite.)
>
> Thanks!
>
> --
> Aymeric.
>
>
>
> > On 22 févr. 2015, at 16:29, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >
> > **tl;dr** I can run the full test suite in 85 seconds on SQLite, a 4.8x
> speedup.
> >
> >
> > Hello,
> >
> > Since I last wrote about this project, I improved parallelization by:
> >
> > - reworking the IPC to avoid exchanging tracebacks
> > - implementing database duplication for SQLite, PostgreSQL and MySQL
> >
> > The code is still rough. Several options of runtests.py don't work with
> > parallelization.
> >
> > Initially performance was disappointing. I couldn’t max out my cores
> during
> > the whole run. With some basic monitoring I noticed that the CPU load
> > plummeted when there were spikes of disk writes. This led me to believe
> I was
> > disk I/O bound even with an in-memory database and a SSD.
> >
> > So I started optimizing disk I/O, which means doing less I/O and doing
> it only
> > in a RAM-mounted temporary directory. Writing to RAM instead of writing
> to
> > disk helps a lot. It’s as simple as creating a RAMdisk (that depends on
> your
> > OS) and pointing the TMPDIR environment variable to the RAMdisk.
> >
> > Unfortunately, i18n and migrations management commands write in the
> > application directories. I haven't found (yet) a way to point them to a
> > temporary directory instead.
> >
> > My 2012 MacBook Pro with a 2.3 GHz Intel Core i7 (4 cores, 8 threads)
> takes:
> >
> > - 30 seconds for creating the two databases
> >
> > - 240 seconds to run the actual tests in a single process
> > - 72 seconds in 4 processes (3.3x faster)
> > - 60 seconds in 6 processes (4x faster)
> > - 55 seconds in 8 processes (4.4x faster)
> >
> > That looks quite close to what my hardware can do. Hyperthreading
> doesn't help
> > as much as multiple cores when it comes to running multiple processes
> and the
> > synchronization costs increase with the number of processes.
> >
> > Creating the database accounts for more than a third of the total
> runtime. I'm
> > not sure how much time is spent in the migrations framework and how much
> doing
> > the table creations. --keepdb helps but it requires an on-disk database.
> An
> > on-RAMdisk database would most likely be the best option. I haven't
> tried it
> > yet. It should work with at least SQLite and PostgreSQL (using
> tablespaces).
> >
> > If you want to help, I'd be interested in:
> >
> > - reports of whether parallelization works for test suites other than
> Django's
> >   own -- apply my pull request and run `django-admin test --parallel` or
> >   `django-admin test --parallel-num=N`
> > - a patch implementing database duplication on Oracle
> >
> > Let me know if you have questions or concerns.
> >
> > --
> > Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.co

Re: Making the test suite run faster

[Aymeric Augustin]

On 31 août 2015, at 06:12, Andrew Godwin  wrote:

> I'm on board merging this without Oracle support - if I understand right, it 
> means we just have to run those non-parallel for now, so we're not losing 
> anything, right?

Right.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/72EC1F48-9CD7-496B-BFA0-017B9AA93B75%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Making the test suite run faster

[Tino de Bruijn]

Hi Aymeric,

Really cool that you pushed this further. Out of curiosity I have two
questions:

- What happens when two SerializeMixin tests try to lock the same file?
Does one wait for the other (probably not), or is a lockfile exception
raised?
- How does this work in combination with the --keepdb flag

Thanks,


Tino

On Mon, Aug 31, 2015 at 8:13 AM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> On 31 août 2015, at 06:12, Andrew Godwin  wrote:
>
> > I'm on board merging this without Oracle support - if I understand
> right, it means we just have to run those non-parallel for now, so we're
> not losing anything, right?
>
> Right.
>
> --
> Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/72EC1F48-9CD7-496B-BFA0-017B9AA93B75%40polytechnique.org
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANQFsQCejkrXOo2DpPZrmo0cG-Q5LzBvDvK%3DT_RKpkFfvtyumw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.