Re: [Python-Dev] very bad network performance

2008-04-21 Thread Ralf Schmitt
On Mon, Apr 21, 2008 at 8:10 PM, Gregory P. Smith <[EMAIL PROTECTED]> wrote: > > > > The 64K hunch is wrong. The system limit can be found using > getsockopt(...SO_RCVBUF...). It can easily be (and often is) set to many > megabytes either at a system default level or on a per socket level by the

Re: [Python-Dev] very bad network performance

2008-04-21 Thread Gregory P. Smith
On Mon, Apr 14, 2008 at 4:41 PM, Curt Hagenlocher <[EMAIL PROTECTED]> wrote: > On Mon, Apr 14, 2008 at 4:19 PM, Guido van Rossum <[EMAIL PROTECTED]> > wrote: > > > > But why was imaplib apparently specifying 10MB? Did it know there was > > that much data? Or did it just not want to bother looping

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Curt Hagenlocher
On Mon, Apr 14, 2008 at 4:19 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > But why was imaplib apparently specifying 10MB? Did it know there was > that much data? Or did it just not want to bother looping over all the > data in smaller buffer increments (e.g. 64K, which is probably the max >

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Ralf Schmitt
On Tue, Apr 15, 2008 at 1:19 AM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > But why was imaplib apparently specifying 10MB? Did it know there was > that much data? Or did it just not want to bother looping over all the > data in smaller buffer increments (e.g. 64K, which is probably the max >

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Guido van Rossum
On Mon, Apr 14, 2008 at 3:57 PM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > > > On Tue, Apr 15, 2008 at 12:19 AM, Curt Hagenlocher <[EMAIL PROTECTED]> > wrote: > > > > On Mon, Apr 14, 2008 at 2:29 PM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > > > > > Sorry to reply on the mailing list. But this

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Ralf Schmitt
On Tue, Apr 15, 2008 at 12:19 AM, Curt Hagenlocher <[EMAIL PROTECTED]> wrote: > On Mon, Apr 14, 2008 at 2:29 PM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > > > Sorry to reply on the mailing list. But this change is wrong. > > e.g. if you're using a buffer size of 16 bytes and try to read 256 > by

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Curt Hagenlocher
On Mon, Apr 14, 2008 at 2:29 PM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > Sorry to reply on the mailing list. But this change is wrong. > e.g. if you're using a buffer size of 16 bytes and try to read 256 bytes, it > should call recv with a value of 256 and not call recv 16 times with a value >

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Ralf Schmitt
On Mon, Apr 14, 2008 at 11:18 PM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > On Mon, Apr 14, 2008 at 8:22 PM, Guido van Rossum <[EMAIL PROTECTED]> > wrote: > > > Eek! Please use the bug tracker. > > > > I 've made some comments on: http://bugs.python.org/issue1092502 (which is > the original issu

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Ralf Schmitt
On Mon, Apr 14, 2008 at 8:10 PM, Curt Hagenlocher <[EMAIL PROTECTED]> wrote: > On Mon, Apr 14, 2008 at 9:12 AM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > > > I've tracked it down to this change: > > http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd > > this is svn revision 61009. > > [...] > >

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Ralf Schmitt
On Mon, Apr 14, 2008 at 8:22 PM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Eek! Please use the bug tracker. > I 've made some comments on: http://bugs.python.org/issue1092502 (which is the original issue). However I cannot reopen this issue. Regards, - Ralf __

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Curt Hagenlocher
On Mon, Apr 14, 2008 at 12:17 PM, A.M. Kuchling <[EMAIL PROTECTED]> wrote: > On Mon, Apr 14, 2008 at 11:10:12AM -0700, Curt Hagenlocher wrote: > > while True: > > left = size - buf_len > > ! recv_size = max(self._rbufsize, left) > >

Re: [Python-Dev] very bad network performance

2008-04-14 Thread A.M. Kuchling
On Mon, Apr 14, 2008 at 11:10:12AM -0700, Curt Hagenlocher wrote: > while True: > left = size - buf_len > ! recv_size = max(self._rbufsize, left) > data = self._sock.recv(recv_size) What version is this patch against? (The last 2.5

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Bill Janssen
There's some really convoluted code in socket._fileobject.__init__() here. When initializing a _fileobject, if the 'bufsize' parameter is explicitly given as zero, that's turned into an _rbufsize of 1, which, combined with the 'min' change, will produce the read-one-byte behavior. The code for se

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Guido van Rossum
Eek! Please use the bug tracker. On Mon, Apr 14, 2008 at 11:10 AM, Curt Hagenlocher <[EMAIL PROTECTED]> wrote: > On Mon, Apr 14, 2008 at 9:12 AM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > > > I've tracked it down to this change: > > http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd > > this

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Curt Hagenlocher
On Mon, Apr 14, 2008 at 9:12 AM, Ralf Schmitt <[EMAIL PROTECTED]> wrote: > > I've tracked it down to this change: > http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd > this is svn revision 61009. > [...] > self._rbufsize if 1, and so the code reads one byte at a time The change is correct, but ex

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Guido van Rossum
Ralf, Terry is right. Please file a bug. I do think there may be a problem with that change but I don't have the time to review it in depth. Hopefully others will. I do recall that sockets reading one byte at a time has been a problem before -- I recall a bug about this in the 1.5.2 era for Window

Re: [Python-Dev] very bad network performance

2008-04-14 Thread Terry Reedy
"Ralf Schmitt" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] | Hi all, | | I'm using mercurial with the release25-maint branch. I noticed that checking | out a local repository now takes more than | 5 minutes (it should be around 30s). | | I've tracked it down to this change: | htt

[Python-Dev] very bad network performance

2008-04-14 Thread Ralf Schmitt
Hi all, I'm using mercurial with the release25-maint branch. I noticed that checking out a local repository now takes more than 5 minutes (it should be around 30s). I've tracked it down to this change: http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd this is svn revision 61009. Here is the diff