Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

> Seems to me that while all this is fine for developers and Python users
> it's completely unsatisfactory for people who just want to use Python
> applications. For them it's much easier if each application comes with
> all dependencies including the interpreter.

I think it depends on your engineering principles. If you absolutely
have to use the latest version of any software package just because it
is there, you will certainly end up with a wobbly setup where only
a single combination of libraries can cooperate at all. OTOH, if you
develop with backwards compatibility in mind, it may be more painful
to develop, but much easier to deploy. Many of the larger libraries
and applications (Twisted, Roundup, MoinMoin) support a wide range
of Python releases, and an equally wide range of underlying libraries
(in case they need any) - so it can be done.

Of course, if you have just a single installation for your application,
developing it portably across various versions could be wasted effort.
However, in that case, I found it sufficient to just make it work on
(the Debian packages of) that single installation. Every time I upgrade
to a new Debian release, I have to see what breaks in my applications.
Most of the time, very little if any effort is needed.

So I disagree that application developers homogeneously prefer
packaging everything into a single stand-alone package. For some,
it's just not an option, for others, it's unnecessary.

I do develop applications myself, and had only once in ten years
the desire to package everything in a stand-alone way, and then ended
up using freeze. I'm genuinely curious what the scenarios are where
people desire such packaging - I did hear the desire often, but never
fully explained.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

> I do develop applications myself, and had only once in ten years
> the desire to package everything in a stand-alone way, and then ended
> up using freeze. I'm genuinely curious what the scenarios are where
> people desire such packaging - I did hear the desire often, but never
> fully explained.
> 

There is one case where I understand that desire: py2exe on Windows
(which is similar to my own case, except that I wanted to support
Solaris). It's the case where you produce an application to download
by end users, and you don't want to bother end users with installing
Python first.

However, I think that py2exe also solves this case sufficiently. One
might debate whether freeze would be better, or whether it would be
better to produce MSI instead of exe, but in essence, the result
will be always the same. I don't think a cross-platform packaging
solution can help here in any way.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Non-Core project: IDLE

[Mark Summerfield]

On 2009-03-23, Guilherme Polo wrote:
> On Mon, Mar 23, 2009 at 5:39 PM, Terry Reedy  wrote:
> > Guilherme Polo wrote:
> >> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy  wrote:
> >>> IDLE needs lots of attention -- more than any one experienced person is
> >>> likely to have
> >>
> >> I'm willing to step up as a student for this but I still have to write
> >> a good proposal for it.
> >> My actual concern is about mentor availability, is someone around
> >> interested on being an IDLE mentor ?
> >
> > If I could, I would, and would have said so.  But I have only read about
> > tk and have not actually used it.  If I did decide to dive into it, you'd
> > be mentoring me ;-).  What I can and would do is give ideas for changes,
> > read and comment on a proposal, and user test patched versions.
>
> That is very nice Terry. Do you have some specific ideas that you want
> to share publicly (or in private) about IDLE ? Your expectations about
> what should be addressed first, or areas that should be improved.. you
> know, anything.

I have one suggestion that I think might be widely appreciated:

Add somewhere in the configuration dialog when users can enter a block
of Python code to be executed at startup and whenever Restart Shell is
executed.

Use case: for people who use IDLE for calculations/experiments they
might like to always have certain module imported. For me personally, it
would be:

import os
import re
import sys
from math import *

but of course the whole point is that people can write any code they
like. (Some people might want to do various from __future__ imports in
Python 2.6 to get various Python 3 features for example.)

I know that you can use the -c option, but that only works at startup,
not every time you Restart Shell.

[snip]

-- 
Mark Summerfield, Qtrac Ltd, www.qtrac.eu
C++, Python, Qt, PyQt - training and consultancy
"Rapid GUI Programming with Python and Qt" - ISBN 0132354187

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Stephen J. Turnbull]

Steve Holden writes:

 > Seems to me that while all this is fine for developers and Python
 > users it's completely unsatisfactory for people who just want to
 > use Python applications.

?  I use lots of Python applications under Mac OS X, MacPorts, Debian,
and Gentoo without looking too closely at what the cat drags in.  In
fact, in many cases I don't even know and don't care that they're
Python apps.  As long as the cat stays in her corner, I'm happy.

 > For them it's much easier if each application comes with all
 > dependencies including the interpreter.  This may seem wasteful,
 > but it removes many of the version compatibility issues that
 > otherwise bog things down.

For that version of the application.  As soon as you start thinking
about version upgrades of the application, the package distribution
problem reappears (in the sense that somebody must manage a whole set
of packages in the application's installed tree, make sure they get
removed at upgrade, deal with multiple concurrent installations so
that beta versions can be tested and compared with one's current
production installation, etc).  This is really the same as the problem
that Python currently faces and solves with the batteries-included
stdlib, just specialized to a particular application.  But those
applications are similarly likely to experience bitrot in their
bundled dependencies, and *some* but not all users will find that
somewhere between annoying and disastrous, just like the
Python-with-stdlib distribution itself.  Degree of each kind of issue
will change, of course, but the issues do remain.

As far as I can *all* of these various solutions have their uses, and
each is going to be strongly favored by some groups of users, and more
or less unacceptable to others.  These preferences may or may not be
correlated with specific apps written in Python.  "Batteries included"
is the status quo for the (C) Python project, and I personally don't
see a strong reason to change that policy until those who favor a
package-manager-based solution have a package manager that satisfies
them!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Hrvoje Niksic]

Nick Coghlan wrote:

Many of the routines in abstract.c check their parameters for NULL, as a
sanity check, and throw a SystemError if NULL is detected.  I'm tempted
to suggest making these checks only when Py_DEBUG is defined, but I
suspect if you wanted it that way, you would have changed it already. ;)

Assuming you really want the NULL checks in production Python, there are
5 checks for NULL even though there are only two parameters.  Seems like
overkill?


The main problem is that many of these methods are not only used
internally, but are *also* part of the public C API made available to
extension modules. We want misuse of the latter to trigger exceptions,
not segfault the interpreter.


Agreed, and more importantly, I have yet to be convinced that those NULL 
checks introduce a measurable slowdown.  Daniel, have you tried 
measuring the performance difference with only the NULL checks removed?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Antoine Pitrou]

Hrvoje Niksic  avl.com> writes:
> 
> Agreed, and more importantly, I have yet to be convinced that those NULL 
> checks introduce a measurable slowdown.  Daniel, have you tried 
> measuring the performance difference with only the NULL checks removed?

I think it would be fine to add a special case for lists (*) and dicts in
PyObject_GetItem(), provided it does make a significant difference in
performance for these two types.

(*) only for integers, not for slices

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Raymond Hettinger]

Agreed, and more importantly, I have yet to be convinced that those NULL 
checks introduce a measurable slowdown.  Daniel, have you tried 
measuring the performance difference with only the NULL checks removed?


I think it highly unlikely that there is a performance difference.
These tend to branch the same way every time, so the 
processor's branch prediction will tend to reduce the check time
to near zero. 




I think it would be fine to add a special case for lists (*) and dicts in
PyObject_GetItem(), provided it does make a significant difference in
performance for these two types.


-1

The API confusion and clutter isn't worth the micro-optimization.
Also, the checks probably do have some value in early detection
of programming errors; it would be ashamed to lose them in 
non-debug builds.  When we get bug reports that are due to 
problems with third-party extensions, it will be harder to know

whether the issue is with the extension or with us.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Antoine Pitrou]

Raymond Hettinger  rcn.com> writes:
> 
> -1
> 
> The API confusion and clutter isn't worth the micro-optimization.

The API wouldn't change, there would only be a short path for long-indexing of
lists, exactly as there is today in 2.x's BINARY_SUBSCR implementation.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] tracker status options

[R. David Murray]

On Tue Mar 24 02:06:29 CET 2009, Tennessee Leeuwenburg  wrote:

Perhaps some developers have a well-established workflow and interpret these
flag in a particular, consistent fashion, however part of the purpose of the
issue tracker is to allow a diverse group to work on development as a group.
On that basis, I feel that more documentation, and clearer terminology, is
required in order to support the bug resolution process.

I have identified some gaps in the workflow process which impact me
personally in classifying issues. These issues will not impact on all
developers; clearly they will be entirely superfluous, perhaps even
confusing, for some individuals. However the impact still remains for some.
There appears to be a general agreement that ability to distinguish between
issues on the following bases would be useful:
   * Whether the nature of the requirement is still under debate or whether
it is well-understood


So, having triaged a few issues, here are my thoughts.

The current workflow is roughly:

o test needed
o patch needed
o patch review
o commit review

One can look at these and see what needs to be done "next".  I think
that in practice the above list actually expands something like this:

o consensus needed
o test needed
o patch needed
o patch needs work
o patch review
o commit review

The first of these additional items is equivalent to your bullet item
above.  I would propose that the issue, regardless of whether or not
it is a bug fix or a feature request, goes into 'consensus needed'
whenever there is significant debate as to the validity of the bug or
feature request.  It moves to an appropriate later stage when agreement
has been reached (which may be by fiat of a developer or the BDFLso
the triage people would need to know who can "pronounce").

The second addition is not as clearly useful. 'patch needs work' could
be the result of a developer review, or of any other review.  That is, at
that stage we are trying to reach consensus that the patch is the correct
solution to the problem or feature request.  An issue could bounce back
and forth between 'patch review' and 'patch needs work' multiple times
(and it would probably be best if the patch submitter could request
'patch review').  But one could argue that the issue could just as easily
be bounced back and forth between 'patch review' and 'patch needed',
since one would need to read the comment stream to figure out what was
actually going on anyway.  I think it would be a useful addition, since
it would enable someone to search for 'patch needed' in order to look
for issues to pick up, whereas 'patch needs work' would indicate someone
was working on it.  (Of course, that someone could also stop working
on it...but if one wished to find such issues, one could simply sort
'patch needs work' issues by last activity date.)  Hmm.  Or perhaps it
should be "patch needs consensus", given the issue I'm looking at where
I want to set it to this stage :)


   * Whether the issue is "up for grabs" by any developer or whether
responsibility lies with an individual or group already


Isn't that covered by 'assigned to'?


   * Whether the issue is ready for more serious consideration by more
experienced or core developers


Hmm.  Theoretically that's covered by 'patch review'.  Given that
'commit review' is (currently?) reserved for issues being considered
for addition to a release candidate or final release, perhaps we need
an additional stage 'core review' that would come after 'patch review'.
Then triage could promote issues from 'patch review' to 'core review'
if triage thinks it is ready for review by someone with commit privileges.

This of course assumes that people other than core developers are actually
doing patch review.  I certainly intend to, but how many of us are there
really going to be?

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Steve Holden]

Martin v. Löwis wrote:
>> I do develop applications myself, and had only once in ten years
>> the desire to package everything in a stand-alone way, and then ended
>> up using freeze. I'm genuinely curious what the scenarios are where
>> people desire such packaging - I did hear the desire often, but never
>> fully explained.
>>
> 
> There is one case where I understand that desire: py2exe on Windows
> (which is similar to my own case, except that I wanted to support
> Solaris). It's the case where you produce an application to download
> by end users, and you don't want to bother end users with installing
> Python first.
> 
> However, I think that py2exe also solves this case sufficiently. One
> might debate whether freeze would be better, or whether it would be
> better to produce MSI instead of exe, but in essence, the result
> will be always the same. I don't think a cross-platform packaging
> solution can help here in any way.
> 
I'm not convinced we do need a cross-platform packaging solution, so I
may have explained my views badly. I regard application developers as
Python users, so I did not intend to suggest that the requirement for
stand-alone installation came from them.

My main concern is that if Linux and Unix (Lunix) application
installation results, as is the case with setuptools, in the download
and/or installation of arbitrary support packages then we may end up
condemning Python app users to our own version of DLL hell (package
purgatory?).

As far as Lunix distributions are concerned I'd be happy enough to rely
on the distributed Python, and let people who replace it suffer the
consequences and maintain their own Python packages as required. So I
suppose that we *could* take the interpreter as a given.

But I don't think that a "normal" Lunix user should ever have to involve
themselves directly with distutils or setuptools (it's OK to invoke them
from the installation routine, but not to require the user to unpack the
application then run setup.py). I am afraid that distutils, and
setuptools, are not really the answer to the problem, since while they
may (as intended) guarantee that Python applications can be installed
uniformly across different platforms they also more or less guarantee
that Python applications are installed differently from all other
applications on the platform.

Mike Driscoll did some work providing Windows installers for various
Python packages and extension modules, and people were amused that he
provided executable installers for pure Python libraries. But I saw that
as a sensible decision, since it meant that Windows users (and even
Windows Python application developers) used the same mechanism to
install everything.

I don't claim to have all the answers, but distutils and setuptools are
for the savvy users - people who build distros and those who maintain
their own systems independent of their distribution's packaging
standard. As time goes by, however, and the Lunix installed base
continues to grow, the majority of users will expect to rely on the
standard installation mechanisms of their distribution, and that will
never be setuptools or distutils.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Daniel Stutzbach]

On Tue, Mar 24, 2009 at 4:30 AM, Hrvoje Niksic wrote:

> Agreed, and more importantly, I have yet to be convinced that those NULL
> checks introduce a measurable slowdown.  Daniel, have you tried measuring
> the performance difference with only the NULL checks removed?


I'll play around with different permutations and report back on their
impact.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[R. David Murray]

On Tue, 24 Mar 2009 at 07:53, Steve Holden wrote:

I don't claim to have all the answers, but distutils and setuptools are
for the savvy users - people who build distros and those who maintain
their own systems independent of their distribution's packaging
standard. As time goes by, however, and the Lunix installed base
continues to grow, the majority of users will expect to rely on the
standard installation mechanisms of their distribution, and that will
never be setuptools or distutils.


+1.

Or any other Python-only package manager.

And that goes at least double for Windows :)

I am (among other things) a software developer, and I am quite capable of
maintaining my own python installation outside the system one, and do so
for testing and development.  But for anything else, I rely on the system
package manager to install my Python packages.  If the extension hasn't
been packaged for my distribution, I typically will either not use it,
or I will install it somewhere other than the system default location.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Test failures under Windows?

[Antoine Pitrou]

Hello,

I've just tried running test_io under Windows (in a virtual machine) and I get
the following failures:

==
ERROR: test_error_through_destructor (test.test_io.CTextIOWrapperTest)
--
Traceback (most recent call last):
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1543, in test_error_through_d
estructor
self.assertRaises(AttributeError, f)
  File "Z:\py3k\__svn__\lib\unittest.py", line 457, in failUnlessRaises
callableObj(*args, **kwargs)
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1541, in f
self.TextIOWrapper(rawio).xyzzy
OSError: [Errno 9] Bad file descriptor

==
ERROR: test_error_through_destructor (test.test_io.PyTextIOWrapperTest)
--
Traceback (most recent call last):
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1543, in test_error_through_d
estructor
self.assertRaises(AttributeError, f)
  File "Z:\py3k\__svn__\lib\unittest.py", line 457, in failUnlessRaises
callableObj(*args, **kwargs)
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1541, in f
self.TextIOWrapper(rawio).xyzzy
  File "Z:\py3k\__svn__\lib\_pyio.py", line 1356, in __init__
encoding = os.device_encoding(buffer.fileno())
OSError: [Errno 9] Bad file descriptor

--

Am I the only one getting those?
By the way, what happened to the Windows buildbots?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[David Cournapeau]

On Tue, Mar 24, 2009 at 8:53 PM, Steve Holden  wrote:

> I'm not convinced we do need a cross-platform packaging solution, so I
> may have explained my views badly. I regard application developers as
> Python users, so I did not intend to suggest that the requirement for
> stand-alone installation came from them.
>
> My main concern is that if Linux and Unix (Lunix) application
> installation results, as is the case with setuptools, in the download
> and/or installation of arbitrary support packages then we may end up
> condemning Python app users to our own version of DLL hell (package
> purgatory?).

There already is a bit of a DLL hell in python. The whole idea to
solve the dependency problems by installing multiple version of the
same software is fundamentally flawed, it just does not work for
general deployment on multiple machines. Many systems outside python,
with more resource, have tried - and failed. By enabling a general,
system-wide installation of multiple version of the same package,
setuptools has made the situation worse. I am quite puzzled than many
people don't realize this fundamental issue, it is a simple
combinatory problem.

If the problem is to get a recent enough version of the library, then
the library would better be installed "locally", for the application.
If it is too much a problem because the application depends on
billions of libraries which are 6 months old, the problem is to allow
such a dependency in the first place. What kind of nightmare would it
be if programs developed in C would required a C library which is 6
months old ? That's exactly what multiple-versions installations
inflict on us. That's great for testing, development. But for
deployment on end-user machines, the whole thing is a failure IMO.

> I am afraid that distutils, and
> setuptools, are not really the answer to the problem, since while they
> may (as intended) guarantee that Python applications can be installed
> uniformly across different platforms they also more or less guarantee
> that Python applications are installed differently from all other
> applications on the platform.

I think they should be part of the solution, in the sense that they
should allow easier packaging for the different platforms (linux,
windows, mac os x and so on). For now, they make things much harder
than they should (difficult to follow the FHS, etc...). But otherwise,
I agree. Python applications which care about non-savy users should be
distributed as .dmg, .exe, .rpm, .deb. There will always be some work
to do that correctly: there is no way to provide a general, automatic
environment to build installers which provide a good experience on all
platforms. AFAIK, It does not even exist in the commercial landscape,
so I see little chance to see this in the python ecosystem.

cheers,

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Core Python development tools

[Daniel (ajax) Diniz]

Nick Coghlan wrote:
> Everything I've seen from Daniel so far seems to be about either making
> things we already do more efficient, or else providing additional
> features in ways that don't make the tools any more confusing for others
> already used to a particular way of doing things. So he seems to be
> navigating this particular minefield pretty well so far.

Thanks a lot Nick! :)

BTW, it seems there's a procedure to follow if my navigation fails[1].

> Then again, I may be a little biased - some of the recent bug tracker
> changes are things I had wished for in the past, but never chipped in
> any code to help make them happen :)

Not a problem, sir, we accept RFEs devoid of any code bits in this store :)

Cheers,
Daniel

[1]
George: If we do happen to step on a mine, Sir, what do we do?

Capt. Blackadder: Normal procedure, Lieutenant, is to jump two hundred
feet in the air and scatter oneself over a wide area.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Jean-Paul Calderone]

On Tue, 24 Mar 2009 13:49:28 + (UTC), Antoine Pitrou  
wrote:

Hello,

[snip]
By the way, what happened to the Windows buildbots?


It looks like some of them are suffering from problems which I think are
common with buildbot on Windows - primarily difficulty dealing with runaway
processes or timeouts.  Perhaps BuildBot/Windows improvements would make a
good GSoC project? :)

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Core Python development tools

[Daniel (ajax) Diniz]

"Martin v. Löwis" wrote:
> Yes, I'm also quite grateful for the contributions I have received from
> Daniel.

Thank you Martin. I'm sure I'd still be going around in circles if it
weren't for your guidance, and I'd be MIA after the first time I broke
the tracker too. So thanks a lot for the support.

> I hope he'll stay around for some time without exhausting.
Me too. I'm trying to leave a big audit trail so other people can join
efforts or take over parts of roadmap easily, but that's also a backup
for long fieldwork trips or the eventual burn out.

Best regards,
Daniel
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Toshio Kuratomi]

Stephen J. Turnbull wrote:
> Chris Withers writes:
>
>  > - debian has an outdated and/or broken version of your package.
>
> True, but just as for the package system you are advocating, it's
> quite easy to set up your apt to use third-party repositories of
> Debian-style packages.  The question is whether those repositories
> exist.  Introducing yet another, domain-specific package manager will
> make it less likely that they do, and it will cause more work for
> downstream distributors like Debian and RH.
>
I haven't seen this mentioned so --

For many sites (including Fedora, the one I work on), the site maintains
a local yum/apt repository of packages that are necessary for getting
certain applications to run.  This way we are able to install a system
with a distribution that is maintained by other people and have local
additions that add more recent versions only where necessary.  This has
the following advantages:

1) We're able to track our changes to the base OS.
2) If the OS vendor releases an update that includes our fixes, we're
able to consume it without figuring out on which boxes we have to delete
what type of locally installed file (egg, jar, gem,
/usr/local/bin/program, etc).
3) We're using the OS vendor package management system for everything so
junior system admins can bootstrap a new machine with only familiarity
with that OS.  We don't have to teach them about rpm + eggs + gems +
where to find our custom repositories of each.
4) If we choose to, we can separate out different repositories for
different sets of machines.  Currently we have the main local repo and
one repo that only the builders pull from.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Daniel Stutzbach]

On Tue, Mar 24, 2009 at 7:30 AM, Daniel Stutzbach <
dan...@stutzbachenterprises.com> wrote:

> On Tue, Mar 24, 2009 at 4:30 AM, Hrvoje Niksic wrote:
>
>> Agreed, and more importantly, I have yet to be convinced that those NULL
>> checks introduce a measurable slowdown.  Daniel, have you tried measuring
>> the performance difference with only the NULL checks removed?
>
>
> I'll play around with different permutations and report back on their
> impact.


After playing around with it, I see that I was barking up the wrong tree.
That's what I get for tracing through code visually instead of using a
debugger.

PyList implements tp_as_mapping, so it's mp_subscript method is called and
the PySequence line is never used.  I tried many variations of special
casing at various points in the path and removing extra checks.

It looks like function calls are the biggest expense.  Calls to functions
within the same module appear to be cheap (presumably because gcc inlines
them).

100 nanoseconds, py3k trunk:
ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)

86 nanoseconds, by special-casing PyLong in list_subscript
ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
PyLong_AsSsize_t (longobject.c)
cost: could slow down
mylist[some_PyNumber_that_is_not_a_long_yet_still_a_valid_index] (who
cares?)

75 nanoseconds, by special-casing PyList and PyLong in PyObject and using
PyList_GET_ITEM
ceval -> PyObject_GetItem (object.c) -> PyLong_AsSsize_t (longobject.c)
cost: could slow down not_a_list[x]

75 nanoseconds, by special-casing PyList and PyLong in ceval and using
PyList_GET_ITEM
ceval -> PyLong_AsSsize_t (longobject.c)
cost: could slow down not_a_list[x]



I had been hoping to find something general to speed up all uses of
__getitem__, but that doesn't seem possible.  Oh well. :-(

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Toshio Kuratomi]

Steve Holden wrote:

> Seems to me that while all this is fine for developers and Python users
> it's completely unsatisfactory for people who just want to use Python
> applications. For them it's much easier if each application comes with
> all dependencies including the interpreter.
> 
> This may seem wasteful, but it removes many of the version compatibility
> issues that otherwise bog things down.
> 
The upfront cost of bundling is lower but the maintenance cost is
higher.  For instance, OS vendors have developed many ways of being
notified of and dealing with security issues.  If there's a security
issue with gtkmozdev and the python bindings to it have to be
recompiled, OS vendors will be alerted to it and have the opportunity to
release updates on zero day, the day that the security announcement goes
out.

Bundled applications suffer in two ways here:
1) the developers of the applications are unlikely to be on vendor-sec
and so the opportunity for zero day fixes is lower.

2) the developer becomes responsible for fixing problems with the
libraries, something that they often do not.  This is especially true
when developers start depending, not only on newer features of some
libraries, but older versions of others (for API changes).  It's not
clear to many developers that requiring a newer version of a library is
at least supported by upstream whereas requiring an older version leaves
them as the sole responsible party.

3) Over time, bundled libraries tend to become forked versions.  And
worse, privately forked versions.  If three python apps all use slightly
different older versions of libfoo-python and have backported fixes,
added new features, etc it is a nightmare for a system administrator or
packager to get them running with a single version from the system
library or forward port them.  And because they're private forks the
developers lose out on collaborating on security, bugfixes, etc because
they are doing their work in isolation from the other forks.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Core Python development tools

[Daniel (ajax) Diniz]

Antoine Pitrou wrote:
> Well, it seems to me that most of these steps are separated by manual
> intervention (e.g. compile and run the test suite to check that everything 
> works
> smoothly)

Agreed.

> so there's no real point in making a script out of them.

The idea would be to provide scripts for each step that people think
should be easier/automated. Like running the provided testcase before
and after applying the corresponding patch. Or making sure to build
before running tests for a patch that touches C code.

> The real issues with svnmerge are its occasional bugs or failures (it forgot
> some changesets when merging in the io-c branch!)

Who's handling those bugs? Is any of them avoidable/easier to
catch/easier to workaround with a script that checks for, e.g.,
svnmerge version, svnmerge correct usage and completeness of the
merge? I think this proposal could help reproducing those issues and
fixing/mitigating them until a better solution is available.

> its slowness,

Maybe something could be improved in svnmerge, as Martin suggests. Or
merging could be made a bit easier, so people would backport their
commits themselves more often and the laundry list merges could be a
little smaller.

> and its
> limitations (which are really inherent to the SVN model: e.g., if someone
> commits to the branch you have just started doing an svnmerge to, you have to
> revert everything and start over with the latest updates).

We have discussed this off-list and I have no answer regarding SVN limitations.

If a svnmerge-on-top-of-hg or svnmerge-on-top-of-bzr script can be
used to merge SVN branches with the exact same results of the real
svnmerge, only faster and prettier, maybe it should be considered for
GSoC too.

If/when the main Python repository moves to a DVCS, scripts that allow
people to perform the same steps they currently perform to achieve the
same results might help the transition.

I won't mention the bzr-on-top-of-git part :)

Cheers,
Daniel
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Mark Dickinson]

2009/3/24 Daniel Stutzbach :
> [...]
> 100 nanoseconds, py3k trunk:
> ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
> PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)
> [more timings snipped]

Does removing the PyLong_Check call in PyLong_AsSsize_t
make any noticeable difference to these timings?

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[R. David Murray]

On Tue, 24 Mar 2009 at 07:59, Toshio Kuratomi wrote:

3) Over time, bundled libraries tend to become forked versions.  And
worse, privately forked versions.  If three python apps all use slightly
different older versions of libfoo-python and have backported fixes,
added new features, etc it is a nightmare for a system administrator or
packager to get them running with a single version from the system
library or forward port them.  And because they're private forks the
developers lose out on collaborating on security, bugfixes, etc because
they are doing their work in isolation from the other forks.


This is one of the things that I really disliked about Java when I
had to work with it:  the culture there is that you ship a zip bundle
that has all the libraries in it, at some unknown version or another.
I fortunately did not run into any of those security or version-drift/bug
issues in the relatively short time I worked with it, but I could feel
those issues looming in the background and it made my skin crawl :(

I'm very happy that Gentoo keeps the libraries separate when it packages
Java applications.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[David Cournapeau]

2009/3/24 Toshio Kuratomi :
> Steve Holden wrote:
>
>> Seems to me that while all this is fine for developers and Python users
>> it's completely unsatisfactory for people who just want to use Python
>> applications. For them it's much easier if each application comes with
>> all dependencies including the interpreter.
>>
>> This may seem wasteful, but it removes many of the version compatibility
>> issues that otherwise bog things down.
>>
> The upfront cost of bundling is lower but the maintenance cost is
> higher.  For instance, OS vendors have developed many ways of being
> notified of and dealing with security issues.  If there's a security
> issue with gtkmozdev and the python bindings to it have to be
> recompiled, OS vendors will be alerted to it and have the opportunity to
> release updates on zero day, the day that the security announcement goes
> out.

I don't think bundling should be compared to depending on the system
libraries, but as a lesser evil compared to requiring multiple,
system-wide installed libraries.

>
> 3) Over time, bundled libraries tend to become forked versions.  And
> worse, privately forked versions.  If three python apps all use slightly
> different older versions of libfoo-python and have backported fixes,
> added new features, etc it is a nightmare for a system administrator or
> packager to get them running with a single version from the system
> library or forward port them.  And because they're private forks the
> developers lose out on collaborating on security, bugfixes, etc because
> they are doing their work in isolation from the other forks.

This is a purely technical problem, and can be handled by good source
control systems, no ?

cheers,

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] About adding a new iterator method called "shuffled"

[Roy Hyunjin Han]

I know that Python has iterator methods called "sorted" and "reversed" and
these are handy shortcuts.


Why not add a new iterator method called "shuffled"?

>>> for x in shuffled(range(5)):
>>>print x
>>> 3
>>> 1
>>> 2
>>> 0
>>> 4


Currently, a person has to do the following because random.shuffle() does
not return the actual shuffled list.  It is verbose.

>>> import random
>>> x = range(5)
>>> random.shuffle(x)
>>> for x in x:
>>> print x
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Daniel Stutzbach]

On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson  wrote:

> 2009/3/24 Daniel Stutzbach :
> > [...]
> > 100 nanoseconds, py3k trunk:
> > ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
> > PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)
> > [more timings snipped]
>
> Does removing the PyLong_Check call in PyLong_AsSsize_t
> make any noticeable difference to these timings?
>

Making no other changes from the trunk, removing the PyLong_Check and NULL
check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
trunk is around 100 nanoseconds).

Here's what I'm testing with, by the way:

./python.exe Lib/timeit.py -r 10 -s 'x = list(range(10))' 'x[5]'

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] About adding a new iterator method called "shuffled"

[Aahz]

On Tue, Mar 24, 2009, Roy Hyunjin Han wrote:
>
> I know that Python has iterator methods called "sorted" and "reversed" and
> these are handy shortcuts.
> 
> Why not add a new iterator method called "shuffled"?

Please do not post ideas like this to python-dev, please use python-ideas
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong."  --GvR, python-ideas, 2009-3-1
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Toshio Kuratomi]

David Cournapeau wrote:
> 2009/3/24 Toshio Kuratomi :
>> Steve Holden wrote:
>>
>>> Seems to me that while all this is fine for developers and Python users
>>> it's completely unsatisfactory for people who just want to use Python
>>> applications. For them it's much easier if each application comes with
>>> all dependencies including the interpreter.
>>>
>>> This may seem wasteful, but it removes many of the version compatibility
>>> issues that otherwise bog things down.
>>>
>> The upfront cost of bundling is lower but the maintenance cost is
>> higher.  For instance, OS vendors have developed many ways of being
>> notified of and dealing with security issues.  If there's a security
>> issue with gtkmozdev and the python bindings to it have to be
>> recompiled, OS vendors will be alerted to it and have the opportunity to
>> release updates on zero day, the day that the security announcement goes
>> out.
> 
> I don't think bundling should be compared to depending on the system
> libraries, but as a lesser evil compared to requiring multiple,
> system-wide installed libraries.
> 
Well.. I'm not so sure it's even a win there.  If the libraries are
installed system-wide, at least the consumer of the application knows:

1) Where to find all the libraries to audit the versions when a security
issue is announced.
2) That the library is unforked from upstream.
3) That all the consumers of the library version have a central location
to collaborate on announcing fixes to the library.

With my distribution packager hat on, I can say I dislike both multiple
versions and bundling but I definitely dislike bundling more.

>> 3) Over time, bundled libraries tend to become forked versions.  And
>> worse, privately forked versions.  If three python apps all use slightly
>> different older versions of libfoo-python and have backported fixes,
>> added new features, etc it is a nightmare for a system administrator or
>> packager to get them running with a single version from the system
>> library or forward port them.  And because they're private forks the
>> developers lose out on collaborating on security, bugfixes, etc because
>> they are doing their work in isolation from the other forks.
> 
> This is a purely technical problem, and can be handled by good source
> control systems, no ?
> 
No.  This is a social problem.  Good source control only helps if I am
tracking upstream's trunk so I'm aware of the direction that their
changes are headed.  But there's a wide range of reasons that
application developers that bundle libraries don't do that:

1) not enough time in a day.  I'm working full-time on making my
application better.  Plus I have to update all these bundled libraries
from time to time, testing that the updates don't break anything.  I
don't have time to track trunk for all these libraries -- I barely have
time to track releases.

2) My release schedule doesn't mesh with all of the upstream libraries
I'm bundling.  When I want to release Foo-1.0, I want to have some
assurance that the libraries I'm bundling with will do the right thing.
   Since releases see more testing than trunk, tracking trunk for twenty
bundled libraries is a lot less attractive than tracking release branches.

3) This doesn't help with the fact that my bundled version of the
library and your bundled version of the library are being developed in
isolation from each other.  This needs central coordination which people
who believe bundling libraries are very unlikely to pursue.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

R. David Murray wrote:
> On Tue, 24 Mar 2009 at 07:53, Steve Holden wrote:
>> I don't claim to have all the answers, but distutils and setuptools are
>> for the savvy users - people who build distros and those who maintain
>> their own systems independent of their distribution's packaging
>> standard. As time goes by, however, and the Lunix installed base
>> continues to grow, the majority of users will expect to rely on the
>> standard installation mechanisms of their distribution, and that will
>> never be setuptools or distutils.

I'll note that such users are mostly going to be installing desktop
applications, which make a good case for the "batteries included" story,
but one which ignores the "system" Python altogether, and bundles the
interpreter.  I think that building such a self-contained bundle would
be eased by a strong story for managing out-of-core distributions.

> +1.
> 
> Or any other Python-only package manager.
> 
> And that goes at least double for Windows :)

I just did an install for a customer yesterday (actually, the customer
did it, with me as "air traffic controller"), which started from the
current Plone Windows installer, and then used the zc.buildout machinery
shipped with Plone to add on the extra distributions (seven or so
"community-maintained", plus one containing custom code).  The buildout
was driven from a text-based config file I supplied to the customer,
which I developed and tested first on Linux.

This method was *way* less painful and error-prone than downloading and
running eight separate installers, *and* it ensured repeatability,
because the extra distributions were all installed from a PyPI-style
index I set up for the customer:  I can recreate the environment at will
(well, all the bits on disk under the main installation directory).

If I need to fix one of the distributions which make up the application,
ar add a new one, I will roll a new 'sdist' for it (no C extensions in
play), upload it to the index, rebuild the metadata, and tell the
customer to re-run the 'buildout.exe'.

> I am (among other things) a software developer, and I am quite capable of
> maintaining my own python installation outside the system one, and do so
> for testing and development.  But for anything else, I rely on the system
> package manager to install my Python packages.  If the extension hasn't
> been packaged for my distribution, I typically will either not use it,
> or I will install it somewhere other than the system default location.

At which point you can benefit from the distributions-with-dependencies-
in-an-index format provided, with varying levels of success, by
setuptools today.

I think the confusion-in-terms here may be that many Python developers
don't use a large number of out-of-core distributions, and thus are
mostly OK with managing the few they do use as part of their
application:  the rest they get from a "batteries include" Python.
Mostly that is a reasonable choice, although I note that the Python 2.5
releases before 2.5.2 broke applicatinos using the tarfile module, and
the 2.6 release broke applications using asyncore / asynchat:  if those
packages were shipped outside the stdlib, I could have fixed the
breakage by updating them separately.

Those of us who do need to deploy applications with larger dependency
sets need tools which make it possible to do the deployments repeatably.
 The "use .deb / .rpm / Windows installer" answer just moves the effort
around (and adds to it, in my experience), because in that world I end
up running the equivalent of a PyPI index anyway:  I can't rely on an
upstream distro to package the correct versions for me, or to avoid
breaking my application by the packaging choices they make.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyRPu+gerLs4ltQ4RAjgPAKDB12YNxmAJX7VFN9WGvMR3H8KPHgCgt0NN
vCKUCTkVcxjP09/LQVBP8G4=
=2qbi
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI

[INADA Naoki]

Hi. I'm Japanese and non-ascii charactor user. (cp932)

We have to use "IME" to input non-ascii charactor in Windows.
When "> chcp 65001" in cmd.exe, we cannot use IME on cmd.exe.

So setting codepage to 65001 make output universal but make input ascii-only.
Sit!!!

I hope PyQtShell  become good
IDLE alternative.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

David Cournapeau wrote:

> There already is a bit of a DLL hell in python. The whole idea to
> solve the dependency problems by installing multiple version of the
> same software is fundamentally flawed, it just does not work for
> general deployment on multiple machines. Many systems outside python,
> with more resource, have tried - and failed. By enabling a general,
> system-wide installation of multiple version of the same package,
> setuptools has made the situation worse. I am quite puzzled than many
> people don't realize this fundamental issue, it is a simple
> combinatory problem.

Many of us using setuptools extensively tend to adopt an "isolated
environment" strategy (e.g., pip, virtualenv, zc.buildout).  We don't
install the packages used by different applications into shared
directories at all.  Instead, each environment uses a restricted subset
of packages known to work together.

> If the problem is to get a recent enough version of the library, then
> the library would better be installed "locally", for the application.
> If it is too much a problem because the application depends on
> billions of libraries which are 6 months old, the problem is to allow
> such a dependency in the first place. What kind of nightmare would it
> be if programs developed in C would required a C library which is 6
> months old ? That's exactly what multiple-versions installations
> inflict on us. That's great for testing, development. But for
> deployment on end-user machines, the whole thing is a failure IMO.

It is wildly successful, even on platforms such as Windows, when you
abandon the notion that separate applications should be sharing the
libaries they need.

>> I am afraid that distutils, and
>> setuptools, are not really the answer to the problem, since while they
>> may (as intended) guarantee that Python applications can be installed
>> uniformly across different platforms they also more or less guarantee
>> that Python applications are installed differently from all other
>> applications on the platform.
> 
> I think they should be part of the solution, in the sense that they
> should allow easier packaging for the different platforms (linux,
> windows, mac os x and so on). For now, they make things much harder
> than they should (difficult to follow the FHS, etc...).

FHS is something which packagers / distributors care about:  I strongly
doubt that the "end users" will ever notice, particularly for silliness
like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.

> But otherwise,
> I agree. Python applications which care about non-savy users should be
> distributed as .dmg, .exe, .rpm, .deb. There will always be some work
> to do that correctly: there is no way to provide a general, automatic
> environment to build installers which provide a good experience on all
> platforms. AFAIK, It does not even exist in the commercial landscape,
> so I see little chance to see this in the python ecosystem.

As a counter-example, I offer the current Plone installer[1], which lays
down a bunch of egg-based packages in a cross-platform way (Windows,
MacOSX, Linux, BSDs).  It uses zc.buildout, which makes
configuration-driven (repeatable) installation of add-ons easy.

[1] http://plone.org/products/plone


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyRZH+gerLs4ltQ4RAu3qAKDXhkY73S2htl6Ytz+ZPVyRgYSHLACdGo+N
XNMrpg7ue0SX3ODgiOjHTmg=
=Rk1D
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[David Cournapeau]

On Wed, Mar 25, 2009 at 1:45 AM, Toshio Kuratomi  wrote:
> David Cournapeau wrote:
>> 2009/3/24 Toshio Kuratomi :
>>> Steve Holden wrote:
>>>
 Seems to me that while all this is fine for developers and Python users
 it's completely unsatisfactory for people who just want to use Python
 applications. For them it's much easier if each application comes with
 all dependencies including the interpreter.

 This may seem wasteful, but it removes many of the version compatibility
 issues that otherwise bog things down.

>>> The upfront cost of bundling is lower but the maintenance cost is
>>> higher.  For instance, OS vendors have developed many ways of being
>>> notified of and dealing with security issues.  If there's a security
>>> issue with gtkmozdev and the python bindings to it have to be
>>> recompiled, OS vendors will be alerted to it and have the opportunity to
>>> release updates on zero day, the day that the security announcement goes
>>> out.
>>
>> I don't think bundling should be compared to depending on the system
>> libraries, but as a lesser evil compared to requiring multiple,
>> system-wide installed libraries.
>>
> Well.. I'm not so sure it's even a win there.  If the libraries are
> installed system-wide, at least the consumer of the application knows:
>
> 1) Where to find all the libraries to audit the versions when a security
> issue is announced.
> 2) That the library is unforked from upstream.
> 3) That all the consumers of the library version have a central location
> to collaborate on announcing fixes to the library.

Yes, those are problems, but installing multi libraries have a lot of
problems too:
 - quickly, by enabling multiple version installed, people become very
sloppy to handle versions of the dependencies, and this increases a
lot the number of libraries installed - so the advantages above for
system-wide installation  becomes intractable quite quickly
 - bundling also supports a real user-case which cannot be solved by
rpm/deb AFAIK: installation without administration privileges.
 - multi-version installation give very fragile systems. That's
actually my number one complain in python: setuptools has caused me
numerous headache, and I got many bug reports because you often do not
know why one version was loaded instead of another one.

So I am not so convinced multiple-version is better than bundling - I
can see how it sometimes can be, but I am not sure those are that
important in practice.

> No.  This is a social problem.  Good source control only helps if I am
> tracking upstream's trunk so I'm aware of the direction that their
> changes are headed.  But there's a wide range of reasons that
> application developers that bundle libraries don't do that:
>
> 1) not enough time in a day.  I'm working full-time on making my
> application better.  Plus I have to update all these bundled libraries
> from time to time, testing that the updates don't break anything.  I
> don't have time to track trunk for all these libraries -- I barely have
> time to track releases.

Yes, but in that case, there is nothing you can do. Putting everything
in one project is always easier than splitting into modules, coding
and deployment-wise. That's just one side of the speed of development
vs maintenance issue IMHO.

> 3) This doesn't help with the fact that my bundled version of the
> library and your bundled version of the library are being developed in
> isolation from each other.  This needs central coordination which people
> who believe bundling libraries are very unlikely to pursue.

As above, I think that in that case, it will happen whatever tools are
available, so it is not a case worth being pursued.

cheers,

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[David Cournapeau]

On Wed, Mar 25, 2009 at 2:20 AM, Tres Seaver  wrote:

>
> Many of us using setuptools extensively tend to adopt an "isolated
> environment" strategy (e.g., pip, virtualenv, zc.buildout).  We don't
> install the packages used by different applications into shared
> directories at all.  Instead, each environment uses a restricted subset
> of packages known to work together.

Is that a working solution when you want to enable easy installation
on a large number of "customers" ? In those discussions, I often see
different solutions depending on the kind of projects people do. I
don't know anything about plone, but I can imagine the deployment
issues are quite different from the projects I am involved in (numpy
and co).

Everytime I tried to understand what buildout was about, I was not
even sure it could help for my own problems at all. It seems very
specific to web development - I may completely miss the point ?
virtualenv, pip, yolk, those are useful tools for development/testing,
but I don't see how they can help me to make the installation of a
numpy environment easier on many different kind of platforms.

>
>> If the problem is to get a recent enough version of the library, then
>> the library would better be installed "locally", for the application.
>> If it is too much a problem because the application depends on
>> billions of libraries which are 6 months old, the problem is to allow
>> such a dependency in the first place. What kind of nightmare would it
>> be if programs developed in C would required a C library which is 6
>> months old ? That's exactly what multiple-versions installations
>> inflict on us. That's great for testing, development. But for
>> deployment on end-user machines, the whole thing is a failure IMO.
>
> It is wildly successful, even on platforms such as Windows, when you
> abandon the notion that separate applications should be sharing the
> libaries they need.

Well, I may not have been clear: I meant that in my experience,
deploying something with several dependencies was easier with bundling
than with a mechanism ala setuptools with *system-wide* installation
of multiple versions of the same library. So I think we agree here:
depending on something stable (python stdlib + a few well known
things) system-wide is OK, for anything else, not sharing is easier
and more robust in the current state of things, at least when one
needs to stay cross platform.

Almost every deployment problem I got from people using my own
softwares was related to setuptools, and in particular the multiple
version thing. For end-users who know nothing about python package
mechanism, and do not care about it, that's really a PITA to debug,
and give bad mouth taste. The fact that those problems happen when my
software was not even *using* setuptools/etc... was a real deal
breaker for me, and I am strongly biased against setuptools ever
since.

>
> FHS is something which packagers / distributors care about:  I strongly
> doubt that the "end users" will ever notice, particularly for silliness
> like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.

That's not silly, and that's a bit of a fallacy. Of course end users
do not care about the FHS in itself, but following the FHS enables
good integration in the system, which end users do care about. I like
finding my doc in /usr/share/doc whatever thing I install - as I am
sure every window user like to find his installed software in the
panel control.

>
> As a counter-example, I offer the current Plone installer[1], which lays
> down a bunch of egg-based packages in a cross-platform way (Windows,
> MacOSX, Linux, BSDs).  It uses zc.buildout, which makes
> configuration-driven (repeatable) installation of add-ons easy.

But zc.buildout is not a solution to deploy applications, right ? In
my understanding, it is a tool to deploy plone instances on
server/test machines, but that's quite a different problem from
installing "applications" for users who may not even know what python
is.

cheers,

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Mark Dickinson]

On Tue, Mar 24, 2009 at 3:50 PM, Daniel Stutzbach
 wrote:
> On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson  wrote:
>> Does removing the PyLong_Check call in PyLong_AsSsize_t
>> make any noticeable difference to these timings?
>
> Making no other changes from the trunk, removing the PyLong_Check and NULL
> check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
> trunk is around 100 nanoseconds).

Thanks.  I'd call that a noticeable difference.  I'd be +1 on changing
this particular check to an assert and so disabling it in non-debug builds.
I'd like to bet that the majority of calls to PyLong_AsSsize_t are
internal.

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[David Bolen]

Antoine Pitrou  writes:

> Am I the only one getting those?
> By the way, what happened to the Windows buildbots?

Oops - sorry, mine (XP-4) apparently got stuck with C++ debug
assertion dialogs (an assertion from the internal close.c module) from
python_d during some earlier build.  I try to catch these but have been
a bit preoccupied this month.  Should be back now.

-- David

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Antoine Pitrou]

Mark Dickinson  gmail.com> writes:
> 
> > Making no other changes from the trunk, removing the PyLong_Check and NULL
> > check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
> > trunk is around 100 nanoseconds).
> 
> Thanks.  I'd call that a noticeable difference.

4% on a micro-micro-benchmark is hardly compelling...

cheers

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

David Cournapeau wrote:
> On Wed, Mar 25, 2009 at 2:20 AM, Tres Seaver  wrote:
> 
>> Many of us using setuptools extensively tend to adopt an "isolated
>> environment" strategy (e.g., pip, virtualenv, zc.buildout).  We don't
>> install the packages used by different applications into shared
>> directories at all.  Instead, each environment uses a restricted subset
>> of packages known to work together.
> 
> Is that a working solution when you want to enable easy installation
> on a large number of "customers" ? In those discussions, I often see
> different solutions depending on the kind of projects people do. I
> don't know anything about plone, but I can imagine the deployment
> issues are quite different from the projects I am involved in (numpy
> and co).

Plone is downloaded and installed on many-many systems, across all the
"mainline" platforms.  In each case (since Plone 3.2), the installer is
based on (and includes) zc.buildout, and documents[1] how to add new
bits to the installed Plone by modifying the buildout.cfg file.

> Everytime I tried to understand what buildout was about, I was not
> even sure it could help for my own problems at all. It seems very
> specific to web development - I may completely miss the point ?

I think so:  it is largely a way to get repeatable / scripted deployment
of software to disk.  It uses setuptools to install Python package
distributions, but also can use other means (e.g, configure-make-make
install to install a C library such as libxml2).  The end result is a
self-contained directory tree:

- - Scripts in the 'bin' directory are configured to have the specific
  Python pacakges (and versions) they require on the PYTHONPATH.

- - By convention, released package distributions are installed into the
  'eggs' subdirectory', which is *not* on the PYTHONPATH, nor is it a
  'site' directory for Python.

- - Other bits are typically in their own subdirectories, often under
  'parts'.

> virtualenv, pip, yolk, those are useful tools for development/testing,
> but I don't see how they can help me to make the installation of a
> numpy environment easier on many different kind of platforms.

When not doing Plone / Zope-specific work (where zc.buildout is a de
facto standard), I use 'virtualenv' to create isolated environments into
which I install the libraries for a given application.  If your
application ships as Python package distributions, then documenting the
use of 'virtualenv' as a "supported" way to install it might reduce your
support burden.  You can even ship a virtualenv-derived script which
pre-installs your own packages into such an environment, isolated from
the other pacakges installed on the machine.

>>> If the problem is to get a recent enough version of the library, then
>>> the library would better be installed "locally", for the application.
>>> If it is too much a problem because the application depends on
>>> billions of libraries which are 6 months old, the problem is to allow
>>> such a dependency in the first place. What kind of nightmare would it
>>> be if programs developed in C would required a C library which is 6
>>> months old ? That's exactly what multiple-versions installations
>>> inflict on us. That's great for testing, development. But for
>>> deployment on end-user machines, the whole thing is a failure IMO.
>> It is wildly successful, even on platforms such as Windows, when you
>> abandon the notion that separate applications should be sharing the
>> libaries they need.
> 
> Well, I may not have been clear: I meant that in my experience,
> deploying something with several dependencies was easier with bundling
> than with a mechanism ala setuptools with *system-wide* installation
> of multiple versions of the same library. So I think we agree here:
> depending on something stable (python stdlib + a few well known
> things) system-wide is OK, for anything else, not sharing is easier
> and more robust in the current state of things, at least when one
> needs to stay cross platform.

You can think of zc.buildout or the virtualenv-based script as a form of
bundling, which bootstraps from another already-installed Python, but
remains isolated from anything in its 'site-packages'.

> Almost every deployment problem I got from people using my own
> softwares was related to setuptools, and in particular the multiple
> version thing.

I never even use that switch manually.  zc.buildout does, but that is
because it wants to control the PYTHONPATH by generating the script
code:  it doesn't ask users to tweak that.

> For end-users who know nothing about python package
> mechanism, and do not care about it, that's really a PITA to debug,
> and give bad mouth taste. The fact that those problems happen when my
> software was not even *using* setuptools/etc... was a real deal
> breaker for me, and I am strongly biased against setuptools ever
> since.

I don't know why anybody who was not writing a pac

Re: [Python-Dev] Test failures under Windows?

[Kristján Valur Jónsson]

These issues should be resolved in the py3k branch, but it will need porting to 
2.6.
Dialogue boxes are annoying, but do they pop up if you run your buildslave as a 
service without access to the console?
K

-Original Message-
From: python-dev-bounces+kristjan=ccpgames@python.org 
[mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of David 
Bolen
Sent: 24. mars 2009 18:02
To: python-dev@python.org
Subject: Re: [Python-Dev] Test failures under Windows?

Antoine Pitrou  writes:

> Am I the only one getting those?
> By the way, what happened to the Windows buildbots?

Oops - sorry, mine (XP-4) apparently got stuck with C++ debug
assertion dialogs (an assertion from the internal close.c module) from
python_d during some earlier build.  I try to catch these but have been
a bit preoccupied this month.  Should be back now.

-- David

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Antoine Pitrou]

David Bolen  gmail.com> writes:
> 
> Antoine Pitrou  pitrou.net> writes:
> 
> > Am I the only one getting those?
> > By the way, what happened to the Windows buildbots?
> 
> Oops - sorry, mine (XP-4) apparently got stuck with C++ debug
> assertion dialogs (an assertion from the internal close.c module) from
> python_d during some earlier build.

:( This is unfortunately the kind of problem I wish the Windows buildbots had
been able to report unambiguously...

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Toshio Kuratomi]

Tres Seaver wrote:
> David Cournapeau wrote:
>>> I am afraid that distutils, and
>>> setuptools, are not really the answer to the problem, since while they
>>> may (as intended) guarantee that Python applications can be installed
>>> uniformly across different platforms they also more or less guarantee
>>> that Python applications are installed differently from all other
>>> applications on the platform.
>> I think they should be part of the solution, in the sense that they
>> should allow easier packaging for the different platforms (linux,
>> windows, mac os x and so on). For now, they make things much harder
>> than they should (difficult to follow the FHS, etc...).
> 
> FHS is something which packagers / distributors care about:  I strongly
> doubt that the "end users" will ever notice, particularly for silliness
> like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.
> 
That's because you're thinking of a different class of end-user than FHS
 is targeting.  Someone who wants to install a web application on a
limited number of machines (one in the home-desktop scenario) or someone
who makes their living helping people to install the software they've
written has a whole different view on things than someone who's trying
to install and maintain the software on fifteen computer labs in a
campus or the person who is trying to write software that is portable to
tens of different platforms in their spare time and every bit of
answering end user's questions, tracking other upstreams for security
bugs, etc, is time taken away from coding.

Following FHS means that the software will work for both "end-users" who
don't care about the nitty-gritty of the FHS and system administrators
of large sites.  Disregarding the FHS because it is "silliness" means
that system administrators are going to have to special-case your
application, decide not to install it at all, or pay someone else to
support it.

Note that those things do make sense sometimes.  For instance, when an
application is not intended to be distributed to a large number of
outside entities (facebook, flikr, etc) or when your revenue stream is
making money from installing and administering a piece of software for
other companies.

-Toshio



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[David Bolen]

Kristján Valur Jónsson  writes:

> These issues should be resolved in the py3k branch, but it will need
> porting to 2.6.  Dialogue boxes are annoying, but do they pop up if
> you run your buildslave as a service without access to the console?

Not sure what the MSVC++ runtime does with assertion dialogs if run
without interactive access (they aren't console messages, but UI dialog
popups).

The buildslave already disables all normal MS runtime error dialogs,
but can't stop the internal C runtime from putting up its own, at
least I don't think so.  It can be disabled from within Python itself
at startup, but not external to the process.

There was a discussion about this on the py3k mailing list back in
mid-2007 ("buildbots" thread) and perhaps later as well, at which
point I believe Martin added an "-n" option to regrtest and the
buildbot test.bat file to disable the assertions.  Is that the py3k
branch piece you are referring to?

Although in looking now I can't find that option in use in any of the
current buildbot branches as part of tools/buildbot/test.bat.  Looks
like it got lost in one of the merges (unless it was intentional)?  I
wasn't paying close attention, so maybe there's some other mechanism
in place at this point?

-- David

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[David Cournapeau]

On Wed, Mar 25, 2009 at 3:15 AM, Tres Seaver  wrote:

>
>> Everytime I tried to understand what buildout was about, I was not
>> even sure it could help for my own problems at all. It seems very
>> specific to web development - I may completely miss the point ?
>
> I think so:  it is largely a way to get repeatable / scripted deployment
> of software to disk.  It uses setuptools to install Python package
> distributions, but also can use other means (e.g, configure-make-make
> install to install a C library such as libxml2).  The end result is a
> self-contained directory tree:
>
> - - Scripts in the 'bin' directory are configured to have the specific
>  Python pacakges (and versions) they require on the PYTHONPATH.
>
> - - By convention, released package distributions are installed into the
>  'eggs' subdirectory', which is *not* on the PYTHONPATH, nor is it a
>  'site' directory for Python.
>
> - - Other bits are typically in their own subdirectories, often under
>  'parts'.

Ok - but I don't think it helps much, see below.


> When not doing Plone / Zope-specific work (where zc.buildout is a de
> facto standard), I use 'virtualenv' to create isolated environments into
> which I install the libraries for a given application.  If your
> application ships as Python package distributions, then documenting the
> use of 'virtualenv' as a "supported" way to install it might reduce your
> support burden.

I now realize why we don't understand each other - in my case, the one
doing the installation is the user, who cannot be assumed to know much
about python.q11 That's what I mean by "application deployment vs
webapp deployment". Ideally, the user has to enter one command/click
one button, and he can use the application - he may not even be a
programmer (I deploy things based on numpy/scipy for scientific
people, who often have little patience for things which take more than
1 minute to set up software-wise).

That's the user case I am mostly interested in - and I think it is
quite general, and quite different from plone deployment kind of
things.

>
> You can think of zc.buildout or the virtualenv-based script as a form of
> bundling, which bootstraps from another already-installed Python, but
> remains isolated from anything in its 'site-packages'.

Yes, I know what virtualenv is, I use it sometimes - but it is
definitely too complicated for the people I want to distribute my
software to.

>
> I never even use that switch manually.  zc.buildout does, but that is
> because it wants to control the PYTHONPATH by generating the script
> code:  it doesn't ask users to tweak that.

Well, that's the thing: neither do I :) but if my software is a
dependency of another software, then I am bothered for problems with
software which are not used at all by my own package (because
setuptools makes an egg of my software automatically, screw things up,
and I am the one blamed for it).

>
> I don't know why anybody who was not writing a packaging tool, or
> packaging a library for something like .deb / .rpm, would even use the
> multi-version option for setuptools:  I don't see any sane way to
> install conflicting requirements into a shared 'site-packages'.

But that's the problem: it often happens *even if you don't use
setuptools yourself*. I would not be surprised if that's one reason
why so many people have a seemingly unfair bias against setuptools.
That's certainly the reason in my case.

cheers,

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Collin Winter]

2009/3/24 Daniel Stutzbach :
> On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson  wrote:
>>
>> 2009/3/24 Daniel Stutzbach :
>> > [...]
>> > 100 nanoseconds, py3k trunk:
>> > ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
>> > PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)
>> > [more timings snipped]
>>
>> Does removing the PyLong_Check call in PyLong_AsSsize_t
>> make any noticeable difference to these timings?
>
> Making no other changes from the trunk, removing the PyLong_Check and NULL
> check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
> trunk is around 100 nanoseconds).
>
> Here's what I'm testing with, by the way:
>
> ./python.exe Lib/timeit.py -r 10 -s 'x = list(range(10))' 'x[5]'

What difference does it make on real applications? Are you running any
macro-benchmarks against this?

Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Curt Hagenlocher]

On Tue, Mar 24, 2009 at 11:49 AM, David Bolen  wrote:
>
> Kristján Valur Jónsson  writes:
>
>> These issues should be resolved in the py3k branch, but it will need
>> porting to 2.6.  Dialogue boxes are annoying, but do they pop up if
>> you run your buildslave as a service without access to the console?
>
> Not sure what the MSVC++ runtime does with assertion dialogs if run
> without interactive access (they aren't console messages, but UI dialog
> popups).

At least at some point in the past, it didn't check to see if there
was an attached window station.  But the dialog can be disabled in
debug mode by using the _set_error_mode function.
(http://msdn.microsoft.com/en-us/library/sas1dkb2(VS.71).aspx)

--
Curt Hagenlocher
c...@hagenlocher.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Toshio Kuratomi]

David Cournapeau wrote:
> On Wed, Mar 25, 2009 at 1:45 AM, Toshio Kuratomi 
wrote:
>> David Cournapeau wrote:
>>> 2009/3/24 Toshio Kuratomi :
 Steve Holden wrote:

> Seems to me that while all this is fine for developers and Python
users
> it's completely unsatisfactory for people who just want to use Python
> applications. For them it's much easier if each application comes with
> all dependencies including the interpreter.
>
> This may seem wasteful, but it removes many of the version
compatibility
> issues that otherwise bog things down.
>
 The upfront cost of bundling is lower but the maintenance cost is
 higher.  For instance, OS vendors have developed many ways of being
 notified of and dealing with security issues.  If there's a security
 issue with gtkmozdev and the python bindings to it have to be
 recompiled, OS vendors will be alerted to it and have the
opportunity to
 release updates on zero day, the day that the security announcement
goes
 out.
>>> I don't think bundling should be compared to depending on the system
>>> libraries, but as a lesser evil compared to requiring multiple,
>>> system-wide installed libraries.
>>>
>> Well.. I'm not so sure it's even a win there.  If the libraries are
>> installed system-wide, at least the consumer of the application knows:
>>
>> 1) Where to find all the libraries to audit the versions when a security
>> issue is announced.
>> 2) That the library is unforked from upstream.
>> 3) That all the consumers of the library version have a central location
>> to collaborate on announcing fixes to the library.
>
> Yes, those are problems, but installing multi libraries have a lot of
> problems too:
>  - quickly, by enabling multiple version installed, people become very
> sloppy to handle versions of the dependencies, and this increases a
> lot the number of libraries installed - so the advantages above for
> system-wide installation  becomes intractable quite quickly

This is somewhat true.  Sloppiness and increased libraries are bad.  But
there are checks on this sloppiness.  Distributions, for instance, are
quite active about porting software to use only a subset of versions.
So in the open source world, there's a large number of players
interested in keeping the number of versions down.  Using multiple
libraries will point people at where work needs to be done whereas
bundling hides it behind the monolithic bundle.

>  - bundling also supports a real user-case which cannot be solved by
> rpm/deb AFAIK: installation without administration privileges.

This is only sortof true.  You can install rpms into a local directory
without root privileges with a commandline switch.  But rpm/deb are
optimized for system administrators so the documentation on doing this
is not well done.  There can also be code issues with doing things this
way but those issues can affect bundled apps as well. And finally, since
rpm's primary use is installing systems, the toolset around it builds
systems.  So it's a lot easier to build a private root filesystem than
it is to cherrypick a single package.  It should be possible to create a
tool that merges a system rpmdb and a user's local rpmdb using the
existing API but I'm not aware of any applications built to do that yet.

>  - multi-version installation give very fragile systems. That's
> actually my number one complain in python: setuptools has caused me
> numerous headache, and I got many bug reports because you often do not
> know why one version was loaded instead of another one.
>
I won't argue for setuptools' implementation of multi-version.  It
sucks.  But multi-version can be done well.  Sonames in C libraries are
a simple system that does this better.

> So I am not so convinced multiple-version is better than bundling - I
> can see how it sometimes can be, but I am not sure those are that
> important in practice.
>
Bundling is always harmful.  Whether multiple versioning is any better
is certainly debatable :-)

>> No.  This is a social problem.  Good source control only helps if I am
>> tracking upstream's trunk so I'm aware of the direction that their
>> changes are headed.  But there's a wide range of reasons that
>> application developers that bundle libraries don't do that:
>>
>> 1) not enough time in a day.  I'm working full-time on making my
>> application better.  Plus I have to update all these bundled libraries
>> from time to time, testing that the updates don't break anything.  I
>> don't have time to track trunk for all these libraries -- I barely have
>> time to track releases.
>
> Yes, but in that case, there is nothing you can do. Putting everything
> in one project is always easier than splitting into modules, coding
> and deployment-wise. That's just one side of the speed of development
> vs maintenance issue IMHO.
>
>> 3) This doesn't help with the fact that my bundled version of the
>> library and your bundled version of the library are 

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

> But I don't think that a "normal" Lunix user should ever have to involve
> themselves directly with distutils or setuptools (it's OK to invoke them
> from the installation routine, but not to require the user to unpack the
> application then run setup.py).

I completely agree. It would be best if the Linux distribution provided
packages for most of the stuff (in RPM or deb format as appropriate),
and volunteers provided a repository of packages in a format that the
native package management tool actually understands (rather than
PyPI - although it might be useful to have PyPI provide a Debian
package repository for all projects that upload .deb files)

> I am afraid that distutils, and
> setuptools, are not really the answer to the problem, since while they
> may (as intended) guarantee that Python applications can be installed
> uniformly across different platforms they also more or less guarantee
> that Python applications are installed differently from all other
> applications on the platform.

Ah, so we are clearly in agreement here.

> Mike Driscoll did some work providing Windows installers for various
> Python packages and extension modules, and people were amused that he
> provided executable installers for pure Python libraries. But I saw that
> as a sensible decision, since it meant that Windows users (and even
> Windows Python application developers) used the same mechanism to
> install everything.

The Windows story is indeed sad, as none of the Windows packaging
formats provides support for dependencies (MSI has some support,
but as far as I understand it, it's pretty useless). But yes, for
Windows, you want .exe or .msi installers, not something proprietary.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

> If the problem is to get a recent enough version of the library, then
> the library would better be installed "locally", for the application.
> If it is too much a problem because the application depends on
> billions of libraries which are 6 months old, the problem is to allow
> such a dependency in the first place. What kind of nightmare would it
> be if programs developed in C would required a C library which is 6
> months old ? That's exactly what multiple-versions installations
> inflict on us. That's great for testing, development. But for
> deployment on end-user machines, the whole thing is a failure IMO.

I see this as a consequence of Web 2.0, and open source: Publish
early, publish often, make the pieces you publish as tiny as possible
to maximize reuse :-(

If people invent a new marshaling format every year (XML-RPC, SOAP,
JSON, YAML, ...), you *have* to use a just-published library to follow
the hype curve. If you have been raised in this tradition, you
are shocked by the number of modules in the Python library, and wish
for this dinosaur to die.

Apparently, deployment to end user machines is not so much of an issue
usually, since the mess will live on the web servers, and not on
everybody's end user machine.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[David Cournapeau]

>
> This is only sortof true.  You can install rpms into a local directory
> without root privileges with a commandline switch.  But rpm/deb are
> optimized for system administrators so the documentation on doing this
> is not well done.  There can also be code issues with doing things this
> way but those issues can affect bundled apps as well. And finally, since
> rpm's primary use is installing systems, the toolset around it builds
> systems.  So it's a lot easier to build a private root filesystem than
> it is to cherrypick a single package.  It should be possible to create a
> tool that merges a system rpmdb and a user's local rpmdb using the
> existing API but I'm not aware of any applications built to do that yet.

Building private root kind of defeat the purpose :) Deploying linux
packages in a reliable way without requiring admin privileges is an
"interesting" experience. The tools just don't support this well - or
there exists some magical tools that I've never seen mention of.

> I won't argue for setuptools' implementation of multi-version.  It
> sucks.  But multi-version can be done well.  Sonames in C libraries are
> a simple system that does this better.

I would say simplistic instead of simple :) what works for C won't
necessarily work for python - and even in C, library versioning is not
used that often except for a few core libraries. Library versioning
works in C because C model is very simple. It already breaks for C++.
More high-level languages like C# already have a more complicated
scheme (GAC) - and my impression is that it did not work that well.
The SxS for dll on recent windows to handle multiple version is a
nightmare too in my (limited) experience.

>>
> I'm confused -- if it will happen whatever tools are available, how does
> "good source control" solve the issue?  I'm saying that this is not an
> issue that can be solved by having good source control... it's a social
> issue that has to be solved by people learning to avoid bad practices.

I meant that whatever technology is available, bundling everything
will always be easier. And sometimes, given the time/resource
constraints, that's even the only realistic option. I could tell you
many stories about wasted hours related to some fortran libraries
which were hopelessly broken (missing symbols) on most distributions,
or ABI conflicts - for which bundling sometimes is the only path to
keep its sanity (short of giving up support for the platform). When
you need to solve the problem now because you want to demo things
tomorrow, not in 6 months, that's the only solution. It is not always
bad practice.

IMHO, one should focus on making it easier to avoid bundling
everything - robust and simple dependency scheme, adapting distutils
installation scheme to the various OS conventions, be it FHS, windows,
etc... But we can't and shouldn't prevent it totally, and tools are
already there to help minimizing the problems of bundling. For
multiple system-wide libraries, I have yet to encounter anything which
makes it somewhat reliable - it has only caused problems for me, and
not solved any single problem.

cheers,

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding PEP consistent aliases for names that don't currently conform

[Guido van Rossum]

Please don't do this. We need stable APIs. Trying to switch the entire
community to use CapWord APIs for something as commonly used as
datetime sounds like wasting a lot of cycles with no reason except the
mythical "PEP 8 conformance". As I said, it's a pity we didn't change
this at the 3.0 point, but I think going forward we should try to be
more committed to slow change. Additions of new functionality are of
course fine. But renamings (even if the old names remain available)
are just noise.

--Guido

On Mon, Mar 23, 2009 at 2:00 PM, Jess Austin  wrote:
> On Tue, Mar 3 at 19:25:21 Guido van Rossum  wrote:
>> On Tue, Mar 3, 2009 at 5:15 PM, Brett Cannon  wrote:
>>>
>>>
>>> On Tue, Mar 3, 2009 at 05:13,  wrote:

 On Tue, 3 Mar 2009 at 06:01, Ivan Krsti?~G wrote:
>
> On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>>
>> > > ?PS.: so is datetime.datetime a builtin then? :)
>> > > ?Another historic accident. Like socket.socket. :-(
>> >
>> ?A pity this stuff wasn't addressed for 3.0. Way too late now, though.
>>
>> A pity indeed.
>>
> It may be too late to rename the existing accidents, but why not add
> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and
> strongly encourage their use in new code?
>>>
>>> Or make the old names aliases for the new names and start a
>>> PendingDeprecationWarning on the old names so they can be switched in the
>>> distant future?
>>
>> +1, if it's not done in a rush and only for high-visibility modules --
>> let's start with socket and datetime.
>>
>> We need a really long lead time before we can remove these. I
>> recommend starting with a *silent* deprecation in 3.1 combined with a
>> PR offensive for the new names.
>
> I've uploaded a patch for the datetime module with respect to this
> issue at http://bugs.python.org/issue5530 . I would appreciate it if
> experienced developers could take a look at it and provide some
> feedback.  Since I've only been hacking on CPython for about a month,
> please be kind!  I'm happy to make changes to this.
>
> As it stands now, the patch changes the current objects to have
> CapWords names, and subclasses these objects to provide objects with
> the old names. Use of methods (including __new__) of the derived
> objects causes PendingDeprecations (if one has warning filters
> appropriately set).
>
> A warning: this patch requires the patch to the test refactoring at
> Issue 5520 to completely apply.  It will fail one test without the
> patch at Issue 5516.  Both of these are (inexpertly) linked from the
> roundup page for this issue.
>
> I hope this will be helpful.
>
> cheers,
> Jess Austin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding PEP consistent aliases for names that don't currently conform

[Jess Austin]

On Tue, Mar 24, 2009 at 3:14 PM, Guido van Rossum  wrote:
> Please don't do this. We need stable APIs. Trying to switch the entire
> community to use CapWord APIs for something as commonly used as
> datetime sounds like wasting a lot of cycles with no reason except the
> mythical "PEP 8 conformance". As I said, it's a pity we didn't change
> this at the 3.0 point, but I think going forward we should try to be
> more committed to slow change. Additions of new functionality are of
> course fine. But renamings (even if the old names remain available)
> are just noise.

OK, I had misunderstood your earlier message.  Sorry for the confusion.

thanks,
Jess
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Terry Reedy]

R. David Murray wrote:


So, having triaged a few issues, here are my thoughts.

The current workflow is roughly:

o test needed
o patch needed
o patch review
o commit review

One can look at these and see what needs to be done "next".  I think
that in practice the above list actually expands something like this:

o consensus needed
o test needed
o patch needed
o patch needs work
o patch review
o commit review


As a point of comparison, here are the GNUnet mantis status codes.
The following status codes are used in Mantis:

* New
A new bug, developers did not look into these yet.
* Feedback
Developers require feedback from users reporting the bug to resolve 
it. Also used if a general discussion between the researches is needed 
on how to address a problem.

* Acknowledged
Developers have seen the bug.
* Confirmed
Developers are convinced that the bug is a problem that needs to be 
fixed.

* Assigned
Some developer has started working on the problem. Note that 
developers may give up on problems, putting the bug back to confirmed, 
or feedback.

* Resolved
The bug has been fixed in some version in Subversion or in a patch 
attached to the bug report.

* Closed
Resolved bugs are closed after the bugfix has made it into a full 
release of GNUnet.


tjr

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding PEP consistent aliases for names that don't currently conform

[Guido van Rossum]

On Tue, Mar 24, 2009 at 1:23 PM, Jess Austin  wrote:
> On Tue, Mar 24, 2009 at 3:14 PM, Guido van Rossum  wrote:
>> Please don't do this. We need stable APIs. Trying to switch the entire
>> community to use CapWord APIs for something as commonly used as
>> datetime sounds like wasting a lot of cycles with no reason except the
>> mythical "PEP 8 conformance". As I said, it's a pity we didn't change
>> this at the 3.0 point, but I think going forward we should try to be
>> more committed to slow change. Additions of new functionality are of
>> course fine. But renamings (even if the old names remain available)
>> are just noise.
>
> OK, I had misunderstood your earlier message.  Sorry for the confusion.

No problem! I was probably using shorthand that only experienced devs
understood.

I hope you'll find other things to contribute!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Martin v. Löwis]

> o consensus needed
> o test needed
> o patch needed
> o patch needs work
> o patch review
> o commit review
> 
> The first of these additional items is equivalent to your bullet item
> above.  I would propose that the issue, regardless of whether or not
> it is a bug fix or a feature request, goes into 'consensus needed'

Well, there is always the "unset" state, which means "not triaged".
So if you propose that this should be the default initial state,
I'm opposed.

Instead, would it help to add a "confirmed" state? For a bug, that
would mean that it was successfully reproduced; for a patch, it
was confirmed as desirable.

If the person performing triage cannot confirm the bug, they should
reject the issue (or recommend rejection, in case they are unsure).
Somebody performing triage should never conclude with "I don't
know"; this would be time-wasting.

If, for a bug, the reviewer disagrees that it would be a bug if the
behavior actually exists, then the issue should be closed (resolution
not-a-bug/invalid). If the reviewer agrees that it would be a bug,
but fails to reproduce it, a test is needed.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Martin v. Löwis]

> There was a discussion about this on the py3k mailing list back in
> mid-2007 ("buildbots" thread) and perhaps later as well, at which
> point I believe Martin added an "-n" option to regrtest and the
> buildbot test.bat file to disable the assertions.  Is that the py3k
> branch piece you are referring to?

I don't quite remember the -n flag, but I believe that Kristjan
just removed all that stuff, ie. there is now no way to block
CRT assertions anymore.

> I
> wasn't paying close attention, so maybe there's some other mechanism
> in place at this point?

Unfortunately, not. There was a long debate, I lost, and Kristjan
removed all the changes to support this setup, claiming that it is
good to get assertions :-(

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI

[Martin v. Löwis]

> Hi. I'm Japanese and non-ascii charactor user. (cp932)
> 
> We have to use "IME" to input non-ascii charactor in Windows.
> When "> chcp 65001" in cmd.exe, we cannot use IME on cmd.exe.
> 
> So setting codepage to 65001 make output universal but make input ascii-only.
> Sit!!!

Is there a code page that still allows IME input, but supports all
of Unicode? I know there is GB18030 - is it any good?

In any case, for *that* problem, I think you need to look for a
different terminal program.

> I hope PyQtShell  become good
> IDLE alternative.

Could well be. To see it integrated into Python (and perhaps replace
IDLE), a lot of steps would need to happen, though. But perhaps you
weren't asking for it to be included in Python - it can also be a good
alternative if it's not included (like several other IDEs)

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Guido van Rossum]

On Sat, Mar 21, 2009 at 2:54 PM, Greg Ewing  wrote:
> Antoine Pitrou wrote:
>
>> Do we really want to add a syntactic feature which has such a complicated
>> expansion? I fear it will make code using "yield from" much more difficult
>> to
>> understand and audit.
>
> As I've said before, I don't think the feature itself is
> difficult to understand. You're not meant to learn about
> it by reading the expansion -- that's only there to pin
> down all the details for language lawyers.
>
> For humans, almost all the important information is
> contained in one paragraph near the top:
>
> "When the iterator is another generator, the effect is the same as if
> the body of the subgenerator were inlined at the point of the ``yield
> from`` expression. Furthermore, the subgenerator is allowed to execute
> a ``return`` statement with a value, and that value becomes the value of
> the ``yield from`` expression."
>
> Armed with this perspective, do you still think there will
> be difficulty in understanding or auditing code?

Well, hmm... I've been out of the loop due to other commitments
(sorry), but I really don't like to have things whose semantics is
defined in terms of code inlining -- even if you don't mean that as
the formal semantics but just as a mnemonic hint. It causes all sorts
of confusion about scopes.

What happened to the first-order approximation

  "yield from X" means roughly the same as "for _x in X: yield x"

?

The more specialized semantics in some cases can probably be put off
until later in the document.

FWIW I am okay with the notion that if the immediate subiterator
returns a value, that value becomes the value of the
yield-from-expression. Suitable semantics that make this effect pass
through multiple layers of sub-iterators are fine too. But the exact
semantics in the light of try/except or try/finally blocks on the
stack are incredibly (perhaps impossibly) tricky to get right -- and
it probably doesn't matter all that much what exactly happens as long
as it's specified in sufficient detail that different implementations
behave the same way (apart from obvious GC differences, alas).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

David Cournapeau wrote:
> On Wed, Mar 25, 2009 at 3:15 AM, Tres Seaver  wrote:
> 
>>> Everytime I tried to understand what buildout was about, I was not
>>> even sure it could help for my own problems at all. It seems very
>>> specific to web development - I may completely miss the point ?
>> I think so:  it is largely a way to get repeatable / scripted deployment
>> of software to disk.  It uses setuptools to install Python package
>> distributions, but also can use other means (e.g, configure-make-make
>> install to install a C library such as libxml2).  The end result is a
>> self-contained directory tree:
>>
>> - - Scripts in the 'bin' directory are configured to have the specific
>>  Python pacakges (and versions) they require on the PYTHONPATH.
>>
>> - - By convention, released package distributions are installed into the
>>  'eggs' subdirectory', which is *not* on the PYTHONPATH, nor is it a
>>  'site' directory for Python.
>>
>> - - Other bits are typically in their own subdirectories, often under
>>  'parts'.
> 
> Ok - but I don't think it helps much, see below.
> 
> 
>> When not doing Plone / Zope-specific work (where zc.buildout is a de
>> facto standard), I use 'virtualenv' to create isolated environments into
>> which I install the libraries for a given application.  If your
>> application ships as Python package distributions, then documenting the
>> use of 'virtualenv' as a "supported" way to install it might reduce your
>> support burden.
> 
> I now realize why we don't understand each other - in my case, the one
> doing the installation is the user, who cannot be assumed to know much
> about python.q11 That's what I mean by "application deployment vs
> webapp deployment". Ideally, the user has to enter one command/click
> one button, and he can use the application - he may not even be a
> programmer (I deploy things based on numpy/scipy for scientific
> people, who often have little patience for things which take more than
> 1 minute to set up software-wise).
> 
> That's the user case I am mostly interested in - and I think it is
> quite general, and quite different from plone deployment kind of
> things.

OK, but I'm still not convinced that the Plone installers don't fit this
case.  It's just that they also allow add-on installation afterward:
the "main" Plone application is installed via a GUI wizard (on Windows /
OSX).

>> You can think of zc.buildout or the virtualenv-based script as a form of
>> bundling, which bootstraps from another already-installed Python, but
>> remains isolated from anything in its 'site-packages'.
> 
> Yes, I know what virtualenv is, I use it sometimes - but it is
> definitely too complicated for the people I want to distribute my
> software to.

Have you tried using the script you can generate from it?  E.g.,:

 >>> MY_ADDON_CODE = """\
 ...
 def after_install(options, home_dir):
 subprocess.call([join(home_dir, 'bin', 'easy_install'),
 'MyPackage'])
 """
 >>> from virtualenv import create_bootstrap_script
 >>> f = open('/tmp/myinstaller.py', 'w')
 >>> f.write(create_bootstrap_script(MY_ADDON_CODE)
 >>> f.close()

Users would then be able to run that script and get a virtualenv, with
your 'MyPackage' distribution already installed in it.

>> I never even use that switch manually.  zc.buildout does, but that is
>> because it wants to control the PYTHONPATH by generating the script
>> code:  it doesn't ask users to tweak that.
> 
> Well, that's the thing: neither do I :) but if my software is a
> dependency of another software, then I am bothered for problems with
> software which are not used at all by my own package (because
> setuptools makes an egg of my software automatically, screw things up,
> and I am the one blamed for it).

Hmm, if your package uses "pure" distutils facilities, then it should be
installable by setuptools as an egg without problems.  Why would your
users be trying to to '--multi-version' installs of your package?

>> I don't know why anybody who was not writing a packaging tool, or
>> packaging a library for something like .deb / .rpm, would even use the
>> multi-version option for setuptools:  I don't see any sane way to
>> install conflicting requirements into a shared 'site-packages'.
> 
> But that's the problem: it often happens *even if you don't use
> setuptools yourself*. I would not be surprised if that's one reason
> why so many people have a seemingly unfair bias against setuptools.
> That's certainly the reason in my case.

Again:  I've *never* seen packages installed in multi-version fashion,
except in the very different mode (not in sys.path, nor in a directory
significant to site.py) used by zc.buildout.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATUR

Re: [Python-Dev] tracker status options

[R. David Murray]

On Tue, 24 Mar 2009 at 21:27, "Martin v. L?wis" wrote:


o consensus needed
o test needed
o patch needed
o patch needs work
o patch review
o commit review

The first of these additional items is equivalent to your bullet item
above.  I would propose that the issue, regardless of whether or not
it is a bug fix or a feature request, goes into 'consensus needed'


Well, there is always the "unset" state, which means "not triaged".
So if you propose that this should be the default initial state,
I'm opposed.


No, I was not suggesting it be the default state.


Instead, would it help to add a "confirmed" state? For a bug, that
would mean that it was successfully reproduced; for a patch, it
was confirmed as desirable.


So, 'confirmed' instead of 'consensus needed' (or confirmed/approved,
to cover feature requests), and 'patch is appropriate' that comes...I'm
not quite sure where?


If the person performing triage cannot confirm the bug, they should
reject the issue (or recommend rejection, in case they are unsure).
Somebody performing triage should never conclude with "I don't
know"; this would be time-wasting.


The cases I was considering are cases where in the comments on the ticket
there is disagreement either on whether or not the bug is a bug or (more
often) whether or not the feature is desirable, and at the patch stage
whether or not the patch is the appropriate fix.  I think currently
things sit in 'patch needed' until consensus is reached on the patch,
but I haven't watched enough tickets progress yet to be sure :)
Adding another stage here may be more confusing than it is helpful,
seeing as I can't really figure out where it would go.

Did I guess correctly that the process for "recommending rejection"
is to set the stage to 'commit/reject', the resolution to 'invalid'
(or whatever is appropriate) and the status to 'pending'?  That
seemed to work for the issue I did it to, in that someone came
along and closed it shortly after that.


If, for a bug, the reviewer disagrees that it would be a bug if the
behavior actually exists, then the issue should be closed (resolution
not-a-bug/invalid). If the reviewer agrees that it would be a bug,
but fails to reproduce it, a test is needed.


OK, so I guess I now understand how the current workflow is supposed to
work: if its stage is 'unassigned', then it hasn't been accepted as a
bug/enhancement yet, and triage should make that accept/reject judgement.

The tricky bit here for me is that I, as a new person doing triage,
don't always feel comfortable in passing judgement on a bug or feature
request.  So what would be the mechanism for a triage person to request a
"passing of judgement" from someone with more experience/authority?  Post
to python-dev?  Given such a mechanism, I think I would be comfortable
with the current workflow, with the expectation that I would need to
call for assistance less and less frequently over time, and ultimately
only for those things where discussion among the devs really is needed.

Hmm.  Maybe I should write a short "guide to performing triage" and
post it for feedback.

--
R. David Murray   http://www.bitdance.com___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Kristján Valur Jónsson]

There was a recent test added to test_fileio.py which triggered runtime checks 
on windows.
I fixed this and other related issues in a series of revisions around 70580.
Now, I know that this msvc behaviour can be disabled, but it was decided that 
it was not appropriate to meddle with runtime flags of the whole process for 
python.
It is unfortunate that a dialogue box is put up, even when the process is 
running without a UI.  I wonder if there is a way to fix that without changing 
the way python is compiled...

K

-Original Message-
From: python-dev-bounces+kristjan=ccpgames@python.org 
[mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Curt 
Hagenlocher
Sent: 24. mars 2009 19:14
To: David Bolen
Cc: python-dev@python.org
Subject: Re: [Python-Dev] Test failures under Windows?

On Tue, Mar 24, 2009 at 11:49 AM, David Bolen  wrote:
>
> Kristján Valur Jónsson  writes:
>
>> These issues should be resolved in the py3k branch, but it will need
>> porting to 2.6.  Dialogue boxes are annoying, but do they pop up if
>> you run your buildslave as a service without access to the console?
>
> Not sure what the MSVC++ runtime does with assertion dialogs if run
> without interactive access (they aren't console messages, but UI dialog
> popups).

At least at some point in the past, it didn't check to see if there
was an attached window station.  But the dialog can be disabled in
debug mode by using the _set_error_mode function.
(http://msdn.microsoft.com/en-us/library/sas1dkb2(VS.71).aspx)

--
Curt Hagenlocher
c...@hagenlocher.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Martin v. Löwis]

> So, 'confirmed' instead of 'consensus needed' (or confirmed/approved,
> to cover feature requests), and 'patch is appropriate' that comes...I'm
> not quite sure where?

If the person who did the triage actually thinks the patch is ready to
be committed, it would be "commit review". Not sure in what other way
a patch might be appropriate.

> The cases I was considering are cases where in the comments on the ticket
> there is disagreement either on whether or not the bug is a bug or (more
> often) whether or not the feature is desirable, and at the patch stage
> whether or not the patch is the appropriate fix.

If the person doing the triage has made a final call, the issue can
enter the next stage. There should never be debate on the tracker, IMO
(although there often is). It might be that people disagree with a
triage, then they should appeal to python-dev, not on the issue itself.
It might be that they think the reviewer misunderstood the issue,
then they should clarify, and the reviewer should revert the status.

IMO, the reviewer should *always* take action, either by asking for
more information, or by advancing the issue to the next stage.

> Did I guess correctly that the process for "recommending rejection"
> is to set the stage to 'commit/reject', the resolution to 'invalid'
> (or whatever is appropriate) and the status to 'pending'?  That
> seemed to work for the issue I did it to, in that someone came
> along and closed it shortly after that.

If you have permission to do so, you should just close the issue
(in that manner). If you don't have permission, you can leave a message
saying "I recommend to close the issue".

If you are unsure, you can set it to Pending, and ask for help on
python-dev. In that case, you haven't actually done triage, but merely
considered it, and determined that it looks too difficult.

> The tricky bit here for me is that I, as a new person doing triage,
> don't always feel comfortable in passing judgement on a bug or feature
> request.  So what would be the mechanism for a triage person to request a
> "passing of judgement" from someone with more experience/authority?  Post
> to python-dev?

Exactly. Never assign to somebody, unless that somebody has explicitly
requested such an assignment, or unless he gave blank permission to
be assigned certain kinds of issues.

> Hmm.  Maybe I should write a short "guide to performing triage" and
> post it for feedback.

That can't hurt. Different people make different experiences.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[R. David Murray]

On Tue, 24 Mar 2009 at 22:41, "Martin v. L?wis" wrote:

If the person doing the triage has made a final call, the issue can
enter the next stage. There should never be debate on the tracker, IMO
(although there often is). It might be that people disagree with a
triage, then they should appeal to python-dev, not on the issue itself.
It might be that they think the reviewer misunderstood the issue,
then they should clarify, and the reviewer should revert the status.


OK, so given this then I revise the way I understand what is happening
in the ticket I'm looking at: a reviewer has said "this patch needs
work" and the submitter has not responded.  Since the behavior has been
accepted as a valid bug this means...I can either work on the patch, or
post a message asking if the submitter wants to either revise the patch
or discuss it on python-dev.  In the latter case if the submitter does
not respond, then the issue continues to languish.  IMO it shouldn't be
closed, because it really is a bug.

Does this match what you would expect a reviewer to do?  (A person
really doing triage would of course not work on the patch themselves :)


IMO, the reviewer should *always* take action, either by asking for
more information, or by advancing the issue to the next stage.


Did I guess correctly that the process for "recommending rejection"
is to set the stage to 'commit/reject', the resolution to 'invalid'
(or whatever is appropriate) and the status to 'pending'?  That
seemed to work for the issue I did it to, in that someone came
along and closed it shortly after that.


If you have permission to do so, you should just close the issue
(in that manner). If you don't have permission, you can leave a message
saying "I recommend to close the issue".

If you are unsure, you can set it to Pending, and ask for help on
python-dev. In that case, you haven't actually done triage, but merely
considered it, and determined that it looks too difficult.


OK, so I guess I've been given more power than I was expecting, and I'll
just have to step up the bar and learn to use it appropriately :)

--
R. David Murray   http://www.bitdance.com___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Raymond Hettinger]

4% on a micro-micro-benchmark is hardly compelling...


I concur!  This is utterly insignificant and certainly does
not warrant removing the checks.

-1 on these sort of fake optimizations.  We should focus
on algorithmic improvements and eliminating redundant
work and whatnot.  Removing checks that were put 
there for a reason doesn't seem useful at all.



Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Ben Finney]

"Martin v. Löwis"  writes:

> I see this as a consequence of Web 2.0, and open source: Publish
> early, publish often, make the pieces you publish as tiny as
> possible to maximize reuse :-(

I'm all in favour of re-use. But re-use should mean having the library
in a *single* predictable location (per declared version) on the
system, so that any code that needs it can declare the dependency and
find it there, and so that bugs in the library only need to be found
and fixed in *one* place.

> If people invent a new marshaling format every year (XML-RPC, SOAP,
> JSON, YAML, ...), you *have* to use a just-published library to
> follow the hype curve.

That's still no reason why that new version of the library can't be
installed to a central, predictable location.

> If you have been raised in this tradition, you are shocked by the
> number of modules in the Python library, and wish for this dinosaur
> to die.

It's more shocking to realise that there are even *more* libraries
installed privately by applications, where they can't be easily found
and fixed by a single update of the library.

-- 
 \  “The process by which banks create money is so simple that the |
  `\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It |
_o__)   Came, Where It Went_, 1975 |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Mark Hammond]

On 25/03/2009 6:51 AM, "Martin v. Löwis" wrote:

Mike Driscoll did some work providing Windows installers for various
Python packages and extension modules, and people were amused that he
provided executable installers for pure Python libraries. But I saw that
as a sensible decision, since it meant that Windows users (and even
Windows Python application developers) used the same mechanism to
install everything.


The Windows story is indeed sad, as none of the Windows packaging
formats provides support for dependencies (MSI has some support,
but as far as I understand it, it's pretty useless). But yes, for
Windows, you want .exe or .msi installers, not something proprietary.


Isn't this discussion slightly at cross-purposes?

* py2exe doesn't create *installers*, just the target application.  It 
(poorly) manages dependencies on the build machine.  There doesn't seem 
to be a need for runtime dependency management in this tool.


* The creation of an *installer* is something quite different.  An 
installer for a py2exe based tool also doesn't need dependency 
management.  An installer for a pure-python package that made no attempt 
to bundle dependencies might be nice, but I don't quite see how that 
falls outside the scope of distutils/setuptools/etc.  In other words, I 
don't see why the installer can't bootstrap the 'normal' dependency 
management which would be used if the package was installed any other 
way or on other platforms.


* distutils already has the ability to create Windows installer 
executables for pure-python apps/libs.  I agree it would be nice if it 
was an MSI but that is an implementation detail rather than 
implementation requirement.  How were Mike's packages fundamentally 
different than that?


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Martin v. Löwis]

> OK, so given this then I revise the way I understand what is happening
> in the ticket I'm looking at: a reviewer has said "this patch needs
> work" and the submitter has not responded.  Since the behavior has been
> accepted as a valid bug this means...I can either work on the patch, or
> post a message asking if the submitter wants to either revise the patch
> or discuss it on python-dev.  In the latter case if the submitter does
> not respond, then the issue continues to languish.  IMO it shouldn't be
> closed, because it really is a bug.
> 
> Does this match what you would expect a reviewer to do?  (A person
> really doing triage would of course not work on the patch themselves :)

Right. The issue would be in the "patch needs work" stage until somebody
contributes. No further review needed, and yes, the issue will languish
until somebody improves the patch.

> OK, so I guess I've been given more power than I was expecting, and I'll
> just have to step up the bar and learn to use it appropriately :)

:-) With power comes responsibility, of course.

However, review/triage doesn't really help unless it gets issues to be
resolved, eventually. In the specific case: if the bug has already been
confirmed, there is nothing else to be done for the review.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Mark Hammond]

On 25/03/2009 7:34 AM, "Martin v. Löwis" wrote:

I don't quite remember the -n flag, but I believe that Kristjan
just removed all that stuff, ie. there is now no way to block
CRT assertions anymore.


I
wasn't paying close attention, so maybe there's some other mechanism
in place at this point?


Unfortunately, not. There was a long debate, I lost, and Kristjan
removed all the changes to support this setup, claiming that it is
good to get assertions :-(


Note that I previously opened http://bugs.python.org/issue5116 with a 
patch to enable this to be controlled from Python via the msvcrt module. 
 This would enable the test suite to disable assertions for its entire run.


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Antoine Pitrou]

Mark Hammond  gmail.com> writes:
> 
> Note that I previously opened http://bugs.python.org/issue5116 with a 
> patch to enable this to be controlled from Python via the msvcrt module. 
>   This would enable the test suite to disable assertions for its entire run.

We certainly don't want to disable assertions during the test suite if it
silences bugs which can crash the user's interpreter.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

> * The creation of an *installer* is something quite different.  An
> installer for a py2exe based tool also doesn't need dependency
> management.

Right. I wasn't really talking about py2exe (anymore), but about
installers for libraries.

> An installer for a pure-python package that made no attempt
> to bundle dependencies might be nice, but I don't quite see how that
> falls outside the scope of distutils/setuptools/etc.  In other words, I
> don't see why the installer can't bootstrap the 'normal' dependency
> management which would be used if the package was installed any other
> way or on other platforms.

Perhaps that could be a solution. However, in package management
systems that solve this properly, you also have proper uninstallation,
which includes:
- uninstallation is rejected if packages still depend on the
  to-be-removed package (or else offers to remove the relying packages
  as well)
- uninstallation reference-counts, causing an automatically-installed
  package to be uninstalled if it is no longer needed, or else offers
  to compute-then-uninstall all packages which are no longer needed.
The .exe/.msi installers do support uninstallation, but, alas, no
dependency management.

> * distutils already has the ability to create Windows installer
> executables for pure-python apps/libs.  I agree it would be nice if it
> was an MSI but that is an implementation detail rather than
> implementation requirement.

Right - plus, distutils also supports creating MSIs.

>  How were Mike's packages fundamentally
> different than that?

Dunno.

Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Curt Hagenlocher]

On Tue, Mar 24, 2009 at 3:45 PM, Antoine Pitrou  wrote:
> Mark Hammond  gmail.com> writes:
>>
>> Note that I previously opened http://bugs.python.org/issue5116 with a
>> patch to enable this to be controlled from Python via the msvcrt module.
>>   This would enable the test suite to disable assertions for its entire run.
>
> We certainly don't want to disable assertions during the test suite if it
> silences bugs which can crash the user's interpreter.

I assume the goal would be to write them to stdout or stderr instead
of hanging the process by throwing up a dialog box.  For that matter,
you could use CrtSetReportFile to redirect them to any given file and
then assert at the end of the test run that the file is empty.

--
Curt Hagenlocher
c...@hagenlocher.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[Martin v. Löwis]

> Note that I previously opened http://bugs.python.org/issue5116 with a
> patch to enable this to be controlled from Python via the msvcrt module.
>  This would enable the test suite to disable assertions for its entire run.

This patch is fine with me. It might need some documentation, though.

However, I would still be unhappy if Python on Windows would pop up
might pop up windows in cases where you get a proper exception on Unix.
So I'd rather see these bugs fixed instead of silencing them.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Greg Ewing]

Guido van Rossum wrote:


I really don't like to have things whose semantics is
defined in terms of code inlining -- even if you don't mean that as
the formal semantics but just as a mnemonic hint.


Think about it the other way around, then. Take any chunk
of code containing a yield, factor it out into a separate
function (using whatever techniques you would normally
use when performing such a refactoring to deal with
references to variables in the surrounding scope) and
call it using yield-from. The result should be the same
as the original unfactored code.

That's the fundamental reason behind all of this -- to
make such refactorings possible in a straightforward way.


What happened to the first-order approximation

  "yield from X" means roughly the same as "for _x in X: yield x"


Everybody's reaction to that when it's been suggested
before has been "that's trivial, why bother?" So I've
been trying to present it in a way that doesn't make it
appear so trivial.

Also, my understanding is that a PEP is not meant to be
a tutorial for naive users, but a document for communicating
ideas between core Python developers, who are presumably
savvy enough not to need such watered-down material.

But I'll be happy to add a paragraph about this at the
beginning if you think it would help.


But the exact
semantics in the light of try/except or try/finally blocks on the
stack are incredibly (perhaps impossibly) tricky to get right -- and
it probably doesn't matter all that much what exactly happens as long
as it's specified in sufficient detail that different implementations
behave the same way (apart from obvious GC differences, alas).


This is part of the reason I've been emphasising the
inlining principle. When pondering what should happen
in such cases, I've been able to think to myself
"What would happen if the subgenerator were inlined?"
Most of the time that makes the answer fairly obvious,
at least in the case where the subiterator is another
generator. Then it's a matter of generalising it to
other iterators.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Test failures under Windows?

[David Bolen]

Kristján Valur Jónsson  writes:

> Now, I know that this msvc behaviour can be disabled, but it was
> decided that it was not appropriate to meddle with runtime flags of
> the whole process for python.

I must have missed that discussion, but I can't see what the problem
is if such an override only occurs during the buildbot non-interactive
test runs?  Isn't that what using -n in the buildbot test.bat did?

It's not like it hides the error, as I believe the process will still
terminate if it would have otherwise, thus causing a test failure, but
the error itself just goes to stderr (or wherever) without popping up
a blocking UI dialog.

In that way it's no different than disabling the Windows (non-CRT)
dialog boxes, which I do in my buildslave for all builds (via code in
buildbot).  The process still fails, but it doesn't hang waiting for
UI interaction.

I can't see the downside.

-- David

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[M.-A. Lemburg]

On 2009-03-24 23:47, Martin v. Löwis wrote:
>> An installer for a pure-python package that made no attempt
>> to bundle dependencies might be nice, but I don't quite see how that
>> falls outside the scope of distutils/setuptools/etc.  In other words, I
>> don't see why the installer can't bootstrap the 'normal' dependency
>> management which would be used if the package was installed any other
>> way or on other platforms.
> 
> Perhaps that could be a solution. However, in package management
> systems that solve this properly, you also have proper uninstallation,
> which includes:
> - uninstallation is rejected if packages still depend on the
>   to-be-removed package (or else offers to remove the relying packages
>   as well)
> - uninstallation reference-counts, causing an automatically-installed
>   package to be uninstalled if it is no longer needed, or else offers
>   to compute-then-uninstall all packages which are no longer needed.
> The .exe/.msi installers do support uninstallation, but, alas, no
> dependency management.

Question is: who really needs such dependency management ?

* It may be helpful to developers who wrap up 3rd party code for
an application (e.g. Miro).

* It may also help users that want to install a few plugins for an
already installed application (e.g. Zope).

* It will help users who use OSes that rely on software management
tools to keep the initial distribution size small and prefer
sharing over application isolation (e.g. Ubuntu).

* It won't simplify things if such a system gets in the way of
how the users or developer usually works or wants to work in
a project.

* It introduces dependencies on network resources that may
potentially not be trusted.

* If the package dependencies are not managed with lots of quality
assurance, it can easily ruin your complete installation or
simply prevent you from installing two sets of packages at
the same time.

There are both lots of reasons for wanting dependency checking
and against such dependency checking.

As a result, there is no definite answer as to whether it's
good or bad and there is no single system that would satisfy
all users/developers.

Instead, there needs to be freedom of choice and
distutils provides this freedom of choice by allowing you
to ship .exes, .msi installers, binary drop-in ZIP archives,
RPM packages, Debian packages, etc. etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 24 2009)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2009-03-19: Released mxODBC.Connect 1.0.1  http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Daniel (ajax) Diniz]

R. David Murray wrote:
> I understood from posts I saw go by earlier from Daniel that 'pending'
> meant 'close pending unless there is feedback to the contrary' (and I
> just used it that way).  It sounds like that is indeed correct but not
> universally known, and thus I would suggest that at a minimum this status
> be changed to 'close pending' to make it clearer.

Aha, at least one post I feel I can reply to. Sort of. I do have a set
of triaging principles and beliefs about each issue property, but if
you explained to me that 'closed' actually requires unsetting
versions, priority and keywords, I'd buy it and would add that to my
set of triaging beliefs. OK, maybe I'd ask for an explanation :)

ISTM that each tracker user understands different things from each
option, each assignee reacts differently to changes in their issues. I
kinda try to understand what people mean with the options, not what
the options should mean.

If I set issues to pending and simultaneously declare my
interpretation of pending, in part that's because I expect different
people to see different meanings in the status change. I've seen
issues in the top 10 most active that lived in a pending status for
most of their existence.

I'm having trouble posting in these threads because I actually can't
tell whether adding a stage, removing a component or renaming a status
is better or worse than the status quo.

I feel we should make the tracker more useful for core developers,
volunteers and end-users. I also think having a clear workflow helps a
lot. Yet, I'd rather have a tracker that allowed users with different
interpretations to work as they feel most comfortable than one that
requires a change of assumptions.

On my first triaging sprint Brett asked whether a new stage would make
things better and Martin said he could add any new components that
could make triaging easier. I told them I was unsure about it, that I
might have requests for new options after some thinking, but that I
had a couple of RFEs. It's been over a month, but new options still
make me uneasy.

Let me hit send before I conclude (again) this adds nothing to the
discussion and discard it :)

Regards,
Daniel
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Guido van Rossum]

On Tue, Mar 24, 2009 at 4:02 PM, Greg Ewing  wrote:
> Guido van Rossum wrote:
>> I really don't like to have things whose semantics is
>> defined in terms of code inlining -- even if you don't mean that as
>> the formal semantics but just as a mnemonic hint.
>
> Think about it the other way around, then. Take any chunk
> of code containing a yield, factor it out into a separate
> function (using whatever techniques you would normally
> use when performing such a refactoring to deal with
> references to variables in the surrounding scope) and
> call it using yield-from. The result should be the same
> as the original unfactored code.

The way I think of it, that refactoring has nothing to do with
yield-from. It's not just variable references -- I used "scope" as a
shorthand for everything that can be done within a function body,
including control flow: try/except/finally,
continue/break/raise/return.

> That's the fundamental reason behind all of this -- to
> make such refactorings possible in a straightforward way.

Well, it solves one particular detail.

>> What happened to the first-order approximation
>>
>>  "yield from X" means roughly the same as "for _x in X: yield x"
>
> Everybody's reaction to that when it's been suggested
> before has been "that's trivial, why bother?" So I've
> been trying to present it in a way that doesn't make it
> appear so trivial.

Maybe you're confusing motivation with explanation? That feedback
seems to tell me that the *motivation* needs more work; but IMO the
*explanation* should start with this simple model and then expand upon
the edge cases.

> Also, my understanding is that a PEP is not meant to be
> a tutorial for naive users, but a document for communicating
> ideas between core Python developers, who are presumably
> savvy enough not to need such watered-down material.

Not quite. PEPs aren't *just* for core developers -- they are also for
communicating to (savvy) developers outside the core group. A good PEP
needs to summarize both the motivation and specification concisely so
prospective readers can quickly determine what it is about, and
whether they care.

> But I'll be happy to add a paragraph about this at the
> beginning if you think it would help.
>
>> But the exact
>> semantics in the light of try/except or try/finally blocks on the
>> stack are incredibly (perhaps impossibly) tricky to get right -- and
>> it probably doesn't matter all that much what exactly happens as long
>> as it's specified in sufficient detail that different implementations
>> behave the same way (apart from obvious GC differences, alas).
>
> This is part of the reason I've been emphasising the
> inlining principle. When pondering what should happen
> in such cases, I've been able to think to myself
> "What would happen if the subgenerator were inlined?"
> Most of the time that makes the answer fairly obvious,
> at least in the case where the subiterator is another
> generator. Then it's a matter of generalising it to
> other iterators.

This is a good way of thinking about use cases, because it helps
deciding how the edge cases should be specified that the simplest
model (my one-liner above) doesn't answer in a useful way. But it
should not be confused with an explanation of the semantics.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Roundup / Python Tracker enhancements

[Daniel (ajax) Diniz]

  This proposal has two main goals: making the Python bug tracker more
efficient for core developers and improving Roundup in areas that
don't directly concern the PSF trackers. Most of the code would land
in Roundup's repositories, but many instance-level changes would be
specific to our tracker.

Justification
 Python Tracker
  Some feedback is necessary to find out unvoiced problems,
shortcomings and RFEs core developers have in mind regarding the
tracker. From my work on old issues and with a couple of meta-tracker
tickets, I can highlight a few pain points.
  Quoting myself:
[A] relevant time-sink consists of inadequacies of the current [tracker]
interface. Many search features are hard to use or notice, among them
date spans and entering multiple inputs as lists. Other search features
are lacking, mostly simple boolean relations, e.g. those including more
than one keyword, full search terms, type, component, etc. Besides
searching, the lack of interfaces (and backend support) for selecting
and working with multiple issues tends to waste considerable amounts
of time.
  To that, I can add some RFEs and bugs with the email interface,
support for third-party authentication and  many possible UI
improvements. See a list of these below.
 Roundup
  The project has suffered from lack of developer time for a while and
has grown a list of important improvements that would help it's
adoption and revitalize the community. Correct handling of different
charsets, DB performance and alternative indexers and templating
engines are examples of often requested features. Making Roundup a
better tool for developers and end-users, with a productive community
can only benefit the PSF.

Pre-Requisites and Status
  We have been working in this area recently, so many ideas are well
developed, with a few fleshed out in great detail.
  Regarding the Python tracker, Martin is guiding my work on simple
tracker issues, Brett gave us documents and many discussions about the
issue handling workflow, Tennessee Leeuwenburg and R. David Murray are
willing to triage issues and contribute to a better workflow.
  On the Roundup side, Stefan Seefeld and Richard Jones are leading
new wave of features and fixes, with contributions from Tobias Herp,
Bernhard Reiter and others.

Student
  With the caveat that I'm probably unable to assess this objectively,
a student should be comfortable with Python and have a grasp of
general bug/issue handling procedures. Familiarity with Roundup is a
huge plus, but the code is newbie-friendly.

Mentors
  Under the condition that enough attention is given to Roundup
itself, Stefan Seefeld is willing to mentor this proposal. I'll be
available to help mentors and the student in many tasks. Martin von
Löwis would be able to discuss many important parts of the work.

Deliverables:
  Patches for upstream Roundup and for the Python instance.


For a given proposal, a major goal should be declared, e.g. 'improve
and clean up the infrastructure' or 'work on integration with third
party services and tools'. Under that goal,  task groups like
'refactor the different frontends' or 'implement VCS integration'
should organize the developement effort in somewhat isolated steps.

Some suggested topics, with examples:
* means the idea is well developed and suitable for implementation
# means a simple task or group of tasks, easy to implement

  Clean up and improve the base infrastructure
Audit and fix any potential security issues *
Fix support of diverse character encodings

  Performance improvements
Reduce the number of RDBMS calls
Improve caching across requests

  Improve end-user UI and features
Enhance search abilities, better indexing
Boolean operators
Make forms easier to use and understand
Support upload of multiple files in one request *#
Support for third-party authentication, e.g. OpenID *

  Add developer-focused features to Roundup
Support for milestones, tasks, workgroups
Integration with code review tools
Integration with Rietveld *
Update/implement [D]VCS integration
SVN integration
Mercurial integration
Interface to continuous integration tools
Buildbot

  Administration enhancements
Mass-update/batch-editing support *#
UI for Class administration

Discussion of more detailed tasks and open issues on roundup-devel[5]
is highly encouraged.

[1] https://lists.sourceforge.net/lists/listinfo/roundup-devel
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] MutableSet and Set named methods

[Raymond Hettinger]

Does anyone remember the reason that most of the named methods were omitted from the ABC for mutablesets and sets?  The update() 
method in particular would be nice to have.



RAymond 


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MutableSet and Set named methods

[Guido van Rossum]

On Tue, Mar 24, 2009 at 4:38 PM, Raymond Hettinger  wrote:
> Does anyone remember the reason that most of the named methods were omitted
> from the ABC for mutablesets and sets?  The update() method in particular
> would be nice to have.

Because it's the same as +=, and if you want it as a callable, you can
use operator.__ior__? :-)

(And frozen sets don't have it.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI

[Glenn Linderman]

On approximately 3/24/2009 10:16 AM, came the following characters from 
the keyboard of INADA Naoki:

Hi. I'm Japanese and non-ascii charactor user. (cp932)

We have to use "IME" to input non-ascii charactor in Windows.
When "> chcp 65001" in cmd.exe, we cannot use IME on cmd.exe.

So setting codepage to 65001 make output universal but make input ascii-only.
Sit!!!

I hope PyQtShell  become good
IDLE alternative.



Thanks for the feedback.

So at least one version of the code I posted shows that 
programmatically, the code page can be set differently for input and 
output, although the last version brought both to 65001.  It seems that 
the chcp 65001 always does both.  If the IME only works for cp932, then 
leave input at cp932, and set output to 65001?


I have no idea if that could be a solution for you, but I would be 
interested in your results if you find that it is, or isn't, as that 
would add to the collective knowledge base about the subject.  This is 
idea 2, below, where I tried to cover the solution space more broadly.


Looking briefly at the definition of cp932, it seems that it covers most 
of the Unicode characters... so perhaps any or several of the following 
could happen:


1) the IME could be converted to produce UTF-8 instead of cp932, 
allowing use of 65001 for input and output
2) the split code page could be used to avoid the conversion of Unicode 
to cp932 for output.
3) Unicode could be converted to cp932 for output, allowing use of cp932 
for both input and output.


These are listed in the order of increased overhead for character handling.

Perhaps you could enlighten us all as to the issues with each of these 
ideas.


I realize the IME exists today, and is likely coded to use cp932, and 
that it would take some work to convert it to produce Unicode.  However, 
there seems to be a straightforward conversion chart between cp932 and 
Unicode at Wikipedia, so perhaps that isn't a huge effort.


It seems that the long term goal of having all software speak Unicode 
would increase the efficiency of all software when dealing with 
multi-lingual issues, as a common solution can be applied universally, 
rather than re-inventing solutions that only work for particular code pages.


But I'm not fully aware of whether or not the design or implementation 
of Unicode precludes universal solutions: I have heard rumors that 
certain characters must be interpreted differently in different locale 
contexts, which seems to be counter to the "one solution fits all" 
possibility.


--
Glenn -- http://nevcal.com/
===
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Roundup / Python Tracker enhancements

[Aahz]

On Tue, Mar 24, 2009, Daniel (ajax) Diniz wrote:
>
>   This proposal has two main goals: making the Python bug tracker more
> efficient for core developers and improving Roundup in areas that
> don't directly concern the PSF trackers. Most of the code would land
> in Roundup's repositories, but many instance-level changes would be
> specific to our tracker.

Is this for GSoC?  If yes, please make sure to include that tag in the
Subject: line to make it easier to track.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-3-22
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC 2009: Roundup / Python Tracker enhancements

[Daniel (ajax) Diniz]

Aahz wrote:
> Is this for GSoC?  If yes, please make sure to include that tag in the
> Subject: line to make it easier to track.

Oops, makes a lot of sense :)

Daniel
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] In case I seem to disappear...

[Guido van Rossum]

I've received several emails recently indicating that someone is
trying to get me unsubscribed from python-dev. I'm sure they're just
pranksters, but just in case they succeed, if I seem to be absent for
a really long time, let me know. After all mailman passwords are not
secure. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Non-Core project: IDLE

[Guilherme Polo]

On Mon, Mar 23, 2009 at 7:52 PM, Scott David Daniels
 wrote:
> Guilherme Polo wrote:
>>
>> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy  wrote:
>>>
>>> IDLE needs lots of attention -- more than any one experienced person is
>>> likely to have
>>
>> I'm willing to step up as a student for this but I still have to write
>> a good proposal for it.
>> My actual concern is about mentor availability, is someone around
>> interested on being an IDLE mentor ?
>
> You might want to talk to Bruce Sherwood, as VPython suffered through
> a bunch of Idle problems.  I got the impression he had a person or two
> who were his Idle experts (so I am thinking they might be the mentors
> you are looking for).  In any case, even a charter of "unit tests to 50%
> coverage" of Idle would be a huge improvement.
>

Thanks for the direction Scott. I have talked with Bruce yesterday and
now again, he included a list of issues that is considered important
to be fixed but unfortunately several of them are related to mac only
and I won't be able to properly address them.

He also pointed me to the person that worked on some issues regarding
IDLE and VPython, waiting now to hear if he is interested on gsoc.

> I've run after specific bugs in Idle, but don't really know the lay of
> the land.
>
>
> --Scott David Daniels
> scott.dani...@acm.org

Thanks,


-- 
-- Guilherme H. Polo Goncalves
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Ron Adam]

Tennessee Leeuwenburg wrote:

Hi all,

I'm continuing to (slowly) work through issues. I have been looking 
particularly at a lot of the open issues regarding strftime.


It would be great to put in some of those extra status options that were 
discussed recently...


   "Open/New"
   "Needs help / Chatting"
   "Under development"
   "Pending feedback"
   "Closed"

For everyone's reference, after some debate, the above list of status 
options was where the conversation finished. So the two options "Needs 
help / chatting" and "Under development" would need to be added. Issues 
that are "Under development" should revert to "Needs help / chatting" 
after a month of inactivity.


Would it be possible to get these put in? Maybe I'm one of a small 
number of people who are nibbling at the bottom end of the bugs, but I 
find it somewhat frustrating not to be able to classify the issues that 
I find into "needs help / chatting" vs "under development" to help make 
more sense of the search results.


Cheers,
-T




Maybe a flow chart can help unambiguize things a bit.  What follows is 
maybe one starting place.  I tried to separate steps that might be done by 
users from developers so that there is a clear order to the process.  This 
seems to follow the general way that python issues are resolved from what 
I've observed.


While this chart separates the descriptive/brain storming and patch making 
parts of an issue, in actuality, a patch (tests, fix, and docs) can be 
developed partially or fully in the discussion faze for the purpose of 
helping the discussion, then later reused and fine tuned in the patch 
making faze.


Hope this is helpful and doesn't get too mangled in sending.

(new issue)
|
v

[discuss issue] <-+
| |
v |
  |
{request issue review}|
| |
v |
  |
 (yes)--> [close issue]  |
   (no)|  |
|  v  |
|   (closed)  |
v |
  |
 (no)--+
  (yes)
|
v

[develop patch]  <--+
|   |
v   |
|
{request patch review}  |
|   |
v   |
|
 (no) ---+
  (yes)
|
v

[apply patch]
|
v

[close issue]
|
v

(closed)



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Non-Core project: IDLE

[Guilherme Polo]

On Tue, Mar 24, 2009 at 4:26 AM, Mark Summerfield  wrote:
> On 2009-03-23, Guilherme Polo wrote:
>> On Mon, Mar 23, 2009 at 5:39 PM, Terry Reedy  wrote:
>> > Guilherme Polo wrote:
>> >> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy  wrote:
>> >>> IDLE needs lots of attention -- more than any one experienced person is
>> >>> likely to have
>> >>
>> >> I'm willing to step up as a student for this but I still have to write
>> >> a good proposal for it.
>> >> My actual concern is about mentor availability, is someone around
>> >> interested on being an IDLE mentor ?
>> >
>> > If I could, I would, and would have said so.  But I have only read about
>> > tk and have not actually used it.  If I did decide to dive into it, you'd
>> > be mentoring me ;-).  What I can and would do is give ideas for changes,
>> > read and comment on a proposal, and user test patched versions.
>>
>> That is very nice Terry. Do you have some specific ideas that you want
>> to share publicly (or in private) about IDLE ? Your expectations about
>> what should be addressed first, or areas that should be improved.. you
>> know, anything.
>
> I have one suggestion that I think might be widely appreciated:
>
> Add somewhere in the configuration dialog when users can enter a block
> of Python code to be executed at startup and whenever Restart Shell is
> executed.
>
> Use case: for people who use IDLE for calculations/experiments they
> might like to always have certain module imported. For me personally, it
> would be:
>
>    import os
>    import re
>    import sys
>    from math import *
>
> but of course the whole point is that people can write any code they
> like. (Some people might want to do various from __future__ imports in
> Python 2.6 to get various Python 3 features for example.)
>
> I know that you can use the -c option, but that only works at startup,
> not every time you Restart Shell.
>

Looks like a good suggestion to me, Mark.
I would recommend adding it as a feature request on the typical place
(bugs.python.org) because although I could just go and do it, I
believe you are aware that new features in IDLE are subject to
approval or disapproval by other members involved with IDLE. Hope you
understand my position.

> [snip]
>
> --
> Mark Summerfield, Qtrac Ltd, www.qtrac.eu
>    C++, Python, Qt, PyQt - training and consultancy
>        "Rapid GUI Programming with Python and Qt" - ISBN 0132354187
>
>

Regards,


-- 
-- Guilherme H. Polo Goncalves
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Steve Holden]

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>> I really don't like to have things whose semantics is
>> defined in terms of code inlining -- even if you don't mean that as
>> the formal semantics but just as a mnemonic hint.
> 
> Think about it the other way around, then. Take any chunk
> of code containing a yield, factor it out into a separate
> function (using whatever techniques you would normally
> use when performing such a refactoring to deal with
> references to variables in the surrounding scope) and
> call it using yield-from. The result should be the same
> as the original unfactored code.
> 
> That's the fundamental reason behind all of this -- to
> make such refactorings possible in a straightforward way.
> 
>> What happened to the first-order approximation
>>
>>   "yield from X" means roughly the same as "for _x in X: yield x"
> 
> Everybody's reaction to that when it's been suggested
> before has been "that's trivial, why bother?" So I've
> been trying to present it in a way that doesn't make it
> appear so trivial.
> 
There is one non-trivial extension that I've been chewing over for a
while. What if you want to yield not the values from the generator but
some function of those values? The present proposal appears to have no
way to specify that. What about extending the syntax somewhat to

  yield expr for x from X

The idea is that x should be a a bound variable in expr, but the "expr
for x" could be optional to yield the existing proposal as a degenerate
case.

> Also [...]

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Tennessee Leeuwenburg]

Hi Ron,

Good flowchart.

Cheers,
-T

On Wed, Mar 25, 2009 at 1:11 PM, Ron Adam  wrote:

>
>
> Tennessee Leeuwenburg wrote:
>
>> Hi all,
>>
>> I'm continuing to (slowly) work through issues. I have been looking
>> particularly at a lot of the open issues regarding strftime.
>>
>> It would be great to put in some of those extra status options that were
>> discussed recently...
>>
>>   "Open/New"
>>   "Needs help / Chatting"
>>   "Under development"
>>   "Pending feedback"
>>   "Closed"
>>
>> For everyone's reference, after some debate, the above list of status
>> options was where the conversation finished. So the two options "Needs help
>> / chatting" and "Under development" would need to be added. Issues that are
>> "Under development" should revert to "Needs help / chatting" after a month
>> of inactivity.
>>
>> Would it be possible to get these put in? Maybe I'm one of a small number
>> of people who are nibbling at the bottom end of the bugs, but I find it
>> somewhat frustrating not to be able to classify the issues that I find into
>> "needs help / chatting" vs "under development" to help make more sense of
>> the search results.
>>
>> Cheers,
>> -T
>>
>
>
>
> Maybe a flow chart can help unambiguize things a bit.  What follows is
> maybe one starting place.  I tried to separate steps that might be done by
> users from developers so that there is a clear order to the process.  This
> seems to follow the general way that python issues are resolved from what
> I've observed.
>
> While this chart separates the descriptive/brain storming and patch making
> parts of an issue, in actuality, a patch (tests, fix, and docs) can be
> developed partially or fully in the discussion faze for the purpose of
> helping the discussion, then later reused and fine tuned in the patch making
> faze.
>
> Hope this is helpful and doesn't get too mangled in sending.
>
>(new issue)
>|
>v
>
>[discuss issue] <-+
>| |
>v |
>  |
>{request issue review}|
>| |
>v |
>  |
> (yes)--> [close issue]  |
>   (no)|  |
>|  v  |
>|   (closed)  |
>v |
>  |
> (no)--+
>  (yes)
>|
>v
>
>[develop patch]  <--+
>|   |
>v   |
>|
>{request patch review}  |
>|   |
>v   |
>|
> (no) ---+
>  (yes)
>|
>v
>
>[apply patch]
>|
>v
>
>[close issue]
>|
>v
>
>(closed)
>
>
>
>


-- 
--
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Steve Holden]

Martin v. Löwis wrote:
>> * The creation of an *installer* is something quite different.  An
>> installer for a py2exe based tool also doesn't need dependency
>> management.
> 
> Right. I wasn't really talking about py2exe (anymore), but about
> installers for libraries.
> 
>> An installer for a pure-python package that made no attempt
>> to bundle dependencies might be nice, but I don't quite see how that
>> falls outside the scope of distutils/setuptools/etc.  In other words, I
>> don't see why the installer can't bootstrap the 'normal' dependency
>> management which would be used if the package was installed any other
>> way or on other platforms.
> 
> Perhaps that could be a solution. However, in package management
> systems that solve this properly, you also have proper uninstallation,
> which includes:
> - uninstallation is rejected if packages still depend on the
>   to-be-removed package (or else offers to remove the relying packages
>   as well)
> - uninstallation reference-counts, causing an automatically-installed
>   package to be uninstalled if it is no longer needed, or else offers
>   to compute-then-uninstall all packages which are no longer needed.
> The .exe/.msi installers do support uninstallation, but, alas, no
> dependency management.
> 
>> * distutils already has the ability to create Windows installer
>> executables for pure-python apps/libs.  I agree it would be nice if it
>> was an MSI but that is an implementation detail rather than
>> implementation requirement.
> 
> Right - plus, distutils also supports creating MSIs.
> 
>>  How were Mike's packages fundamentally
>> different than that?
> 
> Dunno.

As far as memory serves, Mike built the installers precisely by using
distutils to build Windows installers. He then had to suffer criticism
from people who suggested this was inappropriately complex for pure
Python modules.

In so far as "end users" won't wish to install individual Python modules
this argument may have had some merit, but I personally thought the
criticism unjustified since Mike's technique gave a uniform install
procedure for everything.

I don't think anyone was suggesting that py2exe would be helpful for
writing installers--simply to reduce complexity from the user's point of
view and bundle dependencies in isolation from other applications. It
seems to do that quite well.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Steve Holden]

R. David Murray wrote:
> On Tue, 24 Mar 2009 at 21:27, "Martin v. L�wis" wrote:
> 
>>> o consensus needed
>>> o test needed
>>> o patch needed
>>> o patch needs work
>>> o patch review
>>> o commit review
>>>
>>> The first of these additional items is equivalent to your bullet item
>>> above.  I would propose that the issue, regardless of whether or not
>>> it is a bug fix or a feature request, goes into 'consensus needed'
>>
>> Well, there is always the "unset" state, which means "not triaged".
>> So if you propose that this should be the default initial state,
>> I'm opposed.
> 
> No, I was not suggesting it be the default state.
> 
>> Instead, would it help to add a "confirmed" state? For a bug, that
>> would mean that it was successfully reproduced; for a patch, it
>> was confirmed as desirable.
> 
> So, 'confirmed' instead of 'consensus needed' (or confirmed/approved,
> to cover feature requests), and 'patch is appropriate' that comes...I'm
> not quite sure where?
> 
>> If the person performing triage cannot confirm the bug, they should
>> reject the issue (or recommend rejection, in case they are unsure).
>> Somebody performing triage should never conclude with "I don't
>> know"; this would be time-wasting.
> 
> The cases I was considering are cases where in the comments on the ticket
> there is disagreement either on whether or not the bug is a bug or (more
> often) whether or not the feature is desirable, and at the patch stage
> whether or not the patch is the appropriate fix.  I think currently
> things sit in 'patch needed' until consensus is reached on the patch,
> but I haven't watched enough tickets progress yet to be sure :)
> Adding another stage here may be more confusing than it is helpful,
> seeing as I can't really figure out where it would go.
> 
> Did I guess correctly that the process for "recommending rejection"
> is to set the stage to 'commit/reject', the resolution to 'invalid'
> (or whatever is appropriate) and the status to 'pending'?  That
> seemed to work for the issue I did it to, in that someone came
> along and closed it shortly after that.
> 
>> If, for a bug, the reviewer disagrees that it would be a bug if the
>> behavior actually exists, then the issue should be closed (resolution
>> not-a-bug/invalid). If the reviewer agrees that it would be a bug,
>> but fails to reproduce it, a test is needed.
> 
> OK, so I guess I now understand how the current workflow is supposed to
> work: if its stage is 'unassigned', then it hasn't been accepted as a
> bug/enhancement yet, and triage should make that accept/reject judgement.
> 
> The tricky bit here for me is that I, as a new person doing triage,
> don't always feel comfortable in passing judgement on a bug or feature
> request.  So what would be the mechanism for a triage person to request a
> "passing of judgement" from someone with more experience/authority?  Post
> to python-dev?  Given such a mechanism, I think I would be comfortable
> with the current workflow, with the expectation that I would need to
> call for assistance less and less frequently over time, and ultimately
> only for those things where discussion among the devs really is needed.
> 
> Hmm.  Maybe I should write a short "guide to performing triage" and
> post it for feedback.
> 
Might I suggest studying the status values used by the Django team in
their Trac tracker instance. They seem to have a very methodical workflow.

http://docs.djangoproject.com/en/dev/internals/contributing/#ticket-triage

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[P.J. Eby]

At 10:22 PM 3/24/2009 -0400, Steve Holden wrote:

There is one non-trivial extension that I've been chewing over for a
while. What if you want to yield not the values from the generator but
some function of those values? The present proposal appears to have no
way to specify that. What about extending the syntax somewhat to

  yield expr for x from X

The idea is that x should be a a bound variable in expr, but the "expr
for x" could be optional to yield the existing proposal as a degenerate
case.


That would be spelled:

   yield from (expr for x in X)

And the compiler could optionally optimize away the 
genexpr.  Assuming, of course, that this is considered valuable 
enough to implement in the first place, which I don't think it 
is...  especially not with the return bit factored in.


Now, if somebody came up with a different way to spell the extra 
value return, I wouldn't object as much to that part.  I can just see 
people inadvertently writing 'return x' as a shortcut for 'yield x; 
return', and then having what seem like mysterious off-by-one errors, 
or being confused by receiving a generator object instead of their 
desired non-generator return value.


It also seems weird that the only syntactically-supported way to get 
the generator's "return value" is to access it inside *another* 
generator...  which *also* can't return the return value to anyone!


But if it were spelled 'raise Return(value)' or 'raise 
StopIteration(value)' or something similar (or even had its own 
syntax!), I wouldn't object, as it would then be obvious how to get 
the value, and there could be no possible confusion with a regular 
return value.


The unusual spelling would also signal that something unusual (i.e., 
multitasking) is taking place, similar to the way some frameworks use 
things like 'yield Return(value)' to signal the end of a task and its 
return value, in place of a value in the stream.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Guido van Rossum]

On Tue, Mar 24, 2009 at 8:35 PM, P.J. Eby  wrote:
> Now, if somebody came up with a different way to spell the extra value
> return, I wouldn't object as much to that part.  I can just see people
> inadvertently writing 'return x' as a shortcut for 'yield x; return', and
> then having what seem like mysterious off-by-one errors, or being confused
> by receiving a generator object instead of their desired non-generator
> return value.
>
> It also seems weird that the only syntactically-supported way to get the
> generator's "return value" is to access it inside *another* generator...
>  which *also* can't return the return value to anyone!
>
> But if it were spelled 'raise Return(value)' or 'raise StopIteration(value)'
> or something similar (or even had its own syntax!), I wouldn't object, as it
> would then be obvious how to get the value, and there could be no possible
> confusion with a regular return value.
>
> The unusual spelling would also signal that something unusual (i.e.,
> multitasking) is taking place, similar to the way some frameworks use things
> like 'yield Return(value)' to signal the end of a task and its return value,
> in place of a value in the stream.

I'm sympathetic to this point of view.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Greg Ewing]

Steve Holden wrote:

What about extending the syntax somewhat to

  yield expr for x from X


I can't see much advantage that would give you
over writing

  for x in X:
yield expr

There would be little or no speed advantage,
since you would no longer be able to shortcut
the intermediate generator during next().

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Mike Klaas]

On 24-Mar-09, at 3:15 PM, Raymond Hettinger wrote:




4% on a micro-micro-benchmark is hardly compelling...


I concur!  This is utterly insignificant and certainly does
not warrant removing the checks.

-1 on these sort of fake optimizations.  We should focus
on algorithmic improvements and eliminating redundant
work and whatnot.  Removing checks that were put there for a reason  
doesn't seem useful at all.


To be fair, the main proposed optimization(s) would speed up the  
microbenchmark by 15-25% (Daniel already stated that the NULL checks  
didn't have a significant impact).  This seems significant,  
considering that tight loops whose cost is heavily due to array access  
are common.


-Mike
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tracker status options

[Stephen J. Turnbull]

"Martin v. Löwis" writes:

 > > o consensus needed
 > > o test needed
 > > o patch needed
 > > o patch needs work
 > > o patch review
 > > o commit review
 > > 
 > > The first of these additional items is equivalent to your bullet item
 > > above.  I would propose that the issue, regardless of whether or not
 > > it is a bug fix or a feature request, goes into 'consensus needed'

All of these passives are making me aggressive

Sure, these are real statuses internally, but they seem mostly of
interest to people who are actually working on the issue.  They'll be
getting nosy mail and following the issue in their browsers.  Is this
really needed for them?

On the other hand, for a non-developer who wants to know what's
happening to their issue, these are all rather fuzzy.  The submitter
of a bug wants to know

(1) Does python-dev admit it's an issue?
No -> resolved because invalid
Yes -> confirmed
Don't know -> waiting for more information from submitter (I would
  call this "pending response from OP")
Not triaged -> "-- unset --" or new
(2) Has anybody accepted responsibility for working on this?
Yes -> assigned
(3) Has anybody done any work recently?
No -> sleeping
(4) Is there a solution?
Yes -> resolved because implemented
(5) Can I get the solution by normal upgrading?
Yes -> closed

So submitters already have *at least* seven statuses that they would
like to be able to recognize at a glance.  I think these correlate
well with what developers who accept broad responsibility (release
engineers, for example) need to know, too.

Does workflow coordination require more than that?  Almost certainly,
yes.  But is it a good idea to pack all that into status?

Also, note that code, doc, and test can be done in any order.  A
developer may write a test so he can automatically elicit the problem
behavior, then document the desired behavior, then come up with code
to implement it.  Or the OP could submit a patch, then the maintainer
decides what parts of the patch are specified behavior and what parts
are implementation details and documents it, and finally they come up
with a test.  That means you have at least *eight* different statuses,
or maybe 27 (ie, each component is tri-state: no improvement needed,
improvement needed, implemented).

I'm not suggesting that this kind of thing should be implemented;
rather, until the use case and who needs it is clear, the status field
should be kept as simple as possible.

 > Well, there is always the "unset" state, which means "not triaged".

I've had complaints from my users about the "-- not specified --"
status; they prefer "new".  They seem to think that "unspecified"
means the tracker is broken.

 > Instead, would it help to add a "confirmed" state? For a bug, that
 > would mean that it was successfully reproduced; for a patch, it
 > was confirmed as desirable.

Yes.  Lack of such a state is a PR disaster.  There needs to be a
signal that triage has happened, but the person doing triage typically
will not accept responsibility for fixing the bug, only for improving
the description of the reproduction recipe (including adding a test)
etc.

 > If the person performing triage cannot confirm the bug, they should
 > reject the issue (or recommend rejection, in case they are unsure).

If they're unsure, they should ping the submitter and request more
information, and set the status to "waiting for response".  As you say:

 > Somebody performing triage should never conclude with "I don't
 > know"; this would be time-wasting.
 > 
 > If, for a bug, the reviewer disagrees that it would be a bug if the
 > behavior actually exists, then the issue should be closed (resolution
 > not-a-bug/invalid).

Maybe set it "resolved" (if there's such a status separate from
"closed").  The idea is that if the issue gets "me toos", then it
should be added to the FAQ.  Also, such issues should be easy to
search for, but most standard searches restrict to "open" issues.

 > If the reviewer agrees that it would be a bug, but fails to
 > reproduce it, a test is needed.

By "reviewer" you mean the person doing triage, right?  By "test is
needed", you really mean "cooperation from the submitter in defining a
test", right?  (Tests are always needed to prevent regressions.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] speeding up PyObject_GetItem

[Raymond Hettinger]

4% on a micro-micro-benchmark is hardly compelling...


I concur!  This is utterly insignificant and certainly does
not warrant removing the checks.

-1 on these sort of fake optimizations.  We should focus
on algorithmic improvements and eliminating redundant
work and whatnot.  Removing checks that were put there for a reason  
doesn't seem useful at all.


To be fair, the main proposed optimization(s) would speed up the  
microbenchmark by 15-25% (Daniel already stated that the NULL checks  
didn't have a significant impact).  This seems significant,  
considering that tight loops whose cost is heavily due to array access  
are common.


I thought people used PyList_GET_ITEM or something similar
in those use situations.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Greg Ewing]

Guido van Rossum wrote:


The way I think of it, that refactoring has nothing to do with
yield-from.


I'm not sure what you mean by that. Currently it's
*impossible* to factor out code containing a yield.
Providing a way to do that is what led me to invent
this particular version of yield-from in the first
place.

I wanted a way of writing suspendable functions that
can call each other easily. (You may remember I
originally wanted to call it "call".) Then I noticed
that it would also happen to provide the functionality
of earlier "yield from" suggestions, so I adopted that
name.

But for me, factorability has always been the fundamental
idea, and the equivalence, in one particular restricted
situation, to a for loop containing a yield is just a
nice bonus.

That's what I've tried to get across in the PEP, and
it's the reason I've presented things in the way I have.


It's not just variable references -- I used "scope" as a
shorthand for everything that can be done within a function body,
including control flow: try/except/finally,
continue/break/raise/return.


Same answer applies -- use the usual techniques.

When I talk about inlining, I mean inlining the
*functionality* of the code, not its literal text. I'm
leaving the reader to imagine performing the necessary
transformations to preserve the semantics.


Maybe you're confusing motivation with explanation? That feedback
seems to tell me that the *motivation* needs more work; but IMO the
*explanation* should start with this simple model and then expand upon
the edge cases.


Perhaps what I should do is add a Motivation section before
the Proposal and move some of the material from the beginning
of the Rationale sectiomn there.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 (yield from a subgenerator) comments

[Greg Ewing]

P.J. Eby wrote:

Now, if somebody came up with a different way to spell the extra value 
return, I wouldn't object as much to that part.  I can just see people 
inadvertently writing 'return x' as a shortcut for 'yield x; return', 


Well, they need to be educated not to do that. I'm
not sure they'll need much education about this
anyway. They've already been taught not to say
'return' when they mean 'yield', so I don't see why
they should suddenly start doing so now.

I'd be disappointed to lose that part of the proposal.
Part of my philosophy is that suspendable functions
should have the same rights and privileges as ordinary
ones, and that includes the ability to return values
using 'return'.

It also seems weird that the only syntactically-supported way to get the 
generator's "return value" is to access it inside *another* 
generator...  which *also* can't return the return value to anyone!


Would you be happier if some syntactic way to do that
were provided?

It could perhaps be done by enhancing the part of the
'for' loop that gets executed upon normal termination
of the iterator.

  for x in my_iter:
do_something_with(x)
  else v:
handle_return_value(v)

The unusual spelling would also signal that something unusual (i.e., 
multitasking) is taking place, similar to the way some frameworks use 
things like 'yield Return(value)' to signal the end of a task and its 
return value, in place of a value in the stream.


Difference in philosophy again. To me, the need for such
an unusual construct when using these frameworks is a
wart, not a feature.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com