Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
On Fri, Dec 07, 2012 at 01:18:40AM -0500, PJ Eby wrote: > On Thu, Dec 6, 2012 at 1:49 AM, Toshio Kuratomi wrote: > > On Wed, Dec 05, 2012 at 07:34:41PM -0500, PJ Eby wrote: > >> On Wed, Dec 5, 2012 at 6:07 PM, Donald Stufft > >> wrote: > >> > >> Nobody has actually proposed a better one, outside of package renaming > >> -- and that example featured an author who could just as easily have > >> used an obsoleted-by field. > >> > > How about pexpect and pextpect-u as a better example? > > Perhaps you could explain? I'm not familiar with those projects. > pexepect was last released in 2008. Upstream went silent with unanswered bugs in its tracker and no mailing list. A fork of pexpect was created that addressed the issue of unicode type in python2, a python3 port, and has slowly evolvd since then. I see that the original upstream has made some commits to their source repository since the fork was created although there has still been no new release. > > Note that although well-managed Linux distros attempt to control random > > forking internally, the distro package managers don't prevent people from > > installing from third parties. So Ubuntu PPAs, upstreams that provide their > > own rpms/debs, and major third party repos (for instance, rpmfusion as > > an add-on repo to Fedora) all have and sometimes (mis)use the ability to > > Obsolete packages in the base repository. > > But in each of these cases, the packages are being defined *with > reference to* some underlying vision of what the distro (or even "a > distro") is. An Ubuntu PPA, if I understand correctly, is still > *building an Ubuntu system*. Python packaging as a whole lacks such > frames of reference. A forked distro is still a distro, and it's a > fork *of something*. Rpmfusion is defining an enhanced Fedora, not > slinging random unrelated packages about. > Uhm that's both true and false as any complex system is. rpm and deb are just packaging formats. So: *) Not all packages built build on top of that system. There are rpm packages provided by upstreams that users attempt (to greater and lesser degrees of success) to install on SuSE, RHEL, Fedora, Mandriva, etc. There are debs built for Ubuntu that people attempt to install onto Debian. *) PPAs and rpmfusion may both build on top of an existing system but they can change the underlying structure, replacing components that other pieces of the base system depend on. You talk about the setuptools and distribute problem on pypi there's absolutley nothing that prevents someone from building a PPA or a package in a third-party rpm repository that packages a setuptools that Obsoletes: distribute or a distribute package that Obsoletes: setuptools. > > The install tools can then choose how they wish to deal with those caveats. > > Some example strategies: choose to prompt the user as to which to install, > > choose to always treat the fields as human-informational only, mark some > > repositories as being trusted to contain packages where these fields are > > active and other repositories where the fields are ignored. > > A peculiar phenomenon: every defense of these fields seems to refer > almost exclusively to how the problems could be fixed or why the > problems aren't that bad, rather than *how useful the fields would be* > in real-world scenarios. In some cases, the argument for the fields' > safety actually runs *counter* to their usefulness, e.g., the fields > aren't that bad because we could make them have a limited function or > no function at all. Isn't lack of usefulness generally considered an > argument for *not* including a feature? ;-) > If you constantly forget why the fields are useful, then I suppose you'll always believe that :-) -Toshio pgpL8jo6bw502.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2012-11-30 - 2012-12-07)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open3816 (+24)
closed 24597 (+31)
total 28413 (+55)
Open issues with patches: 1671
Issues opened (36)
==
#16582: Tkinter calls SystemExit with string
http://bugs.python.org/issue16582 opened by Abraham Karplus
#16584: unhandled IOError filecmp.cmpfiles() if file not readable
http://bugs.python.org/issue16584 opened by till
#16587: Py_Initialize breaks wprintf on Windows
http://bugs.python.org/issue16587 opened by makegho
#16591: RUNSHARED wrong for OSX no framework
http://bugs.python.org/issue16591 opened by grobian
#16594: SocketServer should set SO_REUSEPORT along with SO_REUSEADDR w
http://bugs.python.org/issue16594 opened by Andy.Zeldis
#16595: Add resource.prlimit
http://bugs.python.org/issue16595 opened by christian.heimes
#16596: Skip stack unwinding when "next", "until" and "return" pdb co
http://bugs.python.org/issue16596 opened by asvetlov
#16597: file descriptor not being closed with context manager on IOErr
http://bugs.python.org/issue16597 opened by udoprog
#16598: Docs: double newlines printed in some file iteration examples
http://bugs.python.org/issue16598 opened by lost-theory
#16599: unittest: Access test result from tearDown
http://bugs.python.org/issue16599 opened by techtonik
#16601: Restarting iteration over tarfile continues from where it left
http://bugs.python.org/issue16601 opened by mbirtwell
#16602: weakref can return an object with 0 refcount
http://bugs.python.org/issue16602 opened by eltoder
#16603: Sporadic test_socket failures: testFDPassCMSG_SPACE on Mac OS
http://bugs.python.org/issue16603 opened by haypo
#16608: immutable subclass constructor call error does not show subcla
http://bugs.python.org/issue16608 opened by LambertDW
#16609: float loses precision when passed to str()
http://bugs.python.org/issue16609 opened by sleepycal
#16611: multiple problems with Cookie.py
http://bugs.python.org/issue16611 opened by jdennis
#16612: Integrate "Argument Clinic" specialized preprocessor into CPyt
http://bugs.python.org/issue16612 opened by larry
#16613: ChainMap.new_child could use improvement
http://bugs.python.org/issue16613 opened by vinay.sajip
#16614: argparse should have an option to require un-abbreviated optio
http://bugs.python.org/issue16614 opened by Michael.Edwards
#16615: gcc 4.7 unused-but-set warnings
http://bugs.python.org/issue16615 opened by jcea
#16616: test_poll.PollTests.poll_unit_tests() is dead code
http://bugs.python.org/issue16616 opened by sbt
#16618: Different glob() results for strings and bytes
http://bugs.python.org/issue16618 opened by serhiy.storchaka
#16620: Avoid using private function glob.glob1() in msi module and to
http://bugs.python.org/issue16620 opened by serhiy.storchaka
#16621: sched module enhancement request
http://bugs.python.org/issue16621 opened by carlosmf.pt
#16623: argparse help formatter does not honor non-breaking space
http://bugs.python.org/issue16623 opened by roysmith
#16624: subprocess.check_output should allow specifying stdin as a str
http://bugs.python.org/issue16624 opened by zwol
#16626: Infinite recursion in glob.glob('*:') on Windows
http://bugs.python.org/issue16626 opened by serhiy.storchaka
#16628: leak in ctypes.resize()
http://bugs.python.org/issue16628 opened by pitrou
#16629: IDLE: Calltips test fails due to int docstring change
http://bugs.python.org/issue16629 opened by serwy
#16630: IDLE: Calltip fails if __getattr__ raises exception
http://bugs.python.org/issue16630 opened by serwy
#16631: tarfile.extractall() doesn't extract everything if .next() was
http://bugs.python.org/issue16631 opened by techtonik
#16632: Enable DEP and ASLR
http://bugs.python.org/issue16632 opened by christian.heimes
#16633: os.environ updates only one copy of env vars under Windows (Ge
http://bugs.python.org/issue16633 opened by eudoxos
#16634: urllib.error.HTTPError.reason is not documented
http://bugs.python.org/issue16634 opened by berker.peksag
#16635: Interpreter not closing stdout/stderr on exit
http://bugs.python.org/issue16635 opened by filip.zyzniewski
#16636: codecs: readline() followed by readlines() returns trunkated r
http://bugs.python.org/issue16636 opened by laurynas
Most recent 15 issues with no replies (15)
==
#16636: codecs: readline() followed by readlines() returns trunkated r
http://bugs.python.org/issue16636
#16634: urllib.error.HTTPError.reason is not documented
http://bugs.python.org/issue16634
#16628: leak in ctypes.resize()
http://bugs.python.org/issue16628
#16626: Infinite recursion in glob.glob('*:') on Windows
http://bugs.python.org/issue16626
#16623: argparse help formatter does not honor non-breaking space
http://bugs.python.org/
Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
On Fri, Dec 7, 2012 at 12:01 PM, Toshio Kuratomi wrote: > On Fri, Dec 07, 2012 at 01:18:40AM -0500, PJ Eby wrote: >> On Thu, Dec 6, 2012 at 1:49 AM, Toshio Kuratomi wrote: >> > On Wed, Dec 05, 2012 at 07:34:41PM -0500, PJ Eby wrote: >> >> Nobody has actually proposed a better one, outside of package renaming >> >> -- and that example featured an author who could just as easily have >> >> used an obsoleted-by field. >> >> >> > How about pexpect and pextpect-u as a better example? >> >> Perhaps you could explain? I'm not familiar with those projects. >> > > pexepect was last released in 2008. Upstream went silent with unanswered > bugs in its tracker and no mailing list. A fork of pexpect was created that > addressed the issue of unicode type in python2, a python3 port, and has > slowly evolvd since then. > > I see that the original upstream has made some commits to their source > repository since the fork was created although there has still been no new > release. And what problem are you saying which fields would have solved (or which benefits they would have provided), for whom? If the packages have files in conflict, they won't be both installed. If they don't have files in conflict, there's nothing important to be informed of. If one is installing pexpect-u, then one does not need to discover that it is a successor of pexpect. If one is installing pexpect, it might be useful to know that pexpect-u exists, but one can't simply discover that from an Obsoletes field on pexpect-u. However, even if one did discover it, this would merely constitute an *advertisement* of pexpect-u's existence, not a *requirement* that it be used in place. A tool cannot know, without other affirmative user action, that it is actually a good assumption to use the advertised replacement. In the distro world, a user has *already* taken this affirmative action by choosing which repository to source packages from, on an implicit contract that this source is up to the job of managing his needs across multiple packages. Or, if they choose to source an off-brand or upstream package, they are taking affirmative action to risk it. In the Python world, there is no notion of a "repository", aside from a handful of managed Python distros, which have their own, distinct packaging methods and distribution tools. So there is no affirmative contract of trust regarding *inter-project* relationships. It is precisely this lack that is why the metadata spec has gone mostly unused since its inception about a decade ago. Nobody really knows what to "provide" or "require", or in what context they would actually be "obsoleting" anything that isn't their own package, or a package they've forked. But if you live mainly in the distro world, this concept seems absurd, and the fields *obviously* useful. But that's because you're swimming in an ocean of context that doesn't exist on dry land. You're saying that *of course* swimming fins are useful... if you live in the ocean. And I, living on dry land, am saying that *sure* they are... but only in a swimming pool or a pond, and we don't have very many of those here in dry Python-land. And the people who run the swimming pools have thoughtfully already provided their own. Do we need to standardize swim fin sizes for people who mostly live on dry land? The flip side of this, btw, is that there's an implicit contract in the Python world that there is generally only "the" package - not "the package as patched and re-packaged by vendors X, Y, and Z". If I install python project foo, version 1.2, I expect it to be the *same* foo-1.2, with the *same metadata*, *no matter where I got it from*. And so, this assumption is our "air" to your "water". We know that pools and ponds (curated Python distros) are different, as an exception to this rule, just as you know that reefs and islands (uncurated repositories, search engines, and upstream-built packages) are different, as an exception to your assumption that "the package I get is intended to play well with everything else in my system." (This of course is why many distro managers are suspicious of language-specific or other sorts of vertical package management tools - they seem as pointless as wheels in the water, solving problems you don't have, and creating new problems for you at the same time. Unfortunately, people on land will keep inventing them, because they have a different set of problems to solve -- some of which are actually created by the ocean-oriented tools. For example, virtualenv and its predecessors were developed to solve the "problem" of a single integrated environment, even though that integrated environment is the "solution" from a distro perspective.) > *) Not all packages built build on top of that system. There are rpm > packages provided by upstreams that users attempt (to greater and lesser > degrees of success) to install on SuSE, RHEL, Fedora, Mandriva, etc. There > are debs built for Ubuntu that people atte
Re: [Python-Dev] PEP 3145 (With Contents)
On Tue, Sep 15, 2009 at 9:24 PM, wrote: > On 04:25 pm, eric.pru...@gmail.com wrote: > >> I'm bumping this PEP again in hopes of getting some feedback. >> > This is useful, indeed. ActiveState recipe for this has 10 votes, which is high for ActiveState (and such hardcore topic FWIW). > On Tue, Sep 8, 2009 at 23:52, Eric Pruitt wrote: >> >>> PEP: 3145 >>> Title: Asynchronous I/O For subprocess.Popen >>> Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson >>> Type: Standards Track >>> Content-Type: text/plain >>> Created: 04-Aug-2009 >>> Python-Version: 3.2 >>> >>> Abstract: >>> >>>In its present form, the subprocess.Popen implementation is prone to >>>dead-locking and blocking of the parent Python script while waiting >>> on data >>>from the child process. >>> >>> Motivation: >>> >>>A search for "python asynchronous subprocess" will turn up numerous >>>accounts of people wanting to execute a child process and communicate >>> with >>>it from time to time reading only the data that is available instead >>> of >>>blocking to wait for the program to produce data [1] [2] [3]. The >>> current >>>behavior of the subprocess module is that when a user sends or >>> receives >>>data via the stdin, stderr and stdout file objects, dead locks are >>> common >>>and documented [4] [5]. While communicate can be used to alleviate >>> some of >>>the buffering issues, it will still cause the parent process to block >>> while >>>attempting to read data when none is available to be read from the >>> child >>>process. >>> >>> Rationale: >>> >>>There is a documented need for asynchronous, non-blocking >>> functionality in >>>subprocess.Popen [6] [7] [2] [3]. Inclusion of the code would >>> improve the >>>utility of the Python standard library that can be used on Unix based >>> and >>>Windows builds of Python. Practically every I/O object in Python has >>> a >>>file-like wrapper of some sort. Sockets already act as such and for >>>strings there is StringIO. Popen can be made to act like a file by >>> simply >>>using the methods attached the the subprocess.Popen.stderr, stdout and >>>stdin file-like objects. But when using the read and write methods of >>>those options, you do not have the benefit of asynchronous I/O. In >>> the >>>proposed solution the wrapper wraps the asynchronous methods to mimic >>> a >>>file object. >>> >>> Reference Implementation: >>> >>>I have been maintaining a Google Code repository that contains all of >>> my >>>changes including tests and documentation [9] as well as blog >>> detailing >>>the problems I have come across in the development process [10]. >>> >>>I have been working on implementing non-blocking asynchronous I/O in >>> the >>>subprocess.Popen module as well as a wrapper class for >>> subprocess.Popen >>>that makes it so that an executed process can take the place of a >>> file by >>>duplicating all of the methods and attributes that file objects have. >>> >> > "Non-blocking" and "asynchronous" are actually two different things. From > the rest of this PEP, I think only a non-blocking API is being introduced. > I haven't looked beyond the PEP, though, so I might be missing something. I suggest renaming http://www.python.org/dev/peps/pep-3145/ to 'Non-blocking I/O for subprocess' and continue. IMHO on this stage is where examples with deadlocks that occur with current subprocess implementation are badly needed. There are two base functions that have been added to the >>> subprocess.Popen >>>class: Popen.send and Popen._recv, each with two separate >>> implementations, >>>one for Windows and one for Unix based systems. The Windows >>>implementation uses ctypes to access the functions needed to control >>> pipes >>>in the kernel 32 DLL in an asynchronous manner. On Unix based >>> systems, >>>the Python interface for file control serves the same purpose. The >>>different implementations of Popen.send and Popen._recv have identical >>>arguments to make code that uses these functions work across multiple >>>platforms. >>> >> > Why does the method for non-blocking read from a pipe start with an "_"? > This is the convention (widely used) for a private API. The name also > doesn't suggest that this is the non-blocking version of reading. > Similarly, the name "send" doesn't suggest that this is the non-blocking > version of writing. The implementation is based on http://code.activestate.com/recipes/440554/which is more clearly illustrates integrated functionality. _recv() is a private base function, which is takes stdout or stderr as parameter. Corresponding user-level functions to read from stdout and stderr are .recv() and .recv_err() I thought about renaming API to .asyncread() and .asyncwrite(), but that may mean that you call method and then result asynchronously start to fill some buffer, which is not
Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
On Fri, Dec 7, 2012 at 3:47 PM, PJ Eby wrote: > In effect, a "conflicts" field actually *creates* conflicts and > maintenance burdens where they did not previously exist, because even > after the conflict no longer really existed, an automated tool would > have prevented PyDispatch from being installed, or, per your > suggestion above, unnecessarily *uninstalled* it after a user > installed RuleDispatch. > That's not what a Conflicts field is for. It's to allow a project to say *they don't support* installing in parallel with another package. It doesn't matter why it's unsupported, it's making a conflict perceived by the project explicit in their metadata. Such a field is designed to convey information to users about *supported* configurations, regardless of whether or not they happen to work for a given use case. If a user believes a declared conflict is in error, and having the two installed in parallel is important to them, they can: 1. Use virtual environments to keep the two projects isolated from each other 2. Use an installer that ignores Conflicts information (which will be all of them, since that's the status quo) 3. Make their case to the upstream project that the conflict has been resolved, and installing the two in parallel no longer causes issues Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
On Friday, December 7, 2012 at 8:33 PM, Nick Coghlan wrote: > That's not what a Conflicts field is for. It's to allow a project to say > *they don't support* installing in parallel with another package. It doesn't > matter why it's unsupported, it's making a conflict perceived by the project > explicit in their metadata. > > Such a field is designed to convey information to users about *supported* > configurations, regardless of whether or not they happen to work for a given > use case. If a user believes a declared conflict is in error, and having the > two installed in parallel is important to them, they can: > 1. Use virtual environments to keep the two projects isolated from each other > 2. Use an installer that ignores Conflicts information (which will be all of > them, since that's the status quo) > 3. Make their case to the upstream project that the conflict has been > resolved, and installing the two in parallel no longer causes issues 4. Use the eventual --force flag that any installer that supported conflicts is likely to include ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
On Sat, Dec 8, 2012 at 8:02 AM, PJ Eby wrote: > > *) Not all packages built build on top of that system. There are rpm > > packages provided by upstreams that users attempt (to greater and lesser > > degrees of success) to install on SuSE, RHEL, Fedora, Mandriva, etc. > There > > are debs built for Ubuntu that people attempt to install onto Debian. > > Sure. But the reference points still exist, and there is a layer of > indirection between "packager" and "developer", even in the case where > the packager and developer are the same person or organization. In > the Python case, there is usually no such indirection, outside of > curated systems like SciPy et al. (Even there, most of what > third-party packaging is about in the Python world is taking care of > binary builds.) > > Again, it's islands in the ocean vs. pools on land. > To strain the analogy, the main value of these fields exists on the beach: at the point where you need to impedance match between the Python community and another packaging community. The ideal is to be able to get a point where you can point an automated tool at a project on PyPI and say "give me that, only packaged as RPM/deb/whatever, with appropriate distro specific metadata". Such a tool is obviously going to be distro specific, since it is going to have to do some remapping based on file names to pick up existing distro packages, but it *also* needs upstream metadata. Even in a distro, a "Conflicts:" field often *does* denote runtime conflicts (e.g. over a particular network port), because, as you say, filesystem level conflicts will usually be picked up automatically. The distro philosophy is to say "OK, we simply won't let you install conflicting projects at the same time, so you won't be surprised later by a conflict that only shows up if you happen to run them both at the same time". It's designed to turn a complex, hard to debug, problem into a simple, explicit error at installation time. People build complex systems (especially web apps) based on the PyPI ecosystem, and the upstream communities *can* assist in flagging potential issues in advance. If people start putting bad metadata in their projects, then that's just a bug to be dealt with like any other. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
On Fri, Dec 7, 2012 at 8:33 PM, Nick Coghlan wrote: > That's not what a Conflicts field is for. It's to allow a project to say > *they don't support* installing in parallel with another package. If that's the actual intended use case, the PEP needs some revision. In particular, if there's a behavioral recommendation for installer tools, it should be to avoid installing the project that *declares* the conflict, rather than the one that is the object of that declaration. ;-) In any case, as I said before, I don't have an issue with the fields all being declared as being for informational purposes only. My issue is only with recommendations for automated tool behavior that permit one project's author to exercise authority over another project's installation. If the fields are defined in such a way that an author can only shoot *themselves* in the foot with a bad declaration, that's fine by me. So if package A includes a "Conflicts: B" declaration, I recommend the following: * An attempt to install A with B already present refuses to install A without a warning and confirmation * An attempt to install B informs the user of the conflict, and optionally offers to uninstall A In this way, any collateral damage to B is avoided, while still making the intended "lack of support" declaration clear. How does that sound? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
