[Python-Dev] Patch 1644818: Allow importing built-in submodules
Hi list, A month and a half ago, I submitted patch 1644818 to the CPython Patch Tracker: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1644818&group_id=5470 On several occassions I have been advised to mention the patch in this list, so here it is: The problem: Importing built-in submodule extensions (e.g. PyQt4.QtCore and PyQt4.QtGui in my case) does not work. By built-in I mean statically linked against the Python executable. The cause: find_module in import.c only checks the module name (e.g. QtCore) against the built-in list, which should contain the full name (e.g. PyQt4.QtCore) instead. Also, the above check is performed after the code to check if the parent module is frozen, which would have already exited in that case. Solution: By moving the "is_builtin()" check to earlier in find_module and using the "fullname" variable instead of "name", I can build PyQt4.QtCoreand PyQt4.QtGui into the interpreter and import and use them with no problem whatsoever, even if their parent module (PyQt4) is frozen. When I submitted the patch, I ran the regression tests and everything seemed Ok. Also, the patch is for Python-2.5, but it a month and a half ago it was applicable to the svn trunk with only a one-line offset. As I am completely new to CPython development, perhaps this problem has already been discussed and/or fixed I may have done something incorrectly. Please let me know if that is the case. Regards, Miguel Lobo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
> As I am completely new to CPython development, perhaps this problem has > already been discussed and/or fixed I may have done something > incorrectly. Please let me know if that is the case. I looked at it briefly. If I understand correctly, the proposed feature is fine, but lacks a test case. I have now added a test case. Please let me know if anything else is needed. Thanks, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
Hi list, Sorry for the repetition, but since nobody has commented on patch 1644818 for about a week I thought perhaps I should ask again. Is there anything I need to do before the patch is ready for inclusion? As a remainder, this patch (which can be seen at https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1644818&group_id=5470 ) fixes built-in submodules. At the moment importing built-in submodules does not work whether the parent submodule is itself built-in, a normal module or a frozen module. This small patch fixes all three cases and has been tested to work under Windows. This is useful for example with the PyQt project, which provides a package called PyQt4 containing a number of extension modules (PyQt4.QtCore, PyQt4.QtGui, etc). It should be possible to create a bespoke Python interpreter that has these extensions statically linked, but without this patch importing these extensions fails in such an interpreter. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
No; basically, you just need to be patient now. Do you have an urgent need to get this patch included? Well, not personally, but I suspect that it is quite possible that other people will waste time trying to figure out why their imports don't work. Also, seeing that there are almost 400 open patches in the patch tracker, I'm concerned that this patch will be forgotten and left to rot, which would be a pity IMO. One issue is whether this patch should be backported to the 2.5 branch; as it is arguably a new feature, it probably should not. As 2.6 is still quite some time ahead, I can't see any urgency. I see this more as a fix rather than a new feature, as there is no reason or explanation in the documentation (that I've been able to find) to suggest that importing built-in submodules shouldn't work. The small changes to the code in the patch also "feel" more like a fix than like a new feature. So yes, ideally I would like to see this patch in 2.5.1, although I will of course defer to the people who are in charge of making that decision. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
Personally, I can't apply it as-is right now, since a) I would have to check that the test case conditionalization works fine, and b) I would have to come up with a patch for the Windows build process. Sorry, I couldn't understand the second point. Why would you have to patch the Windows build process? If you want to prioritize it over the other 300 patches, I can offer you a fast-track procedure if you in turn review 5 other patches. Well, it's not really my place to ask that this patch is prioritised above others, since as I said if I'm asking for its inclusion it is in order to benefit other Python users who may easily hit the same problem. As far as I'm concerned personally, I have no problem at all applying the patch manually. Anyway, I'm intrigued about this "review 5 other patches" procedure you suggest. What exactly would be involved in such a review? Please note that I hadn't touched CPython code before I wrote my patch and I haven't been following CPython development closely. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
This is how we suck you in... ;) I see :-). Funny I didn't see this procedure mentioned in the patch submission guidelines ;-) You don't have to be an expert to review patches. The following procedure would qualify you: 1. Find a patch that it appears no one has ever touched (0 comments, assigned to nobody, etc.) 2. Pretty much every patch should include a unit test and documentation. If something is missing from the patch you're looking at, post a comment that says "Incomplete, no docs/tests". My own patch does not include documentation. I assume documentation would only be needed for patches that add new functionality (as opposed to fixing problems)? 3. Repeat until you've commented on five patches. If you find such clerical work beneath you, you can go further--build Python from source, apply patches, and verify that they work. It's not hard (google "python developer faq"). But it's not required. I might try to do this, as time permits. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
I need to integrate the extra test file into a project file (probably pythoncore). The change to pythoncore.vcproj is already in the patch I posted. Otherwise I wouldn't have been able to run my test under Windows. Yet, the same can be said for most other patches: they are all for the benefit of users running into the same respective problems. Agreed. What I mean is that this fasttrack system where the submitter has to do some extra work seems to imply that accepting the patch somehow benefits the submitter. In fact I'm probably the person the patch will benefit least, because I have already run into the problem and know how to solve it. I feel responsible for defending the patch since I've written it and I know the problem it fixes and my solution better than anybody else, but I don't see how that responsibility extends to having to do extra unrelated work to have the patch accepted. I'm not complaining or anything, and no offence meant to anyone, just explaining my point of view. I might still try to do the 5 patch review thing, depending on how long it takes me. But if I choose not to do so, leaving my patch to rot only harms CPython, not me. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
It won't benefit the Python core either, because we just don't use builtin submodules. In fact, I find the notion of builtin submodules somewhat strange. Please excuse my curiosity, but why do you find it strange? P.S. Thanks to all for the considerate responses. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
Normally, the builtin modules are the ones that are shipped in Python core. I know you can get a bigger builtins list through freeze, or through a custom Setup.local, but it is fairly uncommon to do that. Also, having extension modules in a namespace is something that I would normally not do. I couldn't tell off-hand whether having extension modules in a package would even work (but apparently, it does), Well, it quite doesn't, without the patch ;-) so I would make them global modules just to be safe. Also, they normally have a canonical naming: if they wrap a library called libfoo, the extension module would normally be called foo.so or foomodule.so (on a Unix system; foo.pyd on Windows). As DLLs don't have hierarchical names, extension modules don't need hierarchical names, either. Perhaps one example would help to clarify what I mean. I see that there is an xml.parsers.expat module, with the following content: """Interface to the Expat non-validating XML parser.""" __version__ = '$Revision: 17640 $' from pyexpat import * Then, there is a pyexpat.c module in Modules, which provides the contents for the aforementioned xml.parsers.expat. Wouldn't it be simpler and less invasive of the user's namespace if the Python module at xml.parsers.expat was removed, and pyexpat.c declared a module name of "xml.parsers.expat" instead? Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
For me, the personal benefit of getting a patch applied would be so that I didn't have to keep re-applying it to new versions of Python, and that I could distribute code relying on the patch to others without requiring *them* to use a patched version of Python as well. What you describe is probably fairly common, but in this particular case, the patch is only needed if you are going to build a bespoke Python interpreter. This is a complicated enough process that the difficulty of having to apply a patch is probably insignificant. The potential savings of this patch lay mainly in avoiding the waste of time for people who will face the same problem and not understand why. As anecdotal evidence, just a couple of days after I had figured out what the problem was and had the patch ready, another guy found the same problem completely independently and posted some questions to the PyQt development list about it. Regards, Miguel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reindenting the C code base?
> I think we should not do this. We should use 4 space indents for new > files, but existing files should not be reindented. If you reindent, > much of the history of the file is essentially lost -- "svn blame" > will blame whoever reindented the code, and it's a pain to go back. I believe "svn blame -x -w" ignores whitespace changes. -- Miguel Check out Gleam, an LGPL sound synthesizer library, at http://gleamsynth.sf.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
