Re: [Python-Dev] bytes-like objects
On 06/10/14 10:33, Georg Brandl wrote: > bytes-like object Howdy, two small comments: 1) just as a quick check, I did a Python search for exactly that phrase https://docs.python.org/3/search.html?q=bytes-like+object&check_keywords=yes&area=default with zero results. Maybe it would be a good thing if glossary entries were always found via the search page. 2) And about this glossary entry: "An object that supports the Buffer Protocol" - can I take that for granted, as a real definition, meaning """an object is bytes-like iff it supports the buffer protocol"""? cheers - Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://www.pydica.net/ 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing 2.7.x
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 06.10.14 20:55, Zachary Ware wrote: > On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily wrote: >> 3. security: "fixing issues exploitable by attackers such as crashes, >> privilege escalation and, optionally, other issues such as denial of >> service attacks. Any other changes are not considered a security risk >> and thus not backported to a security branch." >>= 3.2.x and 3.3.x > > 3.1 is still in this category, is it not? According to PEP 375, it's > a few months past due for its last release. > > http://legacy.python.org/dev/peps/pep-0375/#maintenance-releases > I don't think that the rules should be implicitly considered compatible between the 2.X and 3.X series. Python 2.X has a history that extends to X==6, X==5 and even X==4, as a really conservative POV with an extent over more than 10 years. I believe, such a thing does not exist for the Python 3.X series at all. My impression is that no 3.X user ever would want to stick with any older version. Is that true, or am I totally wrong? cheers -- Chris - -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://www.pydica.net/ 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUMur7AAoJEOcwEVD7e+4OW0UIAJk2ZTFX3OjBrt1G0RZc9nPb hHVxGJNXNeBellM9BpmoW9t9hgk94lAIgmh5hop5uMt32o9CH47s97rKw7K1ekl5 sML/4hl5/BLRiHXgwSB1ZltqZrvG/xsE6AE1v37BcPf/X3T4UfPhW30z+43eaBJw Q3b21EwxxUJGJ//GWwi2+buCfkfRuePBIB4MQiMm3/JI9h03EPbRoQ0/53huKLeW I7oAemVzprQHw7coaTf6EOHFTlmUfHvm5K9ywpabX10/Ediz1suJfPMPdzUigHG3 G3ABMKAA1YockpfIDU/7rpmDcliblpjU5MT4BsZycuYXOyUesV6uDaLMOdO5fEY= =ZuoT -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] shebang policy, and pip
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Howdy, this question is a bit about general policy which is not yet covered in the python recommendations: I see projects which do check-ins like "get rid of shebang lines" and they remove those lines from non-script sources. It is not always clear to me what to do, so I tend to leave those lines in per default, in order not to waste time thinking about it, but well, today I was confronted with that. Digging a bit deeper shows the following: python docs: No mention of shebang, but for Windows. https://docs.python.org/3/search.html?q=shebang&check_keywords=yes&area=default https://docs.python.org/3/using/windows.html?highlight=shebang Google's python style guide also says when a shebang is needed, but does not forbid it. Pep 394 explains how to use shebang, but still nothing about not using it. http://legacy.python.org/dev/peps/pep-0394/ So is there anything officially preferred, and should that go into pep 8? Special case with pip - -- I was looking through my installed packages and wondered quite much about pip: Pip has a shebang in the __init__ file, but no shebang in the __main__ file. I guess this is wrong and should be in the executable file, which is __main__ . cheers - Chris - -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://www.pydica.net/ 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUNQ8FAAoJEOcwEVD7e+4Of5MH/13YyXRzPVXfsLDbfe/CRkFF ORVAPkRB90OaEmbLTvBXPhFlAgEwcQnpdO+tmqigvlORd0cSXQKIxoPOiqq601gs XV56aREqBCT26XMaKTuoPdu4DaW+TkwyWSn70eq4U/P7YjF3ZlNt8IkA5mteM7an ycRYMnknEaIvP/xpZdGp+v4pq5LA42LWAY1awnBk4eMP04uDowSmcuLELpZrmSCr iMkw6wPUdZxGVtQNwSses0mh3DuaQuwrubhHMnLoOKn/lqRjckG2Ii2BIHlWy9lQ 5X3y8IdWPh7awio8xaibNqsWaP+0DT97g2H7QZf8YIG7/DkHL/iacSr7NAPBXXQ= =IODc -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] shebang policy, and pip
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 08.10.14 14:20, Donald Stufft wrote: > >> On Oct 8, 2014, at 6:16 AM, Christian Tismer wrote: >> >> >> ... >> >> So is there anything officially preferred, and should that go into pep 8? > > Some editors can use shebang lines to control syntax highlighting or linting > (mine for example will lint different for python2 vs python3 shebangs). > Interesting use case, of course! At first sight, at least ;-) But it becomes relatively awkward when virtualenv is used: There is no textual distinction between python 2 and 3, and the text editor would have to do more analysis to get things right than to guess from the shebang line. Which then makes it useless, and the editor would need to grab the real python interpreter to find things out. cheers - Chris - -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://www.pydica.net/ 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUNZV1AAoJEOcwEVD7e+4OlpcIAN3556PP4sBlFeDDPn4bq3fO ScXRhB8tPkS3Ftgyux2swE91yxOZFSV8Zyae04wTB+GlzKZ6v9aXMKn5mt+b6sIr aq1vfPxG7Qoz3q7zhaAfps8ErD9f1PriC7/4F2P4FOOko07eHt6/e8Sxg4qAgMPz nADLj8/2MtvQYFiw3m4Zsqs31wjCTTICH52FnRgla9+u5IhStdQE7OFTiHZaPNK3 tsUohUoKqhMPIhwuB669JE7rwcRB9dA5Iatiy3uU0wqCYkekT3I4DwCtAS+DZkJX Fe0wpLGGHOprwUTQu0SMFGQRxPX3HxL0RzTNLKjCcDNLDWRcwZRPOZ3K4DVoggQ= =i5dg -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unittest isolation and warnings
Hi guys,
when writing tests, I suddenly discovered that unittest
is not isolated to warnings.
Example:
One of my tests emits warnings when a certain condition is
met. Instead of reporting the error immediately, it uses
warnings, and at the end of the test, an error is produced
if there were warnings.
if hasattr(__main__, "__warningregistry__"):
raise RuntimeError("There are errors, see above.")
By chance, I discovered that an error was suddenly triggered without
a warning. That must mean the warning existed already from
another test as a left-over.
My question:
Is that known, and is that intended?
To what extent are the test cases isolated from each other?
I do admit that my usage of warnings is somewhat special.
But it is very convenient to report many errors on remote servers.
Cheers -- Chris
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest isolation and warnings
Thanks a lot!
Good to know.
Ciao -- Chris
On 17.11.17 16:44, Brett Cannon wrote:
> Tests are not isolated from the warnings system, so things will leak
> out. Your best option is to use the context manager in the warnings
> module to temporarily make all warnings raise exceptions and test for
> the exception (I'm at the airport, hence why I don't know the name of
> the context manager; the warnings module docs actually have a sample on
> how best to write tests the involve warnings).
>
>
> On Fri, Nov 17, 2017, 01:34 Christian Tismer, <mailto:tis...@stackless.com>> wrote:
>
> Hi guys,
>
> when writing tests, I suddenly discovered that unittest
> is not isolated to warnings.
>
> Example:
> One of my tests emits warnings when a certain condition is
> met. Instead of reporting the error immediately, it uses
> warnings, and at the end of the test, an error is produced
> if there were warnings.
>
> if hasattr(__main__, "__warningregistry__"):
> raise RuntimeError("There are errors, see above.")
>
> By chance, I discovered that an error was suddenly triggered without
> a warning. That must mean the warning existed already from
> another test as a left-over.
>
> My question:
> Is that known, and is that intended?
> To what extent are the test cases isolated from each other?
>
> I do admit that my usage of warnings is somewhat special.
> But it is very convenient to report many errors on remote servers.
>
> Cheers -- Chris
>
> --
> Christian Tismer :^) tis...@stackless.com
> <mailto:tis...@stackless.com>
> Software Consulting : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : https://github.com/PySide
> 14482 Potsdam : GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776 fax +49 (30) 700143-0023
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org <mailto:Python-Dev@python.org>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] subprocess not escaping "^" on Windows
Hi Guys,
yes I know there was a lengthy thread on python-dev in 2014
called "subprocess shell=True on Windows doesn't escape ^ character".
But in the end, I still don't understand why subprocess does
escape the double quote when shell=True but not other special
characters like "^"?
Yes I know that certain characters are escaped under certain
Windows versions and others are not. And it is not trivial to make
that work correctly in all cases. But I think if we support
some escaping at all, then we should also support all special
cases. Or what sense should an escape make if it works sometimes
and sometimes not?
The user would have to know which cases work and which not. But
I thought we want to remove exactly that burden from him?
-
As a side note: In most cases where shell=True is found, people
seem to need evaluation of the PATH variable. To my understanding,
>>> from subprocess import call
>>> call(("ls",))
works in Linux, but (with dir) not in Windows. But that is misleading
because "dir" is a builtin command but "ls" is not. The same holds for
"del" (Windows) and "rm" (Linux).
So I thought that using shell=True was a good Thing on windows,
but actually it is the start of all evil.
Using regular commands like "git" works fine on Windows and Linux
without the shell=True parameter.
Perhaps it would be a good thing to emulate the builtin programs
in python by some shell=True replacement (emulate_shell=True?)
to match the normal user expectations without using the shell?
Cheers - Chris
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess not escaping "^" on Windows
That is true.
list2cmdline escapes partially, but on NT and Windows10, the "^" must
also be escaped, but is not. The "|" pipe symbol must also be escaped
by "^", as many others as well.
The effect was that passing a rexexp as parameter to a windows program
gave me strange effects, and I recognized that "^" was missing.
So I was asking for a coherent solution:
Escape things completely or omit "shell=True".
Yes, there is a list of chars to escape, and it is Windows version
dependent. I can provide it if it makes sense.
Cheers -- Chris
On 07.01.18 18:20, Guido van Rossum wrote:
> I assume you're talking about list2cmdline()? That seems to be used to
> construct a string that can be passed to `cmd /c "{}"` -- it gets
> substituted instead of the {}, i.e. surrounded by ". I honestly can't
> say I follow that code completely, but I see that it escapes double
> quotes. Why is there a need to escape other characters? Is there a
> definitive list of special characters somewhere?
>
> On Sun, Jan 7, 2018 at 8:17 AM, Christian Tismer <mailto:tis...@stackless.com>> wrote:
>
> Hi Guys,
>
> yes I know there was a lengthy thread on python-dev in 2014
> called "subprocess shell=True on Windows doesn't escape ^ character".
>
> But in the end, I still don't understand why subprocess does
> escape the double quote when shell=True but not other special
> characters like "^"?
>
> Yes I know that certain characters are escaped under certain
> Windows versions and others are not. And it is not trivial to make
> that work correctly in all cases. But I think if we support
> some escaping at all, then we should also support all special
> cases. Or what sense should an escape make if it works sometimes
> and sometimes not?
>
> The user would have to know which cases work and which not. But
> I thought we want to remove exactly that burden from him?
>
> -
>
> As a side note: In most cases where shell=True is found, people
> seem to need evaluation of the PATH variable. To my understanding,
>
> >>> from subprocess import call
> >>> call(("ls",))
>
> works in Linux, but (with dir) not in Windows. But that is misleading
> because "dir" is a builtin command but "ls" is not. The same holds for
> "del" (Windows) and "rm" (Linux).
>
> So I thought that using shell=True was a good Thing on windows,
> but actually it is the start of all evil.
> Using regular commands like "git" works fine on Windows and Linux
> without the shell=True parameter.
>
> Perhaps it would be a good thing to emulate the builtin programs
> in python by some shell=True replacement (emulate_shell=True?)
> to match the normal user expectations without using the shell?
>
> Cheers - Chris
>
> --
> Christian Tismer-Sperling :^) tis...@stackless.com
> <mailto:tis...@stackless.com>
> Software Consulting : http://www.stackless.com/
> Karl-Liebknecht-Str <http://www.stackless.com/ Karl-Liebknecht-Str>.
> 121 : https://github.com/PySide
> 14482 Potsdam : GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776 fax +49
> (30) 700143-0023
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org <mailto:Python-Dev@python.org>
> https://mail.python.org/mailman/listinfo/python-dev
> <https://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> <https://mail.python.org/mailman/options/python-dev/guido%40python.org>
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess not escaping "^" on Windows
By "normal user expectations" I meant the behavior when the builtin commands
were normal programs.
Using "shell=True" is everywhere recommended to avoid, and I believe
we could avoid it by giving them replacements for build-ins.
But I don't care if the shell escaping is correct. And that is not
trivial, either.
On 07.01.18 18:22, Guido van Rossum wrote:
> On Sun, Jan 7, 2018 at 8:17 AM, Christian Tismer <mailto:tis...@stackless.com>> wrote:
>
> As a side note: In most cases where shell=True is found, people
> seem to need evaluation of the PATH variable. To my understanding,
>
> >>> from subprocess import call
> >>> call(("ls",))
>
> works in Linux, but (with dir) not in Windows. But that is misleading
> because "dir" is a builtin command but "ls" is not. The same holds for
> "del" (Windows) and "rm" (Linux).
>
> So I thought that using shell=True was a good Thing on windows,
> but actually it is the start of all evil.
> Using regular commands like "git" works fine on Windows and Linux
> without the shell=True parameter.
>
> Perhaps it would be a good thing to emulate the builtin programs
> in python by some shell=True replacement (emulate_shell=True?)
> to match the normal user expectations without using the shell?
>
>
> That feels like a terrible idea to me. How do you define "normal user
> expectations" here? If people want shell builtins they should just use
> shell=True. (Also note IIUC there are several quite different shells
> commonly used on Windows, e.g. PowerShell.)
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess not escaping "^" on Windows
Ok, I thought only about Windows where people often use shell=True.
I did not see that as a Linux problem, too.
Not meant as a proposal, just loud thinking... :-)
But as said, the incomplete escaping is a complete mess.
Ciao -- Chris
On 07.01.18 19:54, Christian Tismer wrote:
> By "normal user expectations" I meant the behavior when the builtin commands
> were normal programs.
>
> Using "shell=True" is everywhere recommended to avoid, and I believe
> we could avoid it by giving them replacements for build-ins.
>
> But I don't care if the shell escaping is correct. And that is not
> trivial, either.
>
> On 07.01.18 18:22, Guido van Rossum wrote:
>> On Sun, Jan 7, 2018 at 8:17 AM, Christian Tismer > <mailto:tis...@stackless.com>> wrote:
>>
>> As a side note: In most cases where shell=True is found, people
>> seem to need evaluation of the PATH variable. To my understanding,
>>
>> >>> from subprocess import call
>> >>> call(("ls",))
>>
>> works in Linux, but (with dir) not in Windows. But that is misleading
>> because "dir" is a builtin command but "ls" is not. The same holds for
>> "del" (Windows) and "rm" (Linux).
>>
>> So I thought that using shell=True was a good Thing on windows,
>> but actually it is the start of all evil.
>> Using regular commands like "git" works fine on Windows and Linux
>> without the shell=True parameter.
>>
>> Perhaps it would be a good thing to emulate the builtin programs
>> in python by some shell=True replacement (emulate_shell=True?)
>> to match the normal user expectations without using the shell?
>>
>>
>> That feels like a terrible idea to me. How do you define "normal user
>> expectations" here? If people want shell builtins they should just use
>> shell=True. (Also note IIUC there are several quite different shells
>> commonly used on Windows, e.g. PowerShell.)
>>
>> --
>> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>
>
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess not escaping "^" on Windows
Ok, then I'm happy to improve the escaping! I was confused because I could not understand that nobody than me should have run into this problem before. There are many special cases. I'll try my very best :) Cheers -- Chris On 07.01.18 21:59, Guido van Rossum wrote: > On Sun, Jan 7, 2018 at 12:30 PM, Gregory P. Smith <mailto:g...@krypto.org>> wrote: > > the best way to improve shell escaping on windows is to send a PR > against the list2cmdline code that escapes everything you believe it > should when running on windows. With hyperlinks to the relevant msdn > info about what might need escaping. > > > Agreed. FWIW the call to list2cmdline seems to compound the problem, > since it just takes args and puts double quotes around it, mostly > undoing the work of list2cmdline. For example if I use (args=['a', 'b > c'], shell=True) I think list2cmdline turns that to args='a "b c"', and > then the format() expression constructs the command: > > cmd.exe /c "a "b c"" > > I really have no idea what that means on Windows (and no quick access to > a Windows box to try it) but on Windows that would create *two* > arguments, the first one being 'a b' and the second one 'c'. > > At this point I can understand that Christian recommends against > shell=True -- it's totally messed up! But the fix should really be to > fix this, not inventing a new feature. > > -- > --Guido van Rossum (python.org/~guido <http://python.org/~guido>) > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com > -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes
On 03.02.18 08:12, Steven D'Aprano wrote: > On Sat, Feb 03, 2018 at 12:50:11AM -0500, Terry Reedy wrote: >> On 2/2/2018 7:01 PM, asrp asrp wrote: >>> I don't know if this is the right place to post this. Please redirect as >>> needed. >> >> This list is for development *of* cpython. Development *with* python in >> general belongs on python-list. > > This list is for development of Python the language, not just CPython > the interpreter. It seems to me that announcing a new Python > interpreter, especially one designed for the purpose of allowing rapid > experimentation with the language, is on topic for this list. > > Well spoken! -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes
Hi user with no real name yet, On 03.02.18 01:01, asrp asrp wrote: > Hello, > > I don't know if this is the right place to post this. Please redirect as > needed. > > I've made a small Python interpreter in Python with runtime AST node > semantics and edit-and-continue. I thought it could make prototyping language > changes more easily and visualize usage before writing them in C. > > Its here: https://github.com/asrp/python_terp > > So, for example, redefining the for_stmt function in the right scope changes > the behaviour of future for loops at runtime. > > Although from discussion I've read in PEPs, actual implementation always look > like a non-issue (which seems like magic to me) so maybe no-one here actually > needs this. > > (I really needed edit-and-continue for one of my projects but of course, > running it in this extra interpreter is much too slow.) > > asrp In the readme to python_terp you say: """ python_terp is intended to make language modification to Python easier to preview changes more quickly and is not intended for full CPython compatibility. However, a large subset of Python is already included. In particular, enough to run the first stage of its parser. """ This needs clarification. What do you mean by subset? A real subset or also things that are different and will stay different? To what extent are you planning to stay compatible, and where do you plan to deviate? The reason that I'm asking is that by compatible I mean the compatibility of PyPy. If you can reach that, and be it just by a subset, then it makes sense to speak of Python. Cheers - Chris -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PyIndex_Check conflicts with PEP 384
Hi friends, when implementing the limited API for PySide2, I recognized a little bug where a function was replaced by a macro. The file number.rst on python 3.6 says """ .. c:function:: int PyIndex_Check(PyObject *o) Returns ``1`` if *o* is an index integer (has the nb_index slot of the tp_as_number structure filled in), and ``0`` otherwise. """ Without notice, this function was replaced by a macro a while ago, which reads #define PyIndex_Check(obj) \ ((obj)->ob_type->tp_as_number != NULL && \ (obj)->ob_type->tp_as_number->nb_index != NULL) This contradicts PEP 384, because there is no way for non-heaptype types to access the nb_index field. If nobody objects, I would like to submit a patch that adds the function back when the limited API is active. I think to fix that before Python 3.7 is out. Ciao -- Chris -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyIndex_Check conflicts with PEP 384
Hi Nate, I just did that, and I got some follow-up work, too, which is fine with me. Just as a note for you: Qt not itself, but PyQt5 did that already and submitted a stable ABI for Python 3.5 and up. I was challenged end of last December to try that, and I succeeded after a long struggle, because I also needed to convert all of PySide2 to using heaptypes, and that was really hard. Actually, I succeeded yesterday, after 5 months. And now I know all the subtle things that people need to know when converting existing types to heaptypes. Since QtC has adopted PySide2 in 2016, including myself as a consultant, now it is really a Qt product, and the limited API is due to my work. It comes naturally that I also should fix the problems which showed up during that process. I also think to submit a paper to python.org where I document all the subtle problems which occured during the conversion process. It looks simple, but it really is not. All the best -- Chris On 01.06.18 17:18, Nathaniel Smith wrote: > Indeed, that sounds like a pretty straightforward bug in the stable ABI. > You should file an issue on bugs.python.org <http://bugs.python.org> so > it doesn't get lost (and if it's the main new stable ABI break in 3.7 > then you should probably mark that bug as a release blocker so that Ned > notices it). > > Unfortunately, very few people use the stable ABI currently, so it's > easy for things like this to get missed. Hopefully it Qt starts using it > then that will help us shake these things out... But it also means we > need your help to catch these kinds of issues :-). Thanks! > > On Fri, Jun 1, 2018, 06:51 Christian Tismer <mailto:tis...@stackless.com>> wrote: > > Hi friends, > > when implementing the limited API for PySide2, I recognized > a little bug where a function was replaced by a macro. > > The file number.rst on python 3.6 says > > """ > .. c:function:: int PyIndex_Check(PyObject *o) > > Returns ``1`` if *o* is an index integer (has the nb_index slot > of the > tp_as_number structure filled in), and ``0`` otherwise. > """ > > Without notice, this function was replaced by a macro a while > ago, which reads > > #define PyIndex_Check(obj) \ > ((obj)->ob_type->tp_as_number != NULL && \ > (obj)->ob_type->tp_as_number->nb_index != NULL) > > This contradicts PEP 384, because there is no way for non-heaptype > types to access the nb_index field. > > If nobody objects, I would like to submit a patch that adds the > function back when the limited API is active. > I think to fix that before Python 3.7 is out. > > Ciao -- Chris > > -- > Christian Tismer-Sperling :^) tis...@stackless.com > <mailto:tis...@stackless.com> > Software Consulting : http://www.stackless.com/ > Karl-Liebknecht-Str. 121 : http://pyside.org > 14482 Potsdam : GPG key -> 0xFB7BEE0E > phone +49 173 24 18 776 fax +49 (30) 700143-0023 > > ___ > Python-Dev mailing list > Python-Dev@python.org <mailto:Python-Dev@python.org> > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/njs%40pobox.com > -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable ABI
On 02.06.18 05:47, Nick Coghlan wrote: > On 2 June 2018 at 03:45, Jeroen Demeyer <mailto:j.deme...@ugent.be>> wrote: > > On 2018-06-01 17:18, Nathaniel Smith wrote: > > Unfortunately, very few people use the stable ABI currently, so it's > easy for things like this to get missed. > > > So there are no tests for the stable ABI in Python? > > > Unfortunately not. > > https://bugs.python.org/issue21142 is an old issue suggesting automating > those checks (so we don't inadvertently add or remove symbols for > previously published stable ABI definitions), but it's not yet made it > to the state of being sufficiently well automated that it can be a > release checklist item in PEP 101. > > Cheers, > Nick. Actually, I think we don't need such a test any more, or we could use this one as a heuristic test: I have written a script that scans all relevant header files and analyses all sections which are reachable in the limited API context. All macros that don't begin with an underscore which contain a "->tp_" string are the locations which will break. I found exactly 7 locations where this is the case. My PR will contain the 7 fixes plus the analysis script to go into tools. Preparind that in the evening. cheers -- Chris -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable ABI
On 03.06.18 13:18, Ronald Oussoren wrote: > > >> On 3 Jun 2018, at 12:03, Christian Tismer wrote: ... >> >> I have written a script that scans all relevant header files >> and analyses all sections which are reachable in the limited API >> context. >> All macros that don't begin with an underscore which contain >> a "->tp_" string are the locations which will break. >> >> I found exactly 7 locations where this is the case. >> >> My PR will contain the 7 fixes plus the analysis script >> to go into tools. Preparind that in the evening. > > Having tests would still be nice to detect changes to the stable ABI when > they are made. > > Writing those tests is quite some work though, especially if those at least > smoke test the limited ABI by compiling snippets the use all symbols that > should be exposed by the limited ABI. Writing those tests should be fairly > simple for someone that knows how to write C extensions, but is some work. > > Writing a tests that complain when the headers expose symbols that shouldn’t > be exposed is harder, due to the need to parse headers (either by hacking > something together using regular expressions, or by using tools like gccxml > or clang’s C API). What do you mean? My script does that with all "tp_*" type fields. What else would you want to check? -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PySequence_Check but no __len__
Hi friends, there is a case in the Python API where I am not sure what to do: If an object defines __getitem__() only but no __len__(), then PySequence_Check() already is true and does not care. So if I define no __len__, it simply fails. Is this intended? I was mislead and thought this was the unlimited case, but it seems still to be true that sequences are always finite. Can someone please enlighten me? -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PySequence_Check but no __len__
Hi Brett,
because you did not understand me, I must have had a fundamental
misunderstanding. So I started a self-analysis and came to the
conclusion that this was my error since maybe a decade:
When iterators and generators came into existence, I somehow
fell into the trap to think that there are now sequences with
undetermined or infinite length. They would be exactly those sequences
which have no __len__ attribute.
I understand now that sequences are always of fixed length
and adjusted myself.
-
My problem is to find out how to deal with a class which has
__getitem__ but no __len__.
The documentation suggests that the length of a sequence can always
be obtained by len().
https://docs.python.org/3/reference/datamodel.html
But the existence of __len__ is not guaranteed or enforced.
And if you look at the definition of PySequence_Fast(), you find that
a sequence can be turned into a list with iteration only and no __len__.
So, is a sequence valid without __len__, if iteration is supported,
instead?
There is the whole chapter about sequence protocol
https://docs.python.org/3/c-api/sequence.html?highlight=sequence
but I cannot find out an exact definition what makes up a sequence?
Sorry if I'm again the only one who misunderstands the obvious :)
Best -- Chris
On 21.06.18 18:29, Brett Cannon wrote:
> Sorry, I don't quite follow.
>
> On Thu, 21 Jun 2018 at 08:50 Christian Tismer <mailto:tis...@stackless.com>> wrote:
>
> Hi friends,
>
> there is a case in the Python API where I am not sure what to do:
>
> If an object defines __getitem__() only but no __len__(),
> then PySequence_Check() already is true and does not care.
>
>
> Which matches
> https://docs.python.org/3/c-api/sequence.html#c.PySequence_Check .
>
> From Objects/abstract.c:
>
> int
> PySequence_Check(PyObject *s)
> {
> if (PyDict_Check(s))
> return 0;
> return s != NULL && s->ob_type->tp_as_sequence &&
> s->ob_type->tp_as_sequence->sq_item != NULL;
> }
>
>
>
>
> So if I define no __len__, it simply fails. Is this intended?
>
>
> What is "it" in this case that is failing? It isn't PySequence_Check()
> so I'm not sure what the issue is.
>
> -Brett
>
>
>
> I was mislead and thought this was the unlimited case, but
> it seems still to be true that sequences are always finite.
>
> Can someone please enlighten me?
> --
> Christian Tismer-Sperling :^) tis...@stackless.com
> <mailto:tis...@stackless.com>
> Software Consulting : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : http://pyside.org
> 14482 Potsdam : GPG key -> 0xE7301150FB7BEE0E
> phone +49 173 24 18 776 fax +49 (30)
> 700143-0023
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org <mailto:Python-Dev@python.org>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
>
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : http://pyside.org
14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PySequence_Check but no __len__
Answering myself:
PySequence_Check determines a sequence. See the docs.
len() can but does not have to exist.
The size is always limited.
After evicting my initial fault, this is now obvious.
Sorry about the noise.
On 22.06.18 13:17, Christian Tismer wrote:
> Hi Brett,
>
> because you did not understand me, I must have had a fundamental
> misunderstanding. So I started a self-analysis and came to the
> conclusion that this was my error since maybe a decade:
>
> When iterators and generators came into existence, I somehow
> fell into the trap to think that there are now sequences with
> undetermined or infinite length. They would be exactly those sequences
> which have no __len__ attribute.
>
> I understand now that sequences are always of fixed length
> and adjusted myself.
>
> -
>
> My problem is to find out how to deal with a class which has
> __getitem__ but no __len__.
>
> The documentation suggests that the length of a sequence can always
> be obtained by len().
> https://docs.python.org/3/reference/datamodel.html
>
> But the existence of __len__ is not guaranteed or enforced.
> And if you look at the definition of PySequence_Fast(), you find that
> a sequence can be turned into a list with iteration only and no __len__.
>
> So, is a sequence valid without __len__, if iteration is supported,
> instead?
>
> There is the whole chapter about sequence protocol
> https://docs.python.org/3/c-api/sequence.html?highlight=sequence
>
> but I cannot find out an exact definition what makes up a sequence?
>
> Sorry if I'm again the only one who misunderstands the obvious :)
>
> Best -- Chris
>
>
> On 21.06.18 18:29, Brett Cannon wrote:
>> Sorry, I don't quite follow.
>>
>> On Thu, 21 Jun 2018 at 08:50 Christian Tismer > <mailto:tis...@stackless.com>> wrote:
>>
>> Hi friends,
>>
>> there is a case in the Python API where I am not sure what to do:
>>
>> If an object defines __getitem__() only but no __len__(),
>> then PySequence_Check() already is true and does not care.
>>
>>
>> Which matches
>> https://docs.python.org/3/c-api/sequence.html#c.PySequence_Check .
>>
>> From Objects/abstract.c:
>>
>> int
>> PySequence_Check(PyObject *s)
>> {
>> if (PyDict_Check(s))
>> return 0;
>> return s != NULL && s->ob_type->tp_as_sequence &&
>> s->ob_type->tp_as_sequence->sq_item != NULL;
>> }
>>
>>
>>
>>
>> So if I define no __len__, it simply fails. Is this intended?
>>
>>
>> What is "it" in this case that is failing? It isn't PySequence_Check()
>> so I'm not sure what the issue is.
>>
>> -Brett
>>
>>
>>
>> I was mislead and thought this was the unlimited case, but
>> it seems still to be true that sequences are always finite.
>>
>> Can someone please enlighten me?
>> --
>> Christian Tismer-Sperling :^) tis...@stackless.com
>> <mailto:tis...@stackless.com>
>> Software Consulting : http://www.stackless.com/
>> Karl-Liebknecht-Str. 121 : http://pyside.org
>> 14482 Potsdam : GPG key -> 0xE7301150FB7BEE0E
>> phone +49 173 24 18 776 fax +49 (30)
>> 700143-0023
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org <mailto:Python-Dev@python.org>
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
>>
>
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
>
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : http://pyside.org
14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting the 3.5+ Windows build project files to 2.7
Hi Zack, On 23.06.15 06:42, Zachary Ware wrote: > On Mon, Jun 22, 2015 at 10:37 PM, Nick Coghlan wrote: >> I'd suggest explicitly reaching out to the Stackless folks to get >> their feedback. As I believe the switched to a newer compiler and VC >> runtime for Windows a while back, I suspect it will make their lives >> easier rather than harder, but it's still worth asking for their >> input. > From a glance at the stackless repo, it looks like it still uses > VS2008's project files (which pretty much means using MSVC 9), but I > could have easily missed something. > > Christian, what say you? Would having the project files from 3.5 > backported to 2.7 (but still using MSVC 9) be positive, negative, or > indifferent for Stackless? > I am very positive about your efforts. We wanted to create a Stackless 2.8 version end of 2013 with support for newer compilers, to begin with. After a while, we stopped this, and I left the branch in a private, unmaintained repository. https://bitbucket.org/stackless-dev/stackless-private/branch/2.8-slp Maybe you can make use of this, use it as you like it. You will get an invitation. cheers - Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://www.pydica.net/ 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Migrating to Python 3: the python 3 install issue
Great, that this finally happens. I think this was a silent revolution, initiated by nagging people, distros and larger companies about how mega-out Python2 is, until they finally started to believe it ;-) cheers -- Chris [since 2012 on Py3, charging an extra for back-porting] On 03/10/15 01:05, Victor Stinner wrote: > (grr, again i sent a draft by mistake, sorry about that) > > Fedora 23 (scheduled for the end of this month) will only come with > python3 (/usr/bin/python3), no python2 (nor python), *in the base > system*. Obviously, it will be possible to install Python 2 to install > applications not compatible with Python 3 yet. > > https://fedoraproject.org/wiki/Releases/23/ChangeSet#Python_3_as_the_Default_Implementation > https://twitter.com/_solotraveller/status/645559393627435008 > > Ubuntu is also working on a similar change. I don't know when it will happen. > > Victor > > 2015-10-03 0:55 GMT+02:00 Brett Cannon : >> Thanks for the info, Terry! Glad people are realizing that Python 3 is now >> available widely enough that applications can seriously consider dropping >> Python 2 support now. I still think 2016 is going to see this happen more >> and more once the Linux distros make their switches to Python 3. >> >> On Fri, 2 Oct 2015 at 15:16 Terry Reedy wrote: >>> >>> On python-list, Chris Warrick reported (thread title): >>> "The Nikola project is deprecating Python 2.7 (+2.x/3.x user survey >>> results)" This is for the November release, with 2.7 dropped in the >>> next version next year. (Nikola is a cross-platform unicode-based app >>> for building static websites and blogs from user-written templates and >>> (marked-up) text files. https://getnikola.com/ ) >>> >>> Since users do not write code to use Nikola, the survey was about >>> installation of Python 3. At present, 1/2 have 3.x only, 1/3 2.x only, >>> and 1/6 both. (So much for 'nobody uses 3.x for real work'.) Most of >>> the 2.x only people are able and willing to install 3.x. >>> >>> https://getnikola.com/blog/env-survey-results-and-the-future-of-python-27.html >>> >>> When Stefan Behnel asked why they did not drop the hard-to-maintain 2.7 >>> version once they ported to 3.3, Chris answered >>> >>> > We did it now because it all started with frustration with 2.7 [0]. >>> > Also, doing it back in 2012/2013 would be problematic, because back >>> > then not all Linux distros had an easily installable Python 3 stack >>> > (and RHEL 7 still doesn’t have one in the default repos) >>> > >>> > [0]: >>> http://ralsina.me/weblog/posts/floss-decision-making-in-action.html >>> >>> -- >>> Terry Jan Reedy >>> >>> >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> https://mail.python.org/mailman/options/python-dev/brett%40python.org >> >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com >> > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com > -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The pysandbox project is broken
On 13/11/13 00:49, Josiah Carlson wrote: Python-dev is for the development of the Python core language, the CPython runtime, and libraries. Your sandbox, despite using and requiring deep knowledge of the runtime, is not developing those things. If you had a series of requests for the language or runtime that would make your job easier, then your thread would be on-topic. I think you should consider to re-define you perception of the purpose of the python-dev list. Simple feature-requests is not everything. Instead, this list also touches the general direction where python should go, and discusses the current hard-to-solve problems. The sand-boxing feature via rexec, bastion etc. was perceived as a useful, quite safe thing, until it was proven to be completely broken (Samuele Pedroni et. at., 2003 I think). After that, CPython simply removed those features and failed completely to provide a better solution. I appreciate very much that Victor tried his best to fill that old gap. And after that breakage happened again, I think it is urgent to have an in-depth discussion how that situation should be treated in the future. -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The pysandbox project is broken
On 16.11.13 01:35, Guido van Rossum wrote: On Fri, Nov 15, 2013 at 4:31 PM, Nick Coghlan <mailto:ncogh...@gmail.com>> wrote: "Use an OS level sandbox" *is* better from a security point of view. It's just not portable :P Honestly, I don't believe in portable security. :-) BTW, in case it wasn't clear, I think it was a courageous step by Victor to declare defeat. Negative results are also results, and they need to be published. Thanks Victor! Sure it was, and it was great to follow Victor's project! I was about to use it in production, until I saw it's flaws, a while back. Nevertheless, the issue has never been treated as much as to be able to say "this way you implement that security in Python", whatever "that" should be. So I think it is worth discussing, and may it just be to identify the levels of security involved, to help people to even identify their individual needs. My question is, actually: Do we need to address this topic, or is it already crystal clear that something like PyPy's approach is necessary and sufficient to solve the common, undefined problem of "run some script on whatnot, with the following security constraint"? IOW: Do we really need a full abstraction, embedded in a virtual OS, or is there already a compromise that suits 98 percent of the common needs? I think as a starter, categorizing the expectations of some measure of 'secure python' would make sense. And I'm asking the people with better knowledge of these matters than I have. (and not asking those who don't... ;-) ) cheers -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 0404 and VS 2010
Howdy friends, according to pep 404, there will never be an official Python 2.8. The migration path is from 2.7 to 3.x. I agree with this strategy in almost all consequences but this one: Many customers are forced to stick with Python 2.X because of other products, but they require a Python 2.X version which can be compiled using Visual Studio 2010 or better. This is considered an improvement and not a bug fix, where I disagree. So I see many Python 2.7 repositories on BB/GH which are hacked to support VS 2010, but they are not converted with the same quality in mind that fits our standards. My question --- I have created a very clean Python 2.7.6+ based CPython with the Stackless additions, that compiles with VS 2010, using the adapted project structure of Python 3.3.X, and I want to publish that on the Stackless website as the official "Stackless Python 2.8". If you consider Stackless as official ;-) . This compiler change is currently the only deviation from CPython 2.7, but we may support a few easy back-ports on customer demand. We don'd add any random stuff, of course. My question is if that is safe to do: Can I rely on PEP 404 that the "Python 2.8" namespace never will clash with CPython? And if not, what do you suggest then? It will be submitted by end of November, thanks for your quick responses! all the best -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
My question is not answered at all, sorry Joao! I did not ask a teacher for his opinion on Stackless, but the community about the validity of pep 404. I don't want a python 2.7 that does not install correctly, because people don't read instructions. And exactly that will happen if I submit a modified python 2.7 to PyPI. This is a topic on Stackless Python, and I am asking python-dev before I do it. But people know this has its limits. cheers - chris On 20/11/13 22:15, Joao S. O. Bueno wrote: I'd say publishing a high profile installable code with a "python 2.8" name would cause a lot of undesired confusion to start with. I usually lecture on Python to present the language to college students and I.T. workers - and explaining away the current versioning scheme (use either 2.7 or 3.3) is hard already. Having to add an explanation about a downloadable and installable Python 2.8 that would be incompatible with extensions compiled in Pypi would be tough. and I doubt it could even be done without making your project look bad on the process. Can't you just mark it as "visual studio 2010" version instead? js -><- On 20 November 2013 18:52, Christian Tismer <mailto:tis...@stackless.com>> wrote: Howdy friends, according to pep 404, there will never be an official Python 2.8. The migration path is from 2.7 to 3.x. I agree with this strategy in almost all consequences but this one: Many customers are forced to stick with Python 2.X because of other products, but they require a Python 2.X version which can be compiled using Visual Studio 2010 or better. This is considered an improvement and not a bug fix, where I disagree. So I see many Python 2.7 repositories on BB/GH which are hacked to support VS 2010, but they are not converted with the same quality in mind that fits our standards. My question --- I have created a very clean Python 2.7.6+ based CPython with the Stackless additions, that compiles with VS 2010, using the adapted project structure of Python 3.3.X, and I want to publish that on the Stackless website as the official "Stackless Python 2.8". If you consider Stackless as official ;-) . This compiler change is currently the only deviation from CPython 2.7, but we may support a few easy back-ports on customer demand. We don'd add any random stuff, of course. My question is if that is safe to do: Can I rely on PEP 404 that the "Python 2.8" namespace never will clash with CPython? And if not, what do you suggest then? It will be submitted by end of November, thanks for your quick responses! all the best -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com <mailto:tis...@stackless.com>> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org <mailto:Python-Dev@python.org> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
Hey Barry, On 20.11.13 23:30, Barry Warsaw wrote: On Nov 20, 2013, at 09:52 PM, Christian Tismer wrote: Many customers are forced to stick with Python 2.X because of other products, but they require a Python 2.X version which can be compiled using Visual Studio 2010 or better. This is considered an improvement and not a bug fix, where I disagree. I'm not so sure about that. Python 2.7 can still get patches to help extend its useful life by allowing it to be built with newer compiler suites. I believe this has already been done for various Linux compilers. I see no non-technical reason why Python 2.7 can't be taught how to build with VS 2010 or newer. Details are subject to RM approval, IMHO. I have created a very clean Python 2.7.6+ based CPython with the Stackless additions, that compiles with VS 2010, using the adapted project structure of Python 3.3.X, and I want to publish that on the Stackless website as the official "Stackless Python 2.8". If you consider Stackless as official ;-) . This compiler change is currently the only deviation from CPython 2.7, but we may support a few easy back-ports on customer demand. We don'd add any random stuff, of course. I think you're just going to confuse everyone if you call it "Stackless Python 2.8" and it will do more harm than good. Barry, that's a good thing! This way I have a chance to get my build in at all. And that's the question, after re-thinking: Where can I check my change in, if it is going to be accepted as a valid 2.7 bug fix (concerning VS 2008 as a bug, that is is)? I was intending to do this since a year but was stopped by MVL's influence. Maybe this needs to be re-thought, since I think different. What I do not know: Should I simply check this in to a python 2.7.x-VS2010 branch? Or what do you suggest, then? In any case, my question still stands, and I will do something with the Stackless branch by end of November. Please influence me ASAP, I don't intend to do any harm, but that problem is caused simply by my existence, and I want to stick with that for another few decades. If I think something must be done, then I have my way to do it. If you have good reasons to prevent this, then you should speak up in the next 10 days, or it will happen. Is that ok with you? Hugs -- your friend Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
Yes Paul, On 20.11.13 23:15, Paul Moore wrote: On 20 November 2013 22:04, Christian Tismer wrote: My question is not answered at all, sorry Joao! I did not ask a teacher for his opinion on Stackless, but the community about the validity of pep 404. I don't want a python 2.7 that does not install correctly, because people don't read instructions. And exactly that will happen if I submit a modified python 2.7 to PyPI. This is a topic on Stackless Python, and I am asking python-dev before I do it. But people know this has its limits. PEP 404 says there will be no Python 2.8. In my view, If you release something named (Stackless) Python 2.8, that seems to me to be pretty unfriendly given python-dev's clear intentions. Call it Stackless Python 2.7 plus for Visual Studio 2010 if you want, but using the version number 2.8 is bound to confuse at least some newcomers about the status of the Python 2.x line, and that's what PEP 404 was intended to avoid. I see what you intend, but I am not convinced what's best. Building a version that is numbered the same as existing versions, but binary incompatible is IMHO much more confusing that a version 2.8 which clearly says "if you are not 2.8, then you are not compatible". I am addressing Windows users, and they usually click a version, and if it doesn't work, they complain. The reason to go this way _was_ simplicity, moving to an impossible version, that justifies itself by that impossibility. They would have to install a whole series of packages, which they all would get from my site, and it works. And to repeat: Stackless Python is a slightly different, very compatible version, that has never got approval about being official or compatible. The reason that I am asking is to minimize the friction that I had to envision, anyway. It is your chance to minimize that friction by giving me good reasons to re-think my decision. But it will happen, soon, in the one or the other way. So please don't think I am begging for something - I am offering something, but it will happen. Best regards -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
On 21/11/13 19:59, Ethan Furman wrote: On 11/21/2013 10:53 AM, Christian Tismer wrote: So even if VS2010 exists only in the stackless branch, it is very likely to get used as CPython VS 2010, and I again have the naming problem ... What's wrong with calling it CPython VS 2010? And Stackless VS 2010? Sounds tempting in the first place, but then I need to change policy a bit. With the compiler switch, we always produced the identical Python version where Stackless is based upon. Well, this version string is not a problem. You mean I would stick with the latest CPython version numbers, produce maybe different dll names (maybe cpy27vs2010.dll and spy27vs2010.dll) which could even live together? Maybe I would generate a cpython and spython exe and support them both in the same distribution? What would happen with binary extensions on PyPI? Would they install and then fail, or is the distinction based upon the name "python"? If the latter is true, I think I could live with that, and build binary packages for all relevant extensions. At first sight I was about to complain, at second sight I am about to fall in the trap of being happy. I do not want to loose our relation to CPython and do any harm. But if there is such an easy solution, I might think to go for it. In the end, I want to go the path that MvL/Nick proposed. But as a quick solution for now, I would love to have an easy first solution. Please do not take me too seriously, I need to sleep over this to see if the idea stands my nightmares. cheers & thanks -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
On 22.11.13 00:53, Antoine Pitrou wrote: On Thu, 21 Nov 2013 18:43:37 -0500 Barry Warsaw wrote: On Nov 21, 2013, at 06:36 PM, Terry Reedy wrote: As usual, 'I am not a lawyer', but if Christian wants to push forward with using 'Python 2.8', I suggest that he consult the PSF Trademark Committee and lawyer first. Just to make clear, I'm definitely *not* suggesting this particular case ever escalate to such a level. Chris is our friend and a nice guy. :) I'm just saying that I think we should avoid any possible confusion on the part of ours (and Stackless's) users. And based on the discussions here, there are plenty of good alternatives. +1 from me :-) Thank you for that input! It was important and urgent, as I saw myself jumping into the wrong wagon, again. -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
On 21/11/13 22:13, Glenn Linderman wrote: On 11/21/2013 12:23 PM, Christian Tismer wrote: Maybe I would generate a cpython and spython exe and support them both in the same distribution? That sounds cool, if possible. Hooka Hooka! Let's see if the nightmares agree :-) -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
mpatible with your build -- so you should assure that whatever pip/wheel uses to identify the build is set differently in your build (see the relevant PEPs). Yes, I want to make PIP work with it, want to make it very simple to install whatnot, and let people use that stuff. So if you can, please teach me what I need to do or avoid. I don't want to intrude anywhere, I just want to make the Stackless site a useful site where people can try extensions and additions without getting into that DLL hell where I was for ages. Conclusion: -- I do not want to do anything bad. I do not want to solve hard-to-solve ABI problems in a week. I do not want to drive python-dev crazy right now for just that. What I want is a workable CPython path for some customer (!=CCP) to use for the next (maybe 5) years, and I want to build that now, for good. I think you have helped me incredibly much, and we need to talk in private. Cheers -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
... I also think having a 2.8 out there that is exactly the same as 2.7, except that it was built with a different version of a compiler on one particular platform is a very very bad idea. This was not my proposal. I was seeking a way to make a version that produces no collisions with DLLs, and it was a question about Stackless, not Python. But I admit that I was not aware of a license problem. So how CAN this be addressed? This is really a distribution issue, and the distutils-sig has been hard at work building the infrastructure to support things like this. Right now, if you build binary wheels with the two different "official" OS-X builds, you will get two different names, and pop can find the correct one. (last I checked, pip would happily install the wrong one if you asked it to, though I think that is slated to be fixed) So if a different build of 2.7 for Windows is put out there, we need to make sure it reports the platform in a way that wheel and pip can make the distinction. That was the reason to change the number. If other approaches like a different name for certain things is easy to do, I am fine with that, too. It might be nice to patch the win_inst too--IIRC it's still not very smart about even 32 vs 64 bit builds. As for stackless--just to be clear--can you use extensions built with the "regular" python with a stack less binary? If so, I understand the concern. If not, then it seems stackless is a separate ecosystem anyway. Good question, and this _is_ a problem: Minus a few number of things, Stackless is almost completely binary compatible with extensions, and people _will_ want to try Stackless for some things or might want to go back and use CPython, be it only to remove concerns of a customer. People do want to install binary packages which are not built for Stackless, and we always did best efforts to support that. That means: basically we want to improve the situation for Stackless-based project in the first place, but make it easy to go back to CPython. We have a compiler switch that can completely remove the stackless additions and create a 100 % binary identical CPython version! That implies in the end that extension modules which work with Stackless will also be runnable on CPython 2.7.x VS2010 or whatever name it is. The naming problem then comes in again through the back-door, and we cannot shut our eyes and pretend "hey it is Stackless", because that is admittedly close to a fraud. So even if VS2010 exists only in the stackless branch, it is very likely to get used as CPython VS 2010, and I again have the naming problem ... cheers - Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
On 22/11/13 13:47, "Martin v. Löwis" wrote: Am 22.11.13 10:00, schrieb Richard Tew: That there are people who would consider using the trademark to force us to change the name we've been using without harm for 14 years, worries me. It's one thing to perhaps use it to stop someone scamming Python users, and another to suggest using it as a weapon against us for this? Really? Unfortunately, Christian's original question was unclear. If his plan had been to release "Python 2.8" (as his original question suggested), then he would have faced quite some opposition (and, in a sense, releasing a "python28.dll" is quite close to that). The discussion is over, but I cannot let this comment go through without citing my original question, again: My question --- I have created a very clean Python 2.7.6+ based CPython with the Stackless additions, that compiles with VS 2010, using the adapted project structure of Python 3.3.X, and I want to publish that on the Stackless website as the official "Stackless Python 2.8". If you consider Stackless as official ;-) . Can it be that these sentences have morphed into something different when going out to the mailing list? Or maybe there is a filter in the brains? If one removes the word "Stackless" everywhere, the above text reads still almost syntactic correctly, but changes it's meaning a lot. -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] arguments policy: **kwargs.pop()
Hi guys,
I tried to find advice for hours, but failed so fer, so here is my question:
Whenever I think to adopt a new module that does a good job, I always
can't stand the temptation to look at the coding style and certain
principles.
Then I rather often see things like this:
class someclass(object):
# note that there is no comment about argument destruction...
def __init__(self, **kwargs):
first_arg = kwargs.pop('option_1', somedefault)
...
nth_arg = kwargs.pop('option_n', somedefault')
...
That is:
There are arguments "consumed" rigorously by the __init__ of a class,
although it would be equivalent to use kwargs.get(..., ...).
Happened to me again, today and blocked me completely, since I don't
know if I saw bad programming style or if I'm mislead.
I agree there are valid cases when if makes sense to filter out some
arguments
that a subsequently called super() might not be able to handle.
Bu even then, I would probably have a copy and make the filtering more
explicit.
But most of the time, there is no reason for using this pattern, and there
is also no comment stating that the argument dict is modified by the callee
for no good reason.
I always used the policy that arguments are never changed by a function,
unless explicitly stated.
But since I see this pattern quite frequently, I wanted to ask if I am
right by
thinking this way, or if the general policy is more like "arguments may be
destroyed by default".
What do you think? Is this bad style and should be noticed somewhere,
or is the caller supposed to protect the arguments, or are my worries
useless?
Thanks & cheers -- Chris
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] arguments policy: **kwargs.pop()
Ah, now I see it.
For some reason, I forgot that the dict is always newly created.
That was really wrong thinking. Sorry!
On 11/04/14 05:47, Guido van Rossum wrote:
> I'm not sure what you're worried about here. Modifying kwds doesn't
> actually modify the dict that was passed in. So are you just talking about
> the style issue? Or would you also object to this:
>
> def foo(x=1):
> x += 1
>
No, this is ok, although I personally tend not to modify args.
No, it was just my temporary forgetting that the star
args/kwds are always in a new container, and the whole reasoning chain
was pointless, therefore.
Thanks and cheers - Chris
> On Thu, Apr 10, 2014 at 10:12 PM, Christian Tismer
> wrote:
>
>> Hi guys,
>>
>> I tried to find advice for hours, but failed so fer, so here is my
>> question:
>>
>> Whenever I think to adopt a new module that does a good job, I always
>> can't stand the temptation to look at the coding style and certain
>> principles.
>>
>> Then I rather often see things like this:
>>
>> class someclass(object):
>> # note that there is no comment about argument destruction...
>>
>> def __init__(self, **kwargs):
>> first_arg = kwargs.pop('option_1', somedefault)
>> ...
>> nth_arg = kwargs.pop('option_n', somedefault')
>> ...
>>
>> That is:
>> There are arguments "consumed" rigorously by the __init__ of a class,
>> although it would be equivalent to use kwargs.get(..., ...).
>>
>> Happened to me again, today and blocked me completely, since I don't
>> know if I saw bad programming style or if I'm mislead.
>>
>> I agree there are valid cases when if makes sense to filter out some
>> arguments
>> that a subsequently called super() might not be able to handle.
>> Bu even then, I would probably have a copy and make the filtering more
>> explicit.
>>
>> But most of the time, there is no reason for using this pattern, and there
>> is also no comment stating that the argument dict is modified by the callee
>> for no good reason.
>>
>> I always used the policy that arguments are never changed by a function,
>> unless explicitly stated.
>> But since I see this pattern quite frequently, I wanted to ask if I am
>> right by
>> thinking this way, or if the general policy is more like "arguments may be
>> destroyed by default".
>>
>> What do you think? Is this bad style and should be noticed somewhere,
>> or is the caller supposed to protect the arguments, or are my worries
>> useless?
>>
>> Thanks & cheers -- Chris
>>
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] arguments policy: **kwargs.pop()
Thank you too, Tres.
Somehow I had a brain shortcut and forgot that
the dict is locally generated, *because* of the stars.
Good to become adjusted and restarted, sorry about the noise.
ciao - Chris
On 11/04/14 05:48, Tres Seaver wrote:
> On 04/10/2014 10:12 PM, Christian Tismer wrote:
>
>> I always used the policy that arguments are never changed by a
>> function, unless explicitly stated. But since I see this pattern
>> quite frequently, I wanted to ask if I am right by thinking this
>> way, or if the general policy is more like "arguments may be
>> destroyed by default".
>
>> What do you think? Is this bad style and should be noticed
>> somewhere, or is the caller supposed to protect the arguments,
>> or are my worries useless?
>
> The caller can't know or care that the function / method pops
> arguments::
>
> $ python Python 2.7.3 (default, Feb 27 2014, 19:58:35) [GCC 4.6.3]
> on linux2 Type "help", "copyright", "credits" or "license" for
> more information.
>>>> def foo(**kw):
> ... bar = kw.pop('bar', 'BAR') ... print 'bar: %s' % bar
> ... print 'kw: %s' % kw ...
>>>> foo()
> bar: BAR kw: {}
>>>> foo(bar='baz')
> bar: baz kw: {}
>>>> foo(bar='baz', bam='qux')
> bar: baz kw: {'bam': 'qux'}
>>>> mykw = {'bar': 'baz', 'bam': 'qux'} foo(**mykw)
> bar: baz kw: {'bam': 'qux'}
>>>> mykw
> {'bam': 'qux', 'bar': 'baz'}
>
> because the caller gets its own copy of 'kw', even when called
> with an existing dict.
>
>
> Tres.
>
> ___ Python-Dev mailing
> list Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
>
>
>
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] arguments policy: **kwargs.pop()
Hi Chris,
On 11/04/14 21:50, Chris Barker wrote:
> On Thu, Apr 10, 2014 at 7:12 PM, Christian Tismer wrote:
>
>> Then I rather often see things like this:
>>
>> class someclass(object):
>> # note that there is no comment about argument destruction...
>>
>> def __init__(self, **kwargs):
>> first_arg = kwargs.pop('option_1', somedefault)
>> ...
>> nth_arg = kwargs.pop('option_n', somedefault')
>> ...
>>
>
> While it's been clarified that this isn't dangerous, I find it to be a
> really annoying style, as you've lost the opurtuniyt to docuemnt somethign
> in the signature. Is:
>
> def __init__(self, option_1=some_default, option_n=some_default,
> **kwargs):
> first_arg = kwargs.pop('option_1')
> nth_arg = kwargs.pop('option_n')
>
> *that* much harder to write?
>
> And many of these come with virtually no docstring, either.
Thank you for re-validating my rant, after my wrong start was clarified.
Yes, the style is of course annoying, still.
This is the style of
"""hey look how cool I am, just taking an interface, picking args if
they happen to be there and otherwise not treating them""".
So while I'm still ashamed of my mis-interpretion, I am happy to still
not like that very much.
At least for myself, I like to be way more explicit and tell actively
what I expect as arguments, what I do care about and what not, just to
make sure that people see right by looking at the interface what they
may ignore and what they should probably put in as an argument.
Actually, putting so many defaults in without documenting that in the
interface is this new-fangled sloppiness that is perceived as cool-ness.
May be I am getting old, but I dislike this and tend to tell much more
in the interface. And not in the 35th iteration, but when writing the
first public version.
This is because I don't want to throw an interface at somebody, but to
discuss and improve it, and for that I put comments in that invite to
agree or create a better version.
I have these style problems with several modules that I am reluctant to
use, therefore. I know that I'm pretty alone with that. But my idea
of a published module is such that it should try to motivate why it is
doing things in which way, and why it thinks this is good to do.
Doing that not and nothing instead is my definition of "sloppy".
(interested people may get the actual module from me why this came up)
cheers -- Chris
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] arguments policy: **kwargs.pop()
On 12.04.14 01:55, Ethan Furman wrote: > On 04/11/2014 02:01 PM, Christian Tismer wrote: >> >> I have these style problems with several modules that I am reluctant to >> use, therefore. I know that I'm pretty alone with that. > > You are not alone in that. Funny not to be alone in being alone in that :-) hugs -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
Not trying to argue about Mercurial, but if a major amount of startup time is spent in site.py: I think in cases like hg command line scripts there is no need to import site just for hg scripts. Maybe that would improve things if those startup scripts avoid importing site? Or do they, already? cheers - chris On 16.04.14 04:35, Guido van Rossum wrote: > Well, that's the part that does "import site". Anything that speeds up > the code in Lib/site.py might help. :-) > > > On Tue, Apr 15, 2014 at 5:23 PM, Terry Reedy <mailto:tjre...@udel.edu>> wrote: > > On 4/15/2014 5:26 PM, Brett Cannon wrote: > > To finish my timing work I decided to see > where Py_InitializeEx_Private() spends its time. The following > is a > breakdown measured in microseconds running using -E: > > INIT: > setlocale: 11 > envvar: 2 > random init: 2 > interp creation: 15 > thread creation: 6 > GIL: 10 > _Py_ReadyTypes(): 930 > more types: 44 > builtins: 141 > exceptions: 287 > sys: 258 > _PyImport_Init: 15 > import hooks: 4 > _PyWarnings_Init(): 15 > ENTERING import_init(): >PyImport_ImportFrozenModule(_frozen_importlib): 1186 >interp->importlib: 6 >PyInit_imp(): 15 >_imp: 3 >importlib._install(): 876 >_PyImportZip_Init(): 130 > _PyFaulthandler_Init(): 13 > time: 3 > ENTERING initfsencoding(): >codec lookup: 2179 > signals: 120 > tracemalloc: 7 > __main__: 13 > initstdio(): 1568 > warnings module: 4 > initsite(): 9552 > > > It looks like initsite takes half the time. Can it be sped up? > > > -- > Terry Jan Reedy > > > ___ > Python-Dev mailing list > Python-Dev@python.org <mailto:Python-Dev@python.org> > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > -- > --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>) > > > _______ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
On 16/04/14 16:35, Antoine Pitrou wrote: > On Wed, 16 Apr 2014 09:39:34 +0200 > Christian Tismer wrote: >> >> I think in cases like hg command line scripts there is no need >> to import site just for hg scripts. > > If you don't import site you won't be able to import Mercurial in most > cases. Oh is that so? I thought about hg as a set of command-line scripts with a defined interface that should IMO not be dependent from any site-specific settings. Instead, it should be configured at install time, and after that be completely independent from site settings. > Also, talking about "scripts" is a bit silly here - Mercurial is a > full-blown applications with many modules and tens of thousands of > lines of code. Thanks for regarding my comments as silly. In my sense it would be silly if that full-blown app would suffer from or relate to site at all, tho. Apps are meant to be self-contained. regards - Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] devguide: Add myself to developer log and as a Windows expert.
On 11.05.14 15:57, Steve Dower wrote: > Thanks. > > For those who missed the earlier discussions, Martin v. Löwis has > handed over responsibility for the Windows installers. It sounds like > Brett Cannon and I are both in a position to build 2.7 right now, and > I hope to simplify the setup required for 3.5 so that anyone can > produce and test the installers. (Martin is going to look after the > 3.4.x builds.) > > Obviously I'm also here to help out with Windows in general. I haven't > quite managed to get 50% time from my employer (maybe 10%), but my > management at least is very supportive of my participation and keen to > keep Python running well. > Very nice, great to read this. Welcome from me as well! cheers - Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://www.pydica.net/ 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Write vs Read, Understand and Control Flow
On 25.04.18 05:43, Steven D'Aprano wrote: > On Tue, Apr 24, 2018 at 08:10:49PM -0500, Tim Peters wrote: > >> Binding expressions are debugger-friendly in that they _don't_ just >> vanish without a trace. It's their purpose to _capture_ the values of >> the expressions they name. Indeed, you may want to add them all over >> the place inside expressions, never intending to use the names, just >> so that you can see otherwise-ephemeral intra-expression results in >> your debugger ;-) > > That's a fantastic point and I'm surprised nobody has thought of it > until now (that I've seen). > > Chris, if you're still reading this and aren't yet heartedly sick and > tired of the PEP *wink* this ought to go in as another motivating point. Yay, that's like a dream, really fantastic. So sorry that I was way too deep in development in spring and did not read earlier about that PEP. I was actually a bit reluctant about "yet another way to prove Python no longer simple" and now even that Pascal-ish look! :-) But this argument has completely sold me. Marvellous! -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
Well, to my knowledge they did not modify the marshal code. They are in fact heavily dependent from marshal speed since that is used frequently to save and restore state of many actors. But haven't looked further since 2010 ;-) Btw., why are they considering to make the algorithm slower, just because someone wants the algorithm stable? An optional keyword argument would give the stability, and the default behavior would not be changed at all. Cheers - Chris On 12.07.18 12:07, Steve Holden wrote: > Eve is indeed based on stackless 2, and are well capable of ignoring > changes they don't think they need (or were when I was working with > them). At one point I seem to remember they optimised their interpreter > to use singleton floating-point values, saving large quantities of > memory by having only one floating-point zero. > > Steve Holden > > On Thu, Jul 12, 2018 at 9:55 AM, Alex Walters <mailto:tritium-l...@sdamon.com>> wrote: > > > > > -Original Message- > > From: Python-Dev > list=sdamon@python.org <mailto:sdamon@python.org>> On Behalf Of > Victor Stinner > > Sent: Thursday, July 12, 2018 4:01 AM > > To: Serhiy Storchaka mailto:storch...@gmail.com>> > > Cc: python-dev mailto:python-dev@python.org>> > > Subject: Re: [Python-Dev] Can I make marshal.dumps() slower but stabler? > > > > 2018-07-12 8:21 GMT+02:00 Serhiy Storchaka <mailto:storch...@gmail.com>>: > > >> Is there any real application which marshal.dumps() performance is > > >> critical? > > > > > > EVE Online is a well known example. > > > > EVE Online has been created in 2003. I guess that it still uses Python > 2.7. > > > > I'm not sure that a video game would pick marshal in 2018. > > > > EVE doesn't use stock CPython, IIRC. They use a version of stackless 2, > with their own patches. If a company is willing to patch python > itself, I > don't think their practices should be cited without more context > about what > they actually modified. > > > Victor > > ___ > > Python-Dev mailing list > > Python-Dev@python.org <mailto:Python-Dev@python.org> > > https://mail.python.org/mailman/listinfo/python-dev > <https://mail.python.org/mailman/listinfo/python-dev> > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tritium- > <https://mail.python.org/mailman/options/python-dev/tritium-> > > list%40sdamon.com <http://40sdamon.com> > > ___ > Python-Dev mailing list > Python-Dev@python.org <mailto:Python-Dev@python.org> > https://mail.python.org/mailman/listinfo/python-dev > <https://mail.python.org/mailman/listinfo/python-dev> > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com > <https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com> > > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com > -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : http://pyside.org 14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: PEP 594: update 1
On 05.06.19 02:21, Victor Stinner wrote: > So what is happening for this PEP since Python 3.8 beta1 has been > released? Is it too late for Python 3.8 or not? > > It seems like most people are confused by the intent of the PEP. IMHO > it would be better to rewrite "Remove packages from the stdlib" as > "Move some stdlib modules to PyPI". But that would require to rewrite > some parts of the PEP to explain how modules are moved, who become the > new maintainers, how to support modules both in stdlib (old Python > versions) and in PyPI (new Python), etc. And I would like to add something as well: The stdlib has been a set of well-known modules. Maybe not the latest and greatest, but you knew for quite sure that these modules are guaranteed to be stable and quite persistent. With the move to PyPI, I am missing this promise, partially: PyPI has very many good modules, but also some less good ones. With the stdlib, you had almost one choice to choose from. With PyPI, you have way too many modules, and you have no longer the feeling "this seems to be right in BDFL mind". I think what is missing is replacement of this feature: The set of modules in the stdlib has exactly that being in the stdlib as a quality indicator. I need now a structure that replaces that quality, like "This one is eligible to go into stdlib" Do we have such a replacement implemented, already? -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 Python-Dev mailing list -- python-dev(a)python.org To unsubscribe send an email to python-dev-leave(a)python.org https://mail.python.org/mailman3/lists/python-dev.python.org/
[Python-Dev] Re: PEP 594: update 1
On 06.06.19 21:27, Brett Cannon wrote: > > > On Thu, Jun 6, 2019 at 12:25 AM Christian Tismer <mailto:tis...@stackless.com>> wrote: > > On 05.06.19 02:21, Victor Stinner wrote: > > So what is happening for this PEP since Python 3.8 beta1 has been > > released? Is it too late for Python 3.8 or not? > > > > It seems like most people are confused by the intent of the PEP. IMHO > > it would be better to rewrite "Remove packages from the stdlib" as > > "Move some stdlib modules to PyPI". But that would require to rewrite > > some parts of the PEP to explain how modules are moved, who become the > > new maintainers, how to support modules both in stdlib (old Python > > versions) and in PyPI (new Python), etc. > > And I would like to add something as well: > > The stdlib has been a set of well-known modules. > Maybe not the latest and greatest, but you knew for quite sure > that these modules are guaranteed to be stable and quite persistent. > > With the move to PyPI, I am missing this promise, partially: > > PyPI has very many good modules, but also some less good ones. > With the stdlib, you had almost one choice to choose from. > With PyPI, you have way too many modules, and you have no longer > the feeling "this seems to be right in BDFL mind". > > I think what is missing is replacement of this feature: > The set of modules in the stdlib has exactly that being in the > stdlib as a quality indicator. > I need now a structure that replaces that quality, > like > > "This one is eligible to go into stdlib" > > Do we have such a replacement implemented, already? > > > Are you asking for us to bless packages on PyPI as of a quality that the > core devs approve of it? Or something else? If it's the former we do > have links pointing to other projects already (e.g. linking to > 'requests' from > https://docs.python.org/3/library/urllib.request.html#module-urllib.request). Yes, I'm asking for blessing some packages. And I have not spent much time these days with the topic, so please ignore my uninformed question. -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/
[Python-Dev] typing: how to use names in result-tuples?
Hi friends, I am meanwhile the PySide maintainer at The Qt Company, and we are trying to make the mapping from Qt functions to Python functions as comparable as possible. One problem are primitive pointer variables: In Qt, it is natural to use "sometype *varname" to make a mutable variable. In Python, you have to turn such a thing into a result-tuple. Example: void QPrinter::getPageMargins(qreal *left, qreal *top, \ qreal *right, qreal *bottom, QPrinter::Unit unit) const (meanwhile deprecated, but a good example) is mapped to the Python signature def getPageMargins(self, \ unit: PySide2.QtPrintSupport.QPrinter.Unit) \ -> typing.Tuple[float, float, float, float]: ... NOW my question: I would like to retain the variable names in Python! The first idea is to use typing.NamedTuple, but I get the impression that this would be too much, because I would not only need to create an extra named tuple definition for every set of names, but also invent a name for that named tuple. What I would like to have is something that looks like def getPageMargins(self, \ unit: PySide2.QtPrintSupport.QPrinter.Unit) \ -> typing.NamedTuple[left: float, top: float, \ right:float, bottom:float]: ... but that is obviously not a named tuple. Possible would be to derive a name from the function, so maybe a definition could be used like class PageMargingResult(NamedTuple): left: float top: float right: float bottom: float but then I would have some opaque PageMargingResult type. This would work, but as said, this is a bit too much, since I only wanted something like a tuple with names. What do you suggest to do here? Something what I am missing? Cheers -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YGZELVWRGXZ5BTD4ZMATQT7IRAZVQSHR/
[Python-Dev] Re: typing: how to use names in result-tuples?
Yes, maybe I can use that.
With Python >= 3.6, also
def f() -> typing.NamedTuple("__f", x=int, y=int): ...
which is concise, but a slightly weird abuse.
But there are other drawbacks:
>>> typing.Tuple[int, int]
typing.Tuple[int, int]
>>> typing.Tuple[int, int] is typing.Tuple[int, int]
True
versus
>>> typing.NamedTuple("__f", x=int, y=int)
>>> typing.NamedTuple("__f", x=int, y=int) is typing.NamedTuple("__f",
x=int, y=int)
False
I think the whole NameTuple implementation looks a bit half-hearted,
so I was looking to find something that is closer to the perfect
behavior of the other typing types.
Maybe I should try to implement something that keeps proper identity
like Tuple and avoids the useless function name? Like
def f() -> typing.KwTuple(x=int, y=int): ...
or
def f() -> typing.KwTuple([("x", int), ("y", int)]): ...
cheers -- Chris
On 29.07.19 18:00, Guido van Rossum wrote:
> Can't you use the proper inline form of NamedTuple?
>
> def f() -> typing.NamedTuple("__f", [("x", int), ("y", int)]):
> ...
>
> seems to work.
>
> On Mon, Jul 29, 2019 at 8:26 AM Christian Tismer <mailto:tis...@stackless.com>> wrote:
>
> Hi friends,
>
> I am meanwhile the PySide maintainer at The Qt Company,
> and we are trying to make the mapping from Qt functions
> to Python functions as comparable as possible.
>
> One problem are primitive pointer variables:
> In Qt, it is natural to use "sometype *varname" to make a
> mutable variable. In Python, you have to turn such a thing
> into a result-tuple. Example:
>
> void QPrinter::getPageMargins(qreal *left, qreal *top, \
> qreal *right, qreal *bottom, QPrinter::Unit unit) const
>
> (meanwhile deprecated, but a good example)
>
> is mapped to the Python signature
>
> def getPageMargins(self, \
> unit: PySide2.QtPrintSupport.QPrinter.Unit) \
> -> typing.Tuple[float, float, float, float]: ...
>
> NOW my question:
>
>
> I would like to retain the variable names in Python!
> The first idea is to use typing.NamedTuple, but I get the impression
> that this would be too much, because I would not only need to create
> an extra named tuple definition for every set of names, but also
> invent a name for that named tuple.
>
> What I would like to have is something that looks like
>
> def getPageMargins(self, \
> unit: PySide2.QtPrintSupport.QPrinter.Unit) \
> -> typing.NamedTuple[left: float, top: float, \
> right:float, bottom:float]: ...
>
> but that is obviously not a named tuple.
> Possible would be to derive a name from the function, so maybe
> a definition could be used like
>
> class PageMargingResult(NamedTuple):
> left: float
> top: float
> right: float
> bottom: float
>
> but then I would have some opaque PageMargingResult type. This would
> work, but as said, this is a bit too much, since I only
> wanted something like a tuple with names.
>
> What do you suggest to do here? Something what I am missing?
>
> Cheers -- Chris
> --
> Christian Tismer :^) tis...@stackless.com
> <mailto:tis...@stackless.com>
> Software Consulting : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : https://github.com/PySide
> 14482 Potsdam : GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776 fax +49 (30) 700143-0023
>
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> <mailto:python-dev@python.org>
> To unsubscribe send an email to python-dev-le...@python.org
> <mailto:python-dev-le...@python.org>
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
>
> https://mail.python.org/archives/list/python-dev@python.org/message/YGZELVWRGXZ5BTD4ZMATQT7IRAZVQSHR/
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> /Pronouns: he/him/his //(why is my pronoun here?)/
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
>
[Python-Dev] Re: typing: how to use names in result-tuples?
Hi Petr, hi Paul,
the struct sequence objects are in fact interesting!
About the "every instance" issue:
Well, if I used something like the caching approach from Paul's post,
like
cached_namedtuple = lru_cache(None)(namedtuple)
n1 = cached_namedtuple('f', ('a', 'b', 'c'))
n2 = cached_namedtuple('f', ('a', 'b', 'c'))
n1 is n2
which produces uniqueness, then I would need no explicit names.
How about that principle, but with the struct sequence objects?
Cheers -- Chris
On 30.07.19 11:13, Petr Viktorin wrote:
> Hello,
> I'm afraid you're stuck with defining a new type for each of these.
> One reason why that's better is that info about the names will only be
> stored once, in the type; not in every instance.
>
> Since this is in PySide, I assume you're using the C-API. If that's the
> case, check out Struct Sequences, the "C equivalent of named tuples".
> For example, the result of "os.stat()" is a struct sequence.
>
> https://docs.python.org/3/c-api/tuple.html#struct-sequence-objects
>
> Note that like namedtuples, these are immutable, and they're proper
> subclasses of tuple.
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/WL3M2UGYXO5EZ5G7OUQAYUDRLLEG7TRQ/
[Python-Dev] Re: typing: how to use names in result-tuples?
Hi all,
Ok, I am about to implement generation of such structures
automatically using the struct sequence concept.
One more question:
--
Struct sequences are not yet members of the typing types.
I would like to add that, because a major use case is also to
show nice .pyi files with all the functions and types.
* namedtuple has made the transition to NamedTuple
* What would I need to do that for StructSequence as well?
Things get also a bit more complicated since struct sequence
objects can contain unnamed fields.
Any advice would be appreciated, I am no typing expert (yet :-)
cheers -- Chris
On 30.07.19 17:10, Guido van Rossum wrote:
> I think I have to agree with Petr. Define explicit type names.
>
> On Tue, Jul 30, 2019 at 2:45 AM Paul Moore <mailto:p.f.mo...@gmail.com>> wrote:
>
> On Tue, 30 Jul 2019 at 09:33, Christian Tismer <mailto:tis...@stackless.com>> wrote:
> > >>> typing.NamedTuple("__f", x=int, y=int)
> >
> > >>> typing.NamedTuple("__f", x=int, y=int) is typing.NamedTuple("__f",
> > x=int, y=int)
> > False
>
> This appears to go right back to collections.namedtuple:
>
> >>> from collections import namedtuple
> >>> n1 = namedtuple('f', ['a', 'b', 'c'])
> >>> n2 = namedtuple('f', ['a', 'b', 'c'])
> >>> n1 is n2
> False
>
> I found that surprising, as I expected the named tuple type to be
> cached based on the declared name 'f'. But it's been that way forever
> so obviously my intuition here is wrong. But maybe it would be useful
> for this case if there *was* a way to base named tuple identity off
> the name/fields? It could be as simple as caching the results:
>
> >>> from functools import lru_cache
> >>> cached_namedtuple = lru_cache(None)(namedtuple)
> >>> n1 = cached_namedtuple('f', ('a', 'b', 'c')) # A tuple rather
> than a list of field names, as lists aren't hashable
> >>> n2 = cached_namedtuple('f', ('a', 'b', 'c'))
> >>> n1 is n2
> True
>
> Paul
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> /Pronouns: he/him/his //(why is my pronoun here?)/
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GIFRTFWPEGKZ33PTW63YXKGXHHAQJ35I/
>
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/FG6BG5DENW4TTD24XJIHXWS37RW7X5JY/
[Python-Dev] Re: typing: how to use names in result-tuples?
Hi Guido,
If a C++ function already has a return value, plus some primitive
pointer variables that need to be moved into the result in Python,
then we have the case with a first, single unnamed field.
Only one such field can exist.
I'm not sure if that case exists in the ~25000 Qt5 functions, but in
that case, I think to give that single field the name "unnamed"
or maybe better "result".
Thank you very much for pointing me to that example!
Cheers -- Chris
On 08.08.19 06:41, Guido van Rossum wrote:
> Alas, we didn't think of struct sequences when we designed PEP 484. It
> seems they are a hybrid of Tuple and NamedTuple; both of these are
> currently special-cased in mypy in ways that cannot easily be combined.
>
> Do you really need anonymous fields?
>
> I see an example in typeshed/stdlib/3/os/__init__.pyi (in
> github.com/python/typeshed <http://github.com/python/typeshed>), for
> stat_result. It defines names for all the fields, plus a __getitem__()
> method that indicates that indexing returns an int. This doesn't help if
> anonymous fields could have different types, not does it teach the type
> checker about the number of anonymous fields.
>
> --Guido
>
> On Wed, Aug 7, 2019 at 1:51 AM Christian Tismer <mailto:tis...@stackless.com>> wrote:
>
> Hi all,
>
> Ok, I am about to implement generation of such structures
> automatically using the struct sequence concept.
>
>
> One more question:
> --
>
> Struct sequences are not yet members of the typing types.
> I would like to add that, because a major use case is also to
> show nice .pyi files with all the functions and types.
>
> * namedtuple has made the transition to NamedTuple
>
> * What would I need to do that for StructSequence as well?
>
> Things get also a bit more complicated since struct sequence
> objects can contain unnamed fields.
>
> Any advice would be appreciated, I am no typing expert (yet :-)
>
> cheers -- Chris
>
>
> On 30.07.19 17:10, Guido van Rossum wrote:
> > I think I have to agree with Petr. Define explicit type names.
> >
> > On Tue, Jul 30, 2019 at 2:45 AM Paul Moore <mailto:p.f.mo...@gmail.com>
> > <mailto:p.f.mo...@gmail.com <mailto:p.f.mo...@gmail.com>>> wrote:
> >
> > On Tue, 30 Jul 2019 at 09:33, Christian Tismer
> mailto:tis...@stackless.com>
> > <mailto:tis...@stackless.com <mailto:tis...@stackless.com>>>
> wrote:
> > > >>> typing.NamedTuple("__f", x=int, y=int)
> > >
> > > >>> typing.NamedTuple("__f", x=int, y=int) is
> typing.NamedTuple("__f",
> > > x=int, y=int)
> > > False
> >
> > This appears to go right back to collections.namedtuple:
> >
> > >>> from collections import namedtuple
> > >>> n1 = namedtuple('f', ['a', 'b', 'c'])
> > >>> n2 = namedtuple('f', ['a', 'b', 'c'])
> > >>> n1 is n2
> > False
> >
> > I found that surprising, as I expected the named tuple type to be
> > cached based on the declared name 'f'. But it's been that way
> forever
> > so obviously my intuition here is wrong. But maybe it would be
> useful
> > for this case if there *was* a way to base named tuple
> identity off
> > the name/fields? It could be as simple as caching the results:
> >
> > >>> from functools import lru_cache
> > >>> cached_namedtuple = lru_cache(None)(namedtuple)
> > >>> n1 = cached_namedtuple('f', ('a', 'b', 'c')) # A tuple rather
> > than a list of field names, as lists aren't hashable
> > >>> n2 = cached_namedtuple('f', ('a', 'b', 'c'))
> > >>> n1 is n2
> > True
> >
> > Paul
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido <http://python.org/~guido>
> <http://python.org/~guido>)
> > /Pronouns: he/him/his //(why is my pronoun here?)/
> >
>
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
> >
> >
[Python-Dev] Re: What to do about invalid escape sequences
Hey friends, This is IMHO a great idea. If a package claims to be Python 3.8 compatible, then it has to be correct concerning invalid escapes. A new pip version could perhaps even refuse packages with such literals when it claims to be supporting Python 3.8 . But how can it actually happen that a pre-3.8 package gets installed when you install Python 3.8? Does pip allow installation without a section that defines the allowed versions? Ok, maybe packages are claimed for Python 3.8 and not further checked. But let's assume the third-party things that Raymond sees do _not_ come from pip, but elsewhere. Pre-existing stuff that is somehow copied into the newer Python version? Sure, quite possible! But then it is quite likely that those third-party things still have their creation date from pre-3.8 time. What about the simple heuristic that a Python module with a creation date earlier than xxx does simply not issue the annoying warning? Maybe that already cures the disease in enough cases? just a wild idea - \leave \old \code \untouched -- ciao - \Chris On 06.08.19 18:59, Neil Schemenauer wrote: > > Making it an error so soon would be mistake, IMHO. That will break > currently working code for small benefit. When Python was a young > language with a few thousand users, it was easier to make these > kinds of changes. Now, we should be much more conservative and give > people a long time and a lot of warning. Ideally, we should provide > tools to fix code if possible. > > Could PyPI and pip gain the ability to warn and even fix these > issues? Having a warning from pip at install time could be better > than a warning at import time. If linting was built into PyPI, we > could even do a census to see how many packages would be affected by > turning it into an error. > > On 2019-08-05, raymond.hettin...@gmail.com wrote: >> P.S. In the world of C compilers, I suspect that if the relatively >> new compiler warnings were treated as errors, the breakage would >> be widespread. Presumably that's why they haven't gone down this >> road. > > The comparision with C compilers is relevant. C and C++ represent a > fairly extreme position on not breaking working code. E.g. K & R > style functional declarations were supported for decades. I don't > think we need to go quite that far but also one or two releases is > not enough time. > > Regards, > > Neil > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/V2EDFDJGXRIDMKJU3FKIWC2NDLMUZA2Y/ > -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XVJCYXDZ7VPMMCTP2BPNAJ3OO7S4II4V/
[Python-Dev] Re: typing: how to use names in result-tuples?
Hi Ronald, sure, the tuple is usually not very interesting; people look it up once and use that info in the code. But I think things can be made quite efficient and pretty at the same time. Such a return tuple could be hidden like the stat_result example that Guido mentioned: https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi def stat(self, *, follow_symlinks: bool = ...) -> stat_result: ... The stat_result is a huge structure where you don't want to see much unless you are working with a stat_result. Other with common, repeating patterns like (x, y, width, height) or your examples: def getpoint(v: pointless) -> (int, int) def getvalue(v: someclass) -> (bool, int) would be at first sight def getpoint(v: pointless) -> getpoint_result def getvalue(v: someclass) -> getvalue_result But actually, a much nicer, speaking outcome would be written as the single function StructSequence(...) with arguments, producing: def getpoint(v: pointless) -> StructSequence(x=int, y=int) def getvalue(v: someclass) -> StructSequence(result=bool, val=int) That would have the nice effect of a very visible structure in the .pyi file. When you actually get such an object and look at it, then you have >>> getpoint(pointless()) getpoint_result(x=17, y=4) >>> getvalue(someclass()) getvalue_result(result=True, val=42)) And the "magic" function StructSequence would simply index a dict with the given argument tuple that gives the right struct sequence. This is always possible, since only names and types are involved in the lookup. Cheers -- Chris On 08.08.19 14:22, Ronald Oussoren wrote: > Chrisian, > > How would these namedtuple/structseq values be used? I have a similar > design with PyObjC: pass-by-reference “return” values are returned in a > tuple, e.g.: > > void getpoint(pointclass* v, int* x, int *y) => def get > point(v: pointless) -> (int, int) > BOOL getvalue(someclass* v, int* val) => def getvalue(v: > someclass) -> (bool, int) > > I rarely, if ever, see code that actually stores the return tuple as-is. > The return tuple is just deconstructed immediately, like “x, y = > getpoint(mypoint)”. > > Ronald > — > > Twitter: @ronaldoussoren > Blog: https://blog.ronaldoussoren.net/ > >> On 8 Aug 2019, at 10:42, Christian Tismer > <mailto:tis...@stackless.com>> wrote: >> >> Hi Guido, >> >> If a C++ function already has a return value, plus some primitive >> pointer variables that need to be moved into the result in Python, >> then we have the case with a first, single unnamed field. >> Only one such field can exist. >> >> I'm not sure if that case exists in the ~25000 Qt5 functions, but in >> that case, I think to give that single field the name "unnamed" >> or maybe better "result". >> >> Thank you very much for pointing me to that example! >> >> Cheers -- Chris >> >> >> On 08.08.19 06:41, Guido van Rossum wrote: >>> Alas, we didn't think of struct sequences when we designed PEP 484. It >>> seems they are a hybrid of Tuple and NamedTuple; both of these are >>> currently special-cased in mypy in ways that cannot easily be combined. >>> >>> Do you really need anonymous fields? >>> >>> I see an example in typeshed/stdlib/3/os/__init__.pyi (in >>> github.com/python/typeshed <http://github.com/python/typeshed> >>> <http://github.com/python/typeshed>), for >>> stat_result. It defines names for all the fields, plus a __getitem__() >>> method that indicates that indexing returns an int. This doesn't help if >>> anonymous fields could have different types, not does it teach the type >>> checker about the number of anonymous fields. >>> >>> --Guido >>> >>> On Wed, Aug 7, 2019 at 1:51 AM Christian Tismer >> <mailto:tis...@stackless.com> >>> <mailto:tis...@stackless.com>> wrote: >>> >>> Hi all, >>> >>> Ok, I am about to implement generation of such structures >>> automatically using the struct sequence concept. >>> >>> >>> One more question: >>> -- >>> >>> Struct sequences are not yet members of the typing types. >>> I would like to add that, because a major use case is also to >>> show nice .pyi files with all the functions and types. >>> >>> * namedtuple has made the transition to NamedTuple >>> >>> * What would I need to do that for StructSequence as well? >>>
Re: [Python-Dev] Slides from today's parallel/async Python talk
Hi Trent, I just started to try to understand the idea and the implications. Removing almost all of your message since that is already too long to work with: The reference is http://mail.python.org/pipermail/python-dev/2013-March/124690.html On 3/14/13 11:45 AM, Trent Nelson wrote: On Wed, Mar 13, 2013 at 07:05:41PM -0700, Trent Nelson wrote: Just posted the slides for those that didn't have the benefit of attending the language summit today: https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async Someone on /r/python asked if I could elaborate on the "do Y" part of "if we're in a parallel thread, do Y, if not, do X", which I (inadvertently) ended up replying to in detail. I've included the response below. (I'll work on converting this into a TL;DR set of slides soon.) Can you go into a bit of depth about "X" here? That's a huge topic that I'm hoping to tackle ASAP. The basic premise is that parallel 'Context' objects (well, structs) are allocated for each parallel thread callback. The context persists for the lifetime of the "parallel work". So, the remaining challenge is preventing the use case alluded to earlier where someone tries to modify an object that hasn't been "async protected". That's a bit harder. The idea I've got in mind is to instrument the main CPython ceval loop, such that we do these checks as part of opcode processing. That allows us to keep all the logic in the one spot and not have to go hacking the internals of every single object's C backend to ensure correctness. Now, that'll probably work to an extent. I mean, after all, there are opcodes for all the things we'd be interested in instrumenting, LOAD_GLOBAL, STORE_GLOBAL, SETITEM etc. What becomes challenging is detecting arbitrary mutations via object calls, i.e. how do we know, during the ceval loop, that foo.append(x) needs to be treated specially if foo is a main-thread object and x is a parallel thread object? There may be no way to handle that *other* than hacking the internals of each object, unfortunately. So, the viability of this whole approach may rest on whether or that's deemed as an acceptable tradeoff (a necessary evil, even) to the Python developer community. This is pretty much my concern: In order to make this waterproof, as required for CPython, you will quite likely have to do something on very many objects, and this is hard to chime into CPython. If it's not, then it's unlikely this approach will ever see the light of day in CPython. If that turns out to be the case, then I see this project taking the path that Stackless took (forking off and becoming a separate interpreter). We had that discussion quite often for Stackless, and I would love to find a solution that allows to add special versions and use cases to CPython in a way that avoids the forking as we did it. It would be a nice thing if we could come up with a way to keep CPython in place, but to swap the interpreter out and replace it with a specialized version, if the application needs it. I wonder to what extent that would be possible. What I would like to achieve, after having given up on Stackless integration is a way to let it piggyback onto CPython that works like an extension module, although it hat effectively replace larger parts of the interpreter. I wonder if that might be the superior way to have more flexibility, without forcing everything and all go into CPython. If we can make the interpreter somehow pluggable at runtime, a lot of issues would become much simpler. There's nothing wrong with that; I am really excited about the possibilities afforded by this approach, and I'm sure it will pique the interest of commercial entities out there that have problems perfectly suited to where this pattern excels (shared-nothing, highly concurrent), much like the relationship that developed between Stackless and Eve Online. What do you think: does it make sense to think of a framework that allows to replace the interpreter at runtime, without making normal CPython really slower? cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The end of 2.7
On 07.04.13 03:54, Raymond Hettinger wrote: On Apr 6, 2013, at 2:02 PM, Benjamin Peterson <mailto:benja...@python.org>> wrote: we need to talk about how many more 2.7 releases there are going to be. At the release of 2.7.0, I thought we promised 5 years of bugfix maintenance, but my memory may be fuddled. I don't we need to make any "promises" beyond 5 years, but I also think it is likely the 2.7 will end-up being a long-term maintenance version of Python. At this year's Pycon keynote, I surveyed the crowd (approx 2500 people) and all almost everyone indicated that they had tried out Python 3.x and almost no one was using it in production or writing code for it. That indicates that Python 2.7 will continue to be important for a good while. In addition, the other implementations of Python (Jython, PyPy, GAE, and IronPython) are all at or nearly at Python 2.7. So, continued support will be needed for their users as well. After 2.7.4, I expect that the pace of real bug fixes will slow down, but that we'll continue to improve docs, add docstrings, update IDLE, etc. IMO, it is premature to utter the phrase "the end of 2.7". Better to say, "2.7 is stable and is expected to only have minor updates". Future point releases probably ought to occur "on their own schedule" whenever there are a sufficient number of changes to warrant a release, or an important security fix, or whenever the release managers have time. Raimond, although I think you are right, I don't think this statement helps Python to move forward. Your vision is realistic, nevertheless I think it is important to tell everybody that 2013 is the year to move to Python 3. If enough people like you, Alex, Dabeaz, etc are claiming that loudly enough, it will eventually happen! cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The end of 2.7
Hi Skip, On 07.04.13 14:10, Skip Montanaro wrote: I started writing this last night before the flurry of messages which arrived overnight. I thought originally, "Oh, Skip, you're being too harsh." But now I'm not so sure. I think you are approaching the issue of 2.7's EOL incorrectly. Of those discussing the end of Python 2.7, how many of you still use it in your day-to-day work? Have any of you yet to move to Python 3? It sounds like many people at PyCon are still 2.x users. Where I work (a trading firm that uses Python as just one of many different pieces of technology, not a company where Python is the core technology upon which the firm is based) we are only just now migrating from 2.4 to 2.7. I can't imagine we'll have migrated to Python 3 in two years. It's not like we haven't seen this coming, but you can only justify moving so fast with technology that already works, especially if, like Python, you use it with lots of other packages (most/all of which themselves have to be ported to Python 3) and in-house software. I think the discussion should focus on who's left on 2.x and why, not, "yeah, releases every six months for the next couple years ought to do it." when I read this, I was slightly shocked. You know what? """ We are pleased to announce the release of*Python 2.4, final*on November 30, 2004. """ I know that companies try to save (time? money?) something by not upgrading software, and this is extremely annoying. In my own project, which is for a customer, I just managed to do the complete transition from Python 2.7 to 3.3. Well, this was relatively simple because there is just my boss to be convinced, and myself, because honestly the 3.3 support is still not as good as needed. But I think every employee (including you) can quite easily put some pressure on his company by claiming that Python 2.x is a dead end, and everybody is about to move on to 3.x. This does not have to be true, I just recognize that by claiming it and doing it with your projects, the movement becomes a reality. Just say that we all need to move on and cannot care about companies that ignore this necessity. I agree it is hard to push things forward, when certain tools are just supporting 2.x. My way to get over this is ranting, and porting some things, and claiming it was a cake walk. A lie, but it helped. my 2.01 cent -- chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Multiline Strings confusion it tutorial
I wanted to point a bling guy to the Python wiki: http://wiki.python.org/moin/BeginnersGuide/Programmers/SimpleExamples#preview and when reading a little bit, I found the entry on multiline strings. I found that example pretty contorted, because this is not a multiline string. Instead, there are multiple lines which define a single line string! Actually, the construct is even syntactically nothing else than a single line string which is handled by the parser, already. A multiline string is IMHO a string which value covers multiple lines, after whatever pre-processing was done. I don't think the given example is very helpful, but adds confusion. Where would I add such a complaint, usually? Or should I simply fix it? cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More compact dictionaries with faster iteration
Hi Raymond,
On 08.01.13 15:49, Maciej Fijalkowski wrote:
On Mon, Dec 10, 2012 at 3:44 AM, Raymond Hettinger
wrote:
The current memory layout for dictionaries is
unnecessarily inefficient. It has a sparse table of
24-byte entries containing the hash value, key pointer,
and value pointer.
Instead, the 24-byte entries should be stored in a
dense table referenced by a sparse table of indices.
For example, the dictionary:
d = {'timmy': 'red', 'barry': 'green', 'guido': 'blue'}
is currently stored as:
entries = [['--', '--', '--'],
[-8522787127447073495, 'barry', 'green'],
['--', '--', '--'],
['--', '--', '--'],
['--', '--', '--'],
[-9092791511155847987, 'timmy', 'red'],
['--', '--', '--'],
[-6480567542315338377, 'guido', 'blue']]
Instead, the data should be organized as follows:
indices = [None, 1, None, None, None, 0, None, 2]
entries = [[-9092791511155847987, 'timmy', 'red'],
[-8522787127447073495, 'barry', 'green'],
[-6480567542315338377, 'guido', 'blue']]
Only the data layout needs to change. The hash table
algorithms would stay the same. All of the current
optimizations would be kept, including key-sharing
dicts and custom lookup functions for string-only
dicts. There is no change to the hash functions, the
table search order, or collision statistics.
The memory savings are significant (from 30% to 95%
compression depending on the how full the table is).
Small dicts (size 0, 1, or 2) get the most benefit.
For a sparse table of size t with n entries, the sizes are:
curr_size = 24 * t
new_size = 24 * n + sizeof(index) * t
In the above timmy/barry/guido example, the current
size is 192 bytes (eight 24-byte entries) and the new
size is 80 bytes (three 24-byte entries plus eight
1-byte indices). That gives 58% compression.
Note, the sizeof(index) can be as small as a single
byte for small dicts, two bytes for bigger dicts and
up to sizeof(Py_ssize_t) for huge dict.
In addition to space savings, the new memory layout
makes iteration faster. Currently, keys(), values, and
items() loop over the sparse table, skipping-over free
slots in the hash table. Now, keys/values/items can
loop directly over the dense table, using fewer memory
accesses.
Another benefit is that resizing is faster and
touches fewer pieces of memory. Currently, every
hash/key/value entry is moved or copied during a
resize. In the new layout, only the indices are
updated. For the most part, the hash/key/value entries
never move (except for an occasional swap to fill a
hole left by a deletion).
With the reduced memory footprint, we can also expect
better cache utilization.
For those wanting to experiment with the design,
there is a pure Python proof-of-concept here:
http://code.activestate.com/recipes/578375
YMMV: Keep in mind that the above size statics assume a
build with 64-bit Py_ssize_t and 64-bit pointers. The
space savings percentages are a bit different on other
builds. Also, note that in many applications, the size
of the data dominates the size of the container (i.e.
the weight of a bucket of water is mostly the water,
not the bucket).
Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
One question Raymond.
The compression ratios stay true provided you don't overallocate entry
list. If you do overallocate you don't really gain that much (it all
depends vastly on details), or even loose in some cases. What do you
think should the strategy be?
What is the current status of this discussion?
I'd like to know whether it is a considered alternative implementation.
There is also a discussion in python-ideas right now where this
alternative is mentioned, and I think especially for small dicts
as **kwargs, it could be a cheap way to introduce order.
Is this going on, somewhere? I'm quite interested on that.
cheers - chris
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More compact dictionaries with faster iteration
On 15.05.13 14:01, Stefan Drees wrote: Hi Chris, On 15.05.13 13:32 Christian Tismer wrote: Hi Raymond, On 08.01.13 15:49, Maciej Fijalkowski wrote: On Mon, Dec 10, 2012 at 3:44 AM, Raymond Hettinger wrote: The current memory layout for dictionaries is unnecessarily inefficient. It has a sparse table of 24-byte entries containing the hash value, key pointer, and value pointer. ... What is the current status of this discussion? I'd like to know whether it is a considered alternative implementation. There is also a discussion in python-ideas right now where this alternative is mentioned, and I think especially for small dicts as **kwargs, it could be a cheap way to introduce order. Is this going on, somewhere? I'm quite interested on that. +1 I am also interested on the status. Many people seemed to have copied the recipe from the activestate site (was it?) but I wonder if it maybe was to cool to be progressed into "the field" or simply some understandable lack of resources? Right, found the references: http://mail.python.org/pipermail/python-dev/2012-December/123028.html http://stackoverflow.com/questions/14664620/python-dictionary-details http://code.activestate.com/recipes/578375-proof-of-concept-for-a-more-space-efficient-faster/?in=user-178123 cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __future__ and eval()
Hi guys, when developing my dedent tool, I stumbled over the following behavior: In dedent, I had the line from __future__ import print_function Later in the script, I do some exec(the_script) and I was surprised: The exec() script inherited the __future__ statement! It behaved like the future statement were implicitly there. Is that a bug or a feature? You can try the effect by "pip install dedent" and adding the future statement there. I'd like to know if this is a bug (and I think so) -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __future__ and eval()
Ah, interesting!
Thanks for the clarification. So it is really possible to write
code with an implicit future statement in it, or to switch the behavior
off.
Good to know. I will probably not use it, since I can't decide on a good
default, but getting rid of print_statement is tempting...
> https://docs.python.org/2/reference/simple_stmts.html#the-exec-statement
>
> [2] When strings are executed, __future__ directives active in the
> surrounding context will be active for the compiled code also. If this
> is not desired, see the compile() function's dont_inherit parameter.
>
> Would that clarify?
Yes please, that would be a good place to document it. For some reason
I did not look up __future__.
Thanks -- Chris
On 01/10/16 14:17, Chris Angelico wrote:
On Sat, Oct 1, 2016 at 9:39 PM, Christian Tismer wrote:
The exec() script inherited the __future__ statement!
It behaved like the future statement were implicitly there.
Is that a bug or a feature?
It's documented, but not very noisily.
https://docs.python.org/2/reference/simple_stmts.html#future-statements
So if you want to isolate the execution environments, you can use
compile() explicitly. Without isolation:
Python 2.7.12+ (default, Sep 1 2016, 20:27:38)
[GCC 6.2.0 20160822] on linux2
Type "help", "copyright", "credits" or "license" for more information.
exec(compile("print('hello','world')", "-", "exec"))
('hello', 'world')
exec(compile("from __future__ import print_function; print('hello','world')", "-",
"exec"))
hello world
from __future__ import print_function
exec(compile("print('hello','world')", "-", "exec"))
hello world
exec(compile("from __future__ import print_function; print('hello','world')", "-",
"exec"))
hello world
With isolation:
exec(compile("print('hello','world')", "-", "exec", 0, 1))
('hello', 'world')
exec(compile("from __future__ import print_function; print('hello','world')", "-",
"exec", 0, 1))
hello world
So I'd call it a feature, but possibly one that warrants a mention in
the exec and eval docs. Maybe something like:
https://docs.python.org/2/reference/simple_stmts.html#the-exec-statement
[2] When strings are executed, __future__ directives active in the
surrounding context will be active for the compiled code also. If this
is not desired, see the compile() function's dont_inherit parameter.
Would that clarify?
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Debugging Python scripts with GDB on OSX
Hi Alexandru, I stumbled over this question a little late by chance. There is the possibility to use GDB, but it is most likely that you want to use python's pdb module, instead. Only in rare cases, when debugging the interpreter itself, you use gdb. For debugging Python code, use pdb or something better. Sent from my Ei4Steve > On Jul 6, 2016, at 18:14, Alexandru Croitor wrote: > > Hello, > > I'm interested to find out if debugging Python scripts with GDB is supported > on OSX at all? > > I'm referring to the functionality described on > https://wiki.python.org/moin/DebuggingWithGdb and on > http://fedoraproject.org/wiki/Features/EasierPythonDebugging. > > I've tried so far various combinations of pre-compiled GDB from the homebrew > package manager, locally-compiled GDB from homebrew, as well as locally > compiled GDB from MacPorts, together with a pre-compiled Python 2.7, > homebrew-compiled 2.7, and custom compiled Python 2.7 from the official > source tarball. > > My results so far were not successful. The legacy GDB commands to show a > python stack trace or the local variables - do not work. And the new GDB > commands (referenced on the Fedora project page) are not present at all in > any of the GDB versions. > > I've checked the python CI build bot tests, and it seems the new GDB commands > are only successfully tested on Linux machines, and are skipped on FreeBSD, > OS X, and Solaris machines. > > Are the new python <-> GDB commands specific to Linux? > Are there any considerations to take in regards to debug symbols for Python / > GDB on OSX? > > Has anyone attempted what I'm trying to do? > > I would be grateful for any advice. > > And I apologize if my choice of the mailing lists is not the best. > > Regards, Alex. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Debugging Python scripts with GDB on OSX
Hi Alexandru, because I know that you are multi-platform, I can recommend the debugger integration of PTVS very much. I am currently using the WingWare debugger for all my projects, but the Python/C++ integration of PTVS is something that I didn't find anywhere else! Please have a look at the following introduction. https://www.youtube.com/watch?v=JNNAOypc6Ek Starting with minute 22, you will find the kind of debugging that you are looking for. I was pretty amazed by this, and it is probably very helpful in debugging Qt and PySide. Will give it a try, soon. Cheers -- Chris On 14/10/2016 11:12, Alexandru Croitor wrote: > Hi, > > pdb is fine for pure python scripts. > > I was interested in things like getting the python back trace or local > variables from inside GDB, when used in conjunction with c++, so that I > know which parts of C++ calls python functions, and which parts of > python call c++ functions. You can't do that with pdb. > > >> On 13 Oct 2016, at 19:12, Christian Tismer > <mailto:tis...@stackless.com>> wrote: >> >> Hi Alexandru, >> >> I stumbled over this question a little late by chance. >> >> There is the possibility to use GDB, but it is most likely that you >> want to use python's pdb module, instead. >> >> Only in rare cases, when debugging the interpreter itself, you use >> gdb. For debugging Python code, use pdb or something better. >> >> Sent from my Ei4Steve >> >> On Jul 6, 2016, at 18:14, Alexandru Croitor > <mailto:alexandru.croi...@qt.io>> wrote: >> >>> Hello, >>> >>> I'm interested to find out if debugging Python scripts with GDB is >>> supported on OSX at all? >>> >>> I'm referring to the functionality described >>> on https://wiki.python.org/moin/DebuggingWithGdb and >>> on http://fedoraproject.org/wiki/Features/EasierPythonDebugging. >>> >>> I've tried so far various combinations of pre-compiled GDB from the >>> homebrew package manager, locally-compiled GDB from homebrew, as well >>> as locally compiled GDB from MacPorts, together with a pre-compiled >>> Python 2.7, homebrew-compiled 2.7, and custom compiled Python 2.7 >>> from the official source tarball. >>> >>> My results so far were not successful. The legacy GDB commands to >>> show a python stack trace or the local variables - do not work. And >>> the new GDB commands (referenced on the Fedora project page) are not >>> present at all in any of the GDB versions. >>> >>> I've checked the python CI build bot tests, and it seems the new GDB >>> commands are only successfully tested on Linux machines, and are >>> skipped on FreeBSD, OS X, and Solaris machines. >>> >>> Are the new python <-> GDB commands specific to Linux? >>> Are there any considerations to take in regards to debug symbols for >>> Python / GDB on OSX? >>> >>> Has anyone attempted what I'm trying to do? >>> >>> I would be grateful for any advice. >>> >>> And I apologize if my choice of the mailing lists is not the best. >>> >>> Regards, Alex. >>> >>> >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org <mailto:Python-Dev@python.org> >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com > -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __qualname__ format question
Hi friends,
by chance, I stumbled over
meth_get__qualname__
in methodobject.c and
calculate_qualname
in descrobject.c .
The first uses
res = PyUnicode_FromFormat("%S.%s", type_qualname, m->m_ml->ml_name);
and the latter uses
res = PyUnicode_FromFormat("%S.%S", type_qualname, descr->d_name);
To my knowledge, the "%S" character is undefined in C99 and C11.
Q: Why this character, and why this difference?
cheers - Chris
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __qualname__ format question
On 25.06.17 14:41, Serhiy Storchaka wrote:
> 25.06.17 15:06, Christian Tismer пише:
>> by chance, I stumbled over
>>
>> meth_get__qualname__
>>
>> in methodobject.c and
>>
>> calculate_qualname
>>
>> in descrobject.c .
>>
>> The first uses
>>
>> res = PyUnicode_FromFormat("%S.%s", type_qualname,
>> m->m_ml->ml_name);
>>
>> and the latter uses
>>
>> res = PyUnicode_FromFormat("%S.%S", type_qualname, descr->d_name);
>>
>> To my knowledge, the "%S" character is undefined in C99 and C11.
>>
>> Q: Why this character, and why this difference?
>
> Se the documentation of PyUnicode_FromFormat().
>
> https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_FromFormat
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
Ah, thank you very much.
Cheers - Chris
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __signature__ for PySide ready
Hi friends, in the last months, I have developed signature support for PySide. The module creates the same signatures as are known for plain Python functions. As a non-trivial addition, the module also handles multiple signatures as a list. I consider this extension to PySide as quite essential and actually more important as for Python itself, because type info is rather crucial for PySide. Initially, I wrote this as a pure Python 3 extension. Then I was "asked" to port this to Python 2 too, which was quite hairy to do. I'm not sure if I should have done that. Before I publish this module, I want to ask: Is it a bad idea to support signatures in Python 2 as well? Do I introduce a feature that should not exist in Python 2? Or is it fine to do so? Please let me know your opinion, I am happy with any result. Cheers -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __signature__ for PySide ready
Hi Nick, On 19.08.17 09:37, Nick Coghlan wrote: > On 19 August 2017 at 02:55, Jelle Zijlstra wrote: >> 2017-08-18 18:20 GMT+02:00 Yury Selivanov : >>> There's a backport of signature API to Python 2. Although last time I >>> checked it was fairly outdated. >>> >> I wrote a backport earlier this year: https://pypi.python.org/pypi/inspect2. > > Nice. > > Yury was probably referring to https://funcsigs.readthedocs.io/, which > is a partial backport specifically of inspect.Signature and friends. > > Either way, the answer to Christian's original question is that "Yes, > supporting __signature__ is also useful in Python 2.x", as even though > the native inspect module doesn't support it, 3rd party backports do. I did not use a backport, but simply hacked it together, myself: - Take the signature part of Python 3.6 out, - Adjust the syntax to Python 2.7. The hairy part was my way to create the PySide PyCFunction objects in a compatible way like Python 3 does it: the handling of PyCFunction's "self" parameter is different (holds type info, Python 2 does not). That needed much more internal knowledge as intended... Well, I thought the existence of __signature__ might be a good reason to switch to Python 3, but if I support Python 2, the advantage is gone. But if it's ok with you, then I'll publish both versions. Thanks a lot for the feedback. Ciao - Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __signature__ for PySide ready
Hi Brett, On 18.08.17 18:31, Brett Cannon wrote: > > > On Fri, 18 Aug 2017 at 02:05 Christian Tismer <mailto:tis...@stackless.com>> wrote: > ... > Is it a bad idea to support signatures in Python 2 as well? > Do I introduce a feature that should not exist in Python 2? > Or is it fine to do so? > > Please let me know your opinion, I am happy with any result. > > > If you're getting paid to do the port then I don't think it really hurts > anything since it isn't going to magically open Python 2 to more usage. > In fact, if you are filling in the annotation information so that type > hints are being exposed then maybe there's a chance it might help > someone port to Python 3? Well, I took extra precautions to make sure that existing PyCFunctions are not affected in any way. The new types are clones of the original types, and only PySide sees the additional attribute for it's functions. So, no, I don not create magically signatures for Python 2 functions. But I really warned my customer that the feature might not be welcomed for Python 2, and we might only enable it internally for testing. As said, I'm happy with either answer. Cheers -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] limited_api and datetime
Hi Nick, On 23.08.17 07:41, Nick Coghlan wrote: > On 23 August 2017 at 00:09, stackless wrote: >> Hi again, >> >> I am trying to support the limited Api (PEP 384) for PySide. Fortunately, >> everything worked out of the box, just the datetime module gives me >> a problem. Inspection showed that datetime is completely removed >> when the limitation is set. >> >> Can somebody enlighten me why that needs to be so? > > The main problem is that "datetime.h" defines several C structs that > we wouldn't want to add to the stable ABI, and nobody has previously > worked through details of the interface to see which parts of it could > still be used even if those structs were made opaque, and the macros > relying on them were switched to being functions instead. > > There are almost certainly pieces of it that *could* be usefully > exposed, though. > >> And how should I use datetime, instead: Fish the functions out of >> the dynamically imported datetime module? > > Yep, going via the Python level API rather than directly to the C API > is the default answer, as that inherently hides any struct layout > details behind PyObject*. Thank you very much for the clarification. I think we can live with the Python interface for now. Now I'm sure that I'm going the way to go. Cheers -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: typing: how to use names in result-tuples?
On 08.08.19 17:20, Ronald Oussoren via Python-Dev wrote: > > >> On 8 Aug 2019, at 17:12, Christian Tismer > <mailto:tis...@stackless.com>> wrote: >> >> Hi Ronald, >> >> sure, the tuple is usually not very interesting; people look it up >> once and use that info in the code. >> >> But I think things can be made quite efficient and pretty at the >> same time. Such a return tuple could be hidden like the stat_result >> example that Guido mentioned: >> >> https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi >> >> def stat(self, *, follow_symlinks: bool = ...) -> stat_result: ... >> >> The stat_result is a huge structure where you don't want to see much >> unless you are working with a stat_result. >> >> Other with common, repeating patterns like (x, y, width, height) >> or your examples: >> >> def getpoint(v: pointless) -> (int, int) >> def getvalue(v: someclass) -> (bool, int) >> >> would be at first sight >> >> def getpoint(v: pointless) -> getpoint_result >> def getvalue(v: someclass) -> getvalue_result >> >> But actually, a much nicer, speaking outcome would be written as >> the single function StructSequence(...) with arguments, producing: >> >> def getpoint(v: pointless) -> StructSequence(x=int, y=int) >> def getvalue(v: someclass) -> StructSequence(result=bool, val=int) >> >> That would have the nice effect of a very visible structure in >> the .pyi file. When you actually get such an object and look at it, >> then you have > > But will you ever look at these objects, other then when exploring APIs > in the REPL? As I wrote earlier the normal usage for a similar pattern > in PyObjC is to always immediately deconstruct the tuple into its > separate values. Yes, it is for exploring the interface. In Qt5, you have *very* many functions, and they are pretty unpythonic as well. That was the reason at all for me to write that __signature__ module for PySide, that does everything with introspection. When I'm programming with it, then half as a user who wants to see "yes, it really returns such a value" and as a developer "shit, we claim that interface, but lied". In a sense, it was also a way to test this huge library automatically, and I enjoy it when things can explain themselves. That is absolutely not necessary. As the whole typing idea and typeshed is not necessary, but for me it was a huge win to have a typed interface, and IDE users seem to love it when PyCharm suddenly talks to them ;-) > BTW. I’m primarily trying to understand your use case because it is so > similar to what I’m doing in PyObjC, and such understanding can lead to > an improvement in PyObjC ;-). I am happy to learn from your projects as well! All the best -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TUQVQIRTBSI6HCWBRKAZTCCTNVKCLXJJ/
[Python-Dev] Re: typing: how to use names in result-tuples?
Yes, that's what I mean.
Probably retval or whatever people prefer would be adequate,
with a special rule if that name is taken.
I think btw. that using StructSequence(somename=sometype, ..., ) that
does a dict lookup is quite appealing. It returns a class like
stat_result, but the function call shows its arguments (see answer to
Ronald).
Ciao -- Chris
On 08.08.19 17:22, Guido van Rossum wrote:
> OK, yes, having a convention for naming the anonymous field sounds like
> the way to go. But presuming people will want to use it, and probably
> know it from the C++ API, why not name it 'return_value' or 'result' or
> 'retval' or something like that?
>
> On Thu, Aug 8, 2019 at 1:43 AM Christian Tismer <mailto:tis...@stackless.com>> wrote:
>
> Hi Guido,
>
> If a C++ function already has a return value, plus some primitive
> pointer variables that need to be moved into the result in Python,
> then we have the case with a first, single unnamed field.
> Only one such field can exist.
>
> I'm not sure if that case exists in the ~25000 Qt5 functions, but in
> that case, I think to give that single field the name "unnamed"
> or maybe better "result".
>
> Thank you very much for pointing me to that example!
>
> Cheers -- Chris
>
>
> On 08.08.19 06:41, Guido van Rossum wrote:
> > Alas, we didn't think of struct sequences when we designed PEP 484. It
> > seems they are a hybrid of Tuple and NamedTuple; both of these are
> > currently special-cased in mypy in ways that cannot easily be
> combined.
> >
> > Do you really need anonymous fields?
> >
> > I see an example in typeshed/stdlib/3/os/__init__.pyi (in
> > github.com/python/typeshed <http://github.com/python/typeshed>
> <http://github.com/python/typeshed>), for
> > stat_result. It defines names for all the fields, plus a __getitem__()
> > method that indicates that indexing returns an int. This doesn't
> help if
> > anonymous fields could have different types, not does it teach the
> type
> > checker about the number of anonymous fields.
> >
> > --Guido
> >
> > On Wed, Aug 7, 2019 at 1:51 AM Christian Tismer
> mailto:tis...@stackless.com>
> > <mailto:tis...@stackless.com <mailto:tis...@stackless.com>>> wrote:
> >
> > Hi all,
> >
> > Ok, I am about to implement generation of such structures
> > automatically using the struct sequence concept.
> >
> >
> > One more question:
> > --
> >
> > Struct sequences are not yet members of the typing types.
> > I would like to add that, because a major use case is also to
> > show nice .pyi files with all the functions and types.
> >
> > * namedtuple has made the transition to NamedTuple
> >
> > * What would I need to do that for StructSequence as well?
> >
> > Things get also a bit more complicated since struct sequence
> > objects can contain unnamed fields.
> >
> > Any advice would be appreciated, I am no typing expert (yet :-)
> >
> > cheers -- Chris
> >
> >
> > On 30.07.19 17:10, Guido van Rossum wrote:
> > > I think I have to agree with Petr. Define explicit type names.
> > >
> > > On Tue, Jul 30, 2019 at 2:45 AM Paul Moore
> mailto:p.f.mo...@gmail.com>
> > <mailto:p.f.mo...@gmail.com <mailto:p.f.mo...@gmail.com>>
> > > <mailto:p.f.mo...@gmail.com <mailto:p.f.mo...@gmail.com>
> <mailto:p.f.mo...@gmail.com <mailto:p.f.mo...@gmail.com>>>> wrote:
> > >
> > > On Tue, 30 Jul 2019 at 09:33, Christian Tismer
> > mailto:tis...@stackless.com>
> <mailto:tis...@stackless.com <mailto:tis...@stackless.com>>
> > > <mailto:tis...@stackless.com
> <mailto:tis...@stackless.com> <mailto:tis...@stackless.com
> <mailto:tis...@stackless.com>>>>
> > wrote:
> > > > >>> typing.NamedTuple("__f", x=int, y=int)
> > > >
> > > > >>> typing.NamedTuple("__f", x=int, y=int) is
> > typing.NamedTuple("__f",
> > > > x=int, y=int)
>
[Python-Dev] Re: PEP 572 TargetScopeError
On 09.08.19 03:53, Eric V. Smith wrote: > On 8/8/2019 7:28 PM, Barry Warsaw wrote: >> On Aug 8, 2019, at 14:58, Steven D'Aprano wrote: >> >>> It's not a syntax error. There's nothing wrong with the syntax per-say: >>> we still have ``target := expression``. There's a problem with the >>> *semantics* not the syntax. >> >> I’m not sure that distinction is meaningful though. What you wrote is >> disallowed, so you have to change your code (i.e. syntax) to avoid the >> error. > > I agree with Barry here. For all practical purposes, it's a syntax > error. And if we had a more powerful parser, maybe it really would be a > syntax error. See Guido's recent PEG parser articles > (https://medium.com/@gvanrossum_83706/peg-parsers-7ed72462f97c) where he > discusses raising SyntaxError after the parser is finished, in a > subsequent pass. Is it really a syntax error if pgen doesn't object to > it? In current CPython, the answer is yes. ... OT: Thanks for the interesting read! I am excited which way it will continue. -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/K4Y57KYAK7B6DVKWV7R2RQJH55V7V3WF/
[Python-Dev] Re: PEP 572 TargetScopeError
On 09.08.19 15:44, Nick Coghlan wrote: ... > However, a change like that would make the most sense when considered > independently of the specific case of assignment expressions - > instead, we would want to ask "What kinds of exceptions does the > symbol table analysis pass raise?", and then consider whether it might > make sense to define subclasses of syntax error for them. > > Given the naming issue, and the fact that a potential new SyntaxError > subclass would be relevant for more than just the new PEP 572 > exceptions, I think the right near term approach is to go with the > generic SyntaxError as Barry suggests. I'll update my PRs accordingly. Totally agree. It is fine to have SyntaxError now and go for one or more new subclasses for a whole bunch of errors at a later time, fixing more things in a more consistent way. -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/B2YAMST5CQWPIHPM5DS56TV4CYZDTZ73/
[Python-Dev] Re: What is a public API?
On 16.07.19 00:32, Barry Warsaw wrote: > On Jul 13, 2019, at 19:09, Raymond Hettinger > wrote: >> >> In some modules, we've been careful to use both __all__ and to use an >> underscore prefix to indicate private variables and helper functions >> (collections and random for example). IMO, when a module has shown that >> care, future maintainers should stick with that practice. >> >> The calendar module is an example of where that care was taken for many >> years and then a recent patch went against that practice. This came to my >> attention when an end-user questioned which functions were for internal use >> only and posted their question on Twitter. On the tracker, I then made a >> simple request to restore the module's convention but you seem steadfastly >> resistant to the suggestion. >> >> When we do have evidence of user confusion (as in the case with the calendar >> module), we should just fix it. IMO, it would be an undue burden on the >> user to have to check every method in dir() against the contents of __all__ >> to determine what is public (see below). Also, as a maintainer of the >> module, I would not have found it obvious whether the functions were public >> or not. The non-public functions look just like the public ones. >> >> It's true that the practices across the standard library have historically >> been loose and varied (__all__ wasn't always used and wasn't always kept >> up-to-date, some modules took care with private underscore names and some >> didn't). To me this has mostly worked out fine and didn't require a strict >> rule for all modules everywhere. IMO, there is no need to sweep through the >> library and change long-standing policies on existing modules. > > EIBTI > > Shameless plug: https://public.readthedocs.io/en/latest/ Hey, what a fantastic little module! I'll hurry and use that a lot! Especially the builtins idea is really great :-P Cheers - Chris p.s.: How about adding @private as well? There are cases where I would like to do the opposite: __all__ = dir() @private _some_private_func_1(...): ... ... @private _some_private_func_n(...): ... not-too-seriously yours - Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HUSOXEYFEBJRMQPC5CSFRB2DDMAARBFV/
[Python-Dev] Re: typing: how to use names in result-tuples?
On 12.08.19 10:52, Ivan Levkivskyi wrote: > On Thu, 8 Aug 2019 at 17:17, Christian Tismer <mailto:tis...@stackless.com>> wrote: > > Yes, that's what I mean. > Probably retval or whatever people prefer would be adequate, > with a special rule if that name is taken. > > I think btw. that using StructSequence(somename=sometype, ..., ) that > does a dict lookup is quite appealing. It returns a class like > stat_result, but the function call shows its arguments (see answer to > Ronald). > > Ciao -- Chris > > > Just a little comment: there is a (vague) plan to add a feature (key > types) to the type system that will allow user-defined constructs > similar to NamedTuple (for example StructSequence you mention). > However, taking into account current schedule it is unlikely it will be > added before mid-2020, so your best bet is indeed using ad-hoc named tuples. Interesting! Can I read something more about this? I'm curious ;-) But I guess it will anyway take a little time until I can make that transition. Maybe it's better to see what's coming up with typing, mypy, typing_inspect and friends. Well, I will probably start a prototype dev branch for that. Cheers -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IZM2CKBMBUYH3QPOBTDKRBLNQDYACI75/
[Python-Dev] Python 3.8 problem with PySide
Hi guys, during the last few weeks I have been struggling quite much in order to make PySide run with Python 3.8 at all. The expected problems were refcounting leaks due to changed handling of heaptypes. But in fact, the runtime behavior was much worse, because I always got negative refcounts! After exhaustive searching through the different 3.8 commits, I could isolate the three problems with logarithmic search. The hard problem was this: Whenever PySide creates a new type, it crashes in PyType_Ready. The reason is the existence of the Py_TPFLAGS_METHOD_DESCRIPTOR flag. During the PyType_Ready call, the function mro() is called. This mro() call results in a negative refcount, because something behaves differently since this flag is set by default in mro(). When I patched this flag away during the type_new call, everything worked ok. I don't understand why this problem affects PySide at all. Here is the code that would normally be only the newType line: // PYSIDE-939: This is a temporary patch that circumvents the problem // with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved. PyObject *ob_PyType_Type = reinterpret_cast(&PyType_Type); PyObject *mro = PyObject_GetAttr(ob_PyType_Type, Shiboken::PyName::mro()); auto hold = Py_TYPE(mro)->tp_flags; Py_TYPE(mro)->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR; auto *newType = reinterpret_cast(type_new(metatype, args, kwds)); Py_TYPE(mro)->tp_flags = hold; I would really like to understand the reason for this unexpected effect. Does this ring a bell? I have no clue what is wrong with PySide, if it is wrong at all. Thanks -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U46CQTHQDSFMJ3QXHJZT6FXVVKXRF2JB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Python 3.8 problem with PySide
On 08.12.19 09:49, Nick Coghlan wrote: > On Fri., 6 Dec. 2019, 3:31 am Christian Tismer, <mailto:tis...@stackless.com>> wrote: > > Hi guys, > > during the last few weeks I have been struggling quite much > in order to make PySide run with Python 3.8 at all. > > The expected problems were refcounting leaks due to changed > handling of heaptypes. But in fact, the runtime behavior was > much worse, because I always got negative refcounts! > > After exhaustive searching through the different 3.8 commits, I could > isolate the three problems with logarithmic search. > > The hard problem was this: > Whenever PySide creates a new type, it crashes in PyType_Ready. > The reason is the existence of the Py_TPFLAGS_METHOD_DESCRIPTOR > flag. > During the PyType_Ready call, the function mro() is called. > This mro() call results in a negative refcount, because something > behaves differently since this flag is set by default in mro(). > > When I patched this flag away during the type_new call, everything > worked ok. I don't understand why this problem affects PySide > at all. Here is the code that would normally be only the newType line: > > > // PYSIDE-939: This is a temporary patch that circumvents the > problem > // with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved. > PyObject *ob_PyType_Type = reinterpret_cast *>(&PyType_Type); > PyObject *mro = PyObject_GetAttr(ob_PyType_Type, > Shiboken::PyName::mro()); > auto hold = Py_TYPE(mro)->tp_flags; > Py_TYPE(mro)->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR; > auto *newType = reinterpret_cast(type_new(metatype, > args, kwds)); > Py_TYPE(mro)->tp_flags = hold; > > > Isn't this manipulating the flags in the tuple type, rather than > anything on a custom object? Or is "mro" a custom object rather than an > MRO tuple? no, "mro" is the default mro implementation which is a method descriptor of the standard PyTypeType object. The implementation of PyType_Ready just touches the mro in some helper function lookup_maybe_method. This is so funny: This side effect seems to be totally unrelated to PySide, but something we are doing wrong. > If anything, given the combination of factors required to reproduce the > problem, I would guess that there might be a ref counting problem in the > __set_owner__ invocations when called on a new type rather than a > regular instance, and that was somehow affected by the change to > increment the type refcount in PyObject_Init rather than > PyType_GenericAlloc. Thanks a lot! I will try to use that to find finally what's wrong. Cheers -- Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VGHUHPRPUTNEKZL7HT22W5V2VWCL6BOZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Python 3.8 error? (was: Python 3.8 problem with PySide)
On 08.12.19 09:49, Nick Coghlan wrote:
> On Fri., 6 Dec. 2019, 3:31 am Christian Tismer, <mailto:tis...@stackless.com>> wrote:
>
> Hi guys,
>
> during the last few weeks I have been struggling quite much
> in order to make PySide run with Python 3.8 at all.
>
> The expected problems were refcounting leaks due to changed
> handling of heaptypes. But in fact, the runtime behavior was
> much worse, because I always got negative refcounts!
>
> After exhaustive searching through the different 3.8 commits, I could
> isolate the three problems with logarithmic search.
>
> The hard problem was this:
> Whenever PySide creates a new type, it crashes in PyType_Ready.
> The reason is the existence of the Py_TPFLAGS_METHOD_DESCRIPTOR
> flag.
> During the PyType_Ready call, the function mro() is called.
> This mro() call results in a negative refcount, because something
> behaves differently since this flag is set by default in mro().
>
> When I patched this flag away during the type_new call, everything
> worked ok. I don't understand why this problem affects PySide
> at all. Here is the code that would normally be only the newType line:
>
>
> // PYSIDE-939: This is a temporary patch that circumvents the
> problem
> // with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved.
> PyObject *ob_PyType_Type = reinterpret_cast *>(&PyType_Type);
> PyObject *mro = PyObject_GetAttr(ob_PyType_Type,
> Shiboken::PyName::mro());
> auto hold = Py_TYPE(mro)->tp_flags;
> Py_TYPE(mro)->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR;
> auto *newType = reinterpret_cast(type_new(metatype,
> args, kwds));
> Py_TYPE(mro)->tp_flags = hold;
>
>
> Isn't this manipulating the flags in the tuple type, rather than
> anything on a custom object? Or is "mro" a custom object rather than an
> MRO tuple?
>
> If anything, given the combination of factors required to reproduce the
> problem, I would guess that there might be a ref counting problem in the
> __set_owner__ invocations when called on a new type rather than a
> regular instance, and that was somehow affected by the change to
> increment the type refcount in PyObject_Init rather than
> PyType_GenericAlloc.
Hi Nick,
after staring long at the code, I fount something funny in
typeobject.c #286 ff:
static void
type_mro_modified(PyTypeObject *type, PyObject *bases) {
/*
Check that all base classes or elements of the MRO of type are
able to be cached. This function is called after the base
classes or mro of the type are altered.
Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
has a custom MRO that includes a type which is not officially
super type, or if the type implements its own mro() method.
Called from mro_internal, which will subsequently be called on
each subclass when their mro is recursively updated.
*/
Py_ssize_t i, n;
int custom = (Py_TYPE(type) != &PyType_Type);
int unbound;
PyObject *mro_meth = NULL;
PyObject *type_mro_meth = NULL;
if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
return;
if (custom) {
_Py_IDENTIFIER(mro);
mro_meth = lookup_maybe_method(
(PyObject *)type, &PyId_mro, &unbound);
if (mro_meth == NULL)
goto clear;
type_mro_meth = lookup_maybe_method(
(PyObject *)&PyType_Type, &PyId_mro, &unbound);
if (type_mro_meth == NULL)
goto clear;
if (mro_meth != type_mro_meth)
goto clear;
Py_XDECREF(mro_meth);
Py_XDECREF(type_mro_meth);
}
Look at the "if (custom)" clause.
"mro_meth = lookup_maybe_method(" uses lookup_maybe_method which
gives a borrowed reference. The same holds for "type_mro_meth".
But then both are decreffed, which IMHO is not correct.
Cheers -- Chris
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/IRHXB53SQYZML5JLFJJPLYQJNHPC2SXB/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Python 3.8 error?
On 09.12.19 23:26, Nick Coghlan wrote:
>
>
> On Tue., 10 Dec. 2019, 5:17 am MRAB, <mailto:pyt...@mrabarnett.plus.com>> wrote:
>
> On 2019-12-09 18:22, Christian Tismer wrote:
> >
> >
> > Hi Nick,
> >
> > after staring long at the code, I fount something funny in
> > typeobject.c #286 ff:
> >
> >
> > static void
> > type_mro_modified(PyTypeObject *type, PyObject *bases) {
> > /*
> > Check that all base classes or elements of the MRO of type are
> > able to be cached. This function is called after the base
> > classes or mro of the type are altered.
> >
> > Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
> > has a custom MRO that includes a type which is not officially
> > super type, or if the type implements its own mro() method.
> >
> > Called from mro_internal, which will subsequently be called on
> > each subclass when their mro is recursively updated.
> > */
> > Py_ssize_t i, n;
> > int custom = (Py_TYPE(type) != &PyType_Type);
> > int unbound;
> > PyObject *mro_meth = NULL;
> > PyObject *type_mro_meth = NULL;
> >
> > if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
> > return;
> >
> > if (custom) {
> > _Py_IDENTIFIER(mro);
> > mro_meth = lookup_maybe_method(
> > (PyObject *)type, &PyId_mro, &unbound);
> > if (mro_meth == NULL)
> > goto clear;
> > type_mro_meth = lookup_maybe_method(
> > (PyObject *)&PyType_Type, &PyId_mro, &unbound);
> > if (type_mro_meth == NULL)
> > goto clear;
> > if (mro_meth != type_mro_meth)
> > goto clear;
> > Py_XDECREF(mro_meth);
> > Py_XDECREF(type_mro_meth);
> > }
> >
> >
> > Look at the "if (custom)" clause.
> > "mro_meth = lookup_maybe_method(" uses lookup_maybe_method which
> > gives a borrowed reference. The same holds for "type_mro_meth".
> >
> > But then both are decreffed, which IMHO is not correct.
> >
> Look at what happens at the label "clear": it DECREFs them.
>
> If mro_meth != NULL or mro_meth != type_mro_meth, they'll get DECREFed
> at "clear".
>
>
> I believe Christian's point is that this entire "if (custom) {" branch
> looks suspicious, as it assumes "lookup_maybe_method" will increment the
> refcount on the returned object. If that assumption is incorrect, we're
> going to get DECREFs without a preceding INCREF.
>
> The specific code path is also obscure enough that it's plausible the
> test suite may not currently cover it (as it requires doing something
> that calls "type_mro_modified" on a type with a custom metaclass).
Thanks Nick for this nice analysis. And it's exactly this codepath that
is taken in the case of PySide: custom types all the time :-)
what-a-relief - ly y'rs -- Chris
--
Christian Tismer-Sperling:^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://www.qt.io/qt-for-python
14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/5MTJC47BHGZW6RKKU5JDAIFLV5P6W6JA/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [RELEASE] Python 3.8.1rc1 is now available for testing
Hi Łukasz,
tonite I found a critical bug that affects all heaptype extension
classes with a custom (not PyType_Type) type.
the bug is in typeobject.c function type_mro_modified line 309:
if (custom) {
_Py_IDENTIFIER(mro);
mro_meth = lookup_maybe_method(
(PyObject *)type, &PyId_mro, &unbound);
if (mro_meth == NULL)
goto clear;
Py_INCREF(mro_meth);
type_mro_meth = lookup_maybe_method(
(PyObject *)&PyType_Type, &PyId_mro, &unbound);
if (type_mro_meth == NULL)
goto clear;
Py_INCREF(type_mro_meth);
if (mro_meth != type_mro_meth)
goto clear;
Py_XDECREF(mro_meth);
Py_XDECREF(type_mro_meth);
}
This block is wrong because it decrefs a value that comes from
lookup_maybe_method which does not incref.
The code is easily fixed by two Py_INCREF s.
Please let me know how you want to proceed.
This is a critical error, producing negative refcounts.
Cheers -- Chris
On 10.12.19 10:22, Łukasz Langa wrote:
> Python 3.8.1rc1 is the release candidate of the first maintenance
> release of Python 3.8.
>
> The Python 3.8 series is the newest feature release of the Python
> language, and it contains many new features and optimizations. You can
> find Python 3.8.1rc1 here:
>
> https://www.python.org/downloads/release/python-381rc1/
>
> Assuming no critical problems are found prior to *2019-12-16*, the
> scheduled release date for *3.8.1* as well as *Ned Deily's birthday*, no
> code changes are planned between this release candidate and the final
> release.
>
> That being said, please keep in mind that this is a pre-release of 3.8.1
> and as such its main purpose is testing.
>
> See the “What’s New in Python 3.8
> <https://docs.python.org/3.8/whatsnew/3.8.html>” document for more
> information about features included in the 3.8 series. Detailed
> information about all changes made in 3.8.0 can be found in its change log.
>
> Maintenance releases for the 3.8 series will continue at regular
> bi-monthly intervals, with *3.8.2* planned for February 2020.
>
>
> We hope you enjoy Python 3.8!
>
> Thanks to all of the many volunteers who help make Python Development
> and these releases possible! Please consider supporting our efforts by
> volunteering yourself or through organization contributions to the
> Python Software Foundation.
>
> https://www.python.org/psf/
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IGJ6ZOAOT2WFY5ZIPRQNTHOSUMPUAO2H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/6VIEOGQJ433TVQZRYFXQ7SWQM3XRBBKD/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [RELEASE] Python 3.8.1rc1 is now available for testing
Sorry, I sent the fixed version.
These two incref's are missing!
On 10.12.19 14:16, Christian Tismer wrote:
> Hi Łukasz,
>
> tonite I found a critical bug that affects all heaptype extension
> classes with a custom (not PyType_Type) type.
>
> the bug is in typeobject.c function type_mro_modified line 309:
>
> if (custom) {
> _Py_IDENTIFIER(mro);
> mro_meth = lookup_maybe_method(
> (PyObject *)type, &PyId_mro, &unbound);
> if (mro_meth == NULL)
> goto clear;
This one
> Py_INCREF(mro_meth);
> type_mro_meth = lookup_maybe_method(
> (PyObject *)&PyType_Type, &PyId_mro, &unbound);
> if (type_mro_meth == NULL)
> goto clear;
And this one
> Py_INCREF(type_mro_meth);
> if (mro_meth != type_mro_meth)
> goto clear;
> Py_XDECREF(mro_meth);
> Py_XDECREF(type_mro_meth);
> }
>
> This block is wrong because it decrefs a value that comes from
> lookup_maybe_method which does not incref.
>
> The code is easily fixed by two Py_INCREF s.
>
> Please let me know how you want to proceed.
> This is a critical error, producing negative refcounts.
>
> Cheers -- Chris
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/K2SX7F7P5CXJ7PMK2AAIU7JTAG2AH2A5/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [RELEASE] Python 3.8.1rc1 is now available for testing
On 10.12.19 14:28, Łukasz Langa wrote: > >> On 10 Dec 2019, at 14:16, Christian Tismer > <mailto:tis...@stackless.com>> wrote: >> >> Please let me know how you want to proceed. >> This is a critical error, producing negative refcounts. > > Is there a BPO issue for this? If not, there should be, let's discuss > there. Is this a 3.8 regression? > > 3.8.1 proper is next Monday, if this fix is handled quickly it will land > in 3.8.1. Yes, this problem exists since the first 3.8.0 version. I did not create a PR, yet because it took me so long to understand that this time it was really not a PySide problem ;-) I will try to produce one ASAP. -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BWHTDGNDKZFE4G6GK7ASSY4WTANBGMZ5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [python-committers] Re: [RELEASE] Python 3.8.1rc1 is now available for testing
Howdy,
I have produced this:
https://bugs.python.org/issue39016
No idea if it's correct, doing that too rarely.
Cheers -- Chris
On 10.12.19 14:34, Victor Stinner wrote:
> Can you please open an issue at https://bugs.python.org/ and then post
> the link in this thread?
>
> Thanks in advance,
> Victor
>
> Le mar. 10 déc. 2019 à 14:18, Christian Tismer a écrit
> :
>>
>> Hi Łukasz,
>>
>> tonite I found a critical bug that affects all heaptype extension
>> classes with a custom (not PyType_Type) type.
>>
>> the bug is in typeobject.c function type_mro_modified line 309:
>>
>> if (custom) {
>> _Py_IDENTIFIER(mro);
>> mro_meth = lookup_maybe_method(
>> (PyObject *)type, &PyId_mro, &unbound);
>> if (mro_meth == NULL)
>> goto clear;
>> Py_INCREF(mro_meth);
>> type_mro_meth = lookup_maybe_method(
>> (PyObject *)&PyType_Type, &PyId_mro, &unbound);
>> if (type_mro_meth == NULL)
>> goto clear;
>> Py_INCREF(type_mro_meth);
>> if (mro_meth != type_mro_meth)
>> goto clear;
>> Py_XDECREF(mro_meth);
>> Py_XDECREF(type_mro_meth);
>> }
>>
>> This block is wrong because it decrefs a value that comes from
>> lookup_maybe_method which does not incref.
>>
>> The code is easily fixed by two Py_INCREF s.
>>
>> Please let me know how you want to proceed.
>> This is a critical error, producing negative refcounts.
>>
>> Cheers -- Chris
>>
>>
>> On 10.12.19 10:22, Łukasz Langa wrote:
>>> Python 3.8.1rc1 is the release candidate of the first maintenance
>>> release of Python 3.8.
>>>
>>> The Python 3.8 series is the newest feature release of the Python
>>> language, and it contains many new features and optimizations. You can
>>> find Python 3.8.1rc1 here:
>>>
>>> https://www.python.org/downloads/release/python-381rc1/
>>>
>>> Assuming no critical problems are found prior to *2019-12-16*, the
>>> scheduled release date for *3.8.1* as well as *Ned Deily's birthday*, no
>>> code changes are planned between this release candidate and the final
>>> release.
>>>
>>> That being said, please keep in mind that this is a pre-release of 3.8.1
>>> and as such its main purpose is testing.
>>>
>>> See the “What’s New in Python 3.8
>>> <https://docs.python.org/3.8/whatsnew/3.8.html>” document for more
>>> information about features included in the 3.8 series. Detailed
>>> information about all changes made in 3.8.0 can be found in its change log.
>>>
>>> Maintenance releases for the 3.8 series will continue at regular
>>> bi-monthly intervals, with *3.8.2* planned for February 2020.
>>>
>>>
>>> We hope you enjoy Python 3.8!
>>>
>>> Thanks to all of the many volunteers who help make Python Development
>>> and these releases possible! Please consider supporting our efforts by
>>> volunteering yourself or through organization contributions to the
>>> Python Software Foundation.
>>>
>>> https://www.python.org/psf/
>>>
>>>
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/IGJ6ZOAOT2WFY5ZIPRQNTHOSUMPUAO2H/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> --
>> Christian Tismer :^) tis...@stackless.com
>> Software Consulting : http://www.stackless.com/
>> Karl-Liebknecht-Str. 121 : https://github.com/PySide
>> 14482 Potsdam: GPG key -> 0xFB7BEE0E
>> phone +49 173 24 18 776 fax +49 (30) 700143-0023
>>
>> ___
>> python-committers mailing list -- python-committ...@python.org
>> To unsubscribe send an email to python-committers-le...@python.org
>> https://mail.python.org/mailman3/lists/python-committers.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-committ...@python.org/message/6VIEOGQJ433TVQZRYFXQ7SWQM3XRBBKD/
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
>
>
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/6A7UURL224NQQFHRJXRQYKVTRWNHLLGR/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Python 3.8 error?
Hi all,
Sorry for the noise, I was wrong, and I retract.
I was somehow mislead and hunted a phantom.
Best - Chris
On 10.12.19 00:29, Christian Tismer wrote:
> On 09.12.19 23:26, Nick Coghlan wrote:
>>
>>
>> On Tue., 10 Dec. 2019, 5:17 am MRAB, > <mailto:pyt...@mrabarnett.plus.com>> wrote:
>>
>> On 2019-12-09 18:22, Christian Tismer wrote:
>> >
>> >
>> > Hi Nick,
>> >
>> > after staring long at the code, I fount something funny in
>> > typeobject.c #286 ff:
>> >
>> >
>> > static void
>> > type_mro_modified(PyTypeObject *type, PyObject *bases) {
>> > /*
>> > Check that all base classes or elements of the MRO of type are
>> > able to be cached. This function is called after the base
>> > classes or mro of the type are altered.
>> >
>> > Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
>> > has a custom MRO that includes a type which is not officially
>> > super type, or if the type implements its own mro() method.
>> >
>> > Called from mro_internal, which will subsequently be called on
>> > each subclass when their mro is recursively updated.
>> > */
>> > Py_ssize_t i, n;
>> > int custom = (Py_TYPE(type) != &PyType_Type);
>> > int unbound;
>> > PyObject *mro_meth = NULL;
>> > PyObject *type_mro_meth = NULL;
>> >
>> > if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
>> > return;
>> >
>> > if (custom) {
>> > _Py_IDENTIFIER(mro);
>> > mro_meth = lookup_maybe_method(
>> > (PyObject *)type, &PyId_mro, &unbound);
>> > if (mro_meth == NULL)
>> > goto clear;
>> > type_mro_meth = lookup_maybe_method(
>> > (PyObject *)&PyType_Type, &PyId_mro, &unbound);
>> > if (type_mro_meth == NULL)
>> > goto clear;
>> > if (mro_meth != type_mro_meth)
>> > goto clear;
>> > Py_XDECREF(mro_meth);
>> > Py_XDECREF(type_mro_meth);
>> > }
>> >
>> >
>> > Look at the "if (custom)" clause.
>> > "mro_meth = lookup_maybe_method(" uses lookup_maybe_method which
>> > gives a borrowed reference. The same holds for "type_mro_meth".
>> >
>> > But then both are decreffed, which IMHO is not correct.
>> >
>> Look at what happens at the label "clear": it DECREFs them.
>>
>> If mro_meth != NULL or mro_meth != type_mro_meth, they'll get DECREFed
>> at "clear".
>>
>>
>> I believe Christian's point is that this entire "if (custom) {" branch
>> looks suspicious, as it assumes "lookup_maybe_method" will increment the
>> refcount on the returned object. If that assumption is incorrect, we're
>> going to get DECREFs without a preceding INCREF.
>>
>> The specific code path is also obscure enough that it's plausible the
>> test suite may not currently cover it (as it requires doing something
>> that calls "type_mro_modified" on a type with a custom metaclass).
>
> Thanks Nick for this nice analysis. And it's exactly this codepath that
> is taken in the case of PySide: custom types all the time :-)
>
> what-a-relief - ly y'rs -- Chris
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5MTJC47BHGZW6RKKU5JDAIFLV5P6W6JA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/5PBHW3HEQQ4KLRUATZ7YGX5TR3T24R4C/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Python 3.8 error?
On 12.12.19 13:04, Petr Viktorin wrote:
> I'm quite interested in the rest of the story here. PySide is probably
> the biggest open-source user of the limited API, so IMO it is relevant
> to this list.
I guess that PyQt5 is a similar huge user, and it may be that they have
the same problem, since there is no 5.14 version yet ;-)
> On 2019-12-11 23:48, Christian Tismer wrote:
>> Hi all,
>>
>> Sorry for the noise, I was wrong, and I retract.
>> I was somehow mislead and hunted a phantom.
>
> Does that mean that there was never a problem?
> Or that a problem is still there, but we don't know if it's in CPython
> or PySide or their interaction?
> Or was the problem solved in PySide?
> Do we need to document something better, or just aware of some caveat?
The problem is still there.
When PySide creates a new type, then PyType_Ready digs into the new
type, fetches the "mro()" method and uses the
Py_TPFLAGS_METHOD_DESCRIPTOR flag to decide how to handle it.
When I leave the flag in, we crash. So this is the current fix,
and I have no idea why this affects us at all. Here is the last change
to SbkObjectTypeTpNew() #503:
// The meta type creates a new type when the Python programmer
extends a wrapped C++ class.
auto type_new = reinterpret_cast(PyType_Type.tp_new);
// PYSIDE-939: This is a temporary patch that circumvents the problem
// with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved.
// PyType_Ready uses mro(). We need to temporarily remove the flag
from it's type.
// We cannot use PyMethodDescr_Type since it is not exported by
Python 2.7 .
static PyTypeObject *PyMethodDescr_TypePtr = Py_TYPE(
PyObject_GetAttr(reinterpret_cast(&PyType_Type),
Shiboken::PyName::mro()));
auto hold = PyMethodDescr_TypePtr->tp_flags;
PyMethodDescr_TypePtr->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR;
auto *newType = reinterpret_cast(type_new(metatype,
args, kwds));
PyMethodDescr_TypePtr->tp_flags = hold;
I am still not sure where the bug is and who has it.
My understanding is that this flag should have no impact on PySide
at all, but it has.
Thanks for taking care -- Chris
>> On 10.12.19 00:29, Christian Tismer wrote:
>>> On 09.12.19 23:26, Nick Coghlan wrote:
>>>>
>>>>
>>>> On Tue., 10 Dec. 2019, 5:17 am MRAB, >>> <mailto:pyt...@mrabarnett.plus.com>> wrote:
>>>>
>>>> On 2019-12-09 18:22, Christian Tismer wrote:
>>>> >
>>>> >
>>>> > Hi Nick,
>>>> >
>>>> > after staring long at the code, I fount something funny in
>>>> > typeobject.c #286 ff:
>>>> >
>>>> >
>>>> > static void
>>>> > type_mro_modified(PyTypeObject *type, PyObject *bases) {
>>>> > /*
>>>> > Check that all base classes or elements of the MRO of
>>>> type are
>>>> > able to be cached. This function is called after the
>>>> base
>>>> > classes or mro of the type are altered.
>>>> >
>>>> > Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
>>>> > has a custom MRO that includes a type which is not
>>>> officially
>>>> > super type, or if the type implements its own mro()
>>>> method.
>>>> >
>>>> > Called from mro_internal, which will subsequently be
>>>> called on
>>>> > each subclass when their mro is recursively updated.
>>>> > */
>>>> > Py_ssize_t i, n;
>>>> > int custom = (Py_TYPE(type) != &PyType_Type);
>>>> > int unbound;
>>>> > PyObject *mro_meth = NULL;
>>>> > PyObject *type_mro_meth = NULL;
>>>> >
>>>> > if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
>>>> > return;
>>>> >
>>>> > if (custom) {
>>>> > _Py_IDENTIFIER(mro);
>>>> > mro_meth = lookup_maybe_method(
>>>> > (PyObject *)type, &PyId_mro, &unbound);
>>>> > if (mro_meth == NULL)
>>>> > goto clear;
>>>> > type_mro_meth = lookup_maybe_method(
>>>> > (PyObject *)&PyType_Type, &PyId_mro,
[Python-Dev] Re: Python 3.8 error?
On 12.12.19 13:04, Petr Viktorin wrote: > I'm quite interested in the rest of the story here. PySide is probably > the biggest open-source user of the limited API, so IMO it is relevant > to this list. BTW., the Python 3.8 change to type refcounting is already breaking the Limited API a bit, because we have to dynamically figure that out in order to be version-independent. I am not so sure if that whole change was worth to break it? Cheers -- Chris -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Strandstraße 37 : https://github.com/PySide 24217 Schönberg : GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FVUU6DSHC4KRB2X7H2CJHFTHBENNPYZM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Python 3.8 error?
Pardon, I meant "there is no Python 3.8 version, yet".
And this is wrong, the MacOS pip install shows
PyQt5-5.13.2-5.13.2-cp35.cp36.cp37.cp38-abi3-macosx_10_6_intel.whl
So probably we have some bad oversight, somewhere.
Cheers -- Chris
On 12.12.19 13:48, Christian Tismer wrote:
> On 12.12.19 13:04, Petr Viktorin wrote:
>> I'm quite interested in the rest of the story here. PySide is probably
>> the biggest open-source user of the limited API, so IMO it is relevant
>> to this list.
>
>
> I guess that PyQt5 is a similar huge user, and it may be that they have
> the same problem, since there is no 5.14 version yet ;-)
>
>
>> On 2019-12-11 23:48, Christian Tismer wrote:
>>> Hi all,
>>>
>>> Sorry for the noise, I was wrong, and I retract.
>>> I was somehow mislead and hunted a phantom.
>>
>> Does that mean that there was never a problem?
>> Or that a problem is still there, but we don't know if it's in CPython
>> or PySide or their interaction?
>> Or was the problem solved in PySide?
>> Do we need to document something better, or just aware of some caveat?
>
>
> The problem is still there.
> When PySide creates a new type, then PyType_Ready digs into the new
> type, fetches the "mro()" method and uses the
> Py_TPFLAGS_METHOD_DESCRIPTOR flag to decide how to handle it.
>
> When I leave the flag in, we crash. So this is the current fix,
> and I have no idea why this affects us at all. Here is the last change
> to SbkObjectTypeTpNew() #503:
>
> // The meta type creates a new type when the Python programmer
> extends a wrapped C++ class.
> auto type_new = reinterpret_cast(PyType_Type.tp_new);
>
> // PYSIDE-939: This is a temporary patch that circumvents the problem
> // with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved.
> // PyType_Ready uses mro(). We need to temporarily remove the flag
> from it's type.
> // We cannot use PyMethodDescr_Type since it is not exported by
> Python 2.7 .
> static PyTypeObject *PyMethodDescr_TypePtr = Py_TYPE(
> PyObject_GetAttr(reinterpret_cast(&PyType_Type),
> Shiboken::PyName::mro()));
> auto hold = PyMethodDescr_TypePtr->tp_flags;
> PyMethodDescr_TypePtr->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR;
> auto *newType = reinterpret_cast(type_new(metatype,
> args, kwds));
> PyMethodDescr_TypePtr->tp_flags = hold;
>
>
> I am still not sure where the bug is and who has it.
> My understanding is that this flag should have no impact on PySide
> at all, but it has.
>
> Thanks for taking care -- Chris
>
>
>>> On 10.12.19 00:29, Christian Tismer wrote:
>>>> On 09.12.19 23:26, Nick Coghlan wrote:
>>>>>
>>>>>
>>>>> On Tue., 10 Dec. 2019, 5:17 am MRAB, >>>> <mailto:pyt...@mrabarnett.plus.com>> wrote:
>>>>>
>>>>> On 2019-12-09 18:22, Christian Tismer wrote:
>>>>> >
>>>>> >
>>>>> > Hi Nick,
>>>>> >
>>>>> > after staring long at the code, I fount something funny in
>>>>> > typeobject.c #286 ff:
>>>>> >
>>>>> >
>>>>> > static void
>>>>> > type_mro_modified(PyTypeObject *type, PyObject *bases) {
>>>>> > /*
>>>>> > Check that all base classes or elements of the MRO of
>>>>> type are
>>>>> > able to be cached. This function is called after the
>>>>> base
>>>>> > classes or mro of the type are altered.
>>>>> >
>>>>> > Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
>>>>> > has a custom MRO that includes a type which is not
>>>>> officially
>>>>> > super type, or if the type implements its own mro()
>>>>> method.
>>>>> >
>>>>> > Called from mro_internal, which will subsequently be
>>>>> called on
>>>>> > each subclass when their mro is recursively updated.
>>>>> > */
>>>>> > Py_ssize_t i, n;
>>>>> > int custom = (Py_TYPE(type) != &PyType_Type);
>>>>> > int unbound;
>>>>> > PyObject *mro_meth = NULL;
>>>>> > PyObject *typ
[Python-Dev] an unimportant question, ...
... but I'm curious.
Hi Guido,
while working on Psyco, I stumbled over a log entry in modsupport.h:
19-Aug-2002 GvR 1012Changes to string object struct for
interning changes, saving 3 bytes.
The change to stringobject was this (rev. 28308):
Before:
typedef struct {
PyObject_VAR_HEAD
long ob_shash;
PyObject *ob_sinterned;
char ob_sval[1];
} PyStringObject;
After:
typedef struct {
PyObject_VAR_HEAD
long ob_shash;
int ob_sstate;
char ob_sval[1];
} PyStringObject;
Now, the internals are very clear to me. What I don't understand
is where the three saved bytes should be.
Thinking of the time where this change was made, I cannot imagine
that this comment was about the size diff between pointer and int,
and if this was meant, I still don't get how this could save three
bytes?
With unaligned ob_sval, structure packing and ob_sstate being
unsigned char one could save 3 bytes, but we don't do that.
Well, as said, this is no important question. I am just asking
myself what I don't see here, or if the comment is just sub-optimal :-)
all the best -- chris
p.s.: won't make it to PyCon this time, see you soon at the piggies
--
Christian Tismer :^) <mailto:tis...@stackless.com>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an unimportant question, ...
On 3/22/09 8:01 PM, "Martin v. Löwis" wrote: Now, the internals are very clear to me. What I don't understand is where the three saved bytes should be. If you look at the various patches in http://bugs.python.org/issue576101 then there is a three-byte saving in all versions from 1 to 6. Consequentially, the API was changed in those versions (but only starting from version 5, i.e. the first version created by Guido). For some reason, the saving was then removed from the patch that got actually committed (#7). I guess the comment just stayed. That's very impressive! Thank you very much for the enlightment. Whow! cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an unimportant question, ...
On 3/22/09 8:01 PM, "Martin v. Löwis" wrote: Now, the internals are very clear to me. What I don't understand is where the three saved bytes should be. If you look at the various patches in http://bugs.python.org/issue576101 then there is a three-byte saving in all versions from 1 to 6. Consequentially, the API was changed in those versions (but only starting from version 5, i.e. the first version created by Guido). For some reason, the saving was then removed from the patch that got actually committed (#7). I guess the comment just stayed. Yes, funny, actually. At least, I don't find any comment why the char was turned into an int, after all. Are char pointers not on a word boundary problematic on some platforms? Or was it maybe to just keep the string layout on many common platforms compatible, in order to save rebuilding so many windows extension modules? If the latter is true and the only reason, I vote for reclaiming the three bytes. Maybe it saves a tree or two. Maybe it hurts very little if done for Python 3000. In any case, use the version that saves the most energy. :-) not kidding - ciao -- chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] mingw32 and gc-header weirdness
Hi all,
I was hacking to get mingw32 builds of psyco to work
and found a pretty weird thing:
I used mingw32 (stable distro) to build the psyco extension
on top of standard python2.6, built with Visual Studio,
and got weird crashes.
The reason is in objimpl.h:
typedef union _gc_head {
struct {
union _gc_head *gc_next;
union _gc_head *gc_prev;
Py_ssize_t gc_refs;
} gc;
long double dummy; /* force worst-case alignment */
} PyGC_Head;
Mingw32 behaves funny here. The size of long double is 12 ! 8-)
I happened to use the _PyObject_GC_UNTRACK macro in psyco, instead
of the function, and that caused the errors, because psython
and the extension disagreed on the location of the gc pointers...
Using PyObject_GC_Untrack instead fixed the problem,
but there is a bad feeling left.
Question:
Is that considered a mingw bug?
Should we change the above union to a safer construct?
Or maybe I just missed something obvious and made a fool out of me?
cheers - chris
--
Christian Tismer :^) <mailto:tis...@stackless.com>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/22/09 4:56 PM, Roumen Petrov wrote: Martin v. Löwis wrote: [SNIP] No. tim_one changed it to be long double in r25454 to support some system that Dave Abrahams uses, so it needs to stay that way :-) However, we can certainly acknowledge that this is a bug in MingW, and special case it. Either introduce a symbolic type gchead_align_t which gets defined to just double on MingW, or put the #ifdef right into the structure. No this is not GCC bug. GCC support "hardware extended precision" as implement long double and mingw w32api implement long double functions. According to http://msdn.microsoft.com/en-us/library/9cx8xs15.aspx it is MSVC limitation. It might also be useful to assert that sizeof(gchead_align_t) is 8 or 16, and reject 12 as a value. The point is that we need the maximum alignment, and that certainly shouldn't be 12. So look like python bug. The assumption is that the union with long double gives alignment to the largest possible integral type with a power of 2 size, which is then wrong, because of the unexpected size. What do you propose for doing proper alignment, then? I fear this needs to become yet another special case in pyconfig.h cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/22/09 5:17 PM, David Cournapeau wrote: On Thu, Jul 23, 2009 at 4:40 AM, Antoine Pitrou wrote: The size of long double is also 12 under 32-bit Linux. Perhaps mingw disagrees with Visual Studio Yes, mingw and VS do not have the same long double type. This has been the source of some problems in numpy as well, since mingw uses the MS runtime, and everything involving long double and the runtime is broken (printf, math library calls). I wish there was a way to disable this in mingw, but there isn't AFAIK. on some ABI subtleties (is it expected? is mingw supposed to be ABI-compatible with Visual Studio? if yes, you may report a bug to them :-)). I think mostly ABI compatible is the best description :) Well, the source of my problems is simply that I tried to build an extension for a Visual Studio built Python, using mingw. Did that, because it seems to be common practice so far. Maybe the simple solution is to prevent building extensions with mingw, if the python executable was not also built with it? Then, all would be fine I guess. Despite the fact that Python probably has to be changed: If it is true then all the 32-bit Linux Pythons have a 12 byte GC head, IOW they are *all* badly aligned. If that matters, of course. cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/23/09 2:04 AM, Antoine Pitrou wrote: Christian Tismer stackless.com> writes: Despite the fact that Python probably has to be changed: If it is true then all the 32-bit Linux Pythons have a 12 byte GC head, IOW they are *all* badly aligned. Why are they badly aligned? The fact that long double is 12 bytes long doesn't mean it will force a 12-byte alignment - just whatever alignment is enough for a long double on the target machine. This could be 4, 8 or 16 bytes. Things are a bit different: Alignment is not the primary concern of the gc header structure. Note that all the objects are created by malloc (system or python's arena allocator), and therefore all objects are correctly aligned by construction. The point is: The GC header is a structure invisible to the "real" gc allocated objects. It is opaquely prepended to every gc aware object. Therefore, it *needs* to have the correct size, in order to propagate its (already correct) alignment to the real object. It appears that python, compiled with gcc for all x64 32bit Linuxen (and mingw32) produces a 12 byte GC header. Not relevant for the header itself, but all GC objects are misaligned. This may not be recognized so far, because there is no builtin GC-enabled type that embeds a double. But if you create such a type, then the double will be correctly aligned in your object's structure, but then, when the object gets allocated, it is misaligned, because the header size is not a multiple of 8. To Martin: So I disagree. The gc header is not responsible for alignment in the first place, but to propagate it, correctly. And this fails miserably (in principle) since years. Proposal: We should use a simple construct that makes the gc header size simply a multiple of 8 or 16, whatever needed. Even a byte array would be ok. But please no long double :-) cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/23/09 2:27 PM, Antoine Pitrou wrote: Christian Tismer stackless.com> writes: ... I'm not sure a double aligned on a 4-byte boundary is "misaligned" on a x86 CPU. I'm also not sure. Anyway, the result was neither intended nor expected, I guess. Alignment is primarily important to avoid access violations on CPUs and datatypes which don't support arbitrary alignment, although it can also be useful for performance reasons. Performance, performance, of course (that's my job, after all :-) ) ... Of course, it will also make memory consumption a tad bigger for GC-enabled objects (but GC-enabled objects are generally not that small anyway). For that reason, I don't like the addition of the opaque header too much. If there were an option to make the header explicit, we would not have to round up the object size to a multiple of (8, 16), but could arrange embedded doubles as they fit the best. (I disagree, however, that we should remove the "long double". After all, we also want alignment of PyObjects to allow inclusion of a "long double" in them) Well, I doubt that a 12 byte long double causes any other alignment but 4. -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/25/09 7:11 AM, Neil Hodgson wrote: Martin v. Löwis: I propose to add another (regular) double into the union. Adding a regular double as a second dummy gives the same sizes and alignments with Mingw or MSVC as the original definition with MSVC: Great (checked that, too) This makes pretty much sense, also given that this waste happens under Windows, already. In regard to alignment penalties, a simple copy loop for doubles runs about 20% slower when misaligned on an my AMD processor. Other x86 processors can be much worse. As much as 2 to 3.25 times according to I am still unhappy with this waste of memory, just because the GC header has to be rounded up, regardlwss wether that is needed or not. We should keep Martin's hint in mind, that Python 4 could place the gc header at the end of structures, instead. cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/27/09 12:48 AM, Christian Heimes wrote: Christian Tismer wrote: We should keep Martin's hint in mind, that Python 4 could place the gc header at the end of structures, instead. Wow, 3.1 just came out and we already have the first PEP for Python 4k? :) Christian Maybe it's even possible right now, with a couple of tweaks that allow to explicitly say where the header is, for a specific extension type. Might cost a few cycles, tho. Christian -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 and gc-header weirdness
On 7/24/09 5:16 AM, Roumen Petrov wrote: Christian Tismer wrote: ... Did the crash disappear is you add "__attribute__((aligned(8)))" after variable dummy ? Did not try. But the proposed addition of a double does it, see the dev list. cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Where's the Starship's crew?
by utilizing OpenVZ, as fine-grained as makes sense. I am investigating this these days. Right now, the Starship is a single VE. It will turn into a growing set of smaller VE's in the next weeks. And as soon as we get sponsorship, python.net will split itself over multiple machines. After all, my vision is to create the ultimate Python showdown, running everything possible in isolated environments and allowing people to play with different configurations. python.net should become the ultimate python resource site for interactive playing and trying, and providing a home for people to show their development. I will also try to get the PSF interested in that; maybe there is also some PSF funding possible. But this is an option, I will continue Starship without support as well. I believe I can do that, with your help. feeling stronger than before that stroke attack -- sincerely - chris -- This is much more work than I can do alone. Therefore, I am asking for people to help me with this. I also don't want to miss any of the current supporters, and we will name them all on the revised contributors pages to come. In order to support really many Python projects, we will need not only sponsors, but probably the support of the individual project maintainers as well. I am open to make this my goal of life, if there are enough people interested. But they will, I know it. I do believe in Python, Starship, PyPy and Stackless. Please help me to make this life-dream into reality. happily being back to the roots -- chris -- Christian Tismer :^) <mailto:[EMAIL PROTECTED]> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] making python's c iterators picklable (http://bugs.python.org/issue14288)
On 3/13/12 4:05 PM, Antoine Pitrou wrote: On Tue, 13 Mar 2012 22:53:42 + Kristján Valur Jónsson wrote: http://bugs.python.org/issue14288 Raymond suggested that this patch should be discussed here, so here goes: Sounds good on the principle. Of course, the patch needs to be reviewed. I am very much for this patch. Of course I am biased by my stackless history and would be very happy to get most pickling into Python, where it belongs. To improve the patch a bit, I propose to put the hidden types into some module. I agree with Kristjan that the types module would be a good place to bootstrap the hidden iterable types. I even would like to propose a PEP: Whenever stuff is turned into C, which was picklable when implemented in Python, or something new is implemented that makes sense to pickle, a pickle implementation should always be provided. cheers -- chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
