Re: [Python-Dev] hg.python.org cloning troubles after Sep 13 changes
hg 3.1.1 prints comprehensive message that guess to call `ssh-keygen -R hg.python.org` On Fri, Nov 7, 2014 at 2:21 PM, M.-A. Lemburg wrote: > Just FYI (and for the archives), to perhaps save someone a few minutes: > > I've been hitting a problem with hg pull and hg clone on a box recently > and after staring at it for a while, finally found the cause. > > Here's what hg printed: > > HG-Python/cpython> hg pull -u -b 2.7 > abort: no suitable response from remote hg! > > Not much help. > > Using --debug: > > orig/HG-Python> hg --debug clone ssh://h...@hg.python.org/cpython cpython-2.7 > running ssh -C -q -l marc-andre.lemburg -i /home/lemburg/.ssh/id_rsa > h...@hg.python.org "hg -R cpython > serve --stdio" > sending hello command > sending between command > abort: no suitable response from remote hg! > > Still no help. > > Then I remembered that the box was newly setup on Sept 13 and checked > Benjamins email. The server host key had also changed. After removing > the offending key and accepting the new one, things started working > again. > > The hg on the box is 1.7.5, so newer version may include a better > error message. > > Cheers, > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Nov 07 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ > > > : Try our mxODBC.Connect Python Database Interface for free ! :: > >eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >Registered at Amtsgericht Duesseldorf: HRB 46611 >http://www.egenix.com/company/contact/ > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] peps: New PEP 490: Chain exceptions at C level
There is another issue: exception chain is not set up on exception creation in python code, only on throwing. Thus I have to assign `__context__` and `__cause__` attributes manually if I want to call `future.set_exception(exc)` instead of `raise exc`. See aiohttp code for usage example: https://github.com/KeepSafe/aiohttp/blob/931efbd518d0d99522d1cd36b43620657cd07304/aiohttp/client.py#L522 On Thu, Mar 26, 2015 at 11:11 AM, Serhiy Storchaka wrote: > On 26.03.15 10:08, victor.stinner wrote: >> >> https://hg.python.org/peps/rev/7daf3bfd9586 >> changeset: 5741:7daf3bfd9586 >> user:Victor Stinner >> date:Thu Mar 26 09:08:08 2015 +0100 >> summary: >>New PEP 490: Chain exceptions at C level > > >> +Python 3.5 introduces a new private ``_PyErr_ChainExceptions()`` function >> which >> +is enough to chain manually exceptions. > > > It also was added in Python 3.4.3. > > I meditar about adding _PyErr_ReplaceException() in 2.7 for simpler > backporting patches from 3.x. > >> +Functions like ``PyErr_SetString()`` don't chain automatically >> exceptions. To >> +make usage of ``_PyErr_ChainExceptions()`` easier, new functions are >> added: >> + >> +* PyErr_SetStringChain(exc_type, message) >> +* PyErr_FormatChaine(exc_type, format, ...) > > > Typo. > >> +* PyErr_SetNoneChain(exc_type) >> +* PyErr_SetObjectChain(exc_type, exc_value) > > > I would first make these functions private, as _PyErr_ChainExceptions(). > After proofing their usefulness in the stdlib, they can be made public. > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] peps: New PEP 490: Chain exceptions at C level
I think setting context in exception constructor would be fine. On Thu, Mar 26, 2015 at 1:36 PM, Victor Stinner wrote: > 2015-03-26 11:52 GMT+01:00 Andrew Svetlov : >> There is another issue: exception chain is not set up on exception >> creation in python code, only on throwing. > > I'm not suprised of that. > >> Thus I have to assign `__context__` and `__cause__` attributes >> manually if I want to call `future.set_exception(exc)` instead of >> `raise exc`. > > Do you mean that we need an helper to make this task even simpler? Or > do you suggest to set them automatically in the constructor? > > Victor -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Keyword-only parameters
At least asyncio uses keyword-only intensively. On Tue, Apr 14, 2015 at 1:40 PM, Eric V. Smith wrote: > I'm working on adding a numeric_owner parameter to some tarfile methods > (http://bugs.python.org/issue23193), > > In a review, Berker suggested making the parameter keyword-only. I agree > that you'd likely never want to pass just "True", but that > "numeric_owner=True" would be a better usage. > > But, I don't see a lot of keyword-only parameters being added to stdlib > code. Is there some position we've taken on this? Barring someone saying > "stdlib APIs shouldn't contain keyword-only params", I'm inclined to > make numeric_owner keyword-only. > > Is there anything stopping me from making it keyword-only? > > Thanks. > Eric. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Aprender Pythton
I'm sorry. Please use English in the mailing list. People may not understand your chatting. 2015-04-14 20:36 GMT-04:00 Erik Rivera : > Baldomero, > > Veo que perteneces al estado de Puebla, México, existe la lista de Python > México https://mail.python.org/mailman/listinfo/python-mx, habemos varios > de Puebla que te podemos apoyar. > > Saludos. > > El 14 de abril de 2015, 19:50, Baldomero Perez martinez < > bpma...@yahoo.com.dmarc.invalid.mx> escribió: > >> Quiero aprender python quiero empezar agradezco si me pueden ayudar >> >> L.I. Baldomero Pérez Martínez >> Enc. Proy. Informatica >> Fideicomiso Ingenio Atencingo 80326 >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/erik.river%40gmail.com >> >> > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Aprender Pythton
Python-dev is for development OF Python, not for development WITH Python or Python LEARNING, BTW. On Tue, Apr 14, 2015 at 8:46 PM, Raúl Cumplido wrote: > Hi Andrew, > > Is someone asking where to find resources to learn Python. We have > redirected him to the python lists both in english and spanish. > > We should have replied in English if it would have been something related > to python-dev, but we have responded in Spanish as maybe the user doesn't > understand English. > > Kind Regards, > Raúl > > 2015-04-14 20:41 GMT-04:00 Andrew Svetlov : > >> I'm sorry. Please use English in the mailing list. >> >> People may not understand your chatting. >> >> 2015-04-14 20:36 GMT-04:00 Erik Rivera : >> >>> Baldomero, >>> >>> Veo que perteneces al estado de Puebla, México, existe la lista de >>> Python México https://mail.python.org/mailman/listinfo/python-mx, >>> habemos varios de Puebla que te podemos apoyar. >>> >>> Saludos. >>> >>> El 14 de abril de 2015, 19:50, Baldomero Perez martinez < >>> bpma...@yahoo.com.dmarc.invalid.mx> escribió: >>> >>>> Quiero aprender python quiero empezar agradezco si me pueden ayudar >>>> >>>> L.I. Baldomero Pérez Martínez >>>> Enc. Proy. Informatica >>>> Fideicomiso Ingenio Atencingo 80326 >>>> >>>> ___ >>>> Python-Dev mailing list >>>> Python-Dev@python.org >>>> https://mail.python.org/mailman/listinfo/python-dev >>>> Unsubscribe: >>>> https://mail.python.org/mailman/options/python-dev/erik.river%40gmail.com >>>> >>>> >>> >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com >>> >>> >> >> >> -- >> Thanks, >> Andrew Svetlov >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/raulcumplido%40gmail.com >> >> > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Aprender Pythton
Ok, sorry. On Tue, Apr 14, 2015 at 8:59 PM, Ian Cordasco wrote: > > > On Tue, Apr 14, 2015 at 7:54 PM, Andrew Svetlov > wrote: > >> Python-dev is for development OF Python, not for development WITH Python >> or Python LEARNING, BTW. >> >> On Tue, Apr 14, 2015 at 8:46 PM, Raúl Cumplido >> wrote: >> >>> Hi Andrew, >>> >>> Is someone asking where to find resources to learn Python. We have >>> redirected him to the python lists both in english and spanish. >>> >>> We should have replied in English if it would have been something >>> related to python-dev, but we have responded in Spanish as maybe the user >>> doesn't understand English. >>> >>> Kind Regards, >>> Raúl >>> >>> 2015-04-14 20:41 GMT-04:00 Andrew Svetlov : >>> >>>> I'm sorry. Please use English in the mailing list. >>>> >>>> People may not understand your chatting. >>>> >>>> 2015-04-14 20:36 GMT-04:00 Erik Rivera : >>>> >>>>> Baldomero, >>>>> >>>>> Veo que perteneces al estado de Puebla, México, existe la lista de >>>>> Python México https://mail.python.org/mailman/listinfo/python-mx, >>>>> habemos varios de Puebla que te podemos apoyar. >>>>> >>>>> Saludos. >>>>> >>>>> El 14 de abril de 2015, 19:50, Baldomero Perez martinez < >>>>> bpma...@yahoo.com.dmarc.invalid.mx> escribió: >>>>> >>>>>> Quiero aprender python quiero empezar agradezco si me pueden ayudar >>>>>> >>>>>> L.I. Baldomero Pérez Martínez >>>>>> Enc. Proy. Informatica >>>>>> Fideicomiso Ingenio Atencingo 80326 >>>>>> >>>>>> ___ >>>>>> Python-Dev mailing list >>>>>> Python-Dev@python.org >>>>>> https://mail.python.org/mailman/listinfo/python-dev >>>>>> Unsubscribe: >>>>>> https://mail.python.org/mailman/options/python-dev/erik.river%40gmail.com >>>>>> >>>>>> >>>>> >>>>> ___ >>>>> Python-Dev mailing list >>>>> Python-Dev@python.org >>>>> https://mail.python.org/mailman/listinfo/python-dev >>>>> Unsubscribe: >>>>> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Andrew Svetlov >>>> >>>> ___ >>>> Python-Dev mailing list >>>> Python-Dev@python.org >>>> https://mail.python.org/mailman/listinfo/python-dev >>>> Unsubscribe: >>>> https://mail.python.org/mailman/options/python-dev/raulcumplido%40gmail.com >>>> >>>> >>> >> >> >> -- >> Thanks, >> Andrew Svetlov >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com >> >> > Andrew if you translate the text that was sent to Baldomero, you'll see > that's exactly what they said. Please don't be so rude (or lazy) to people > helping someone learn Python, regardless of whether they mistakenly posted > to the wrong list. > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
For now I can use mix asyncio.coroutines and `async def` functions, I mean I can write `await f()` inside async def to call asyncio.coroutine `f` and vise versa: I can use `yield from g()` inside asyncio.coroutine to call `async def g(): ...`. If we forbid to call `async def` from regualr code how asyncio should work? I'd like to push `async def` everywhere in asyncio API where asyncio.coroutine required. On Wed, Apr 22, 2015 at 8:13 PM, Yury Selivanov wrote: > Hi Rajiv, > > On 2015-04-22 12:53 PM, Rajiv Kumar wrote: >> >> I'd like to suggest another way around some of the issues here, with >> apologies if this has already been discussed sometime in the past. >> >> From the viewpoint of a Python programmer, there are two distinct reasons >> for wanting to suspend execution in a block of code: >> >> 1. To yield a value from an iterator, as Python generators do today. >> >> 2. To cede control to the event loop while waiting for an asynchronous >> task >> to make progress in a coroutine. >> >> As of today both of these reasons to suspend are supported by the same >> underlying mechanism, i.e. a "yield" at the end of the chain of "yield >> from"s. PEPs 492 and 3152 introduce "await" and "cocall", but at the >> bottom >> of it all there's effectively still a yield as I understand it. >> >> I think that the fact that these two concepts use the same mechanism is >> what leads to the issues with coroutine-generators that Greg and Yury have >> raised. >> >> With that in mind, would it be possible to introduce a second form of >> suspension to Python to specifically handle the case of ceding to the >> event >> loop? I don't know what the implementation complexity of this would be, or >> if it's even feasible. But roughly speaking, the syntax for this could use >> "await", and code would look just like it does in the PEP. The semantics >> of >> "await " would be analogous to "yield from " today, with the >> difference that the Task would go up the chain of "await"s to the >> outermost >> caller, which would typically be asyncio, with some modifications from its >> form today. Progress would be made via __anext__ instead of __next__. > > > I think that what you propose is a great idea. However, its > implementation will be far more invasive than what PEP 492 > proposes. I doubt that we'll be able to make it in 3.5 if > we choose this route. > > BUT: With my latest proposal to disallow for..in loops and > iter()/list()-like builtins, the fact that coroutines are > based internally on generators is just an implementation > detail. > > There is no way users can exploit the underlying generator > object. Coroutine-objects only provide 'send()' and 'throw()' > methods, which they would also have with your implementation > idea. > > This gives us freedom to consider your approach in 3.6 if > we decide to add coroutine-generators. To make this work > we might want to patch inspect.py to make isgenerator() family > of functions to return False for coroutines/coroutine-objects. > > Thanks a lot for the feedback! > > Yury > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Wed, Apr 22, 2015 at 9:45 PM, Yury Selivanov wrote: > Andrew, > > On 2015-04-22 2:32 PM, Andrew Svetlov wrote: >> >> For now I can use mix asyncio.coroutines and `async def` functions, I >> mean I can write `await f()` inside async def to call >> asyncio.coroutine `f` and vise versa: I can use `yield from g()` >> inside asyncio.coroutine to call `async def g(): ...`. > > > That's another good point that I forgot to add to the list. > Thanks for bringing this up. > >> >> If we forbid to call `async def` from regualr code how asyncio should >> work? I'd like to push `async def` everywhere in asyncio API where >> asyncio.coroutine required. > > > You'll have to use a wrapper that will do the following: > > async def foo(): > return 'spam' > > @asyncio.coroutine > def bar(): > what = yield from foo.__await__(foo, *args, **kwargs) > # OR: > what = yield from await_call(foo, *args, **kwargs) > If I cannot directly use `yield from f()` with `async def f():` then almost every `yield from` inside asyncio library should be wrapped in `await_call()`. Every third-party asyncio-based library should do the same. Also I expect a performance degradation on `await_call()` calls. > Thanks, > Yury -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Wed, Apr 22, 2015 at 10:24 PM, Yury Selivanov wrote: > > > On 2015-04-22 2:53 PM, Andrew Svetlov wrote: >> >> On Wed, Apr 22, 2015 at 9:45 PM, Yury Selivanov >> wrote: > > [...] >>> >>> >>>> If we forbid to call `async def` from regualr code how asyncio should >>>> work? I'd like to push `async def` everywhere in asyncio API where >>>> asyncio.coroutine required. >>> >>> >>> You'll have to use a wrapper that will do the following: >>> >>> async def foo(): >>> return 'spam' >>> >>> @asyncio.coroutine >>> def bar(): >>> what = yield from foo.__await__(foo, *args, **kwargs) >>> # OR: >>> what = yield from await_call(foo, *args, **kwargs) >>> >> If I cannot directly use `yield from f()` with `async def f():` then >> almost every `yield from` inside asyncio library should be wrapped in >> `await_call()`. Every third-party asyncio-based library should do the >> same. >> >> Also I expect a performance degradation on `await_call()` calls. >> > > I think there is another way... instead of pushing > > GET_ITER > ... > YIELD_FROM > > opcodes, we'll need to replace GET_ITER with another one: > > GET_ITER_SPECIAL > ... > YIELD_FROM > > > Where "GET_ITER_SPECIAL (obj)" (just a working name) would check > that if the current code object has CO_COROUTINE and the > object that you will yield-from has it as well, it would > push to the stack the result of (obj.__await__()) > GET_ITER_SPECIAL sounds better than wrapper for `coro.__await__()` call. > Yury -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Wed, Apr 22, 2015 at 10:44 PM, PJ Eby wrote: > On Tue, Apr 21, 2015 at 1:26 PM, Yury Selivanov > wrote: >> It is an error to pass a regular context manager without ``__aenter__`` >> and ``__aexit__`` methods to ``async with``. It is a ``SyntaxError`` >> to use ``async with`` outside of a coroutine. > > I find this a little weird. Why not just have `with` and `for` inside > a coroutine dynamically check the iterator or context manager, and > either behave sync or async accordingly? Why must there be a > *syntactic* difference? IIRC Guido always like to have different syntax for calling regular functions and coroutines. That's why we need explicit syntax for asynchronous context managers and iterators. > > Not only would this simplify the syntax, it would also allow dropping > the need for `async` to be a true keyword, since functions could be > defined via "def async foo():" rather than "async def foo():" > > ...which, incidentally, highlights one of the things that's been > bothering me about all this "async foo" stuff: "async def" looks like > it *defines the function* asynchronously (as with "async with" and > "async for"), rather than defining an asynchronous function. ISTM it > should be "def async bar():" or even "def bar() async:". > > Also, even that seems suspect to me: if `await` looks for an __await__ > method and simply returns the same object (synchronously) if the > object doesn't have an await method, then your code sample that > supposedly will fail if a function ceases to be a coroutine *will not > actually fail*. > > In my experience working with coroutine systems, making a system > polymorphic (do something appropriate with what's given) and > idempotent (don't do anything if what's wanted is already done) makes > it more robust. In particular, it eliminates the issue of mixing > coroutines and non-coroutines. > > To sum up: I can see the use case for a new `await` distinguished from > `yield`, but I don't see the need to create new syntax for everything; > ISTM that adding the new asynchronous protocols and using them on > demand is sufficient. Marking a function asynchronous so it can use > asynchronous iteration and context management seems reasonably useful, > but I don't think it's terribly important for the type of function > result. Indeed, ISTM that the built-in `object` class could just > implement `__await__` as a no-op returning self, and then *all* > results are trivially asynchronous results and can be awaited > idempotently, so that awaiting something that has already been waited > for is a no-op. (Prior art: the Javascript Promise.resolve() method, > which takes either a promise or a plain value and returns a promise, > so that you can write code which is always-async in the presence of > values that may already be known.) > > Finally, if the async for and with operations have to be distinguished > by syntax at the point of use (vs. just always being used in > coroutines), then ISTM that they should be `with async foo:` and `for > async x in bar:`, since the asynchronousness is just an aspect of how > the main keyword is executed. > > tl;dr: I like the overall ideas but hate the syntax and type > segregation involved: declaring a function async at the top is OK to > enable async with/for semantics and await expressions, but the rest > seems unnecessary and bad for writing robust code. (e.g. note that > requiring different syntax means a function must either duplicate code > or restrict its input types more, and type changes in remote parts of > the program will propagate syntax changes throughout.) > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Thu, Apr 23, 2015 at 3:35 AM, Guido van Rossum wrote:
> On Wed, Apr 22, 2015 at 5:12 PM, Greg Ewing
> wrote:
>>
>> Guido van Rossum wrote:
>>>
>>> On Wed, Apr 22, > OTOH I'm still struggling with what you have to do to
>>> wrap a coroutine in a Task, the way its done in asyncio by the Task()
>>> constructor, the loop.create_task() method, and the async() function
>>
>>
>> That's easy. You can always use costart() to adapt a cofunction
>> for use with something expecting a generator-based coroutine,
>> e.g.
>>
>> codef my_task_func(arg):
>> ...
>>
>> my_task = Task(costart(my_task_func, arg))
>>
>> If you're willing to make changes, Task() et al could be made to
>> recognise cofunctions and apply costart() where needed.
>
>
> Hm, that feels backwards incompatible (since currently I can write
> Task(my_task_func(arg)) and also a step backwards in elegance (having to
> pass the args separately).
>
> OTOH the benefit is that it's much harder to accidentally forget to wait for
> a coroutine. And maybe the backward compatibility issue is not really a
> problem because you have to opt in by using codef or async def.
>
> So I'm still torn. :-)
>
> Somebody would need to take a mature asyncio app and see how often this is
> used (i.e. how many place would require adding costart() as in the above
> example).
>
I have not found fresh patch for 3152 to play with, but at least
aiohttp [1] library very often creates new tasks by `async(coro(...))`
call. The same for aiozmq, aioredis, sockjs (aiohttp-based library for
sock.js), aiokafka etc.
My applications created for my job also has a `async(...)` calls or
direct `Task(f(arg))` creations -- the numbers are between 3 and 10
usage lines per application. Not a big deal to fix them all but it's
backward incompatibility.
In opposite, I've finished experimental branch [2] of aiomysql library
(asyncio driver for MySQL database) with support for `async for` and
`async with`.
The main problem with public released version is impossibility to
handle transactions (requires async context manager) and iteration
with async fetching data from cursor (required for server-side cursors
for example).
Now both problems are solved with keeping full backward compatibility.
The library can be used with Python 3.3+ but obviously no new features
are available for old Pythons.
I use asyncio coroutines, not async functions, e.g.:
class Cursor:
# ...
@asyncio.coroutine
def __aiter__(self):
return self
@asyncio.coroutine
def __anext__(self):
ret = yield from self.fetchone()
if ret is not None:
return ret
else:
raise StopAsyncIteration
The whole aiomysql code is correct from Python 3.3+ perspective. For
testing new features I use new syntax of in separate test files, test
runner will skip test modules with syntax errors on old Python but run
those modules on python from PEP 492 branch.
Usage example (table 'tbl' is pre-filled, DB engine is connected to server):
async def go(engine):
async with engine.connect() as conn:
async with (await conn.begin()) as tr:
await conn.execute("DELETE FROM tbl WHERE (id % 2) = 0")
async for row in conn.execute("SELECT * FROM tbl"):
print(row['id'], row['name'])
[1] https://github.com/KeepSafe/aiohttp
[2] https://github.com/aio-libs/aiomysql/tree/await
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
I guess to raise exception on unwinded async generator in destructor even in non-debug mode. Debug mode may have more complex info with source_traceback included, as Victor Stinner does for CoroWrapper. On Thu, Apr 23, 2015 at 4:27 AM, Yury Selivanov wrote: > Greg, > > On 2015-04-22 7:47 PM, Greg Ewing wrote: >> >> Yury Selivanov wrote: >> >>> On the other hand, I hate the idea >>> of grammatically requiring parentheses for 'await' >>> expressions. That feels non-pytonic to me. >> >> >> How is it any different from grammatically requiring >> parens in an ordinary function call? Nobody ever >> complained about that. > > > It is different. > > 1. Because 'await' keyword might be at a great distance > from the object you're really calling: > > await foo.bar.baz['spam']() > +---+ > > Can I chain the calls: > > await foo()() ? > > or await foo().bar()? > > 2. Because there is no other keyword in python > with similar behaviour. > > 3. Moreover: unless I can write 'await future' - your > proposal *won't* work with a lot of existing code > and patterns. It's going to be radically different > from all other languages that implement 'await' too. > > Yury > > _______ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
sal to add async/await in ECMAScript 7 [2]_; >> see also Traceur project [9]_; >> >> * Facebook's Hack/HHVM [6]_; >> >> * Google's Dart language [7]_; >> >> * Scala [8]_; >> >> * proposal to add async/await to C++ [10]_; >> >> * and many other less popular languages. >> >> This is a huge benefit, as some users already have experience with >> async/await, and because it makes working with many languages in one >> project easier (Python with ECMAScript 7 for instance). >> >> >> Why "__aiter__" is a coroutine >> -- >> >> In principle, ``__aiter__`` could be a regular function. There are >> several good reasons to make it a coroutine: >> >> * as most of the ``__anext__``, ``__aenter__``, and ``__aexit__`` >> methods are coroutines, users would often make a mistake defining it >> as ``async`` anyways; >> >> * there might be a need to run some asynchronous operations in >> ``__aiter__``, for instance to prepare DB queries or do some file >> operation. >> >> >> Importance of "async" keyword >> - >> >> While it is possible to just implement ``await`` expression and treat >> all functions with at least one ``await`` as coroutines, this approach >> makes APIs design, code refactoring and its long time support harder. >> >> Let's pretend that Python only has ``await`` keyword:: >> >> def useful(): >> ... >> await log(...) >> ... >> >> def important(): >> await useful() >> >> If ``useful()`` function is refactored and someone removes all >> ``await`` expressions from it, it would become a regular python >> function, and all code that depends on it, including ``important()`` >> would be broken. To mitigate this issue a decorator similar to >> ``@asyncio.coroutine`` has to be introduced. >> >> >> Why "async def" >> --- >> >> For some people bare ``async name(): pass`` syntax might look more >> appealing than ``async def name(): pass``. It is certainly easier to >> type. But on the other hand, it breaks the symmetry between ``async >> def``, ``async with`` and ``async for``, where ``async`` is a modifier, >> stating that the statement is asynchronous. It is also more consistent >> with the existing grammar. >> >> >> Why not a __future__ import >> --- >> >> ``__future__`` imports are inconvenient and easy to forget to add. >> Also, they are enabled for the whole source file. Consider that there >> is a big project with a popular module named "async.py". With future >> imports it is required to either import it using ``__import__()`` or >> ``importlib.import_module()`` calls, or to rename the module. The >> proposed approach makes it possible to continue using old code and >> modules without a hassle, while coming up with a migration plan for >> future python versions. >> >> >> Why magic methods start with "a" >> >> >> New asynchronous magic methods ``__aiter__``, ``__anext__``, >> ``__aenter__``, and ``__aexit__`` all start with the same prefix "a". >> An alternative proposal is to use "async" prefix, so that ``__aiter__`` >> becomes ``__async_iter__``. However, to align new magic methods with >> the existing ones, such as ``__radd__`` and ``__iadd__`` it was decided >> to use a shorter version. >> >> >> Why not reuse existing magic names >> -- >> >> An alternative idea about new asynchronous iterators and context >> managers was to reuse existing magic methods, by adding an ``async`` >> keyword to their declarations:: >> >> class CM: >> async def __enter__(self): # instead of __aenter__ >>
Re: [Python-Dev] async/await in Python; v2
Greg, how waiting for multiple cocalls should look and work? In asyncio when I need to wait for two and more coroutines/futures I use `asyncio.gather()`: yield from gather(coro1(a1, a2), coro2(), fut3) >From my understanding to use cofunctions I must wrap it with costart call: yield from gather(costart(coro1, a1, a2), costart(coro2), fut3) That looks weird. There are other places in asyncio API those accept coroutines or futures as parameters, not only Task() and async(). On Thu, Apr 23, 2015 at 12:25 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> I think that the problem of forgetting 'yield from' is a bit exaggerated. >> Yes, I myself forgot 'yield from' once or twice. But that's it, it has never >> happened since. > > > I think it's more likely to happen when you start with > an ordinary function, then discover that it needs to > be suspendable, so you need to track down all the > places that call it, and all the places that call > those, etc. PEP 3152 ensures that you get clear > diagnostics if you miss any. > > -- > Greg > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Thu, Apr 23, 2015 at 3:10 PM, Greg Ewing wrote: > Andrew Svetlov wrote: >> >> From my understanding to use cofunctions I must wrap it with costart call: >> >> yield from gather(costart(coro1, a1, a2), costart(coro2), fut3) >> >> There are other places in asyncio API those accept coroutines or >> futures as parameters, not only Task() and async(). > > > In a PEP 3152 aware version of asyncio, they would all > know about cofunctions and what to do with them. > But we already have asyncio and code based on asyncio coroutines. To make it work I should always use costart() in places where asyncio requires coroutine. Maybe your proposal is better than current asyncio practice. But now asyncio is built on top of two-step process, as you have mentioned: building coroutine and waiting for it's result. That's why I prefer `await` as replace for well-known `yield from`. > > -- > Greg > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Thu, Apr 23, 2015 at 3:27 PM, Wolfgang Langner wrote: > > > On Thu, Apr 23, 2015 at 12:35 PM, Paul Sokolovsky wrote: >> >> Hello, >> >> On Thu, 23 Apr 2015 12:18:51 +0300 >> Andrew Svetlov wrote: >> >> [] >> >> > > 3. >> > > async with and async for >> > > Bead idea, we clutter the language even more and it is one more >> > > thing every newbie could do wrong. >> > > for x in y: >> > > result = await f() >> > > is enough, every 'async' framework lived without it over years. >> > >> > async for i in iterable: >> > pass >> > >> > is not equal for >> > >> > for fut in iterable: >> > i = yield from fut >> >> But people who used Twisted all their life don't know that! They just >> know that "async for" is not needed and bad. >> > > I don't think it is bad nor not needed, but the syntax is not beautiful and > for the 90% not doing async stuff irritating and one more thing to learn > and do right/wrong. > > I had also a need for async loop. But there are other solutions like > channels, > not needing a new syntax. > By `channels` do you mean something like `asyncio.Queue`? It requires that producer and consumer should be separate tasks. Often it works (with some performance penalty cost) but creating 2 tasks is not always obvious way to solve problem. > Also possible a function returning futures and yield in the loop with a > sentinel. A proposal looks like guess to avoid `for` loop and use `while` everywhere. Just compare `while` loop: it = iter(it) while True: try: i = next(it) process(i) except StopIteration: break with `for` alternative: for i in it: process(i) > > All this goes the road down to a producer consumer pattern. Nothing more. > I think one of the most convenient consumer-producer pattern implementation in Python is `for` loop and iterators concept. It's sometimes too limited but works pretty well in 95% of use cases. > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3152 and yield from Future()
I can live with `cocall fut()` but the difference between `data = yield from loop.sock_recv(sock, 1024)` and `data = cocall (loop.sock_recv(sock, 1024))()` frustrates me very much. — Sent from Mailbox On Thu, Apr 23, 2015 at 4:09 PM, Victor Stinner wrote: > (I prefer to start a new thread, the previous one is too long for me :-)) > Hi, > I'm still trying to understand how the PEP 3152 would impact asyncio. > Guido suggests to replace "yield from fut" with "cocall fut()" (add > parenthesis) and so add a __cocall__() method to asyncio.Future. > Problem: PEP 3152 says "A cofunction (...) does not contain any yield > or yield from expressions". Currently, Future.__iter__() is > implemented using yield: > def __iter__(self): > if not self.done(): > self._blocking = True > yield self # This tells Task to wait for completion. > assert self.done(), "yield from wasn't used with future" > return self.result() # May raise too. > From my understanding, the PEP 3151 simply does not support > asyncio.Future. Am I right? > How is it possible to suspend a cofunction if it's not possible to use yield? > If waiting for a future in a cofunction is not supported, the PEP 3151 > is useless for asyncio, since asyncio completly relies on futures. > Victor > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Fri, Apr 24, 2015 at 3:14 AM, Greg Ewing wrote: > Andrew Svetlov wrote: > >> But we already have asyncio and code based on asyncio coroutines. >> To make it work I should always use costart() in places where asyncio >> requires coroutine. > > > As I understand it, asyncio would require changes to > make it work seamlessly with PEP 492 as well, since > an object needs to have either a special flag or > an __await__ method before it can have 'await' > applied to it. > PEP 492 requires a change of asyncio.Future only. PEP 3152 requires of change in any asyncio-based library, this is the difference. > > -- > Greg > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3152 and yield from Future()
On Fri, Apr 24, 2015 at 11:34 AM, Greg Ewing wrote: > Yury Selivanov wrote: > >> It's a common pattern in asyncio when functions >> return futures. It's OK later to refactor those >> functions to coroutines *and* vice-versa. This >> is a fundamental problem for PEP 3152 approach. > > > Hmmm. So you have an ordinary function that returns > a future, and you want to turn it into a coroutine > function, but still have it return a future in > order to keep the API the same, is that right? No. In asyncio there is no difference between coroutine and regular function returning future. >From caller site next both are equal: @asyncio.coroutine def f(): return 1 def g(): fut = asyncio.Future() fut.set_result(1) return fut Both may be called via `yield from`: ret1 = yield from f() ret2 = yield from g() > > Turning it into a coroutine means you're going > to have to change every site that calls it, so > its API has already changed. Given that, I'm not > sure what advantage there is in keeping the future- > returning part of the API. > > However, if we use the await()-cofunction idea, > then a call to the initial version looks like > >cocall await(f(x)) > > and after the refactoring it becomes > >cocall await(cocall f(x)) > > That doesn't look so bad to me. > > -- > Greg > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
I used to think in the same way but found the result looks like Perl (or Haskell), not Python. On Sat, Apr 25, 2015 at 7:47 AM, Greg Ewing wrote: > Wild idea: > > Let "@" mean "async" when it's directly in front > of a keyword. > > Then we would have: > > @def f(): > ... > > @for x in iter: > ... > > @with context as thing: > ... > > -- > Greg > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
I prefer option #3.
On Mon, Apr 27, 2015 at 4:44 PM, Yury Selivanov wrote:
> Hi Greg,
>
> I don't want this: "await a() * b()" to be parsed, it's not meaningful.
>
> Likely you'll see "await await a()" only once in your life, so I'm fine to
> use parens for it (moreover, I think it reads better with parens)
>
> Yury
>
>
> On 2015-04-27 8:52 AM, Greg Ewing wrote:
>>
>> Yury Selivanov wrote:
>>>
>>> I've done some experiments with grammar, and it looks like
>>> we indeed can parse await quite differently from yield. Three
>>> different options:
>>
>>
>> You don't seem to have tried what I suggested, which is
>> to make 'await' a unary operator with the same precedence
>> as '-', i.e. replace
>>
>>factor: ('+'|'-'|'~') factor | power
>>
>> with
>>
>>factor: ('+'|'-'|'~'|'await') factor | power
>>
>> That would allow
>>
>> await a()
>> res = await a() + await b()
>> res = await await a()
>> if await a(): pass
>> return await a()
>> print(await a())
>> func(arg=await a())
>> await a() * b()
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
— Sent from Mailbox On Wednesday Apr 29, 2015 at 7:14 PM, Yury Selivanov , wrote:Also please see this: https://hg.python.org/peps/rev/d048458307b7 FWIW, 'await await fut' isn't something that you likely to see in your life; 'await -fut' is 99.999% just a bug. Agree. I'm not sure why Greg is pushing his Grammar idea so aggressively. Yury On 2015-04-29 11:33 AM, Yury Selivanov wrote: > Ethan, > > On 2015-04-29 11:29 AM, Ethan Furman wrote: >> Python is not lisp, and await is not a >> function, so parens should not be needed in the common case. > > Which common case you mean? > > Please see this table > https://www.python.org/dev/peps/pep-0492/#syntax-of-await-expression > > Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 492 (async/await)
Congrats, Yury! On Wed, May 6, 2015 at 11:07 AM, Paul Moore wrote: > On 6 May 2015 at 00:58, Guido van Rossum wrote: >> I totally forgot to publicly congratulate Yury on this PEP. He's put a huge >> effort into writing the PEP and the implementation and managing the >> discussion, first on python-ideas, later on python-dev. Congrats, Yury! And >> thanks for your efforts. Godspeed. > > Agreed, congratulations! There's been a lot of debate on this PEP, and > Yury has done a great job of responding where needed and keeping > things on track, which can't have been easy. Thanks for all the work. > > Paul. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ssl] The weird case of IDNA
ssl.match_hostname was added in Python 2.7.9, looks like Python 2 should be fixed as well. On Sat, Dec 30, 2017 at 3:50 PM Antoine Pitrou wrote: > > Thanks. So the change sounds ok to me. > > Regards > > Antoine. > > > On Sat, 30 Dec 2017 14:34:04 +0100 > Christian Heimes wrote: > > On 2017-12-30 11:28, Antoine Pitrou wrote: > > > On Fri, 29 Dec 2017 21:54:46 +0100 > > > Christian Heimes wrote: > > >> > > >> On the other hand ssl module is currently completely broken. It > converts > > >> hostnames from bytes to text with 'idna' codec in some places, but not > > >> in all. The SSLSocket.server_hostname attribute and callback function > > >> SSLContext.set_servername_callback() are decoded as U-label. > > >> Certificate's common name and subject alternative name fields are not > > >> decoded and therefore A-labels. The *must* stay A-labels because > > >> hostname verification is only defined in terms of A-labels. We even > had > > >> a security issue once, because partial wildcard like 'xn*.example.org > ' > > >> must not match IDN hosts like 'xn--bcher-kva.example.org'. > > >> > > >> In issue [2] and PR [3], we all agreed that the only sensible fix is > to > > >> make 'SSLContext.server_hostname' an ASCII text A-label. > > > > > > What are the changes in API terms? If I'm calling wrap_socket(), can I > > > pass `server_hostname='straße'` and it will IDNA-encode it? Or do I > > > have to encode it myself? If the latter, it seems like we are putting > > > the burden of protocol compliance on users. > > > > Only SSLSocket.server_hostname attribute and the hostname argument to > > the SNI callback will change. Both values will be A-labels instead of > > U-labels. You can still pass an U-label to the server_hostname argument > > and it will be encoded with "idna" encoding. > > > > >>> sock = ctx.wrap_socket(socket.socket(), server_hostname=' > www.straße.de <http://www.strasse.de>') > > > > Currently: > > >>> sock.server_hostname > > 'www.straße.de <http://www.strasse.de>' > > > > Changed: > > >>> sock.server_hostname > > 'www.strasse.de' > > > > Christian > > > > ___________ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reminder: Please elaborate commit messages
Sorry for that. I thought that the bpo issue can be skipped because it is tests-only change, no asyncio code was affected. Will be more accurate next time. On Tue, May 22, 2018 at 3:26 PM Victor Stinner wrote: > Hi, > > In https://bugs.python.org/issue33531, Andrew Svetlov wrote "Fixed > failed sendfile tests on Windows (at least I hope so)." without giving > any bpo number or a commit number. So I looked at latest commits and I > found: > > --- > commit e2537521916c5bf88fcf54d4654ff1bcd332be4a > Author: Andrew Svetlov > Date: Mon May 21 12:03:45 2018 +0300 > > Fix asyncio flaky tests (#7023) > --- > > Please try to write better error messages for people who will dig into > the Git history in 1, 5 or 10 years: > > * Usually, it's better to open a bug. Here you could give the full > error message, mention on which platform the test fails, etc. > * Mention which tests are affected > * Maybe even give an extract of the error message of the fixed test in > the commit message > > I know that it's more effort and fixing flaky tests is annoying and > may require multiple iterations, but again, please think to people who > will have to read the Git history later... > > I was able able to rebuild the context of this commit from a comment > of https://bugs.python.org/issue33531 > > Victor > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding c-api async protocol support
I'm with Victor: we are in beta now. Making C API is useful and important but we may wait for new Python release. The same for asycnio acceleration: we definitely need it but it requires inviting C API also I believe. Personally I've concentrated on making third-party libraries on top of asyncio -- aiohttp etc. P.S. Thank you Victor so much for your work on asyncio. Your changes on keeping source tracebacks and raising warnings for unclosed resources are very helpful. On Thu, Jun 25, 2015 at 6:56 PM, Victor Stinner wrote: > It looks like the code is currently moving fast. I suggest to wait > until Python 3.6 to stabilize the Python C API for async/await. It's a > pain to maintain a public API. I hate having to add 2 or 3 versions of > a single function :-( > > Victor > > 2015-06-25 17:43 GMT+02:00 Yury Selivanov : >> Hi Arc, >> >> >> On 2015-06-24 10:36 PM, Arc Riley wrote: >>> >>> A type slot for tp_as_async has already been added (which is good!) but we >>> do not currently seem to have protocol functions for awaitable types. >>> >>> I would expect to find an Awaitable Protocol listed under Abstract Objects >>> Layer, with functions like PyAwait_Check, PyAwaitIter_Check, and >>> PyAwaitIter_Next, etc. >>> >>> Specifically its currently difficult to test whether an object is >>> awaitable >>> or an awaitable iterable, or use said objects from the c-api without >>> relying on method testing/calling mechanisms. >> >> >> The request is reasonable, I created a couple of bug tracker >> issues: >> >> http://bugs.python.org/issue24511 >> http://bugs.python.org/issue24510 >> >> Let's continue the discussion there. >> >> Yury >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Importance of "async" keyword
> Another issue that bothers me, is code reuse. Independent from whether the > 'async def' makes sense or not, it would not allow us to reuse asyncio > functions as if they were normal functions and vice versa (if I understood > that correctly). So, we would have to implement things twice for the asyncio > world and the classic world. To me, it would be important to use one > function in either world where it suits me better. I am uncertain if that > makes sense but right now it does to me. Yes, you cannot call async function from synchronous code. There are two worlds: classic and async. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should asyncio ignore KeyboardInterrupt?
I believe it's a bug #23057 http://bugs.python.org/issue23057
On Sat, Jul 4, 2015 at 10:58 PM, Guido van Rossum wrote:
> It's possible, but AFAIK asyncio.sleep() has nothing in common with
> time.sleep() -- it's implemented as a timeout on select() or on the IOCP
> loop. (I also have no access to Windows ATM.)
>
> On Sat, Jul 4, 2015 at 7:49 PM, R. David Murray
> wrote:
>>
>> Once long ago in Internet time (issue 581232) time.sleep on windows was
>> not interruptible and this was fixed. Is it possible the work on EINTR
>> has broken that fix?
>>
>> (I don't currently have 3.5 installed on windows to test that theory...)
>>
>> On Sat, 04 Jul 2015 17:46:34 +0200, Guido van Rossum
>> wrote:
>> > I think this may be more of a Windows issue than an asyncio issue. I
>> > agree
>> > that ideally ^C should take effect immediately (as it does on UNIX).
>> >
>> > On Sat, Jul 4, 2015 at 9:54 AM, Terry Reedy wrote:
>> >
>> > > Should the loop.run... methods of asyncio respect KeyboardInterrupt
>> > > (^C)?
>> > >
>> > > Developer and user convenience and this paragraph in PEP
>> > >
>> > > "However, exceptions deriving only from BaseException are typically
>> > > not
>> > > caught, and will usually cause the program to terminate with a
>> > > traceback.
>> > > In some cases they are caught and re-raised. (Examples of this
>> > > category
>> > > include KeyboardInterrupt and SystemExit ; it is usually unwise to
>> > > treat
>> > > these the same as most other exceptions.) "
>> > >
>> > > and this examples in the doc (two places)
>> > >
>> > > TCP echo server
>> > > # Serve requests until CTRL+c is pressed
>> > > print('Serving on {}'.format(server.sockets[0].getsockname()))
>> > > try:
>> > > loop.run_forever()
>> > > except KeyboardInterrupt:
>> > > pass
>> > >
>> > > suggest yes. On the other hand, the section on
>> > > "Set signal handlers for SIGINT and SIGTERM"
>> > > suggests not, unless an explicit handler is registered and then only
>> > > on
>> > > Unix.
>> > >
>> > > In any case, Adam Bartos, python-list, "An asyncio example", today
>> > > asks.
>> > > '''
>> > > This is a minimal example:
>> > >
>> > > import asyncio
>> > >
>> > > async def wait():
>> > > await asyncio.sleep(5)
>> > >
>> > > loop = asyncio.get_event_loop()
>> > > loop.run_until_complete(wait())
>> > >
>> > > Ctrl-C doesn't interrupt the waiting, instead KeyboardInterrupt occurs
>> > > after those five seconds. It's 3.5.0b2 on Windows. Is it a bug?
>> > > '''
>> > >
>> > > Using run_forever instead, I found no way to stop other than killing
>> > > the
>> > > process (Idle or Command Prompt).
>> > >
>> > > --
>> > > Terry Jan Reedy
>> > >
>> > > ___
>> > > Python-Dev mailing list
>> > > Python-Dev@python.org
>> > > https://mail.python.org/mailman/listinfo/python-dev
>> > > Unsubscribe:
>> > > https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> > >
>> >
>> >
>> >
>> > --
>> > --Guido van Rossum (python.org/~guido)
>> > ___
>> > Python-Dev mailing list
>> > Python-Dev@python.org
>> > https://mail.python.org/mailman/listinfo/python-dev
>> > Unsubscribe:
>> > https://mail.python.org/mailman/options/python-dev/rdmurray%40bitdance.com
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Rietveld is broken
When I try to press 'retry' on http://bugs.python.org/review/25074/ page and send a message server responds with 500 error. Going through the page for viewing diff for concrete file http://bugs.python.org/review/25074/diff/15535/Lib/test/test_asyncio/test_base_events.py with adding a comment works well. The behavior is very annoying -- I was forced to write missed message twice. -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await behavior on multiple calls
Both Yury's suggestions sounds reasonable. On Tue, Dec 15, 2015 at 10:24 PM, Yury Selivanov wrote: > Hi Roy and Guido, > > On 2015-12-15 3:08 PM, Guido van Rossum wrote: > [..] >> >> >> I don't know how long you have been using async/await, but I wonder if >> it's possible that you just haven't gotten used to the typical usage >> patterns? In particular, your claim "anything that takes an `awaitable` has >> to know that it wasn't already awaited" makes me sound that you're just >> using it in an atypical way (perhaps because your model is based on other >> languages). In typical asyncio code, one does not usually take an awaitable, >> wait for it, and then return it -- one either awaits it and then extracts >> the result, or one returns it without awaiting it. > > > I agree. Holding a return value just so that coroutine can return it again > seems wrong to me. > > However, since coroutines are now a separate type (although they share a lot > of code with generators internally), maybe we can change them to throw an > error when they are awaited on more than one time? > > That should be better than letting them return `None`: > > coro = coroutine() > await coro > await coro # <- will raise RuntimeError > > > I'd also add a check that the coroutine isn't being awaited by more than one > coroutine simultaneously (another, completely different issue, more on which > here: https://github.com/python/asyncio/issues/288). This was fixed in > asyncio in debug mode, but ideally, we should fix this in the interpreter > core. > > Yury > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Asynchronous context manager in a typical network server
I my asyncio code typical initialization/finalization procedures are much more complicated. I doubt if common code can be extracted into asyncio. Personally I don't feel the need for `wait_forever()` or `loop.creae_context_task()`. But even if you need it you may create it from scratch easy, isn't it? On Fri, Dec 18, 2015 at 3:58 PM, Szieberth Ádám wrote: > Hi Developers! > > This is my first post. Please excuse me my poor English. If anyone is > interested, I wrote a small introduction on my homepage. Link is at the > bottom. > > This post is about how to effectively implement the new asynchronous context > manager in a typical network server. > > I would appreciate and welcome any confirmation or critics whether my thinking > is right or wrong. Thanks in advance! > > So, a typical server main code I used to see around is like this: > > srv = loop.run_until_complete(create_server(handler, host, port)) > try: > loop.run_forever() > except KeyboardInterrupt: > pass > finally: > # other tear down code may be here > srv.close() > loop.run_until_complete(srv.wait_closed()) > loop.close() > > Note that `create_server()` here is not necessary > `BaseEventLoop.create_server()`. > > The above code is not prepared to handle `OSError`s or any other `Exception`s > (including a `KeyboardInterrupt` by a rapid Ctr+C) when setting up the server, > it just prints the traceback to the console which is not user friendly. > Moreover, I would expect from a server to handle the SIGTERM signal as well > and tell its clients that it stops serving when not force killed. > > How the main code should create server, maintain the serving, deal with errors > and close properly both the connections and the event loop when exiting > without letting pending tasks around is not trivial. There are many questions > on SO and other places of the internet regarding of this problem. > > My idea was to provide a simple code which is robust in terms of these > concerns by profiting from the new asynchronous context manager pattern. > > The code of the magic methods of a typical awaitable `CreateServer` object > seems rather trivial: > > async def __aenter__(self): > self.server = await self > return self.server > > async def __aexit__(self, exc_type, exc_value, traceback): > # other tear down code may be here > self.server.close() > await self.server.wait_closed() > > However, to make it work, a task has to be created: > > async def server_task(): > async with CreateServer(handler, host, port) as srv: > await asyncio.Future() # wait forever > > I write some remarks regarding the above code to the end of this post. Note > that `srv` is unreachable from outside which could be a problem in some cases. > What is unavoidable: this task has to get cancelled explicitely by the main > code which should look like this: > > srvtsk = loop.create_task(server_task()) > > signal.signal(signal.SIGTERM, lambda si, fr: > loop.call_soon(srvtsk.cancel)) > > while True: > try: > loop.run_until_complete(srvtsk) > except KeyboardInterrupt: > srvtsk.cancel() > except asyncio.CancelledError: > break > except Exception as err: > print(err) > break > loop.close() > > Note that when `CancelledError` gets raised, the tear down process is already > done. > > Remarks: > > * It would be nice to have an `asyncio.wait_forever()` coroutine for dummy > context bodies. > * Moreover, I also imagined an `BaseEventLoop.create_context_task(awithable, > body_coro_func=None)` method. The `body_coro_func` should default to > `asyncio.wait_forever()`, otherwise it should get whatever is returned by > `__aenter__` as a single argument. The returned Task object should also > provide a reference to that object. > > Best regards, > Ádám > > (http://szieberthadam.github.io/) > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] [RELEASED] Python 3.3.4 release candidate 1
Would you to accept fixes for http://bugs.python.org/issue20434 and http://bugs.python.org/issue20437 before 3.3.4 final? On Mon, Jan 27, 2014 at 9:36 AM, Georg Brandl wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On behalf of the Python development team, I'm reasonably happy to announce the > Python 3.3.4 release candidate 1. > > Python 3.3.4 includes several security fixes and over 120 bug fixes compared > to > the Python 3.3.3 release. > > This release fully supports OS X 10.9 Mavericks. In particular, this release > fixes an issue that could cause previous versions of Python to crash when > typing > in interactive mode on OS X 10.9. > > Python 3.3 includes a range of improvements of the 3.x series, as well as > easier > porting between 2.x and 3.x. In total, almost 500 API items are new or > improved > in Python 3.3. For a more extensive list of changes in the 3.3 series, see > > http://docs.python.org/3.3/whatsnew/3.3.html > > and for the detailed changelog of 3.3.4, see > > http://docs.python.org/3.3/whatsnew/changelog.html > > To download Python 3.3.4 rc1 visit: > > http://www.python.org/download/releases/3.3.4/ > > This is a preview release, please report any bugs to > > http://bugs.python.org/ > > The final version is scheduled to be released in two weeks' time, on or about > the 10th of February. > > Enjoy! > > - -- > Georg Brandl, Release Manager > georg at python.org > (on behalf of the entire python-dev team and 3.3's contributors) > -BEGIN PGP SIGNATURE- > Version: GnuPG v2.0.22 (GNU/Linux) > > iEYEARECAAYFAlLmDI4ACgkQN9GcIYhpnLAr6wCePRbHF80k5goV4RUDBA5FfkwF > rLUAnRg0RpL/b6apv+Dt2/sgnUd3hTPA > =Z4Ss > -END PGP SIGNATURE- > _______ > python-committers mailing list > python-committ...@python.org > https://mail.python.org/mailman/listinfo/python-committers -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: inspect.Signature: Add 'Signature.from_callable' classmethod. Closes #17373
And probably the block should be deindented On Thu, Mar 27, 2014 at 6:48 PM, Antoine Pitrou wrote: > On Thu, 27 Mar 2014 17:12:02 +0100 (CET) > yury.selivanov wrote: >> >> +.. classmethod:: Signature.from_callable(obj) >> + >> + Return a :class:`Signature` (or its subclass) object for a given >> callable >> + ``obj``. This method simplifies subclassing of :class:`Signature`: >> + >> + :: >> + >> + class MySignature(Signature): >> + pass >> + sig = MySignature.from_callable(min) >> + assert isinstance(sig, MySignature) >> + > > This needs a "versionadded" tag. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
5 cents about lz4 alternatives: Broli (mentioned above) is widely supported by web. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding mentions it along with gzip and deflate methods. I don't recall lz4 or Zstd metioning in this context. Both Chrome/Chromium and Firefox accepts it by default (didn't check Microsoft products yet). P.S. I worked with lz4 python binding a year ago. It sometimes crashed to core dump when used in multithreaded environment (we used to run compressor/decompresson with asyncio by loop.run_in_executor() call). I hope the bug is fixed now, have no update for the current state. On Thu, Nov 29, 2018 at 12:04 PM INADA Naoki wrote: > On Thu, Nov 29, 2018 at 6:27 AM Steven D'Aprano > wrote: > > > > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > > > PyPI makes getting more algorithms easy. > > > > Can we please stop over-generalising like this? PyPI makes getting > > more algorithms easy for *SOME* people. (Sorry for shouting, but you > > just pressed one of my buttons.) > > I don't think this is over-generalising. > > If "get it from PyPI" is not easy enough, why not adding hundreds of > famous libraries? > Because we can't maintain all of them well. > > When considering adding new format (not only compression, but also > serialization like toml), I think it should be stable, widely used, and > will > be used widely for a long time. If we want to use the format in Python > core > or Python stdlib, it's good reasoning too. gzip and json are good example. > > When we say "we can use PyPI", it means "are there enough reasons > make the package special enough to add to stdlib?" We don't mean > "everyone can use PyPI." > > Regards, > -- > INADA Naoki > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
Neither http.client nor http.server doesn't support compression (gzip/compress/deflate) at all. I doubt if we want to add this feature: for client better to use requests or, well, aiohttp. The same for servers: almost any production ready web server from PyPI supports compression. I don't insist on adding brotli to standard library. There is officiall brotli library on PyPI from google, binary wheels are provided. Unfortunately installing from a tarball requires C++ compiler On other hand writing a binding in pure C looks very easy. On Thu, Nov 29, 2018 at 10:30 PM Gregory P. Smith wrote: > > On Thu, Nov 29, 2018 at 2:58 AM Andrew Svetlov > wrote: > >> 5 cents about lz4 alternatives: Broli (mentioned above) is widely >> supported by web. >> >> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding >> mentions it along with gzip and deflate methods. >> I don't recall lz4 or Zstd metioning in this context. >> >> Both Chrome/Chromium and Firefox accepts it by default (didn't check >> Microsoft products yet). >> > > Acceptance by multiple popular browsers is a good reason to *also* > propose brotli support in the stdlib. Though it'd probably make sense to > actually _support_ Accept-Encoding based on available compression modules > within the stdlib http.client (and potentially server) as a prerequisite > for that reasoning. > https://github.com/python/cpython/blob/master/Lib/http/client.py#L1168. > > -gps > > >> P.S. >> I worked with lz4 python binding a year ago. >> It sometimes crashed to core dump when used in multithreaded environment >> (we used to run compressor/decompresson with asyncio by >> loop.run_in_executor() call). >> I hope the bug is fixed now, have no update for the current state. >> >> On Thu, Nov 29, 2018 at 12:04 PM INADA Naoki >> wrote: >> >>> On Thu, Nov 29, 2018 at 6:27 AM Steven D'Aprano >>> wrote: >>> > >>> > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: >>> > >>> > > PyPI makes getting more algorithms easy. >>> > >>> > Can we please stop over-generalising like this? PyPI makes getting >>> > more algorithms easy for *SOME* people. (Sorry for shouting, but you >>> > just pressed one of my buttons.) >>> >>> I don't think this is over-generalising. >>> >>> If "get it from PyPI" is not easy enough, why not adding hundreds of >>> famous libraries? >>> Because we can't maintain all of them well. >>> >>> When considering adding new format (not only compression, but also >>> serialization like toml), I think it should be stable, widely used, and >>> will >>> be used widely for a long time. If we want to use the format in Python >>> core >>> or Python stdlib, it's good reasoning too. gzip and json are good >>> example. >>> >>> When we say "we can use PyPI", it means "are there enough reasons >>> make the package special enough to add to stdlib?" We don't mean >>> "everyone can use PyPI." >>> >>> Regards, >>> -- >>> INADA Naoki >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com >>> >> >> >> -- >> Thanks, >> Andrew Svetlov >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org >> > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
On Thu, Nov 29, 2018 at 11:22 PM Steve Dower wrote: > FWIW, Brotli has been supported in Microsoft Edge since early last year: > > > https://blogs.windows.com/msedgedev/2016/12/20/introducing-brotli-compression/ > > Thanks, good to know. -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
On Fri, Nov 30, 2018 at 2:12 AM Brett Cannon wrote: > > > On Thu, 29 Nov 2018 at 14:12, Andrew Svetlov > wrote: > >> Neither http.client nor http.server doesn't support compression >> (gzip/compress/deflate) at all. >> I doubt if we want to add this feature: for client better to use requests >> or, well, aiohttp. >> The same for servers: almost any production ready web server from PyPI >> supports compression. >> > > There was actually a PR to add compressions support to http.server but I > closed it in the name of maintainability since http.server, as you said, > isn't for production use so compression isn't critical. > > I remember this PR and agree with your decision. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Return type of datetime subclasses added to timedelta
>From my perspective datetime classes are even more complex than int/float.
Let's assume we have
class DT(datetime.datetime): ...
class TD(datetime.timedelta): ...
What is the result type for the following expressions?
DT - datetime
DT - DT
DT + TD
DT + timedelta
I have a feeling that the question has no generic answer.
For *particular* implementation you can override all __add__, __sub__
and other arithmetic operations, and you can do it right now with the
current datetime module implementation.
P.S.
I think inheritance from datetime classes is a very rare thing, 99.99% of
users don't need it.
On Sun, Jan 6, 2019 at 6:03 PM Paul Ganssle wrote:
> I can think of many reasons why datetime is different from builtins,
> though to be honest I'm not sure that consistency for its own sake is
> really a strong argument for keeping a counter-intuitive behavior - and to
> be honest I'm open to the idea that *all* arithmetic types *should* have
> some form of this change.
>
> That said, I would say that the biggest difference between datetime and
> builtins (other than the fact that datetime is *not* a builtin, and as
> such doesn't necessarily need to be categorized in this group), is that
> unlike almost all other arithmetic types, *datetime* has a special,
> dedicated type for describing differences in datetimes. Using your example
> of a float subclass, consider that without the behavior of "addition of
> floats returns floats", it would be hard to predict what would happen in
> this situation:
>
> >>> F(1.2) + 3.4
>
> Would that always return a float, even though F(1.2) + F(3.4) returns an
> F? Would that return an F because F is the left-hand operand? Would it
> return a float because float is the right-hand operand? Would you walk the
> MROs and find the lowest type in common between the operands and return
> that? It's not entirely clear which subtype predominates. With datetime,
> you have:
>
> datetime - datetime -> timedelta
> datetime ± timedelta -> datetime
> timedelta ± timedelta -> timedelta
>
> There's no operation between two datetime objects that would return a
> datetime object, so it's always clear: operations between datetime
> subclasses return timedelta, operations between a datetime object and a
> timedelta return the subclass of the datetime that it was added to or
> subtracted from.
>
> Of course, the real way to resolve whether datetime should be different
> from int/float/string/etc is to look at why this choice was actually made
> for those types in the first place, and decide whether datetime is like
> them *in this respect*. The heterogeneous operations problem may be a
> reasonable justification for leaving the other builtins alone but changing
> datetime, but if someone knows of other fundamental reasons why the
> decision to have arithmetic operations always create the base class was
> chosen, please let me know.
>
> Best,
> Paul
> On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>
>
>
> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle wrote:
>
>> .. the original objection was that this implementation assumes that the
>> datetime subclass has a constructor with the same (or a sufficiently
>> similar) signature as datetime.
>>
> While this was used as a possible rationale for the way standard types
> behave, the main objection to changing datetime classes is that it will
> make them behave differently from builtins. For example:
>
> >>> class F(float):
> ... pass
> ...
> >>> type(F.fromhex('AA'))
>
> >>> type(F(1) + F(2))
>
>
> This may be a legitimate gripe, but unfortunately that ship has sailed
>> long ago. All of datetime's alternate constructors make this assumption.
>> Any subclass that does not meet this requirement must have worked around it
>> long ago (or they don't care about alternate constructors).
>>
>
> This is right, but the same argument is equally applicable to int, float,
> etc. subclasses. If you want to limit your change to datetime types you
> should explain what makes these types special.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AMD64 Windows8.1 Refleaks 3.x buildbot is back to green!
That's great! I would add that many Windows Proactor bugs were found after implementing sendfile support. If sendfile is not available for some reason asyncio uses a fallback to send a file content chunk-by-chunk reading it in memory. The fallback code was easy and straightforward, it worked just fine on Linux but the same code failed on proactor. Thank you very much, Victor, for finally pinning the problem down. On Wed, Jan 9, 2019 at 7:51 PM Brett Cannon wrote: > Thanks for tracking that down! Been bugging me as well and the one time I > tried to figure it out I got no where, so kudos for sticking with it! > > On Wed, 9 Jan 2019 at 09:13, Victor Stinner wrote: > >> Hi, >> >> The "AMD64 Windows 8.1 Refleaks 3.x" buildbot (which hunts reference >> leaks and memory leaks) was failing on test_asyncio for 1 year: >> >>https://bugs.python.org/issue32710 >> >> It was a leak of a single reference: >> >>test_aiosend leaked [1, 1, 1] references, sum=3 >> >> I tried multiple times since last year (January 2018) to understand >> the leak: it didn't make any sense (well, as any bug at the beginning, >> no?). I checked several times the complex asyncio code: all transports >> are cleanly closed, the event loop is properly closed, etc. The code >> looks correct. >> >> After a long sleepness night... I still failed to reproduce the bug >> :-( But I succeeded to get a way shorter reproducer script. Thanks to >> this script, I was able to loop to get 100 reference leaks instead of >> leaking a single reference. Using tracemalloc, I found the faulty >> line... but it still didn't make sense to me. After additional several >> hours of debug, I found that an overlapped object wasn't released >> properly: an asynchronous WSASend(). >> >> The problem was when an overlapped WSASend() failed immediately, the >> internal buffer was not released, whereas it holds a reference to the >> input byte string. **It means that an asyncio send() failure using the >> ProactorEventLoop can cause a memory leak**... I'm very surprised that >> nobody noticed such **huge** memory leak previously! >> >> The _overlapped bugfix: >> >> >> https://github.com/python/cpython/commit/a234e148394c2c7419372ab65b773d53a57f3625 >> >> Eventually, the "AMD64 Windows 8.1 Refleaks 3.x" buildbot is back to >> green! >> >>https://buildbot.python.org/all/#/builders/80 >> >> It means that it will be easier and faster to spot reference or memory >> leak regressions (specific to Windows, the Gentoo Refleaks buildbot >> was already green for several months!). >> >> Since ProactorEventLoop became the default event loop in Python 3.8 >> (on Windows, it's specific to Windows), I hope that we fixed all most >> obvious bugs! >> >> This story also means that any very tiny buildbot failure (a single >> test method failure on a single very specific buildbot) can hide a >> giant bug ;-) Sadly, we have to fix *all* buildbots failures to find >> them... Hopefully, almost all buildbots are green currently. >> >> Victor >> -- >> Night gathers, and now my watch begins. It shall not end until my death. >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/brett%40python.org >> > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lost sight
That's sad to hear. Get well soon! On Sat, Jan 19, 2019 at 1:22 PM Christian Heimes wrote: > On 19/01/2019 11.12, Serhiy Storchaka wrote: > > I have virtually completely lost the sight of my right eye (and the loss > > is quickly progresses) and the sight of my left eye is weak. That is why > > my activity as a core developer was decreased significantly at recent > > time. My apologies to those who are waiting for my review. I will do it > > slowly. > > Oh, I'm sorry to hear that and hope that you'll get better soon. Please > take care of yourself! > > Christian > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm
I see the chicken and egg problem here. If we are talking about typing module usage -- typeshed is the type hints provider. If PyCharm doesn't want to use it -- it is not CPython problem. I think there is no need to change python code itself but used tooling. On Mon, Apr 22, 2019 at 11:06 PM Brett Cannon wrote: > > > > On Sat, Apr 20, 2019 at 2:10 PM Inada Naoki wrote: >> >> "import typing" is slow too. > > > But is it so slow as to not do the right thing here and use the 'typing' > module as expected? If you have so much work you need to launch some threads > or processes to deal with it then a single import isn't going to be your > biggest bottleneck. > > -Brett > >> >> >> 2019年4月21日(日) 1:43 Ilya Kamenshchikov : >>> >>> alright, so would an import under TYPE_CHECKING guard be an option? like: >>> >>> from typing import TYPE_CHECKING >>> if TYPE_CHECKING: >>> from .process import ProcessPoolExecutor >>> from .thread import ThreadPoolExecutor >>> >>> >>> Perhaps we can have both clarity and performance. >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/brett%40python.org > > ___________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm
I agree that `from typing import TYPE_CHECKING` is not desirable from the import time reduction perspective. >From my understanding code completion *can* be based on type hinting to avoid actual code execution. That's why I've mentioned that typeshed already has the correct type information. if TYPE_CHECKING: import ... requires mypy modification. if False: import ... Works right now for stdlib (mypy ignores stdlib code but uses typeshed anyway) but looks a little cryptic. Requires a comprehensive comment at least. On Tue, Apr 23, 2019 at 1:59 AM Inada Naoki wrote: > > On Tue, Apr 23, 2019 at 4:40 AM Brett Cannon wrote: > > > > On Sat, Apr 20, 2019 at 2:10 PM Inada Naoki wrote: > >> > >> "import typing" is slow too. > > > > But is it so slow as to not do the right thing here and use the 'typing' > > module as expected? > > I don't know it is not a "right thing" yet. It feel it is just a > workaround for PyCharm at the moment. > > __dir__ and __all__ has ProcessPoolExecutor and ThreadPoolExecutor for > interactive shell. So Python REPL can complete them. But we didn't discussed > about "static hinting" version of __all__ in PEP 562. > > If we decide it's a "right way", we can update example code in PEP 562. > > But when we use lazy import, we want to make import faster. > Adding more 3~5ms import time seems not so happy solution. > > Maybe, can we add TYPE_CHECKING=False in builtins? > > > > If you have so much work you need to launch some threads or processes to > > deal with it then a single import isn't going to be your biggest bottleneck. > > Importing futures module doesn't mean the app really need > thread or processes. That's why we defer importing ThreadPoolExecutor > and ProcessPoolExecutor. > > And people who want apps like vim starts quickly (~200ms), we want avoid > every "significant overhead" as possible. Not only "the biggest bottleneck" > is the problem. > > -- > Inada Naoki > _______ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()
the > exception. For example, right now we only pass one object. But there > are cases where a second object would totally makes sense. > > -- > > > Packing arguments into a single extendable object just complicates the > > code and increases the chance of raising an exception or crashing. > > Technically, UnraisableHookArgs is basically a tuple of 4 items. I > consider that there is a low risk that creating the object can fail. > > UnraisableHookArgs creation failure *is* covered by my implementation! > The default hook is called directly in C without using a temporary > UnraisableHookArgs object. The exception which caused the failure is > logged as well. > > IMHO the very little risk of UnraisableHookArgs creation failure is > worth it, compared to the pain to extend an API based on a fixed > number of parameters. > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library
rom `audioop`_ C extension) > Deprecated in > 3.8 > To be removed in > 3.10 > Substitute > *n/a* > > > Future maintenance of removed modules > = > > The main goal of the PEP is to reduce the burden and workload on the Python > core developer team. Therefore removed modules will not be maintained by > the core team as separate PyPI packages. However the removed code, tests and > documentation may be moved into a new git repository, so community members > have a place from which they can pick up and fork code. > > A first draft of a `legacylib <https://github.com/tiran/legacylib>`_ > repository is available on my private Github account. > > It's my hope that some of the deprecated modules will be picked up and > adopted by users that actually care about them. For example ``colorsys`` and > ``imghdr`` are useful modules, but have limited feature set. A fork of > ``imghdr`` can add new features and support for more image formats, without > being constrained by Python's release cycle. > > Most of the modules are in pure Python and can be easily packaged. Some > depend on a simple C module, e.g. `audioop`_ and `crypt`_. Since `audioop`_ > does not depend on any external libraries, it can be shipped in as binary > wheels with some effort. Other C modules can be replaced with ctypes or cffi. > For example I created `legacycrypt <https://github.com/tiran/legacycrypt>`_ > with ``_crypt`` extension reimplemented with a few lines of ctypes code. > > > Discussions > === > > * Elana Hashman and Nick Coghlan suggested to keep the *getopt* module. > * Berker Peksag proposed to deprecate and removed *msilib*. > * Brett Cannon recommended to delay active deprecation warnings and removal > of modules like *imp* until Python 3.10. Version 3.8 will be released > shortly before Python 2 reaches end of lifetime. A delay reduced churn for > users that migrate from Python 2 to 3.8. > * Brett also came up with the idea to keep lib2to3. The package is useful > for other purposes, e.g. `black <https://pypi.org/project/black/>`_ uses > it to reformat Python code. > * At one point, distutils was mentioned in the same sentence as this PEP. > To avoid lengthy discussion and delay of the PEP, I decided against dealing > with distutils. Deprecation of the distutils package will be handled by > another PEP. > * Multiple people (Gregory P. Smith, David Beazley, Nick Coghlan, ...) > convinced me to keep the `wave`_ module. [4]_ > * Gregory P. Smith proposed to deprecate `nntplib`_. [4]_ > > > References > == > > .. [1] https://en.wikipedia.org/wiki/Open_Sound_System#Free,_proprietary,_free > .. [2] https://man.openbsd.org/ossaudio > .. [3] > https://blogs.msmvps.com/installsite/blog/2015/05/03/the-future-of-windows-installer-msi-in-the-light-of-windows-10-and-the-universal-windows-platform/ > .. [4] https://twitter.com/ChristianHeimes/status/1130257799475335169 > .. [5] https://twitter.com/dabeaz/status/1130278844479545351 > > > Copyright > = > > This document has been placed in the public domain. > > > > .. >Local Variables: >mode: indented-text >indent-tabs-mode: nil >sentence-end-double-space: t >fill-column: 70 >coding: utf-8 >End: > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: The semantics of pattern matching for Python
If you don't want the overlapping with the existing syntax angle brackets
can be used:
match eggs:
case Spam: ...
On Sat, Nov 21, 2020 at 7:31 PM David Mertz wrote:
> On Sat, Nov 21, 2020 at 12:23 PM Steven D'Aprano
> wrote:
>
>> Clearly Spam(a=1, b=2) does not necessarily result in an instance with
>> attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
>> equivalent to (roughly):
>>
>> if (instance(obj, Spam)
>> and getattr(obj, a) == 1
>> and getattr(obj, b) == 2)
>>
>> it doesn't imply that obj was *literally* created by a call to
>> the constructor `Spam(a=1, b=2)`, or even that this call would be
>> possible.
>>
>
> I think this explanation makes me not worry about the fact that `Spam(a=1,
> b=2)` in a pattern looks a lot like a constructor. Like some other
> commenters, I was vaguely bothered that the identical spelling might have
> these different meanings in different contexts. But I think a match case
> just clearly enough IS a different context that using slightly different
> intuitions is no real conceptual stretch for remembering or teaching it.
>
> As a strawman, we could use different syntax for "match the thing of class
> Spam that has attributes with these values:
>
> match eggs:
> case Spam[a=1, b=2]: ...
>
> Or:
>
> match eggs:
> case Spam{a=1, b=2}: ...
>
> Well, the square brackets COULD mean something different if PEP 637 is
> adopted. But even supposing the curly braces could be fit into the
> grammar. Yes, it sort of suggests the connection between dictionaries and
> Spam.__dict__. But it still reads as "this is something special that I
> have to think about a little differently."
>
> Even where there are capture variables, I think I'd be completely
> comfortable thinking about the different context for:
>
> match eggs:
> case Spam(a=x, b=2): ...
>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born. Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XSRVX2NTDGUF7CWTPR5SHIOAQPWNXXYZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/45JLJCSQRZ3SCVV6EP546W4P5LBYNUKM/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Pattern Matching controversy: Don't read PEP 635, read DLS'20 paper instead
On Sat, Dec 5, 2020 at 1:53 AM Guido van Rossum wrote: > I hope more of the regulars here jump on this bandwagon. It will be a > great day when Paul posts one of his offensive posts and there is just > deafening silence. > > Paul was in my (very short) kill file for years but I decided to give him > another chance. And he blew it. > > There is a reason why he was banned from micropython, and you all are > seeing it here — he just can’t help himself. > > > Probably he’s hoping to be banned for a COC violation. Let’s not give him > that satisfaction. > Agree. I press "Mute" GMail button when seeing offensive words from Paul in conversation. I did it several times for the last months. > —Guido (speaking for myself only) > > On Tue, Dec 1, 2020 at 15:11 Stephen J. Turnbull < > turnbull.stephen...@u.tsukuba.ac.jp> wrote: > >> Paul Sokolovsky writes: >> >> > Sorry, but there may be a suggestion of tactics of sneaking somebody's >> > "pet feature" past the attention of core developers by employing >> > sycophant techniques. >> >> I've had enough of your random aspersions. They make it impossible >> for me to continue reading your posts. I'll join David with a >> "moderate" -100 on your proposals, and bid you "good day". >> ___ >> Python-Dev mailing list -- python-dev@python.org >> To unsubscribe send an email to python-dev-le...@python.org >> https://mail.python.org/mailman3/lists/python-dev.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-dev@python.org/message/3BAYVC53Z7ZS2UQ4NLAJGIBFJ62DI3NK/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- > --Guido (mobile) > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/RADAUJGFQLUG2YIQBQF622RY3TRZSTKO/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HJEX4XXE3H4STTF5IWNH4QAP2AV6OYTK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] tp_dictoffset and tp_weaklistoffset slots for Stable API
Looking on PEP http://www.python.org/dev/peps/pep-0384/ and docs I don't figure out how to specify this values. Maybe I've missed something? If not I like to solve that problem at us pycon sprints. Hope, Martin von Loewis will visit the conference. -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: New DSL syntax and slightly changed semantics for the Argument Clinic DSL.
On Sun, Mar 17, 2013 at 3:26 PM, Stefan Krah wrote:
> [PEP 436 revised syntax]
>
> While I like the syntax better and appreciate the option to condense the
> function declaration I still fear that the amount of implicitness will
> distract from what is important: programming in C.
>
> This applies especially if people start declaring converters using the
> [python] feature.
>
> So I hope that at least converters can be declared statically in a header
> file, like I suggested in PEP 437.
>
>
> A couple of comments:
>
>
>> As of CPython 3.3, builtin functions nearly always parse their arguments
>> with one of two functions: the original ``PyArg_ParseTuple()``, [1]_ and
>> the more modern ``PyArg_ParseTupleAndKeywords()``. [2]_ The former
>> only handles positional parameters; the latter also accommodates keyword
>> and keyword-only parameters, and is preferred for new code.
>
> What is the source for this? I seem to remember a discussion on python-ideas
> (but cannot find it now) where some developers preferred non-keyword functions
> for some use cases.
>
> For example it's strange to write div(x=10, y=3), or worse, div(y=3, x=10).
> Using positional-only arguments prevents this "feature".
IIRC objection was about functions like abs(5).
If function has single and obvious argument why you need to name that parameter?
The issue has related to documentation for existing one-argument functions only.
>
>
>> /*[clinic]
>> os.stat as os_stat_fn -> stat result
>>
>>path: path_t(allow_fd=1)
>>Path to be examined; can be string, bytes, or open-file-descriptor
>> int.
>
> I do not see where the C initialization or the cleanup are specified. Are
> they part of the converter specification?
>
>
>> /*[clinic]
>> curses.window.addch
>>
>>[
>>x: int
>> X-coordinate.
>>
>>y: int
>> Y-coordinate.
>>]
>
> The parameters appear to be in the wrong order.
>
>
>> The return annotation is also optional. If skipped, the arrow ("``->``")
>> must also be omitted.
>
> Why is it optional? Aren't type annotations important?
>
>
>> Clinic will ship with a number of built-in converters; new converters can
>> also be added dynamically.
>
> How are the converters specified? Inside the preprocessor source? Are
> initialization
> and cleanup part of the specification, e.g. is a converter represented by a
> class?
>
> I would prefer if the converters were in a header file, like I suggested in
> PEP 437. Any tool can read such a file and custom converters can be redeclared
> above their definition.
>
>
>> The default value is dynamically assigned, "live" in the generated C code,
>> and although it's specified as a Python value, it's translated into a native
>> C value in the generated C code. Few default values are permitted, owing to
>> this manual translation step.
>
> I think there should be a table that lists which values are converted and what
> the result of the conversion is.
>
>
>> ``[``
>>Establishes the start of an optional "group" of parameters.
>>Note that "groups" may nest inside other "groups".
>>See `Functions With Positional-Only Parameters`_ below.
>
> I don't quite understand the terminology: Functions with the ``/`` are also
> "positional-only". Why not reserve this syntax exclusively for the legacy
> left-and-right optional case?
>
>
>> ``/``
>>This hints to Argument Clinic that this function is performance-sensitive,
>>and that it's acceptable to forego supporting keyword parameters when
>> parsing.
>>(In early implementations of Clinic, this will switch Clinic from
>> generating
>>code using ``PyArg_ParseTupleAndKeywords`` to using ``PyArg_ParseTuple``.
>>The hope is that in the future there will be no appreciable speed
>> difference,
>>rendering this syntax irrelevant and deprecated but harmless.)
>
> Here I would use "positional-only" and mention that the slash plays
> essentially
> the same role as the vertical bar in the existing syntax. If this isn't the
> intention, then I simply did not understand the paragraph.
>
>
>> types
>>
>>A list of strings representing acceptable Python types for this object.
>>There are also four strings which represent Python protocols:
>
> I don't quite follow: Aren't input types always specified by the converter
> function?
>
&g
Re: [Python-Dev] [Python-checkins] peps: update 2.7.4 release dates
Are you sure about 2.7.4 2012-04-06? I mean 2012 year. On Tue, Mar 19, 2013 at 9:15 PM, benjamin.peterson wrote: > http://hg.python.org/peps/rev/ce17779c395c > changeset: 4810:ce17779c395c > user:Benjamin Peterson > date:Tue Mar 19 23:15:23 2013 -0500 > summary: > update 2.7.4 release dates > > files: > pep-0373.txt | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > > diff --git a/pep-0373.txt b/pep-0373.txt > --- a/pep-0373.txt > +++ b/pep-0373.txt > @@ -56,8 +56,8 @@ > > Planned future release dates: > > -- 2.7.4rc1 2013-02-02 > -- 2.7.4 2012-02-16 > +- 2.7.4rc1 2013-03-23 > +- 2.7.4 2012-04-06 > > Dates of previous maintenance releases: > > > -- > Repository URL: http://hg.python.org/peps > > ___ > Python-checkins mailing list > python-check...@python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (2.7): - Issue #13150: sysconfig no longer parses the Makefile and config.h files
orm()+"-"+sys.version[0:3]' >platform > > +# Create build directory and generate the sysconfig build-time data there. > +# pybuilddir.txt contains the name of the build dir and is used for > +# sys.path fixup -- see Modules/getpath.c. > +pybuilddir.txt: $(BUILDPYTHON) > + $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig > --generate-posix-vars > > # Build the shared modules > # Under GNU make, MAKEFLAGS are sorted and normalized; the 's' for > # -s, --silent or --quiet is always the first char. > # Under BSD make, MAKEFLAGS might be " -s -v x=y". > -sharedmods: $(BUILDPYTHON) > +sharedmods: $(BUILDPYTHON) pybuilddir.txt > @case "$$MAKEFLAGS" in \ > *\ -s*|s*) quiet="-q";; \ > *) quiet="";; \ > @@ -955,7 +960,7 @@ > elsetrue; \ > fi; \ > done > - @for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc > $(srcdir)/Lib/*.egg-info ; \ > + @for i in $(srcdir)/Lib/*.py `cat pybuilddir.txt`/_sysconfigdata.py > $(srcdir)/Lib/*.doc $(srcdir)/Lib/*.egg-info ; \ > do \ > if test -x $$i; then \ > $(INSTALL_SCRIPT) $$i $(DESTDIR)$(LIBDEST); \ > @@ -1133,6 +1138,7 @@ > --install-scripts=$(BINDIR) \ > --install-platlib=$(DESTSHARED) \ > --root=$(DESTDIR)/ > + -rm $(DESTDIR)$(DESTSHARED)/_sysconfigdata.py* > > # Here are a couple of targets for MacOSX again, to install a full > # framework-based Python. frameworkinstall installs everything, the > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -216,6 +216,10 @@ > Library > --- > > +- Issue #13150: sysconfig no longer parses the Makefile and config.h files > + when imported, instead doing it at build time. This makes importing > + sysconfig faster and reduces Python startup time by 20%. > + > - Issue #10212: cStringIO and struct.unpack support new buffer objects. > > - Issue #12098: multiprocessing on Windows now starts child processes > > -- > Repository URL: http://hg.python.org/cpython > > ___ > Python-checkins mailing list > python-check...@python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
No. _decimal is new functionality that will never be backported. On Wed, Mar 27, 2013 at 9:40 AM, Andriy Kornatskyy wrote: > Any plans backport decimal C implementation from 3.3? > > Thanks. > > Andriy Kornatskyy > > > >> Date: Tue, 26 Mar 2013 16:18:34 -0700 >> From: ether@gmail.com >> To: h...@ox.cx >> CC: python-dev@python.org >> Subject: Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1 >> >> On Tue, Mar 26, 2013 at 3:26 PM, Hynek Schlawack wrote: >> > Speakerdeck is slides only. The video is here: >> > http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27 >> >> Sweet thanks! >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/andriy.kornatskyy%40live.com > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix typo
Do you mean something like: «Let's also change the rest of the program to make the new functionality:» ??? On Sat, Apr 6, 2013 at 6:41 AM, Eli Bendersky wrote: > > On Fri, Apr 5, 2013 at 1:40 AM, andrew.svetlov > wrote: >> >> http://hg.python.org/cpython/rev/6cf485ffd325 >> changeset: 83110:6cf485ffd325 >> parent: 83106:94fb906e5899 >> parent: 83109:9610ede72ed2 >> user:Andrew Svetlov >> date:Fri Apr 05 11:40:01 2013 +0300 >> summary: >> Fix typo >> >> files: >> Doc/howto/argparse.rst | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst >> --- a/Doc/howto/argparse.rst >> +++ b/Doc/howto/argparse.rst >> @@ -668,7 +668,7 @@ >> So far, we have been working with two methods of an >> :class:`argparse.ArgumentParser` instance. Let's introduce a third one, >> :meth:`add_mutually_exclusive_group`. It allows for us to specify options >> that >> -conflict with each other. Let's also change the rest of the program make >> the >> +conflict with each other. Let's also change the rest of the program to >> make the >> new functionality makes more sense: > > > > Andrew, while you're at it you can also fix the rest of the sentence, which > makes no sense ;-) > > Eli > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix typo
Done, thanks On Sat, Apr 6, 2013 at 5:41 PM, Eli Bendersky wrote: > It currently says: > > "Let's also change the rest of the program to make the new functionality > makes more sense" > > This can be changed to: > > "Let's also change the rest of the program so that the new functionality > makes more sense". > > Eli > > > > > On Sat, Apr 6, 2013 at 7:25 AM, Andrew Svetlov > wrote: >> >> Do you mean something like: >> «Let's also change the rest of the program to make the new functionality:» >> ??? >> > > > > >> >> On Sat, Apr 6, 2013 at 6:41 AM, Eli Bendersky wrote: >> > >> > On Fri, Apr 5, 2013 at 1:40 AM, andrew.svetlov >> > >> > wrote: >> >> >> >> http://hg.python.org/cpython/rev/6cf485ffd325 >> >> changeset: 83110:6cf485ffd325 >> >> parent: 83106:94fb906e5899 >> >> parent: 83109:9610ede72ed2 >> >> user:Andrew Svetlov >> >> date:Fri Apr 05 11:40:01 2013 +0300 >> >> summary: >> >> Fix typo >> >> >> >> files: >> >> Doc/howto/argparse.rst | 2 +- >> >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> >> >> >> diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst >> >> --- a/Doc/howto/argparse.rst >> >> +++ b/Doc/howto/argparse.rst >> >> @@ -668,7 +668,7 @@ >> >> So far, we have been working with two methods of an >> >> :class:`argparse.ArgumentParser` instance. Let's introduce a third >> >> one, >> >> :meth:`add_mutually_exclusive_group`. It allows for us to specify >> >> options >> >> that >> >> -conflict with each other. Let's also change the rest of the program >> >> make >> >> the >> >> +conflict with each other. Let's also change the rest of the program to >> >> make the >> >> new functionality makes more sense: >> > >> > >> > >> > Andrew, while you're at it you can also fix the rest of the sentence, >> > which >> > makes no sense ;-) >> > >> > Eli >> > >> >> >> >> -- >> Thanks, >> Andrew Svetlov > > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (3.3): Process DEFAULT values in mock side_effect that returns iterator.
My bad, sorry On Mon, Apr 8, 2013 at 8:03 PM, Antoine Pitrou wrote: > On Mon, 8 Apr 2013 10:52:30 +0100 > Michael Foord wrote: >> On 7 April 2013 14:44, andrew.svetlov wrote: >> >> > http://hg.python.org/cpython/rev/18fd64f1de2d >> > changeset: 83179:18fd64f1de2d >> > branch: 3.3 >> > user:Andrew Svetlov >> > date:Sun Apr 07 16:42:24 2013 +0300 >> > summary: >> > Process DEFAULT values in mock side_effect that returns iterator. >> > >> > Patch by Michael Ford. >> > > [...] >> >> This was committed without a NEWS entry. > > And I wonder who that Michael Ford is :-) > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A decade as a core dev
My congratulations! On Thu, Apr 18, 2013 at 8:01 PM, Brett Cannon wrote: > On Thu, Apr 18, 2013 at 12:17 PM, Guido van Rossum wrote: >> Congrats Brett! I'd say you have learned a lot more than programming >> during your time here!! > > Oh yes, such as how to put on a flame-retardant suit, the colours of > the rainbow based on the bikesheds, and how I will never be able to > pronounce your name properly. =) > > Seriously though, yes, I've learned tons about all sorts of things > during my time here. > > -Brett > >> >> On Thu, Apr 18, 2013 at 8:02 AM, Brett Cannon wrote: >>> Today marks my 10 year anniversary as a core developer on Python. I >>> wrote a blog post to mark the occasion >>> (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I >>> wanted to personally thank python-dev for the past decade (and >>> whatever comes in the future). All of you taught me how to really >>> program and for that I will be eternally grateful. And the friendships >>> I have built through this list are priceless. >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> >> >> -- >> --Guido van Rossum (python.org/~guido) > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Add function to signal module for getting main thread id
Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. My proposition is to add function like get_mainthread_id() -> int which return ident for main thread (I know function name is not perfect, please guess better one). Signal module already has required data as internal variable static long main_thread; I just guess to expose this value to python. Thoughts? -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
I missed _MainThread in threading, that's why I've guessed to add function to signal module. threading.main_thread() is much better sure. On Fri, Aug 30, 2013 at 12:39 PM, Antoine Pitrou wrote: > > Le Fri, 30 Aug 2013 12:24:07 +0300, > Andrew Svetlov a écrit : >> Main thread is slightly different from others. >> Signals can be subscribed from main thread only. >> Tulip has special logic for main thread. >> In application code we can explicitly know which thread is executed, >> main or not. >> But from library it's not easy. >> Tulip uses check like >> threading.current_thread().name == 'MainThread' >> This approach has a problem: thread name is writable attribute and can >> be changed by user code. > > Please at least use: > > >>> isinstance(threading.current_thread(), threading._MainThread) > True > > But really, what we need is a threading.main_thread() function. > > (Apologies for the previous incomplete reply (keyboard mishap)) > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
_MainThread can be used as workaround, but adding public function makes value. Oleg, as I understand _MainThread is a class, not class instance, test for threading._MainThread.ident doesn't make sense. On Fri, Aug 30, 2013 at 12:44 PM, Oleg Broytman wrote: > On Fri, Aug 30, 2013 at 12:24:07PM +0300, Andrew Svetlov > wrote: >> Main thread is slightly different from others. >> Signals can be subscribed from main thread only. >> Tulip has special logic for main thread. >> In application code we can explicitly know which thread is executed, >> main or not. >> But from library it's not easy. >> Tulip uses check like >> threading.current_thread().name == 'MainThread' >> This approach has a problem: thread name is writable attribute and can >> be changed by user code. > >You can test > threading.current_thread().__class__ is threading._MainThread >or > threading.current_thread().ident == threading._MainThread.ident > >> My proposition is to add function like get_mainthread_id() -> int >> which return ident for main thread > >threading._MainThread.ident ? > > Oleg. > -- > Oleg Broytmanhttp://phdru.name/p...@phdru.name >Programmers don't die, they just GOSUB without RETURN. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
I've filed http://bugs.python.org/issue18882 for this. On Fri, Aug 30, 2013 at 12:52 PM, Andrew Svetlov wrote: > _MainThread can be used as workaround, but adding public function makes value. > > Oleg, as I understand _MainThread is a class, not class instance, test > for threading._MainThread.ident doesn't make sense. > > On Fri, Aug 30, 2013 at 12:44 PM, Oleg Broytman wrote: >> On Fri, Aug 30, 2013 at 12:24:07PM +0300, Andrew Svetlov >> wrote: >>> Main thread is slightly different from others. >>> Signals can be subscribed from main thread only. >>> Tulip has special logic for main thread. >>> In application code we can explicitly know which thread is executed, >>> main or not. >>> But from library it's not easy. >>> Tulip uses check like >>> threading.current_thread().name == 'MainThread' >>> This approach has a problem: thread name is writable attribute and can >>> be changed by user code. >> >>You can test >> threading.current_thread().__class__ is threading._MainThread >>or >> threading.current_thread().ident == threading._MainThread.ident >> >>> My proposition is to add function like get_mainthread_id() -> int >>> which return ident for main thread >> >>threading._MainThread.ident ? >> >> Oleg. >> -- >> Oleg Broytmanhttp://phdru.name/p...@phdru.name >>Programmers don't die, they just GOSUB without RETURN. >> ___________ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > > > > -- > Thanks, > Andrew Svetlov -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
I've made a patch. It works except scenario described by Christian Heimes. See details in http://bugs.python.org/issue18882 On Fri, Aug 30, 2013 at 3:21 PM, Antoine Pitrou wrote: > Le Fri, 30 Aug 2013 14:06:11 +0200, > Christian Heimes a écrit : >> Am 30.08.2013 11:39, schrieb Antoine Pitrou: >> > >> > Le Fri, 30 Aug 2013 12:24:07 +0300, >> > Andrew Svetlov a écrit : >> >> Main thread is slightly different from others. >> >> Signals can be subscribed from main thread only. >> >> Tulip has special logic for main thread. >> >> In application code we can explicitly know which thread is >> >> executed, main or not. >> >> But from library it's not easy. >> >> Tulip uses check like >> >> threading.current_thread().name == 'MainThread' >> >> This approach has a problem: thread name is writable attribute and >> >> can be changed by user code. >> > >> > Please at least use: >> > >> > >>> isinstance(threading.current_thread(), threading._MainThread) >> > True >> > >> > But really, what we need is a threading.main_thread() function. >> >> What happens, when a program fork()s from another thread than the main >> thread? AFAIR the other threads are suspended and the forking thread >> is the new main thread. Or something similar... > > Yes. We even support it :-) > http://hg.python.org/cpython/file/c347b9063a9e/Lib/test/test_threading.py#l503 > > (well, whoever wrote that test wanted to support it. I don't think > that's me) > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] error in test suite
Sorry, this is mine. This is related to http://bugs.python.org/issue11798 Error happens when tests regrtest executed with -R option. I've temporary disabled this feature until finally fix it. On Sat, Aug 31, 2013 at 8:23 PM, Ethan Furman wrote: > Am I the only one experiencing this? > > 262 tests OK. > 93 tests failed: > test___all__ test_abc test_array test_ast test_asynchat > test_asyncore test_bisect test_buffer test_bufio test_bytes > test_codeccallbacks test_codecs test_colorsys test_compileall > test_configparser test_contextlib test_crypt test_ctypes test_dbm > test_dbm_dumb test_dbm_ndbm test_dictcomps test_enum > test_exceptions test_faulthandler test_file test_fileinput > test_frozen test_future test_future3 test_future5 test_genericpath > test_getargs2 test_getpass test_hash test_hashlib test_heapq > test_idle test_imaplib test_imp test_import test_index test_io > test_ioctl test_ipaddress test_iterlen test_json test_keyword > test_largefile test_locale test_macpath test_multiprocessing_fork > test_multiprocessing_forkserver test_multiprocessing_spawn > test_namespace_pkgs test_ntpath test_operator test_osx_env > test_pdb test_pep352 test_posixpath test_print test_py_compile > test_random test_regrtest test_robotparser test_runpy test_sched > test_set test_shutil test_site test_smtpd test_sndhdr > test_source_encoding test_sqlite test_stat test_strftime > test_sundry test_tarfile test_textwrap test_threading test_time > test_unicode test_univnewlines test_urllib test_urllib2net > test_userstring test_uuid test_warnings test_wave test_webbrowser > test_xml_dom_minicompat test_zipfile > 24 tests skipped: > test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp > test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_gnu > test_devpoll test_gdb test_kqueue test_lzma test_msilib > test_ossaudiodev test_smtpnet test_socketserver test_startfile > test_timeout test_tk test_ttk_guionly test_urllibnet test_winreg > test_winsound test_xmlrpc_net test_zipfile64 > > and the failure appears to always be: > > test [...] crashed -- Traceback (most recent call last): > File "/home/ethan/source/python/issue18780/Lib/test/regrtest.py", line > 1265, in runtest_inner > huntrleaks) > File "/home/ethan/source/python/issue18780/Lib/test/regrtest.py", line > 1381, in dash_R > indirect_test() > File "/home/ethan/source/python/issue18780/Lib/test/regrtest.py", line > 1261, in > test_runner = lambda: support.run_unittest(tests) > File "/home/ethan/source/python/issue18780/Lib/test/support/__init__.py", > line 1683, in run_unittest > _run_suite(suite) > File "/home/ethan/source/python/issue18780/Lib/test/support/__init__.py", > line 1649, in _run_suite > result = runner.run(suite) > File "/home/ethan/source/python/issue18780/Lib/test/support/__init__.py", > line 1548, in run > test(result) > File "/home/ethan/source/python/issue18780/Lib/unittest/suite.py", line > 76, in __call__ > return self.run(*args, **kwds) > File "/home/ethan/source/python/issue18780/Lib/unittest/suite.py", line > 114, in run > test(result) > File "/home/ethan/source/python/issue18780/Lib/unittest/suite.py", line > 76, in __call__ > return self.run(*args, **kwds) > File "/home/ethan/source/python/issue18780/Lib/unittest/suite.py", line > 114, in run > test(result) > TypeError: 'NoneType' object is not callable > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #11798: fix tests for regrtest -R :
regrtest -R runs test suites several times. That's why test cleanup
should be disabled for this case.
Details discussed in issue.
I'll do more expressive commit messages next time.
Thanks.
On Mon, Sep 2, 2013 at 1:58 AM, Eli Bendersky wrote:
>
>
>
> On Sat, Aug 31, 2013 at 9:58 PM, andrew.svetlov
> wrote:
>>
>> http://hg.python.org/cpython/rev/39781c3737f8
>> changeset: 85490:39781c3737f8
>> user:Andrew Svetlov
>> date:Sun Sep 01 07:58:41 2013 +0300
>> summary:
>> Issue #11798: fix tests for regrtest -R :
>>
>> files:
>> Lib/test/regrtest.py| 5 +
>> Lib/unittest/suite.py | 8 ++--
>> Lib/unittest/test/test_suite.py | 8
>> 3 files changed, 19 insertions(+), 2 deletions(-)
>>
>>
>
> Hi Andrew,
>
> It would help if you could add more details into the commit message. This
> would make both post-commit reviews and future code archeology simpler.
>
> Eli
>
>
>>
>> diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
>> --- a/Lib/test/regrtest.py
>> +++ b/Lib/test/regrtest.py
>> @@ -496,6 +496,8 @@
>>
>> if ns.slaveargs is not None:
>> args, kwargs = json.loads(ns.slaveargs)
>> +if kwargs.get('huntrleaks'):
>> +unittest.BaseTestSuite._cleanup = False
>> try:
>> result = runtest(*args, **kwargs)
>> except KeyboardInterrupt:
>> @@ -528,6 +530,9 @@
>> #gc.set_debug(gc.DEBUG_SAVEALL)
>> found_garbage = []
>>
>> +if ns.huntrleaks:
>> +unittest.BaseTestSuite._cleanup = False
>> +
>> if ns.single:
>> filename = os.path.join(TEMPDIR, 'pynexttest')
>> try:
>> diff --git a/Lib/unittest/suite.py b/Lib/unittest/suite.py
>> --- a/Lib/unittest/suite.py
>> +++ b/Lib/unittest/suite.py
>> @@ -16,6 +16,8 @@
>> class BaseTestSuite(object):
>> """A simple test suite that doesn't provide class or module shared
>> fixtures.
>> """
>> +_cleanup = True
>> +
>> def __init__(self, tests=()):
>> self._tests = []
>> self.addTests(tests)
>> @@ -61,7 +63,8 @@
>> if result.shouldStop:
>> break
>> test(result)
>> -self._removeTestAtIndex(index)
>> +if self._cleanup:
>> +self._removeTestAtIndex(index)
>> return result
>>
>> def _removeTestAtIndex(self, index):
>> @@ -115,7 +118,8 @@
>> else:
>> test.debug()
>>
>> -self._removeTestAtIndex(index)
>> +if self._cleanup:
>> +self._removeTestAtIndex(index)
>>
>> if topLevel:
>> self._tearDownPreviousClass(None, result)
>> diff --git a/Lib/unittest/test/test_suite.py
>> b/Lib/unittest/test/test_suite.py
>> --- a/Lib/unittest/test/test_suite.py
>> +++ b/Lib/unittest/test/test_suite.py
>> @@ -303,6 +303,9 @@
>> suite.run(unittest.TestResult())
>>
>> def test_remove_test_at_index(self):
>> +if not unittest.BaseTestSuite._cleanup:
>> +raise unittest.SkipTest("Suite cleanup is disabled")
>> +
>> suite = unittest.TestSuite()
>>
>> suite._tests = [1, 2, 3]
>> @@ -311,6 +314,9 @@
>> self.assertEqual([1, None, 3], suite._tests)
>>
>> def test_remove_test_at_index_not_indexable(self):
>> +if not unittest.BaseTestSuite._cleanup:
>> +raise unittest.SkipTest("Suite cleanup is disabled")
>> +
>> suite = unittest.TestSuite()
>> suite._tests = None
>>
>> @@ -318,6 +324,8 @@
>> suite._removeTestAtIndex(2)
>>
>> def assert_garbage_collect_test_after_run(self, TestSuiteClass):
>> +if not unittest.BaseTestSuite._cleanup:
>> +raise unittest.SkipTest("Suite cleanup is disabled")
>>
>> class Foo(unittest.TestCase):
>> def test_nothing(self):
>>
>> --
>> Repository URL: http://hg.python.org/cpython
>>
>> ___
>> Python-checkins mailing list
>> python-check...@python.org
>> http://mail.python.org/mailman/listinfo/python-checkins
>>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Having Sorted Containers in stdlib?
sortedcollections is basically a specialized BTree. > [2] A wish that came about because I had sorted sets of exact/prefix > strings of 1,000 to 10,000 elements and needed them to interact in various > ways and wanted to preserve their sortedness. > > - > > re: "Is this particular one clearly the “best”[*]?" > > Performance wise, it's probably about the best insofar as a BTree of > depth=2 and high fanout (10,000 or something? I forget what > sortedcontainers defaults to) is. In my limited experience, it's docs and > claims about its performance were pretty accurate[3]. > > API wise, it has its share of idiosyncrasies like anything else. e.g. > SortedList claims to implement MutableSequence, but actually raises errors > when calling e.g. append(), which is a bit misleading / wrong to some > extent (to be fair, given MutableSequence's api contract, it's not like > there's particularly clear/right/good choice one can make here). > > (disclaimer: i've minimal experience with sortedcontainers; when I looked > at it, it was really only for comparing my own toy BTree/RBT > behavior/performance with a known implementation) > > [3] The exception being a claim that its (supposedly) better cache > locality is a significant factor. I think the underlying BTree algorithm is > the dominant factor. But someone who actually knows how to measure cache > locality would be a better judge of that than me, who doesn't know how. > > >> Paul >> ___ >> Python-Dev mailing list -- python-dev@python.org >> To unsubscribe send an email to python-dev-le...@python.org >> https://mail.python.org/mailman3/lists/python-dev.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-dev@python.org/message/5SURNB4C5FGJ6LSXUPVW2EFP22ERKSGB/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/HVZCFPKI7TY6WVLFK43KLVVDTEPIXFFQ/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/F6I4AR735TYIJ2WNAC4C43BEQJQPDJKZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on Callable Syntax PEP
At my job, we have 1577 Callable type hints scattered in 1063 Python files. For comparison, this codebase also has 2754 dict annotations and 1835 list ones. On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker wrote: > note: I wasn't thinking -- typeshed, of course, has a lot more than the > standard lib. But it's still a collection of widely used somewhat general > purpose libraries. So I think my hypothesis is still valid. > > -CHB > > > On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker > wrote: > >> A question that came up for me is: >> >> How common is it to need to use Callable for type hints? particularly >> complex versions, specifying what parameters the Callable takes? A more >> compact and easier to read syntax is nice, but not very important if it >> isn't used much. >> >> My first thought on this was that I can't remember a single time that I >> wrote production code that took a Callable as a function parameter -- or >> returned one -- OK maybe a few times, but it's certainly rare in my >> production code. >> >> So I looked in the PEP to see if that issue was addressed, and indeed it >> is: >> >> "The Callable type is widely used. For example, as of October 2021 it was >> the fifth most common complex type in typeshed," >> >> That did surprise me, but on thinking about it, maybe not so much. It >> strikes me that Callable is most likely to be used in fairly low level, >> general purpose functions, like map(), sort(), various functions in >> itertools, etc. Just the sort of functions that are common in the standard >> library, but may not so much in production code. >> >> I have no idea how to evaluate how common it is in production code -- >> maybe type hinting is common enough now that PyPi could be searched -- but >> even PyPi is a biased sample, as it is full of, by definition, >> libraries for others' use -- i.e. general purpose tools (less general that >> the stad lib, but still not specialty production code, which I suspect is >> the majority of Python code out there). >> >> Perhaps some folks that have been type=hinting their production code >> bases could provide anecdotal evidence. >> >> Anyway, if my hypothesis is correct, then it's not so bad that >> not-so-nice syntax is required to type hint general purpose utilities. >> >> -CHB >> >> -- >> Christopher Barker, PhD (Chris) >> >> Python Language Consulting >> - Teaching >> - Scientific Software Development >> - Desktop GUI and Web Development >> - wxPython, numpy, scipy, Cython >> > > > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > ___________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on Callable Syntax PEP
On Mon, Dec 20, 2021 at 1:42 PM Mark Shannon wrote: > OOI, of those 1577 Callable type hints, how many distinct Callable types? > > Good question. About 30 callables for source code itself and an additional 60 for pytest factory fixtures. > On 20/12/2021 7:52 am, Andrew Svetlov wrote: > > At my job, we have 1577 Callable type hints scattered in 1063 Python > files. > > For comparison, this codebase also has 2754 dict annotations and 1835 > list ones. > > > > On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker <mailto:python...@gmail.com>> wrote: > > > > note: I wasn't thinking -- typeshed, of course, has a lot more than > the standard lib. But it's still a collection of widely used somewhat > general purpose libraries. So I think my hypothesis is still valid. > > > > -CHB > > > > > > On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker < > python...@gmail.com <mailto:python...@gmail.com>> wrote: > > > > A question that came up for me is: > > > > How common is it to need to use Callable for type hints? > particularly complex versions, specifying what parameters the Callable > takes? A more compact and easier to read syntax is nice, but not very > important if it isn't used much. > > > > My first thought on this was that I can't remember a single time > that I wrote production code that took a Callable as a function parameter > -- or returned one -- OK maybe a few times, but it's certainly rare in my > production code. > > > > So I looked in the PEP to see if that issue was addressed, and > indeed it is: > > > > "The Callable type is widely used. For example, as of October > 2021 it was the fifth most common complex type in typeshed," > > > > That did surprise me, but on thinking about it, maybe not so > much. It strikes me that Callable is most likely to be used in fairly low > level, general purpose functions, like map(), sort(), various functions in > itertools, etc. Just the sort of functions that are common in the standard > library, but may not so much in production code. > > > > I have no idea how to evaluate how common it is in production > code -- maybe type hinting is common enough now that PyPi could be > searched -- but even PyPi is a biased sample, as it is full of, by > definition, libraries for others' use -- i.e. general purpose tools (less > general that the stad lib, but still not specialty production code, which I > suspect is the majority of Python code out there). > > > > Perhaps some folks that have been type=hinting their production > code bases could provide anecdotal evidence. > > > > Anyway, if my hypothesis is correct, then it's not so bad that > not-so-nice syntax is required to type hint general purpose utilities. > > > > -CHB > > > > -- > > Christopher Barker, PhD (Chris) > > > > Python Language Consulting > >- Teaching > >- Scientific Software Development > >- Desktop GUI and Web Development > >- wxPython, numpy, scipy, Cython > > > > > > > > -- > > Christopher Barker, PhD (Chris) > > > > Python Language Consulting > >- Teaching > >- Scientific Software Development > >- Desktop GUI and Web Development > >- wxPython, numpy, scipy, Cython > > ___ > > Python-Dev mailing list -- python-dev@python.org python-dev@python.org> > > To unsubscribe send an email to python-dev-le...@python.org python-dev-le...@python.org> > > https://mail.python.org/mailman3/lists/python-dev.python.org/ < > https://mail.python.org/mailman3/lists/python-dev.python.org/> > > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/ > < > https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/ > > > > Code of Conduct: http://python.org/psf/codeofconduct/ < > http://python.org/psf/codeofconduct/> > > > > > > > > -- > > Thanks, > > Andrew Svetlov > > > > ___ > > Python-Dev mailing list -- python-dev@python.org > > To unsubscribe send an email to python-dev-le...@python.org > > https://mail.python.org/mailman3/lists/python-dev.python.org/ > > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/ > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MG2PKYBIZHA3PPEVWJMSXR4WJ4TLBKVZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on Callable Syntax PEP
Perhaps Serhiy did more accurate counting, my estimate is very rough. On Mon, Dec 20, 2021 at 2:15 PM Serhiy Storchaka wrote: > 20.12.21 13:42, Mark Shannon пише: > > OOI, of those 1577 Callable type hints, how many distinct Callable types? > > Around 15-20%. Most of them are in tests which widely use pytest > fixtures, so functions taking and returning callables are common. There > are around 200 occurrences in non-test code, half of them are distinct > types. > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Q4A4ZTJHSS6ALDXHZJEERJBVGJNYC6KV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Function Prototypes
I like the Callable decorator idea very much.
It supports all Python function flavors out of the box, isn't it?
Also, what is about allowing to make callable types from existing functions
(and even methods maybe) with type hints?
def f(a: int, /, b: float) -> str:
return str(a*b)
F = Callable(f)
Could it work?
I'm ok with making an explicit Callable type alias first for every usage.
But if I can create it from an existing function augmented with type hints
without copy-pasting the signature -- it can make my life significantly
easier.
What do you think?
On Fri, Dec 24, 2021 at 11:57 AM Serhiy Storchaka
wrote:
> 24.12.21 00:09, Guido van Rossum пише:
> > Without decorator too (that was Lukasz’ idea). Why bother with the
> > decorator (*if* we were to go there)?
>
> It is errorprone.
>
> Some library provide function foo() which returns an instance of private
> type _Foo and people start using it as a type hint. A new version
> converts foo() into a class. It is usually a safe backward compatible
> change, except that now all type hints became wrong. "a: foo" now means
> an instance of foo, not a callable which returns an instance of _Foo.
>
> There are also issues with subclassing.
>
> >>> foo = Callable[[int], str]
> >>> class A(foo): ...
> ...
> >>> def bar(x: int, /) -> str: pass
> ...
> >>> class B(bar): ...
> ...
> Traceback (most recent call last):
> File "", line 1, in
> TypeError: function() argument 'code' must be code, not str
>
> There are also issues with subscripting:
>
> >>> T = TypeVar('T')
> >>> foo = Callable[[T], list[T]]
> >>> list[foo][int]
> list[typing.Callable[[int], list[int]]]
> >>> def bar(x: T, /) -> list[T]: pass
> ...
> >>> list[bar][int]
> Traceback (most recent call last):
> File "", line 1, in
> TypeError: There are no type variables left in list[__main__.bar]
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ALW5PMZ3MRIG3BGTX5DVIKFO4JS45MBK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/MWWWIIYOWH3TCZGAW7J6OHDCO6IDTJWZ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)
Stefan, do you really need to emulate call stack with positions? Could the __note__ string with generated Cython part of exception traceback solve your needs (https://www.python.org/dev/peps/pep-0678/) ? On Wed, Feb 9, 2022 at 7:46 PM Pablo Galindo Salgado wrote: > I can only say that currently, I am not confident to expose such an API, > at least for co_positions, as the internal implementation is very likely to > heavily change and we want to have the possibility of changing it between > patch versions if required (to address bugs and other things like that). > > On Wed, 9 Feb 2022 at 17:38, Stefan Behnel wrote: > >> Pablo Galindo Salgado schrieb am 09.02.22 um 17:40: >> >> Should there be a getter/setter for co_positions? >> > >> > We consider the representation of co_postions private >> >> Yes, and that's the issue. >> >> >> > so we don't want (for now) to ad >> > getters/setters. If you want to get the position of a instruction, you >> can >> > use PyCode_Addr2Location >> >> What Cython needs is the other direction. How can we provide the current >> source position range for a given piece of code to an exception? >> >> As it stands, the way to do this is to copy the implementation details of >> CPython into Cython in order to let it expose the specific data >> structures >> that CPython uses for its internal representation of code positions. >> >> I would prefer using an API instead that allows exposing this mapping >> directly to CPython's traceback handling, rather than having to emulate >> byte code positions. While that would probably be quite doable, it's far >> from a nice interface for something that is not based on byte code. >> >> And that's not just a Cython issue. The same applies to Domain Specific >> Languages or other programming languages that integrate with Python and >> want to show users code positions for their source code. >> >> Stefan >> >> ___ >> Python-Dev mailing list -- python-dev@python.org >> To unsubscribe send an email to python-dev-le...@python.org >> https://mail.python.org/mailman3/lists/python-dev.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-dev@python.org/message/VQSWX6MFKIA3RYPSX7O6RTVC422LTJH4/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/P7SMK5ZGFAHZMLUKW4WZNNX47CONXIQS/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3RVEOQSOJRWVWEVXHM6NDW6WH7DPBGLF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Feature Request: Python Pipes: Incluye sspipe module
I prefer keeping it as a separate library on PyPI.
On Sat, Sep 14, 2019 at 12:26 AM Juan Telleria wrote:
>
> Could sspipe module be included as part of Python's Standard Library?
>
> https://sspipe.github.io/
>
> https://github.com/sspipe/sspipe
>
> https://pypi.org/project/sspipe/
>
> sspipe allows to use syntax such as:
>
> from sspipe import p, px
> import numpy as np
> import pandas as pd
>
> (
> np.linspace(0, pi, 100)
> | p({'x': px, 'y': np.sin(px)})
> | p(pd.DataFrame)
> | px[px.x > px.y].head()
> | p(print, "Example 6: pandas and numpy support:\n", px)
> )
>
> The issue in Python's Bug Tracker is:
>
> https://bugs.python.org/issue38052
>
> Any Core Python Developer willing to support this PEP?
>
> Thank you,
>
> Juan Telleria
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2MUGDTKV5CFRXZ5LKLIBW5XB7Y3QZV6A/
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/J7RM2DSE5YXPDRP5ZOCVERHXWCIILK4D/
[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?
>From my understanding, _Py_IDENTIFIER was designed for static usage. The overhead is quite low; keeping it as a private (static) module-level internal variable helps to decouple things. This target is very important for keeping public API as small as possible. On Fri, Sep 20, 2019 at 10:32 PM Vinay Sajip via Python-Dev wrote: > > I just ran an analysis of static variable definitions in CPython code, using > clang, on Ubuntu and Windows. The results should be available here: > > https://cpython.red-dove.com/ > > As I understand it, _Py_IDENTIFIER instances are supposed to hold constant > strings that are used in Python - e.g. "__class__", "__dict__" and so on. I > noticed that there are numerous duplicates of these - e.g. 8 instances of > __name__, 11 instances of __dict__, and so on - each instance is defined as > static in its source file and so completely distinct from the others. > > I realise the overall amount of memory used by these structures isn't large, > but is there any particular benefit to having these multiple copies? The > current situation seems a little untidy, at least. What would be the > disadvantage of making them extern in the headers and allocating them once in > some consts.c module? After all, they seem for the most part to be well-known > constant strings that don't need to be encapsulated in any particular C > compilation unit. > > Regards, > > Vinay Sajip > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/GVQOMWXUDYLBXZ2MMIDX5D6X7X42VQV7/ -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E24YQUYJMXC4LZXZEU7YELWIDDVWBDF3/
[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?
On Wed, Dec 11, 2019 at 11:52 PM Antoine Pitrou wrote: > > On Mon, 9 Dec 2019 21:42:36 -0500 > Kyle Stanley wrote: > > > > > There's also a practical use case for having a large number of coroutine > > objects, such as for asynchronously: > > > > 1) Handling a large number of concurrent clients on a continuously running > > web server that receives a significant amount of traffic. > > Not sure how that works? Each client has an accepted socket, which is > bound to a local port number, and there are 65536 TCP port numbers > available. Unless you're using 15+ coroutines per client, you probably > won't reach 1M coroutines that way. > I'm sorry, but the accepted socket has the same local port number as the listening one. Routing is performed by (local_ip, local_port, remote_ip, remote_port) quad. The listening socket can accept hundreds of thousands of concurrent client connections. The only thing that should be tuned for this is increasing the limit of file descriptors. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/O3ZODXHEIJ2SM5SZBOVJ4PIAQMSYNXEJ/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZVTAHRNWGI4ESWRT44PG3JUJLWJBYXFT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Adding a scarier warning to object.__del__?
__del__ is very useful not for interpreter shutdown but as a regular destructor in object's lifecycle. Action on the shutdown is another beast. Personally, I prefer to do all finalization works by explicit calls instead. On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger wrote: > > Hey everyone, > > I just encountered yet another reason to beware of __del__: when it's called > during interpreter shutdown, for reasons which are kind of obvious in > retrospect, if it calls notify() on a threading.Condition, the waiting thread > may or may not ever actually receive it, and so if it does that and then > tries to join() the thread the interpreter may hang in a hard-to-debug way. > > This isn't something that can reasonably be fixed, and (like in most cases > involving __del__) there's a very simple fix of using weakref.finalize > instead. My question for the dev list: How would people feel about changing > the documentation for the method to more bluntly warn people against using > it, and refer them to weakref.finalize and/or atexit.register as an > alternative? The text already has an undertone of "lasciate ogni speranza, > voi ch'entrate" but it may be helpful to be more explicit to avoid people > getting caught in unexpected pitfalls. > > Yonatan > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/AAZQBWD6PHC4PVNCCPX4A2745SS7B3LS/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/57CDW4NIYEQ3JEVX2JVCJDA5TXTC5MBR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Adding a scarier warning to object.__del__?
If the warning text tells about the delayed execution -- I'm +1. -1 for something like "just don't use __del__, it is dangerous". On Wed, Jan 1, 2020, 21:51 Gregory P. Smith wrote: > > > On Wed, Jan 1, 2020 at 6:40 AM Andrew Svetlov > wrote: > >> __del__ is very useful not for interpreter shutdown but as a regular >> destructor in object's lifecycle. >> > > The reason we should warn people against ever implementing __del__ is that > people rarely actually understand object lifecycle. Its presence > guarantees delayed garbage collection and that its code will code execute > in a context way outside of normal control flow that all other methods are > invoked from. > > -gps > > >> >> Action on the shutdown is another beast. >> Personally, I prefer to do all finalization works by explicit calls >> instead. >> >> On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger wrote: >> > >> > Hey everyone, >> > >> > I just encountered yet another reason to beware of __del__: when it's >> called during interpreter shutdown, for reasons which are kind of obvious >> in retrospect, if it calls notify() on a threading.Condition, the waiting >> thread may or may not ever actually receive it, and so if it does that and >> then tries to join() the thread the interpreter may hang in a hard-to-debug >> way. >> > >> > This isn't something that can reasonably be fixed, and (like in most >> cases involving __del__) there's a very simple fix of using >> weakref.finalize instead. My question for the dev list: How would people >> feel about changing the documentation for the method to more bluntly warn >> people against using it, and refer them to weakref.finalize and/or >> atexit.register as an alternative? The text already has an undertone of >> "lasciate ogni speranza, voi ch'entrate" but it may be helpful to be more >> explicit to avoid people getting caught in unexpected pitfalls. >> > >> > Yonatan >> > ___ >> > Python-Dev mailing list -- python-dev@python.org >> > To unsubscribe send an email to python-dev-le...@python.org >> > https://mail.python.org/mailman3/lists/python-dev.python.org/ >> > Message archived at >> https://mail.python.org/archives/list/python-dev@python.org/message/AAZQBWD6PHC4PVNCCPX4A2745SS7B3LS/ >> > Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> >> -- >> Thanks, >> Andrew Svetlov >> ___ >> Python-Dev mailing list -- python-dev@python.org >> To unsubscribe send an email to python-dev-le...@python.org >> https://mail.python.org/mailman3/lists/python-dev.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-dev@python.org/message/57CDW4NIYEQ3JEVX2JVCJDA5TXTC5MBR/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NFGVEU53S37ZQZ2JEUFQIDYHQ6USR6DG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Adding a scarier warning to object.__del__?
I would say that the "recommended" weakref.finalize() shares very many limitations of __del__(), that's why hard to buy it. atexit.register() is not a common thing, the recommendation of using atexit for file descriptor closing *in general* looks weird, while it can be a good solution in some particular case. On Thu, Jan 2, 2020 at 1:05 PM Armin Rigo wrote: > > Hi, > > On Thu, 2 Jan 2020 at 03:59, Yonatan Zunger wrote: > > It is possible (though not recommended!) for the __del__() method to > > postpone destruction of the instance by creating a new reference to it. > > This is called object resurrection. It is implementation-dependent whether > > __del__() is called a second time when a resurrected object is about to be > > destroyed; the current CPython implementation only calls it once. > > "...in most cases." > > > Armin Rigo -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VY2PX376FVNKR3KGFM557KR4OMRH22VL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Adding a scarier warning to object.__del__?
On Sat, Jan 4, 2020 at 5:43 AM Inada Naoki wrote: > > I don't want to recommend atexit or weakref because they have their pitfall. > > I think the context manager should be considered at first when people want > to use __del__. > I support this proposal :) > -- > Inada Naoki > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/RSG6DHNG7ZTJCNYSAMVOFFQSD7HFSGKE/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YA6574IBBBIMAJESCFKJXLISFQXXQ475/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Remove formatter module from Python 3.9, deprecated for 7 years (Python 3.4)
I have no objections, the module raises the explicit deprecation warning since 2013. On Fri, Jan 17, 2020 at 7:31 PM Victor Stinner wrote: > > Hi, > > I proposed https://bugs.python.org/issue39352 to remove the formatter > module. It's deprecated since Python 3.4. The main reason why it's > still around is the PEP 4 rule: > > "In order to facilitate writing code that works in both Python 2 & 3 > simultaneously, any module that exists in both Python 3.5 and Python > 2.7 will not be removed from the standard library until Python 2.7 is > no longer supported as specified by PEP 373." > > https://www.python.org/dev/peps/pep-0004/#for-modules-existing-in-both-python-2-7-and-python-3-5 > > Python 2.7 is not longer supported. So can we now remove the formatter module? > > The module has no test, I failed to find any user in a dummy GitHub > code search and I didn't know that this module even existed. > > We can still revert the module before 3.9.0 final release if someone > shows up and asks to keep it for one more cycle. > > My intent here is to reduce the size of the Python standard library to > reduce the maintenance burden. Python became quite large and it's more > and more expensive to maintain it. Wait... I'm not asking to remove > all modules of the standard library :-) The decision must be taken on > a case by case basis, for each module. Here I'm only taking about the > formatter module. > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/ESSRZXEJ7MWMGNZQKNDURGRWIZ5FQHKP/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BHCWXBTELE354DCELWADMOX6O6CJOJ2A/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Comparing UUID objects to strings: why not?
-1 Implicit type casting leads to hidden errors very often. On Thu, Jan 23, 2020 at 5:46 PM Michael Anckaert via Python-Dev wrote: > > Hello everyone > > I have a usecase where I'm comparing a UUID instance with string quite > often. A case where I have to convert the UUID instance to a string > using str(uuid_obj), leading to redundant code: > > if str(uuid_obj) == uuid: > # do stuff... > > In my experience I can't find a case where you wouldn't want the > implicit conversion to string before comparing a UUID with string. So > I would like to propose improving the __eq__ method of UUID to: > > def __eq__(self, other): > if isinstance(other, UUID): > return self.int == other.int > elif isinstance(other, str): > return str(self) == other > return NotImplemented > > Based on my testing and experience I don't think this is a breaking > change and would be a meaningful change to the way UUID works. Of course > please don't hesitate to change my view or point out any flaws in my > reasoning. > > If other people support this change I'd start the work of creating an > issue and PR to get this change implemented. > > -- > Michael Anckaert > +32 474 066 467 > https://www.sinax.be > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VXLJMWZRZLSFZRNHUITSLAV2O363WG5Q/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Thanks, Andrew Svetlov ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7S6IJLPHIJLMBVKHUBNKYZXWOOHMIGW2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] PEP 3121
What real status of this document? As I figured out py3k trunk uses this ones. Most part of 'battery included' modules don't use this feature, leaving m_traverse, m_clear and m_free NULL pointers. There are only exception: _io. But looks like all underground python machinery is already ported to support PEP. Maybe it's good point to update PEP? "accepted; may not be implemented yet" sounds a bit confusing. Just insert somethere close to top of content: "Implemented by Python itself, but builtin modules should be converted". Next question: are where all 'python standard library' modules looks like "not converted yet" really need to be refactored? Or we can split this huge set to 'looks already good' and 'requires some work'? And just add comment for every PyModuleDef definition like: "Completely satisfy PEP 3121, signed and stamped by Martin von Loewis and GvR".? Last question: PEP refers to "multiple interpreters" feature of python. This feature presented at least in 2.xx branch (don't remember about 1.xx - up to 2003 I used python as scripting language only and not care about C interface). I never saw usage of this one in real use cases. Is there projects (preferable with code sources, of course) what runs multiple interpreter in same process? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 384
Is there some public branch for this PEP? I like the idea and glad to look on implementation. Thanks. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] copying of itertools iterators
using of copy.copy for simple iterators is forbidden >>> import copy >>> copy.copy(iter([1, 2, 3])) Traceback (most recent call last): File "", line 1, in File "/home/andrew/projects/py3k/Lib/copy.py", line 96, in copy return _reconstruct(x, rv, 0) File "/home/andrew/projects/py3k/Lib/copy.py", line 284, in _reconstruct y = callable(*args) File "/home/andrew/projects/py3k/Lib/copyreg.py", line 88, in __newobj__ return cls.__new__(cls, *args) TypeError: object.__new__(list_iterator) is not safe, use list_iterator.__new__() That behavior is safe and clean. But it's possible to copy iterator objects returned by itertools functions: >>> i = itertools.chain([1, 2], [3, 4, 5]) >>> i.__next__() 1 >>> j = copy.copy(i) >>> j.__next__() Traceback (most recent call last): File "", line 1, in StopIteration >>> i.__next__() 2 Looks like itertools object should be protected from usage like that. Folks, what are you think about? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __package__ attribute
What are expected values for module.__package__? I see two different behaviors: importlib and standard python import. importlib processes __package__ pretty simple and clean: - for toplevel modules __package__ is empty string - for packages it's package name - for modules inside packages it's again package name Standard import follows another strategy: - it tries to setup __package__ only in case of relative import (see first 'if' in import.c:get_parent) - otherwise __package__ is untouched and None by default. For me importlib's way is better. I don't see any PEP specifying __package__ behavior. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __package__ attribute
For me it's the real bug in standard python import machinery. I don't see any backward incompatibilities. There are very hard to write any import-depended code based on decision: was module imported in absolute or relative way. If it's a bug I can create issue in python bugtracker and provide a patch (for Python 3.2 and for 2.7 if later is required). On Fri, May 21, 2010 at 10:18 PM, Andrew Svetlov wrote: > What are expected values for module.__package__? > I see two different behaviors: importlib and standard python import. > importlib processes __package__ pretty simple and clean: > - for toplevel modules __package__ is empty string > - for packages it's package name > - for modules inside packages it's again package name > > Standard import follows another strategy: > - it tries to setup __package__ only in case of relative import (see > first 'if' in import.c:get_parent) > - otherwise __package__ is untouched and None by default. > > For me importlib's way is better. > I don't see any PEP specifying __package__ behavior. > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3148 ready for pronouncement
Is there any reason to have Future .cancelled, .done, .running as methods? >From my perspective they are really readonly properties. BTW, is 'cancelled' correct name? Spell-checkers likes only single 'l' form: 'canceled'. On Sun, May 23, 2010 at 2:47 PM, Brian Quinlan wrote: > > On May 23, 2010, at 8:43 PM, Robert Collins wrote: > >> On Sun, May 23, 2010 at 10:15 PM, Brian Quinlan >> wrote: >> Also, you can't fix bugs except by releasing new versions of Python. Therefore the API must be completely stable, and the product virtually bugfree before it should be in stdlib. The best way of ensuring that is to release it as a separate module on PyPI, and let it stabilize for a couple of years. >>> >>> Yeah but that model isn't likely to work with this package. >>> Cheers, >>> Brian >> >> Forgive my ignorance, but why do you say that that model won't work >> with this package? > > As I said in my last message: > > """Simple modules are unlikely to develop a following because it is too easy > to partially replicate their functionality. urlparse and os.path are very > useful modules but I doubt that they would have been successful on PyPI.""" > > Cheers, > Brian > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] contributors survey?
SVN is very bad instrument to contribute or follow an issue patches. And, of course, very long lifecycle of the most issues greatly reduces enthusisasm. On Wed, Mar 2, 2011 at 2:08 PM, Antoine Pitrou wrote: > > Hello, > > On Tue, 01 Mar 2011 19:25:00 -0800 > Westley Martínez wrote: >> >> If I got a message like that in my mailbox I would be rather annoyed, >> mark it as spam, and be less likely to contribute again. > > Yes, I think that's a risk. Do you think of a wording that could > alleviate such perception? > > Thank you > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] contributors survey?
On Wed, Mar 2, 2011 at 2:39 PM, Antoine Pitrou wrote: > > Will Mercurial make things more attractive? > Definitely yes! I welcome upcoming migration. >> And, of course, very long lifecycle of the most issues greatly reduces >> enthusisasm. > > True. I believe we are improving that, but perhaps that's a > misperception on my part. > I understand reasons for that situation and really doubt if process can be significantly accelerated. But it just very unconvenient. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] User conversions in custom string.Formatter
As PEP 3101 http://www.python.org/dev/peps/pep-3101/ says (and current
Python does) user can specify conversions like "{0!s}".
In custom formatters (derived from string.Formatter) he can override
convert_field method to add custom conversions.
I experimented with that last month and found it very convenient.
>From my perspective custom conversions are very close to 'filters'
from html template engines (jinja2, mako, django etc).
While I like to see custom conversions simple and easy I don't wan't
to bring 'cascading' and 'parametrization' to standard formatting.
But why don't relax spec a bit and allow to use any word (with
'identifier' semantix, I mean str.isidentifier()) as conversion name?
Proposed update doesn't make any backward incompatibility (both python
and public C API are not changed), it's clean and obvious.
And it's very funny to use words instead single characters for custom
user-specific conversions.
What do you think?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] tuple[int] optimization
tuple[int] is 30% slower than list[int].
Please let me explain the problem.
(1, 2, 3)[1] has compiled as BINARY_SUBSCR operation.
ceval.c has optimization for list:
case BINARY_SUBSCR:
w = POP();
v = TOP();
if (PyList_CheckExact(v) && PyInt_CheckExact(w)) {
/* INLINE: list[int] */
Py_ssize_t i = PyInt_AsSsize_t(w);
if (i < 0)
i += PyList_GET_SIZE(v);
if (i >= 0 && i < PyList_GET_SIZE(v)) {
x = PyList_GET_ITEM(v, i);
Py_INCREF(x);
}
else
goto slow_get;
}
else
slow_get:
x = PyObject_GetItem(v, w);
Py_DECREF(v);
Py_DECREF(w);
SET_TOP(x);
if (x != NULL) continue;
break;
For tuple code follows slow way via tp_as_mapping slot and calls
tuplesubscript from Objects/tupleobject.c
Looks like tuple is goot applicant to be optimized as list does.
tuples is used in python programs very often and getting element from
tuple by index as common operation as list[int].
Is there reasons to prevent special case for tuples in BINARY_SUBSCR?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tuple[int] optimization
You right. Sorry, I missed changes in ceval.c for py3k.
Please note, simple test like:
from timeit import timeit
print('list', timeit("l[0]", "l = [1]"))
print('tuple', timeit("l[0]", "l = (1,)"))
Has results:
andrew@ocean ~/p/cpython> python2.7 z.py
('list', 0.03479599952697754)
('tuple', 0.046610116958618164)
andrew@ocean ~/p/cpython> python3.2 z.py
list 0.04870104789733887
tuple 0.04825997352600098
For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and
trunk) very close to "unoptimized" 2.7 version.
On Sun, Jul 24, 2011 at 2:27 AM, Antoine Pitrou wrote:
> On Sun, 24 Jul 2011 09:13:07 +1000
> Ryan Kelly wrote:
>>
>> In latest trunk this optimisation seems to have gone away, the code is
>> now:
>>
>> TARGET(BINARY_SUBSCR)
>> w = POP();
>> v = TOP();
>> x = PyObject_GetItem(v, w);
>> Py_DECREF(v);
>> Py_DECREF(w);
>> SET_TOP(x);
>> if (x != NULL) DISPATCH();
>> break;
>>
>> The implementation of PyObject_GetItem doesn't appear to have changed
>> though. Maybe this optimisation was no longer worth it in practice?
>
> The optimization was probably removed because PyInt objects don't exist
> anymore. There's a related but more ambitious patch at
> http://bugs.python.org/issue10044.
>
> In practice however, such micro-optimizations usually have little or no
> effect.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tuple[int] optimization
I undrestand your point. Thank you for explanation.
On Sun, Jul 24, 2011 at 3:18 AM, Antoine Pitrou wrote:
> Le dimanche 24 juillet 2011 à 03:15 +0300, Andrew Svetlov a écrit :
>> You right. Sorry, I missed changes in ceval.c for py3k.
>> Please note, simple test like:
>>
>> from timeit import timeit
>>
>> print('list', timeit("l[0]", "l = [1]"))
>> print('tuple', timeit("l[0]", "l = (1,)"))
>>
>> Has results:
>>
>> andrew@ocean ~/p/cpython> python2.7 z.py
>> ('list', 0.03479599952697754)
>> ('tuple', 0.046610116958618164)
>>
>> andrew@ocean ~/p/cpython> python3.2 z.py
>> list 0.04870104789733887
>> tuple 0.04825997352600098
>>
>> For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and
>> trunk) very close to "unoptimized" 2.7 version.
>
> My point is that on non-trivial benchmarks, the savings are almost zero.
> If you look at the (much more complex) patch I linked to, the savings
> are at most 10% on a couple of select benchmarks, other benchmarks
> showing almost no difference.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394
ArchLinux has used `python` as alias for `python3` while `python2` is still supported. On Mon, Feb 20, 2012 at 4:58 PM, anatoly techtonik wrote: > On Mon, Feb 20, 2012 at 4:58 PM, Nick Coghlan wrote: >> >> PEP 394 >> was at the top of my list recently > > > I've tried to edit it to be a little bit shorter (perhaps cleaner) and > commented (up to revision 2) up to Migration Notes. > http://piratepad.net/pep-0394 > > The main points: > 1. `python2.7` should be `python27` > 2. until platform supports Python 2, `python` should link to python2 binary > 3. python2 should always point to the latest version available on the system > (I didn't write that in comments) > -- > anatoly t. > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP czar for PEP 3144?
I like 'PEP czar' On Mon, Feb 20, 2012 at 6:50 PM, Mark Lawrence wrote: > On 20/02/2012 16:28, Senthil Kumaran wrote: >> >> On Tue, Feb 21, 2012 at 12:07 AM, wrote: >>> >>> "supreme ruler" sounds good to me. I could go for "inquisitor" instead >>> of "czar" as well... >> >> >> But that would be bad for developers from Spain as nobody would expect >> a spanish inquisition. >> >> :-) >> > > How about Big Brother then? As anyone worked in room 101? > > -- > Cheers. > > Mark Lawrence. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Drop the new time.wallclock() function?
On Tue, Mar 13, 2012 at 5:29 PM, Victor Stinner wrote: > I suppose that you can use a manual fallback to time.time() if > time.monotonic() failed. If time.monotonic() fails, it fails directly at the > first call. Example of a fallback working with Python < 3.3: > > try: > time.monotonic() > except (OSError, AttributeError): > get_time = time.time > else: > get_time = time.monotonic > I like 'fallback' solution while `get_time` is not the best name for high precision timer from my perspective. Can you call it `monotonic` or `realtime`? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Drop the new time.wallclock() function?
On Wed, Mar 14, 2012 at 10:21 AM, Guido van Rossum wrote: > +1 for steady(). > +1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] lzma
When I build python from sources I have no lzma support (module _lzma cannot be built). There are lzma packages installed in my Ubuntu 11.10 box: lzma, lzma-dev and lzma-sources. I can see lib files (headers are also can be found in linux-headers): andrew@tiktaalik ~/p/cpython> locate lzma.so /usr/lib/liblzma.so.2 /usr/lib/liblzma.so.2.0.0 I can live with that but if somebody can point me the way to build python with lzma support please do. -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lzma
liblzma-dev has solved my problem. Thank you, Senthil. On Fri, Mar 16, 2012 at 7:38 PM, Senthil Kumaran wrote: > > On Fri, Mar 16, 2012 at 07:30:31PM -0700, Andrew Svetlov wrote: >> When I build python from sources I have no lzma support (module _lzma >> cannot be built). >> > > I have liblzma-dev, liblzma2 and lzma packages installed on ubuntu. I > am able to build and import lzma module. > > Thanks, > Senthil > > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Playing with a new theme for the docs, iteration 2
I like to always see "Quick search" widget without scrolling page to top. Is it possible? Or maybe you can embed some keyboard shortcut for quick jump to search input box? On Sun, Mar 25, 2012 at 11:44 AM, Serhiy Storchaka wrote: > 25.03.12 11:06, Peter Otten написав(ла): > >> * Inlined code doesn't need the gray background. The bold font makes it >> stand out enough. > > > I believe that the gray background is good, but it should make it lighter. > > >> * Instead of the box consider italics or another color for [New in ...] >> text. > > > Yes, the border around "New in" and "Changed in" looks not good-looking. > Maybe a very light colored background with no border or underlined italic > will look better. > > >> * Maybe the Next/Previous Page headers on the left could link to the >> respective page. > > > Do you mean next/previous links in header/footer? > > I totally agree with your other comments, including the admiration of the > current version of the design. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Playing with a new theme for the docs, iteration 2
On Sun, Mar 25, 2012 at 12:04 PM, Stefan Krah wrote: > Andrew Svetlov wrote: >> I like to always see "Quick search" widget without scrolling page to >> top. Is it possible? > > Do you mean a fixed search box like this one? > > http://coq.inria.fr/documentation > No. You are right, it's distracting. Maybe narrow persistent line with searchbox on the top will be better. But just jump to searchbox by shortcut is good enough for me also. > > Please don't do this, I find scrolling exceptionally distracting in the > presence of fixed elements. > > > > Stefan Krah > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Playing with a new theme for the docs, iteration 2
I like to see new schema only for 3.3 as sign of shiny new release. On Sun, Mar 25, 2012 at 10:25 PM, Georg Brandl wrote: > On 25.03.2012 21:11, Steven D'Aprano wrote: >> Georg Brandl wrote: >> >>> Thanks everyone for the overwhelmingly positive feedback. I've committed >>> the >>> new design to 3.2 and 3.3 for now, and it will be live for the 3.3 docs >>> momentarily (3.2 isn't rebuilt at the moment until 3.2.3 final goes out). >>> I'll transplant to 2.7 too, probably after the final release of 2.7.3. >> >> I think it would be better to leave 2.7 with the old theme, to keep it >> visually distinct from the nifty new theme used with the nifty new 3.2 and >> 3.3 >> versions. > > Hmm, -0 here. I'd like more opinions on this from other devs. > > Georg > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] datetime module and pytz with dateutil
I figured out what pytz and dateutil are not mentioned in python docs for datetime module. It's clean why these libs is not a part of Python Libraries — but that's not clean for Docs. From my perspective at least pytz (as py3k compatible) should to be mentioned as the library which contains timezone info, supported carefully and recommended to use with datetime standard module, -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
