[Python-Dev] Re: Exposing Tools/parser/unparse.py in the stdlib?

[Brett Cannon]

Victor Stinner wrote:
> Le mar. 19 nov. 2019 à 22:12, Pablo Galindo Salgado
> pablog...@gmail.com a écrit :
> > Float infinity
> > is replaced with 1e309. Again, maybe someone wants
> > to render this differently? It sounds like an arbitrary choice (which
> > "works" as expected).
> > That is not true. float('inf') is rendered as float('inf´):
> > Unparser(ast.parse("float('inf')"))
> > float('inf')
> > I was thinking at:
> > Unparser(ast.parse("1e999"))
> > 1e309
> > Maybe just move the constant as a class attribute, so it can be
> overriden in a subclass?
> > I would prefer
> > to keep a separated module, like "import ast.unparse"
> > or "import unparse".
> > Why? I think ast.unparse is a natural fit. It will likely be only one 
> > function
> > exposed.
> > It's mostly to minimize the number of imports on "import ast". unparse
> requires extra imports like tokenize which has also tons of
> dependencies.

Why are you specifically worried about that? If you're doing AST 
transformations you're probably either doing it offline/AOT or you're already 
doing extra processing which means you probably aren't stressing over how 
quickly an initial import will take if this is all worries surrounding doing 
live AST transformations on the way to writing out bytecode or loading a module 
on the fly.

> Victor
> Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XOK626U42XMPSL2M4BY5TDM6TXFTTX2E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] Python 3.9.0a1 available for testing

[Steve Dower]

On 19Nov2019 1708, Łukasz Langa wrote:

Go get it here: https://www.python.org/downloads/release/python-390a1/ 



Is it intentional that this link does not appear on 
https://www.python.org/download/pre-releases/ ?


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6C2G5LH7YAEEKPSX7LFJFIDNS6GVOQAH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Exposing Tools/parser/unparse.py in the stdlib?

[Nick Coghlan]

On Thu., 21 Nov. 2019, 3:30 am Brett Cannon,  wrote:

> Victor Stinner wrote:
> > > It's mostly to minimize the number of imports on "import ast". unparse
> > requires extra imports like tokenize which has also tons of
> > dependencies.
>
> Why are you specifically worried about that? If you're doing AST
> transformations you're probably either doing it offline/AOT or you're
> already doing extra processing which means you probably aren't stressing
> over how quickly an initial import will take if this is all worries
> surrounding doing live AST transformations on the way to writing out
> bytecode or loading a module on the fly.
>

The AST module is also the home of "ast.literal_eval", and that is going to
be used in a much wider variety of situations.

Cheers,
Nick.


>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YUAYQMHD4XKF3GSPTGYMNQJFJVLFNS5D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] Python 3.9.0a1 available for testing

[Łukasz Langa]

Good catch, I didn't update that.

-- 
Best regards,
Łukasz Langa

> On 20 Nov 2019, at 18:59, Steve Dower  wrote:
> 
> On 19Nov2019 1708, Łukasz Langa wrote:
>> Go get it here: https://www.python.org/downloads/release/python-390a1/ 
>> 
> 
> Is it intentional that this link does not appear on 
> https://www.python.org/download/pre-releases/ ?
> 
> Cheers,
> Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZGLBBJEU5GDYIEQ2ELUJ7LRNIINU5HHP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Exposing Tools/parser/unparse.py in the stdlib?

[Pablo Galindo Salgado]

Opened https://bugs.python.org/issue38870 to track this.

On Tue, 19 Nov 2019 at 00:40, Pablo Galindo Salgado 
wrote:

> Hi,
>
> What do people feel about exposing Tools/parser/unparse.py in the standard
> library? Here is my initial rationale:
>
> * The tool already needs to be maintained and updated as is tested as part
> of the test suite.
> * I have used the tool almost all the time I needed to deal with AST
> transformations.
> * The public interface will have a very low surface API, keeping
> maintaining it (the public interface) a very small burden IMHO.
>
> We could add the public interface to the ast.py module or a new one if
> people feel strongly about it.
>
> Does anyone feel strongly against this or have any objection that I am not
> contemplating?
>
> Regards from rainy London,
> Pablo Galindo Salgado
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OGLBBPVPWGCJXBAUCWLSVGZ7VQXHCSKY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dangerous default value for reuse_address in asyncio loop.create_datagram_endpoint()

[Giampaolo Rodola']

SO_REUSEADDR was controversial also for socket.create_server(). In the end
I concluded the best solution was to not expose a reuse_address parameter.
See:
https://github.com/python/cpython/blob/94e165096fd65e8237e60de570fb609604ab94c9/Lib/socket.py#L891-L899

It must be noted that right now also asyncio's create_server() method
allows passing reuse_address=True, which on Windows should probably be
turned into a no-op.

As for asyncio's create_datagram_endpoint() I partly agree with Antoine's
solution.
https://bugs.python.org/issue37228#msg357068
My course of action, though, would be the following:

* in 3.8: turn reuse_address parameter into a no-op, update doc
* in 3.9: raise error if reuse_address=True, update doc

Note: differently from TCP / create_server(), with UDP you can set
SO_REUSEADDR manually after calling create_datagram_endpoint() if you
really want to.

On Tue, Nov 19, 2019 at 4:48 AM  wrote:

> When creating UDP servers with asyncio's create_datagram_endpoint(), the
> default value for reuse_address = True, resulting in a dangerous (and
> currently incorrectly documented) situation. I have proposed changing the
> default value but understandably such a change for a core library function
> parameter is not to be taken lightly. Thus I put this up for discussion on
> the list.
>
> As background, when creating TCP servers on UNIX-like systems, it is
> almost boilerplate to set SO_REUSEADDR for all server sockets to make sure
> that a restarting server can immediately bind the socket again. Without the
> SO_REUSEADDR sockopt, the kernel will hold the addr:port in a TIME_WAIT
> state for a while, preventing reuse. Thus, when creating TCP servers with
> loop.create_server(), the parameter reuse_address has a very reasonable
> default value of True.
>
> However things are very different in UDP-land. The kernel does not hold
> UDP server ports in a waiting state so the SO_REUSEADDR sockopt was
> repurposed in Linux (and *BSD afaik) to allow multiple processes to bind
> the SAME addr:port for a UDP server. The kernel will then feed incoming UDP
> packets to all such processes in a semi-fair-roundrobin manner. This is
> very useful in some scenarios, for example I've used it myself in C++
> projects to allow UDP servers to be scaled easily and rolling upgrades to
> be implemented without separate load-balancing. But for this to be the
> default behaviour is quite dangerous.
>
> I discovered this default behaviour accidentally by having 2 separate
> Python programs (both doing SIP over UDP) accidentally configured to use
> the same UDP port. The result was that my 2 processes were indeed "sharing
> the load" - neither of them threw an exception at startup about the port
> being already in use and both started getting ~half of the incoming
> packets. So off to the docs I went and discovered that the documentation
> for create_datagram_endpoint() does not mention this behaviour at all,
> instead it mistakenly refers to the TCP protocol use of SO_REUSEADDR:
> "reuse_address tells the kernel to reuse a local socket in TIME_WAIT state,
> without waiting for its natural timeout to expire. If not specified will
> automatically be set to True on Unix."
>
> https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_datagram_endpoint
>
> What makes this default especially dangerous is,
> - Most people are not aware of this special mode that Linux allows for UDP
> sockets
> - Even if it was documented to be the default, many people would miss it
> unless a big warning was slapped on the docs
> - The problems are unlikely to appear in test scenarios and much more
> likely to pop up in production months or years after rolling out the code
> - If you have never used it on purpose, it is very confusing to debug,
> causing you to doubt your own and the kernel's sanity
> - The behaviour changes again if you happen to use a multicast address...
>
> Thus, my proposal is to change the default value for UDP to False or
> deprecate the function and introduce a new one as suggested by Yuri in my
> original bug report at:  https://bugs.python.org/issue37228
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/TK2NTPWID7RBUUNUU5JYAZHR6FKEABRU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GSWPWB63DF4AT6I4R6O4MCJ5A52UZO3T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hold multiple tracebacks per-object in tracemalloc

[Sümer Cip]

Thanks Victor, I have a better understanding of tracemalloc internals.

On Tue, Nov 19, 2019 at 10:07 PM Victor Stinner  wrote:

> Hi,
>
> Le mar. 19 nov. 2019 à 18:33, Sümer Cip  a écrit :
> > First of all, I would like to thank for such a useful tool for debugging
> memory issues. I am pretty proud as a Python lover that we have such a
> tool: tracemalloc:)
>
> As the module author: you're welcome ;-)
>
>
> > AFAIU, tracemalloc holds a single traceback per-object:
>
> "tracemalloc" name means that a "trace" is stored in mapping
> associated to an allocated memory block. A trace is just: (size,
> traceback): memory block in bytes, and the Python traceback where the
> memory block has been allocated.
>
> tracemalloc is not aware of Python objects at all. For example, it is
> unable to associate an object __dict__ to the object. It works at
> malloc() and free() level, not at the _PyObject_GC_Malloc() level.
>
> One advantage is that memory which is not tracked by the GC is tracked
> by tracemalloc. Another is that the implementation is "simple" (from
> my point of view :-)).
>
> One drawback is that tracemalloc traces are harder to analyze.
>
>
> > """
> > Output:
> > ...
> > 5.py:38: size=9264 B, count=1, average=9264 B
> > ['  File "5.py", line 38', 'a.alloc_more()', '  File "5.py", line
> 32', 'self._d += [1] * 512']
> > """
>
> When the internal storage of a Python list grows, the old memory block
> is released and a new memory block is allocated. From the tracemalloc
> point of view, the old trace is removed, a new trace is created. There
> is no relationship between the two traces, even if realloc() has been
> used. In tracemalloc, realloc() traces work as two isolated
> operations: free() + malloc().
>
>
> > Now, allocation of alloc and alloc_more functions seems to be merged
> into a single traceback. Is it possible we add (maybe a constant number of
> tracebacks) per object? This might be pretty useful to find the responsible
> leak in situations where same object is modified in many places?
>
> You might try to take snapshots frequently and write an heuristic to
> link traces which look similar... good luck with that :-)
>
> --
>
> There are other tools to track Python memory usage which may help you
> to provide a view closer to what you are looking for.
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
>


-- 
Sümer Cip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7ECDFRQBT2ZHVCEOTNP6XPZ7NERIXZJF/
Code of Conduct: http://python.org/psf/codeofconduct/