Re: [Python-Dev] bpo-33257: seeking advice & approval on the course of action

[Chris Barker via Python-Dev]

On Wed, May 2, 2018 at 8:21 PM, Terry Reedy  wrote:

> On 5/2/2018 4:38 PM, Ivan Pozdeev via Python-Dev wrote:
>
>> The bottom line is: Tkinter is currently broken
>>
>
> This is way over-stated.  Many modules have bugs, somethings in features
> more central to their main purpose.


I'll suggest a re-statement:

tkinter is not thread safe, and yet it is documented as being thread safe
(or at least not documented as NOT being thread safe)

This is either a bug(s) in the implementation or the docs.

So what are the solutions?

1) fix the docs -- unless tkInter is made thread safe really soon, and
fixes are back-ported, this seems like a no brainer -- at least temporarily.

2) fix the issues that make tkInter not thread safe -- apparently there is
a thread safe tcl/tk, so it should be possible, though I have to say I'm
really surprised that that's the case for an old C code base -- but great!

The problem here is that we'll need qualified people to submit and review
the code, and it really should get some extensive testing -- that's a lot
of work.

And it's going to take time, so see (1) above.

Another issue:

Many GUI toolkits are not thread safe (I have personal experience with
wxPython), so it's not so bad to simply say "don't do that" for tkInter --
that is, don't make tkInter calls from more than one thread.

However, wxPython (for example) makes this not-too-bad for multi-threaded
programs by providing thread-safe ways to put events on the event queue --
whether with wx.PostEvent, or the utilities wx.CallAfter() and
wx.CallLater(). This makes it pretty easy to keep the GUI in one thread
while taking advantage of threads for long running tasks, etc.

IIUC, tkinter does not have such an easy way to interact with the GUI from
another thread -- so maybe adding that would be a good first step.

I neither use tkinter, nor have the expertise to contribute this -- but
clearly Ivan does -- and maybe others would want to help.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python startup time

[Chris Barker via Python-Dev]

On Fri, May 11, 2018 at 11:05 AM, Ryan Gonzalez  wrote:

>  https://refi64.com/uprocd/ 


very cool -- but *nix only, of course :-(

But it seems that there is a demand for this sort of thing, and a few major
projects are rolling their own. So maybe it makes sense to put something
into the standard library that everyone could contribute to and use.

With regard to forking -- is there another way? I don't have the expertise
to have any idea if this is possible, but:

start up python

capture the entire runtime image as a single binary blob.

could that blob be simply loaded into memory and run?

(hmm -- probably not -- memory addresses would be hard-coded then, yes?) or
is memory virtualized enough these days?

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python startup time

[Chris Barker via Python-Dev]

On Mon, May 14, 2018 at 12:33 PM, INADA Naoki 
wrote:

> It will broke hash randomization.
>
> See also: https://www.cvedetails.com/cve/CVE-2017-11499/


I'm not enough of a security expert to know how much that matters in this
case, but I suppose one could do a bit of post-proccessing on the image to
randomize the hashes? or is that just insane?

Also -- I wasn't thinking it would be a pre-build binary blob that everyone
used -- but one built on the fly on an individual system, maybe once per
reboot, or once per shell instance even. So if you are running, e.g. hg a
bunch of times in a shell, does it matter that the instances are all
identical?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3

[Chris Barker via Python-Dev]

On Sat, May 12, 2018 at 8:14 AM, Skip Montanaro 
wrote:

> > I have found 2to3 conversion to be remarkably easy and painless.
>
> > And the whole Unicode thing is much easier.
>

Another point here:

between 3.0 and 3.6 (.5?) -- py3 grew a lot of minor features that made it
easier to write py2/py3 compatible code. u"string", b'bytes %i' % something
-- and when where the various __future__ imports made available?

If these had been in place in 3.0, the whole process would have been easier
:-(

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Docstrings on builtins

[Chris Barker via Python-Dev]

Over on python-ideas, someone is/was proposing literals for timedeltas.

I don't expect that will come to anything, but it did make me take a look
at the docstring for datetime.timedelta. I use iPython's ? a lot for a
quick overview of how to use a class/function.

This is what I get:

In [8]: timedelta?
Init signature: timedelta(self, /, *args, **kwargs)
Docstring:  Difference between two datetime values.
File:   ~/miniconda2/envs/py3/lib/python3.6/datetime.py
Type:   type


That is, well, not so useful. I'd like to see at least the signature:

datetime.timedelta(days=0, seconds=0, microseconds=0, milliseconds=0,
minutes=0, hours=0, weeks=0

And ideally much of the text in the docs.

I've noticed similarly minimal docstrings on a number of builtin functions
and methods.

If I wanted to contribute a PR to enhance these docstrings, where would
they go?  I've seen mention of "argument clinic", but really don't know
quite what that is, or how it works, but it appears to be related.

Anyway -- more comprehensive docstrings on buildins could really help
Python's usability for command line usage.

Thanks,
-  Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Docstrings on builtins

[Chris Barker via Python-Dev]

On Mon, Jun 4, 2018 at 3:27 PM, Victor Stinner  wrote:

> For Argument Clinic, have a look at
> https://docs.python.org/dev/howto/clinic.html


Thanks Victor -- scanning that page, it is indeed where I needed to look.

You can also try to copy/paste code from other files using Argument
> Clinic and then run "make clinic" to regenerate the generated files.
>

good idea.

Now to find some time to actually work on this...

-CHB




> Victor
>
> 2018-06-04 23:45 GMT+02:00 Chris Barker via Python-Dev <
> python-dev@python.org>:
> > Over on python-ideas, someone is/was proposing literals for timedeltas.
> >
> > I don't expect that will come to anything, but it did make me take a
> look at
> > the docstring for datetime.timedelta. I use iPython's ? a lot for a quick
> > overview of how to use a class/function.
> >
> > This is what I get:
> >
> > In [8]: timedelta?
> > Init signature: timedelta(self, /, *args, **kwargs)
> > Docstring:  Difference between two datetime values.
> > File:   ~/miniconda2/envs/py3/lib/python3.6/datetime.py
> > Type:   type
> >
> >
> > That is, well, not so useful. I'd like to see at least the signature:
> >
> > datetime.timedelta(days=0, seconds=0, microseconds=0, milliseconds=0,
> > minutes=0, hours=0, weeks=0
> >
> > And ideally much of the text in the docs.
> >
> > I've noticed similarly minimal docstrings on a number of builtin
> functions
> > and methods.
> >
> > If I wanted to contribute a PR to enhance these docstrings, where would
> they
> > go?  I've seen mention of "argument clinic", but really don't know quite
> > what that is, or how it works, but it appears to be related.
> >
> > Anyway -- more comprehensive docstrings on buildins could really help
> > Python's usability for command line usage.
> >
> > Thanks,
> > -  Chris
> >
> >
> >
> >
> > --
> >
> > Christopher Barker, Ph.D.
> > Oceanographer
> >
> > Emergency Response Division
> > NOAA/NOS/OR&R(206) 526-6959   voice
> > 7600 Sand Point Way NE   (206) 526-6329   fax
> > Seattle, WA  98115   (206) 526-6317   main reception
> >
> > chris.bar...@noaa.gov
> >
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
> >
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Docstrings on builtins

[Chris Barker via Python-Dev]

OK,

looking a bit deeper:

In [69]: timedelta.__new__.__doc__
Out[69]: 'Create and return a new object.  See help(type) for accurate
signature.'

In [70]: timedelta.__init__.__doc__
Out[70]: 'Initialize self.  See help(type(self)) for accurate signature.'

In [71]: timedelta.__doc__
Out[71]: 'Difference between two datetime values.'

So the none of the docstrings have the proper information.  And:

help(timedelta) returns:

Help on class timedelta in module datetime:

class timedelta(builtins.object)
 |  Difference between two datetime values.
 |
 |  Methods defined here:
 |
 |  __abs__(self, /)
 |  abs(self)
 |
 |  __add__(self, value, /)
 |  Return self+value.


So no signature either.

I'm guessing this is because argument clinic has not been properly applied
-- so Ihave a PR to work on.

but where does help() get its info anyway?

I always thought docstrings were supposed to be used for the basic, well,
docs. And between the class and __new__ and __init__, somewhere in there
you should learn how to initialize an instance, yes?

-CHB





On Mon, Jun 4, 2018 at 6:21 PM, Matthias Bussonnier <
bussonniermatth...@gmail.com> wrote:

>
>
> On Mon, 4 Jun 2018 at 17:29, Ivan Pozdeev via Python-Dev <
> python-dev@python.org> wrote:
>
>> On 05.06.2018 3:09, Matthias Bussonnier wrote:
>>
>> This may even be a bug/feature of IPython,
>>
>> I see that inspect.signature(timedelta) fails, so if  timedelta? says
>> Init signature: timedelta(self, /, *args, **kwargs)
>> Then this may be some IPython internal logic. The timedelta class seem to
>> use __new__ instead of __init__ (not sure why)
>>
>> Because it's an immutable type.
>>
> Ah, yes, thanks.
>
>
>> and __new__ have a meaningful signature,
>> So maybe we should fallback on that during signature inspection.
>>
>> According to https://stackoverflow.com/questions/4374006/check-for-
>> mutability-in-python ,
>> there are no reliable tests for mutability.
>>
> Sure, but we can test if the signature of __init__ is (self,/, *args,
> **kwargs), and if it is,  it is useless we can attempt to get the signature
> from __new__ and show that instead.  We do similar things for docstrings,
> if __init__ have no docstring we look at the class level docstring.
> --
> M
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Docstrings on builtins

[Chris Barker via Python-Dev]

On Tue, Jun 5, 2018 at 8:01 AM, Ivan Pozdeev  wrote:

> In [5]: print(str.__doc__)
> str(object='') -> str
> str(bytes_or_buffer[, encoding[, errors]]) -> str
>
> Create a new string object from the given object. If encoding or
> errors is specified <...>
>
As you can see, the start of the type's docstring contains constructor
> signature(s).
>
> And iPython does the "right thing" here, too:

In [7]: str?
Init signature: str(self, /, *args, **kwargs)
Docstring:
str(object='') -> str
str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or
errors is specified, then the object must expose a data buffer
that will be decoded using the given encoding and error handler.
Otherwise, returns the result of object.__str__() (if defined)
or repr(object).
encoding defaults to sys.getdefaultencoding().
errors defaults to 'strict'.
Type:   type
>
> Timedelta's one should probably do the same.
>
OK, I've found the docstring in the source and will submit a PR.

-CHB





> -CHB
>
>
>
>
>
> On Mon, Jun 4, 2018 at 6:21 PM, Matthias Bussonnier <
> bussonniermatth...@gmail.com> wrote:
>
>>
>>
>> On Mon, 4 Jun 2018 at 17:29, Ivan Pozdeev via Python-Dev <
>> python-dev@python.org> wrote:
>>
>>> On 05.06.2018 3:09, Matthias Bussonnier wrote:
>>>
>>> This may even be a bug/feature of IPython,
>>>
>>> I see that inspect.signature(timedelta) fails, so if  timedelta? says
>>> Init signature: timedelta(self, /, *args, **kwargs)
>>> Then this may be some IPython internal logic. The timedelta class seem
>>> to use __new__ instead of __init__ (not sure why)
>>>
>>> Because it's an immutable type.
>>>
>> Ah, yes, thanks.
>>
>>
>>> and __new__ have a meaningful signature,
>>> So maybe we should fallback on that during signature inspection.
>>>
>>> According to https://stackoverflow.com/ques
>>> tions/4374006/check-for-mutability-in-python ,
>>> there are no reliable tests for mutability.
>>>
>> Sure, but we can test if the signature of __init__ is (self,/, *args,
>> **kwargs), and if it is,  it is useless we can attempt to get the signature
>> from __new__ and show that instead.  We do similar things for docstrings,
>> if __init__ have no docstring we look at the class level docstring.
>> --
>> M
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.
>> barker%40noaa.gov
>>
>>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE
> 
>   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
>
> --
> Regards,
> Ivan
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A more flexible task creation

[Chris Barker via Python-Dev]

Excuse my ignorance (or maybe it's a vocabulary thing), but I'm trying to
understand the problem here.

But if I have this right:

I've been using asyncio a lot lately and have encountered this problem
> several times. Imagine you want to do a lot of queries against a database,
> spawning 1 tasks in parallel will probably cause a lot of them to fail.
>

async is not parallel -- all the tasks will be run in the same thread
(Unless you explicitly spawn another thread), and only one task is running
at once, and the task switching happens when the task specifically releases
itself.

If it matters in what order the tasks are performed, then you should not be
using async.

So why do queries fail with 1 tasks? or ANY number? If the async DB
access code is written right, a given query should not "await" unless it is
in a safe state to do so.

So what am I missing here???

What you need in a task pool of sorts, to limit concurrency and do only 20
> requests in parallel.
>

still wrapping my head around the vocabulary, but async is not concurrent.

If we were doing this synchronously, we wouldn't spawn 1 threads using
> 1 connections,
>

and threads aren't synchronous -- but they are concurrent.


> we would use a thread pool with a limited number of threads and submit the
> jobs into its queue.
>

because threads ARE concurrent, and there is no advantage to having more
threads than can actually run at once, and having many more does cause
thread-switching performance issues.

To me, tasks are (somewhat) logically analogous to threads.
>

kinda -- in the sense that they are run (and completed) in arbitrary order,
But they are different, and that difference is key to this issue.

As Yury expressed interest in this idea, there must be something I'm
missing.

What is it?

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Fri, Jun 22, 2018 at 10:09 AM, Michael Selik  wrote:

> I forgot to add that I don't anticipate changing my lesson plans if this
> proposal is accepted. There's already not enough time to teach everything
> I'd like. Including a new assignment operator would distract from the
> learning objectives.
>

nor would I. For a while, anyway

But once it becomes a more common idiom, students will see it in the wild
pretty early in their path to learning python. So we'll need to start
introducing it earlier than later.

I think this reflects that the "smaller" a language is, the easier it is to
learn.

Python has already grown a fair bit since 1.5 (when I started using it :-)
). Some things, like generators, are special purpose enough that I can wait
pretty far into the program before teaching them. But others, like
comprehensions (and lambda) are common enough that I have to introduce them
pretty early on.

Adding := is not a HUGE change, but it IS an expansion of the language, and
one that we WILL have to introduce in an introductory course once it starts
seeing common use.

I really have no idea how much harder thats going to make the langauge to
teach, but it will make it a bit harder -- I see enough confusion with "is"
vs == already...

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Fri, Jun 22, 2018 at 7:23 PM, Steven D'Aprano 
wrote:

> > But once it becomes a more common idiom, students will see it in the wild
> > pretty early in their path to learning python. So we'll need to start
> > introducing it earlier than later.
>
> Students see many features early in their path. I've had people still
> struggling with writing functions ask about metaclasses. People
> will see async code everywhere. We don't have to teach *everything* at
> once.
>

These are not similar at all -- if you want similar examples, I"d say
comprehensions, and lambda, both of which I DO introduce fairly early

While newbies will *ask* about metaclasses, it's probably because they read
about them somewhere, not because someone actually used a metaclass in a
simple script or answer to a common question on SO.

As for async, you are either doing async or not -- you can't even run an
async def function without an event loop -- so again, it won't show up in
real code newbies need to understand (at least until async becomes common
practice with python...)

-CHB

 So if you absolutely need to teach it to a beginner, it
> shouldn't be difficult once they understand the difference between an
> expression and a statement.
>

probably not, though that's a distinction that's mostly academic in the
early stages of learning, it may become more critical now...

again, not a huge deal, just a little bit more complexity

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Wed, Jun 27, 2018 at 12:01 PM, Eric V. Smith  wrote:

> >>> def test():
> >>>spam = 1
> >>>ham = 2
> >>>vars = [key1+key2 for key1 in locals() for key2 in locals()]
> >>>return vars
> >>>
> >>> Wanna guess what that's gonna return?
>

Guessing aside, messing around with locals() isn't really helpful for the
usual case of common code.

But the real problem I see with all of this the distinction between
generator expressions and comprehensions:

"""
an assignment expression occurring in a list, set or dict comprehension or
in a generator expression (below collectively referred to as
"comprehensions") binds the target in the containing scope, honoring a
nonlocal or global declaration for the target in that scope, if one exists.
For the purpose of this rule the containing scope of a nested comprehension
is the scope that contains the outermost comprehension. A lambda counts as
a containing scope.
"""

It seems everyone agrees that scoping rules should be the same for
generator expressions and comprehensions, which is a good reason for
python3's non-leaking comprehensions:

(py2)

In [*5*]: i = 0


In [*6*]: l = [i *for* i *in* range(3)]


In [*7*]: i

Out[*7*]: 2


In [*8*]: i = 0


In [*9*]: g = (i *for* i *in* range(3))


In [*10*]: i

Out[*10*]: 0


In [*11*]: *for* j *in* g:

...: *pass*

...:


In [*12*]: i

Out[*12*]: 0

so comprehensions and generator expressions behave differently -- not great.

(py3)


In [*4*]: i = 0


In [*5*]: l = [i *for* i *in* range(3)]


In [*6*]: i

Out[*6*]: 0


In [*7*]: g = (i *for* i *in* range(3))


In [*8*]: i

Out[*8*]: 0


In [*9*]: list(g)

Out[*9*]: [0, 1, 2]


In [*10*]: i

Out[*10*]: 0

The loop name doesn't "leak" and comprehensions and generator expressions
are the same this regard -- nice.

So what about:

l = [x:=i for i in range(3)]

vs

g = (x:=i for i in range(3))

Is there any way to keep these consistent if the "x" is in the regular
local scope?

Note that this thread is titled "Informal educator feedback on PEP 572".

As an educator -- this is looking harder an harder to explain to newbies...

Though easier if any assignments made in a "comprehension" don't "leak out".

Which does not mean that we'd need a "proper" new local scope (i.e.
locals() returning something new) -- as long as the common usage was
"intuitive".

>> I'm not singling out Chris here, but these discussions would be easier
> >> to follow and more illuminating if the answers to such puzzles were
> >> presented when they're posed.
>

well, I think the point there was that it wasn't obvious without running
the code -- and that point is made regardless of the answer.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Thu, Jun 28, 2018 at 9:28 AM, Tim Peters  wrote:

> >>> g = (x:=i for i in range(3))
> Common or not, I have no idea why anyone would write a genexp like the one
> you gave, except to contrive an example of silly behavior exhibited by
> silly code ;-)
>

yes, it was a contrived example, but the simplest one I could think of off
the top of my head that re-bound a name in the loop -- which was what I
thought was the entire point of this discussion?

If we think hardly anyone is ever going to do that -- then I guess it
doesn't matter how it's handled.

> So while this is technically the same as the comprehension, it is not
> > the same as a generator function which does get its own scope.
>
> It is the same as a generator function with appropriate scope declarations
> - a generator expression is, after all, implemented _by_ a nested generator
> function.  You can write a workalike to your code above today, but nobody
> worries about that because nobody does that ;-)
>
> def f():
> def bashx(outermost):
> nonlocal x
> for i in outermost:
> x = i
> yield i
>

but here the keyword "nonlocal" is used -- you are clearly declaring that
you are messing with a nonlocal name here -- that is a lot more obvious
than simply using a :=

And "nonlocal" is not used that often, and when it is it's for careful
closure trickery -- I'm guessing := will be far more common. And, of
course, when a newbie encounters it, they can google it and see what it
means -- far different that seeing a := in a comprehension and
understanding (by osmosis??) that it might make changes in the local scope.

And  I don't think you can even do that with generator expressions now --
as they can only contain expressions. Which is my point -- this would allow
the local namespace to be manipulated in places it never could before.

Maybe it's only comprehensions, and maybe it'll be rare to have a confusing
version of those, so it'll be no big deal, but this thread started talking
about educators' take on this -- and as an educator, I think this really
does complicate the language.

Python got much of it's "fame" by being "executable pseudo code" -- its
been moving farther and farther away from those roots. That's generally a
good thing, as we've gain expressiveness in exchangel, but we shouldn't
pretend it isn't happening, or that this proposal doesn't contribute to
that trend.

Did adding ternary `if` (truepart if expression else falsepart) complicate
> the language significantly?
>

I don't think so -- no. For two reasons:

1) the final chosen form is kind of verbose, but therefor more like
"executable pseudo code" :-) As apposed to the C version, for instance.

2) it added one new construct, that if, when someone sees it for the first
(or twenty fifth) time and doesn't understand it, they can look it up, and
find out. and it only effects that line of code.

So adding ANYTHING does complicate the language, by simply making it a bit
larger, but some things are far more complicating than others.

Python has rarely expanded the number of expression forms, but whenever it
> has the sky didn't actually fall despite earnest warnings that disaster was
> inevitable ;-)
>

Well, I've been surprised by what confused students before, and I will
again. But I dont hink there is any doubt that Python 3.7 is a notably
harder to learn that Python 1.5 was...


> > Maybe it’s just me, but re-binding a name seems like a whole new
> > category of side effect.
>
> With no trickery at all, you've always been able to rebind attributes, and
> mutate containers, in comprehensions and genexps.  Because `for` targets
> aren't limited to plain names; e.g.,
>
> g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))
>

sure, but you are explicitly using the names "object" and "a" here -- so
while side effects in comprehension are discouraged, it's not really a
surprised that namespaces specifically named are changed.

and this:

In [*55*]: x = 0


In [*56*]: [x *for* x *in* range(3)]

Out[*56*]: [0, 1, 2]


In [*57*]: x

Out[*57*]: 0

doesn't change x in the local scope -- if that was a good idea, why is a
good idea to have := in a comprehension effect the local scope??

But maybe it is just me.


> And as in my goofy code above, mucking with binding of plain names is also
> possible today.  Indeed, straightforward if that's what you _want_ to do.
> But nobody does.
>
> It's just not one of Python's goals to make it impossible to write useless
> code ;-)
>

 I suppose we need to go back and look at the "real" examples of where/how
folks think they'll use := in comprehensions, and see how confusing it may
be.

One of these conversations was started with an example something like this:

[(f(x), g(f(x))) for x in an_iterable]

The OP didn't like having to call f() twice. So that would become:

[ (temp:=f(x), g(temp)) for x in an_iterable]

so now the question is: should "temp" be created / changed i

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Sun, Jul 1, 2018 at 8:35 AM, Michael Selik  wrote:

> As Mark and Chris said (quoting Mark below), this is just one straw in the
> struggle against piling too many things on the haystack. Unlike some
> changes to the language, this change of such general use that it won't be
> an optional topic. Once widely used, it ain't optional.
>

Exactly -- and I also emphasis that this would complicate the language in a
broad way -- a much bigger deal than adding a self contained new expression
or even the nonlocal keyword.

But to be clear about my take -- it will make the language that little bit
harder to teach, but it will also add a bit of complexity that will effect
us non-newbies as well.

Is it worth it? maybe. I know I like a better way to express the
loop-and-a-half concept in particular.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Sun, Jul 1, 2018 at 5:25 PM, Steven D'Aprano  wrote:

>
> > an optional topic. Once widely used, it ain't optional.
>
> Without knowing the details of your course, and who they are aimed at,
> we cannot possibly judge this comment. Decorators are widely used, but
> surely you don't teach them in a one day introductory class aimed at
> beginners?
>
> Here is the syllabus for a ten week course:
>
> https://canvas.uw.edu/courses/1026775/pages/python-100-course-syllabus
>

That would be mine ;-)


> Note that decorators and even regular expressions don't get touched
> until week ten.


I actually often don't ever get to regex -- why? because they are a topic
all of their own, and not unique to Python. And if code uses a regex, it
doesn't change anything about any other code anywhere. IN sort, regex's are
a library, not a language feature.

Which brings us to decorators (and I'm going to add context managers, as
its similar). Decorators (and even more so context managers) are used
commonly enough that we certainly have to introduce them in a intro class.
But there is a really big distinction between having some idea how to use
them, and knowing how they work / how to write them yourself.

So I introduce:

with open(filename) as the_file:
do_somethign_with(the_file)

early on, with only a hand-wavy explanation of what that with is all about.

And when i get to properties, I teach them:

@property
def a_method(self):


with also a hand-wavy explanation of what that @ symbol is.

and if := catches on, I expect it will be far more common that either of
those, especially in the simple script style codes that newbies are going
to be reading/writing at first.

But in the end, I, at least, am not trying to make the case that assignment
expressions are going to be particularly difficult to understand, but that
they are a significant complication, and any new feature like that makes
the language more complex. That's it.

I taught myself Python with version 1.5 in 1998 -- and have been teaching
it for I think almost ten years. In that time, the language has grown
substantially in complexity, and it does make it harder to teach.

We (UWPCE) recently revamped our 3-class program, and had a lot of debate
about how early to introduce things -- one of the instructors wanted to
leave off lambda and comprehensions 'till the second course in the
sequence, as part of functional programming. I I think he was right, if we
were teaching it in a more isolated environment, but both of those show up
in example code all over the place, so I've found it's necessary to
introduce a lot at a high level early:

decorators
comprehensions
lambda
context managers
*args and **kwargs
(off the top of my head)

So there is basically no way that we won't have to add assignment
expressions early on.

As someone else said: it's another straw on the haystack.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Mon, Jul 2, 2018 at 11:42 PM, Tim Peters  wrote:


> "comprehensions" was always a dubious term, carried over from set theory
> where the term focuses on the optional "if" part rather than the more
> fundamental iterator or computation parts.
>

I always wondered about that :-) -- I'd say for most of us that aren't
familiar with set theory, it's kind of a "sounds something like putting
thing together" word and I just left it at that, and learned what they are.


> So if we had it to do over again I'd sigh and accept "generator
> comprehensions" anyway.  It's been an eternal PITA - and especially in the
> PEP 572 threads! - to keep typing "comprehensions or generator
> expressions".
>

Well, too late to change the official name, but not too late to start using
the term in threads like these -- and other documentation, etc

I find there is a lot of confusion about the word "generator", as it
implies a "thing that generates values on the fly" (like, say the range()
object.

But then, in Python, a generator is something that gets crated by a
generator function, and CAN be an "thing (iterator) that generates things
on the fly", but can also be a more generic coroutine, and can be used in
nifty ways that really have nothing to do with generating a bunch of value.
(like pytest fixtures, for example)

So we have generators, iterators, and iterables, and generators can be
iterators, but aren't always, and any number of iterators can generate
values on the fly, and 

so it's all a bit of a mess to explain to a newbie.

Then again, if I had the power of Guido's time machine, I'd go back more,
> and not use "comprehensions" for anything to begin with.  Instead we'd have
> list, dict, set, and generator twizzlers, affectionately called listwiz,
> dictwiz, setwiz, and gentwiz by the cool kids :-)
>

I'd like that!

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Examples for PEP 572

[Chris Barker via Python-Dev]

On Tue, Jul 3, 2018 at 2:51 PM, Chris Angelico  wrote:

> On Wed, Jul 4, 2018 at 7:37 AM, Serhiy Storchaka 
> wrote:
> > I believe most Python users are not
> > professional programmers -- they are sysadmins, scientists, hobbyists and
> > kids --
>
> [citation needed]


fair enough, but I think we all agree that *many*, if not most, Python
users are "not professional programmers". While on the other hand everyone
involved in discussion on python-dev and python-ideas is a serious (If not
"professional") programmer.

So we do have a bit of a disconnect between much of the user base and folks
making decisions about how the language evolves -- which is probably
inevitable,

Over the years I've been using it (most of its life), Python has evolved to
become much less of a "scripting" language, and much more of a "systems"
language, and this addition is a (pretty significant) step more in that
direction.

Serhiy: FWIW, a number of us made the case about the additional complexity
of this feature in this discussion -- I think it was a bit side-tracked by
the impression that we were only talking about newbies (and also by scope
of comprehensions), but the point was made and I assume considered.


> > In particularly mutating and
> > non-mutating operations are separated. The assignment expression breaks
> > this.
>

I'd call it "local namespace manipulating and non-local namespace
manipulating", but yes, that is my concern as well.

In terms of blending mutating and non-mutating operations, augmented
> assignment is far worse.


yeah -- I've always thought it was a shame that augmented assignment worked
on immutables :-(

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Examples for PEP 572

[Chris Barker via Python-Dev]

On Wed, Jul 4, 2018 at 7:20 AM, David Mertz  wrote:

>
> That said, this is a silly game either way.  And even though you CAN
> (sometimes) bind in an expression pre-572, that's one of those perverse
> corners that one shouldn't actually use.
>

not only shouldn't by hardly anyone ever does / would.

A lot of the argument for this feature is that it doesn't really let you do
things you couldn't do before (like manipulate the local namespace in side
comprehensions, or the nonocal one, or...)

But this IS a silly game -- Python is highly dynamic, you can do all sorts
of metaprogamming tricks. I'm pretty sure you can even alter the byte code
in a running interpreter (but only pretty sure, 'cause why would I ever try
to do that?)

But the point is not that you can do tricky namespace manipulations now,
it's that you need to do advanced (and obvious) thinks like call locals()
or globals() or use nonlocal, or...

I don't think these concerns have been ignored, but I also don't think that
I've heard anyone on the pro side say something along the line of:

"Yes, this does add a significant complication to the language, but we
think it will be unlikely to be mis-used in confusing ways, and that
complication is worth it."

Rather, I've heard a  lot of "but you can already do that" or "but we added
[ternary expressions, augmented assignment, f-strings, ...]" And none of
those add *complication* to the language itself -- they add one more
feature that needs to be looked up when you encounter it -- but only effect
the line of code where there are used.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Naming comprehension syntax [was Re: Informal educator feedback on PEP 572 ...]

[Chris Barker via Python-Dev]

On Mon, Jul 9, 2018 at 3:18 PM, Guido van Rossum  wrote:

> Definitely docs first. And we should keep references to generator
> expressions too, if only to explain that they've been renamed.
>
> Perhaps someone should try it out in a 3rd party tutorial to see how it
> goes?
>

I'm not sure what "trying it out in a tutorial" would look like.

I try to be pretty clear about terminology when I teach newbies -- so I
don't want to tell anyone this new thing is called a "generator
comprehension" if they aren't going to see that term anywhere else.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Naming comprehension syntax [was Re: Informal educator feedback on PEP 572 ...]

[Chris Barker via Python-Dev]

Thanks Nick,

I'll adopt this approach when I update my teaching materials.

If I think of it, I"ll post here when I do that

-CHB


On Sun, Jul 15, 2018 at 12:21 AM, Nick Coghlan  wrote:

> On 13 July 2018 at 15:30, Chris Barker via Python-Dev
>  wrote:
> > On Mon, Jul 9, 2018 at 3:18 PM, Guido van Rossum 
> wrote:
> >>
> >> Definitely docs first. And we should keep references to generator
> >> expressions too, if only to explain that they've been renamed.
> >>
> >> Perhaps someone should try it out in a 3rd party tutorial to see how it
> >> goes?
> >
> >
> > I'm not sure what "trying it out in a tutorial" would look like.
> >
> > I try to be pretty clear about terminology when I teach newbies -- so I
> > don't want to tell anyone this new thing is called a "generator
> > comprehension" if they aren't going to see that term anywhere else.
>
> Nina Zakharenko made the "they're officially called generator
> expressions, but I find it more helpful to think of them as generator
> comprehensions" observation in her PyCon 2018 presentation on "Elegant
> Solutions for Everyday Python Problems":
> https://www.slideshare.net/nnja/elegant-solutions-for-
> everyday-python-problems-pycon-2018/27
>
> The article from Ned Batchelder mentioned in that slide is this one,
> which goes through the history of Raymond originally proposing the
> notion as generator comprehensions, them getting changed to generator
> expressions during the PEP 289 discussions, and then asking if it
> might be worth going back to the originally suggested name:
> https://nedbatchelder.com/blog/201605/generator_comprehensions.html
>
> And then in PEP 572, it was found that being able to group all 4
> constructs (list/set/dict comps + generator expressions) under a
> single term was a genuinely useful shorthand:
> https://www.python.org/dev/peps/pep-0572/#scope-of-the-target
>
> So trying out the terminology in a tutorial context would be to do
> something similar to what Nina did in her talk: introduce the notion
> of list/set/dict/generator comprehensions, and then make a side note
> that the latter are officially referred to as "generator expressions".
>
> This wouldn't be the first time that terminology has differed between
> Python-as-commonly-taught and Python-as-formally-defined, as I've yet
> to hear anyone refer to container displays outside a language design
> discussion - everyone else calls them container literals (or, more
> typically, a literal for the specific builtin container type being
> discussed).
>
> In this case, though, we'd be considering eventually changing the
> language reference as well, and perhaps even some day the AST node
> name (from GeneratorExp to GeneratorComp).
>
> We wouldn't need to change anything in the grammar definition (since
> that already shares the comp_for and comp_if syntax definitions
> between container comprehensions and generator expressions), or the
> AST node structure (since GeneratorExp already uses a "comprehensions"
> attribute, the same as the ListComp/SetComp/DictComp nodes).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Const access to CPython objects outside of GIL?

[Chris Barker via Python-Dev]

On Tue, Jul 17, 2018 at 4:34 AM, Victor Stinner  wrote:

>
> IMHO you need a different approach to implement optimizations. For
> example, use your objects which don't rely on the GIL to be
> consistent. Sadly, I have no experience with that and so cannot
> provide any example.
>

I DO NOT understand the technical details, but IIUC, numpy releases the GIL
while doing pure numpy operations.

(and I have experimented with it, and it does seem to be the case -- that
is, you can get enhanced performance with computationally heavy numpy code
an multiple threads)

So maybe some lessons learned there.

The key differences may be that numpy arrays are full-fledged python
objects, but they are not (generally) containers of python objects -- that
is, the actual values are i memory accessed via a C pointer. Whereas a
PyList holds pointers to Python objects, so even if are confident in the
stability of the list itself, the objects in the list may not be stable.

 For example, os.stat() writes into a memory block allocated
> by Python. But it's a raw memory block, it's much simpler than a
> Python object like a tuple.


exactly. -- so is a numpy .data element

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572 and assert

[Chris Barker via Python-Dev]

>
> I don't see why. As Chris said, side effects in asserts are nothing new
> and
>

Indeed -- this new feature makes it easier to affect the local scope in all
sorts of new places. It was decided that the additional complexity is worth
it to make the language more expressive, and it was also decided not to try
to "lock down" this new feature to only "appropriate" places.

> this PEP is not the one to do something about it.

Hmm --- not sure if it's PEP-worthy, but I'd sure love to see the community
make a strong stance that:

asserts are for tests, and only for tests

Which doesn't necessarily mean that assert statements only exist in test
code (though I personally follow that practice), but it does mean that
assert statements, wherever they live, should not be counted on at run
time, ever.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Error message for wrong number of arguments

[Chris Barker via Python-Dev]

On Mon, Jul 30, 2018 at 5:12 AM, Jeroen Demeyer  wrote:

> I think it has been argued before that it's a feature that self is
> counted.


I suppose it is, as it's technically correct, but it's also a HUGE source
of confusion, particularly for newbies. IF this is being touched anyway, is
it possible for the interpreter to know when this error is generated that
this is a bound method expecting a "self", rather than an arbitrary
function with n parameters? In which case, it would be really nice if the
error message replaced that somehow, maybe something like:

>>> g(1,2)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: append() takes 1 positional argument in addition to the
automatically added instance object --  but 2 were given in addition to the
object instance.

Man -- hard to come up with good wording for that -- but SOMETHING that
lets users know what they actually did wrong would be good :-)

If it's not do-able, then still +1 on making builtins consistent.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Error message for wrong number of arguments

[Chris Barker via Python-Dev]

On Mon, Jul 30, 2018 at 11:39 AM, Jeroen Demeyer  wrote:

> On 2018-07-30 20:22, Chris Barker wrote:
>
>> is it possible for the interpreter to know when this error is
>> generated that this is a bound method expecting a "self", rather than an
>> arbitrary function with n parameters?
>>
>
> That would be quite hard. The error message is generated by the underlying
> function. At that point, the information of how it was called (as bound
> method or not) is already gone.


Thanks, I figured as much.

Oh well. This is a serious usability issue -- but what can you do?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Error message for wrong number of arguments

[Chris Barker via Python-Dev]

On Wed, Aug 1, 2018 at 2:40 PM, Armin Rigo  wrote:

> On 30 July 2018 at 22:19, Chris Barker via Python-Dev
>  wrote:
> > Oh well. This is a serious usability issue -- but what can you do?
>
> I think that argument clinic knows if the built-in function is
> supposed to be a method or a function.  It doesn't look too hard to
> add a new flag METH_IS_METHOD or something, which would be taken in
> consideration in the common cases, and which can be added manually OR
> used automatically by argument clinic.  This would not be a 100%
> solution out of the box, but if the new wording is right, it shouldn't
> be a problem.
>

But can you do the same thing with pure-python methods?

After all, this thread started with trying ot unify what error folks get
regardless of how the method was written.


-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pre- and post-conditions

[Chris Barker via Python-Dev]

This is an appropriate topic for pyton-ideas:

https://mail.python.org/mailman/listinfo/python-ideas

not python-dev.

I'm sure you'll find interest in your idea there.

-CHB


On Wed, Aug 15, 2018 at 1:25 AM, Marko Ristin-Kaufmann <
marko.ris...@gmail.com> wrote:

> Hi python devs,
>
> I would be very interested to bring design-by-contract into python 3. I
> looked at some of the packages I found on pypi and also we rolled our own
> solution (https://github.com/Parquery/icontract/). I also looked into
> https://www.python.org/dev/peps/pep-0316/.
>
> However, all these solutions seem quite clunky to me. The decorators
> involve an unnecessary computational overhead and the implementation of
> icontract became quite tricky once we wanted to get the default values of
> the decorated function.
>
> What do you think about the following solution as an extension to python
> compiler / interpreter?
>
> * We specify pre- and post-conditions in the docstring.
> * Python interpreter parses these conditions and adapts the corresponding
> byte codes at the begining and the end of the function body.
>
> I'm very grateful for any feedback on this!
>
> Marko
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue?

[Chris Barker via Python-Dev]

python used the "timsort" sorting routine:

https://en.wikipedia.org/wiki/Timsort

So you can look at that and confirm that this is correct behaviour (I'm
betting it is :-)

But in general, sorting is O(n log(n)) -- there are going to be more than n
comparisons.

If comparing is slow, you want to use a key function, to reduce your
comparison to a simple and fast one:

sorted(L, key=lambda i: (i.name, i.score))

or something like that.

personally, I advocate adding a "key_fun" attribute to classes you want to
make efficiently sortable, so you'd have:

sorted(L, key=Student.key_fun)

There was a discussion on python-ideas about adding a __sort_key__ protocol
to python, but there were too many downsides.

-CHB




On Wed, Aug 22, 2018 at 3:40 AM, 楼晓峰 <1520306...@qq.com> wrote:

>
> *Why compare twice?*
>
> class Student(object):
>
> def __init__(self, name, score):
> self.name = name
> self.score = score
>
> def __str__(self):
> return '(%s: %s)' % (self.name, self.score)
>
> __repr__ = __str__
>
> def __lt__(self, s):
> #print(self, '--', s)
> if(self.score print(self, '<', s)
> return True
> if(self.score>s.score):
> print(self, '>', s)
> return False
> if(self.score==s.score):
> if(self.name>s.name):
> print(self, '>', s)
> return False
> if(self.name print(self, '<', s)
> return True
> if(self.name==s.name):
> print(self, '==', s)
> return False
>
> def __eq__(self, s):
> return (self.score == s.score) and (self.name == s.name)
> def __gt__(self, s):
> return not ((self == s) or (self < s))
> def __le__(self, s):
> return ((self == s) or (self < s))
> def __ge__(self, s):
> return ((self == s) or (self > s))
> def __nq__(self, s):
> return not (self == s)
>
> L = [Student('Tim', 22), Student('Bob', 33), Student('Kevin', 11),
> Student('Alice', 11)]
> print(sorted(L))
>
> Output:
> (Bob: 33) > (Tim: 22)
> *(Kevin: 11) < (Bob: 33)*
> *(Kevin: 11) < (Bob: 33)*
> (Kevin: 11) < (Tim: 22)
> (Alice: 11) < (Tim: 22)
> (Alice: 11) < (Kevin: 11)
> [(Alice: 11), (Kevin: 11), (Tim: 22), (Bob: 33)]
>
> *Best regards,*
> *Xiaofeng*
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Chris Barker via Python-Dev]

I"d like ot know what thee citations are expected to be used for?

i.e. -- usually, academic papers have a collection of citiations to
acknowledge where you got an idea, or fact, or  It serves both to
jusstify something and make it clear that it is not your own idea (i.e. not
pagerism).

in the enclosed doc, it says:

"""
*If someone publishes research, they will cite the exact major version that
was used, so if someone was trying to recreate the research they would be
able to do it.*
*"""*
That is about reproducible results, which is really a different thing than
the usual citations. In that case, you would want some way to identify the
actual source code (cPython version 3.6.4, and probably a url to the source
-- but how long might that last???) And you would need to post your own
source anyway.

Again, regular citation is about acknowledging the source of an idea or
fact, or something of that nature. I can imagine a paper about computer
language design or some such might need to reference Python -- in which
case it should reference the Language Reference, I suppose. Maybe we should
have a DOI for each version of the standard docs.

But "I wrote some code in Python to produce these statistics" -- does that
need a citation? If so, maybe that would take a different form.

Anyway, hard to make this decision without some idea how the citation is
intended to be used.

-CHB




On Mon, Sep 10, 2018 at 9:51 AM, Gerald Klix  wrote:

> Wouldn't it make sense to ask the developers of the other Python
> implementations too?
>
> Just my 0.02€,
>
>
> Gerald
>
>
>
> Am 09.09.2018 um 21:43 schrieb Jacqueline Kazil:
>
>> The PSF has received a few inquiries asking the question — “How do I cite
>> Python?”So, I am reaching out to you all to figure this out.
>>
>> (For those that don’t know my background, I have been in academia for a
>> bit
>> as a Ph.D student and have worked at the Library of Congress writing code
>> to process Marc records , among
>> other things.)
>>
>> IMHO the citation for Python should be decided upon by the Python
>> developers and should live somewhere on the site.
>>
>> Two questions to be answered…
>>
>> 1. What format should it take?
>> 2. Where does it live on the site?
>>
>> To help frame the first one, I quickly wrote this up —
>> https://docs.google.com/document/d/1R0mo8EYVIPNkmNBImpcZTbk0
>> e78T2oU71ioX5NvVTvY/edit#
>>
>> tldr; Summary of possibilities…
>>
>> 1. Article for one citation (1 DOI, generated by the publication)
>> 2. No article (many DOIs — one for each major version through Zenodo
>>  (or similar service))
>>
>> Discuss.
>>
>> -Jackie
>>
>> Jackie Kazil
>> Board of Directors, PSF
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/python.
>> 00%40klix.ch
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.
> barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Chris Barker via Python-Dev]

On Tue, Sep 11, 2018 at 3:48 AM, Steven D'Aprano 
wrote:

> > That is about reproducible results, which is really a different thing
> than
> > the usual citations.
>
> I don't think it is. I think you are seeing a distinction that is not
> there.


no need for us to agree on that, but there are still multiple reasons /
ways you might want to cite Python, and what you would want to cite would
be different.

Lets say one were to write an article about how different computer
languages express functional programming concepts -- you may want to cite
Python, but you are not trying to identify a specific version for
reproducible results.

And see Wes Turner's note -- it is highly unlikely that a single citation
to a standard document or something will be enough for reproducibility
anyway.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Chris Barker via Python-Dev]

On Tue, Sep 11, 2018 at 2:45 PM, Steven D'Aprano 
wrote:

> I think this thread is about *academic* citations.


yes, I assumed that as well, what in any of my posts made you think
otherwise?


> There's a metric ton of information on the web about citing software,
> there are existing standards, and I really think you are
> over-complicating this. See, for example:
>
> https://www.software.ac.uk/how-cite-software
>
> http://www.citethisforme.com/cite/software
>
> https://openresearchsoftware.metajnl.com/about/#q12


The fact that those posts exist demonstrates that this is anything but a
solved problem.

Its not our job to tell academics how to cite, they already have a
> number of standardized templates that they use, but it is our job to
> tell them what information to fill into the template.
>

yes, of course -- I don't know why this thread got sidetracked into
citation formats, that has nothing to do with it. Or as the op said, that's
"the easy part"

> Lets say one were to write an article about how different computer
> > languages express functional programming concepts -- you may want to cite
> > Python, but you are not trying to identify a specific version for
> > reproducible results.
>
> I don't think we need to lose any sleep over how random bloggers and
> Redditors informally cite Python.


Why in the world would you think "article" meant random bloggers? In
BiBTex, for instance, a paper in a peer reviewed journal is called an
"article", as apposed to a book, or chapter, or inproceedings, or
techreport, or As this whole thread is about academic citations, I
assumed that...

I think the focus here is on academic
> citations, which have rather precise and standard requirements.


not for software, yet.


> No need
> to expand the scope of this problem to arbitrary mentions of Python.
>

I was not expanding it -- I was hoping to contract it -- or at least better
define it.


> Of course it is possible that I've completely misunderstood Jackie's
> request. If so, hopefully she will speak up soon.


I think we're all on the same page about that, actually.

My point, to be more pedantic about it, is that an academic paper might be
*about* Python in some way, or it might describe work that *used* Python as
a tool to accomplish some other understanding. These *may* require a
different citation.

And a citation that satisfies academic criteria for using Python may not be
enough to assure reproducible results.

> And see Wes Turner's note -- it is highly unlikely that a single citation
> > to a standard document or something will be enough for reproducibility
> > anyway.
>
> The academic community seems to think that it is. We don't have to tell
> them that they're wrong.


The Academic community has a really bad track record with reproducible
results for computationally based research -- it is not a solved problem.

And it's not a "they" -- many of us on this list are part of the academic
community.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Chris Barker via Python-Dev]

Thanks Wes.

"""
> Python
>
> Guido van Rossum: Scripting the Web with Python. In "Scripting Languages:
> Automating the Web", World Wide Web Journal, Volume 2, Issue 2, Spring
> 1997, O'Reilly.
>
> Aaron Watters, Guido van Rossum, James C. Ahlstrom: Internet Programming
> with Python. MIS Press/Henry Holt publishers, New York, 1996.
>
> Guido van Rossum: Python Library Reference. May 1995. CWI Report CS-R9524.
>
> Guido van Rossum: Python Reference Manual. May 1995. CWI Report CS-R9525.
>
> Guido van Rossum: Python Tutorial. May 1995. CWI Report CS-R9526.
>
> Guido van Rossum: Extending and Embedding the Python Interpreter. May
> 1995. CWI Report CS-R9527.
>
> Guido van Rossum, Jelke de Boer: Linking a Stub Generator (AIL) to a
> Prototyping Language (Python). Spring 1991 EurOpen Conference Proceedings
> (May 20-24, 1991) Tromso, Norway.
> """
>

Of these, I think the Python Reference Manual is the only one that comes
close as a general citation for the langue itself. And it would be a
particular version, presumably. But those old versions are published as
tech reports by a know institution -- easier to cite than teh PSF.

I've seen folks cite various academic articles to satisfy a citation for a
language or library, but often that is simply because that is the one thing
they could find that is citable in the standard way.

sure -- though I don't think "article" is the correct catagory -- probbaly
technical report.

Looking at the current "official" docs, the copyright is held by the PSF,
but I see no author mentioned (I recall it said Fred Drake many years
back...) I take that back -- the PDF version has an author.

So I'm thinking maybe:

@techreport{techreport,
  author   = {Guido van Rossum and the Python development team},
  title= {Python Language Reference, release 3.7.0},
  institution  = {Python Software Foundation},
  year = 2018,
  address  = {9450 SW Gemini Dr. ECM# 90772, Beaverton, OR 97008 USA},
  number   = 3.7,
  url .= {https://docs.python.org/release/3.7.0/},
}

I used "Guido van Rossum and the Python development team" as the Author, as
that's what it says on the PDF version. Not sure how bibliography software
will deal with that...

And what do we do with version? part of the title, or (ab)use number (Or
volume, or )

And it would be better to cite the entire standard docs, rather than just
the language reference, but I"m not sure what to call it -- there is no
single document with a title.

And a DOI for each release would be nice.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Chris Barker via Python-Dev]

>
> On Saturday, September 15, 2018, Jacqueline Kazil 
> wrote:
>
>> I just got caught up on the thread. This is a really great discussion.
>> Thank you for all the contributions.
>>
>> Before we get into the details, let's go back to the main use case we are
>> trying to solve.
>> *As a user, I am writing an academic paper and I need to cite Python. *
>>
>
ai'd still like to know *why* you need to cite python 0 I can imagine
multiple reasons, and that may influence the best document to cite.


> Let's throw reproducibility out the window for now (<--- something I never
>> thought I would say), because that should be captured in the code, not in
>> the citations.
>>
>
thanks for that clarification.


> So, if we don't need the specific version of Python, then maybe creating
>> one citation is all we need.
>>
>
well, Python does evolve over time, so depending on why you are citing it,
version may matter.

But i suggest hat the language reference be used as the "primary" citation
for Python, and then you can cite the version that is current at the time
of your paper writing (Or the version that's relevant to your paper).

And that gives it some good Google juice as well.
>>
>
> https://scholar.google.com/scholar?hl=en&q=python+van+Rossum *
>

looks like the language reference shows there -- so good to go.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Arbitrary non-identifier string keys when using **kwargs

[Chris Barker via Python-Dev]

On Fri, Oct 5, 2018 at 3:01 PM Brett Cannon  wrote:

> I'm also fine with saying that keys in **kwargs that are not proper
> identifiers is an implementation detail.
>

It's not just **kwargs -- you can also use arbitrary names with setattr() /
getattr() :

In [6]: setattr(foo, "4 not an identifier", "this works")

In [7]: getattr(foo, "4 not an identifier")
Out[7]: 'this works'

Which brings up a question I've had for years -- is the fact that cPython
uses a regular old dict for namespaces (and **kwargs) part of the language
spec, or an implementation detail?

I would say that for the get/setattr() example, it is kinda handy when you
want to use a class instance to model some external data structure that may
have different identifier rules. Though I tend to think that's mingling
data and code too much.

-CHB


>
> On Thu, 4 Oct 2018 at 02:20, Serhiy Storchaka  wrote:
>
>> 04.10.18 11:56, Steven D'Aprano пише:
>> > While keyword arguments have to be identifiers, using **kwargs allows
>> > arbitrary strings which aren't identifiers:
>> >
>> > py> def spam(**kwargs):
>> >  print(kwargs)
>> > 
>> > py> spam(**{"something arbitrary": 1, '\n': 2})
>> > {'something arbitrary': 1, '\n': 2}
>> >
>> >
>> > There is some discussion on Python-Ideas on whether or not that
>> > behaviour ought to be considered a language feature, an accident of
>> > implementation, or a bug.
>> >
>> > Can we get some guidence on this please?
>>
>> This is an implementation detail. Currently CPython doesn't ensure that
>> keyword argument names are identifiers for performance reasons. But this
>> can be changed in future versions or in other implementations.
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Arbitrary non-identifier string keys when using **kwargs

[Chris Barker via Python-Dev]

On Sun, Oct 7, 2018 at 11:42 AM João Santos  wrote:

> *locals *and *globals* are documented as dictionaries (for example exec's
> documentation states that " If only *globals* is provided, it must be a
> dictionary")
>

well, that is specifically about exec() -- it may or may not apply to
everywhere nameapaces are used in the interpreter...


> but __dict__ is described as " [a] dictionary or other mapping object".
>

exactly.

-CHB

On Sun, 7 Oct 2018 at 19:38, Chris Barker via Python-Dev <
> python-dev@python.org> wrote:
>
>> On Fri, Oct 5, 2018 at 3:01 PM Brett Cannon  wrote:
>>
>>> I'm also fine with saying that keys in **kwargs that are not proper
>>> identifiers is an implementation detail.
>>>
>>
>> It's not just **kwargs -- you can also use arbitrary names with setattr()
>> / getattr() :
>>
>> In [6]: setattr(foo, "4 not an identifier", "this works")
>>
>> In [7]: getattr(foo, "4 not an identifier")
>> Out[7]: 'this works'
>>
>> Which brings up a question I've had for years -- is the fact that cPython
>> uses a regular old dict for namespaces (and **kwargs) part of the language
>> spec, or an implementation detail?
>>
>> I would say that for the get/setattr() example, it is kinda handy when
>> you want to use a class instance to model some external data structure that
>> may have different identifier rules. Though I tend to think that's mingling
>> data and code too much.
>>
>> -CHB
>>
>>
>>>
>>> On Thu, 4 Oct 2018 at 02:20, Serhiy Storchaka 
>>> wrote:
>>>
>>>> 04.10.18 11:56, Steven D'Aprano пише:
>>>> > While keyword arguments have to be identifiers, using **kwargs allows
>>>> > arbitrary strings which aren't identifiers:
>>>> >
>>>> > py> def spam(**kwargs):
>>>> >  print(kwargs)
>>>> > 
>>>> > py> spam(**{"something arbitrary": 1, '\n': 2})
>>>> > {'something arbitrary': 1, '\n': 2}
>>>> >
>>>> >
>>>> > There is some discussion on Python-Ideas on whether or not that
>>>> > behaviour ought to be considered a language feature, an accident of
>>>> > implementation, or a bug.
>>>> >
>>>> > Can we get some guidence on this please?
>>>>
>>>> This is an implementation detail. Currently CPython doesn't ensure that
>>>> keyword argument names are identifiers for performance reasons. But
>>>> this
>>>> can be changed in future versions or in other implementations.
>>>>
>>>> ___
>>>> Python-Dev mailing list
>>>> Python-Dev@python.org
>>>> https://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe:
>>>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>>>
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
>>>
>>
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R(206) 526-6959   voice
>> 7600 Sand Point Way NE   (206) 526-6329   fax
>> Seattle, WA  98115   (206) 526-6317   main reception
>>
>> chris.bar...@noaa.gov
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/jmcs%40jsantos.eu
>>
>

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Deprecation" of os.system in favor of subprocess?

[Chris Barker via Python-Dev]

On Wed, Oct 24, 2018 at 9:06 AM, Victor Stinner  wrote:

> I like os.system() and use it everyday.


me too.

Python has been evolved over the years away from a "scripting language",
and becoming more of a "systems language". Which is mostly a good thing,
but no need to gratuitously make quick scripting more difficult.

That being said, if there are examples in the docs that use os.system()
that have a good reason to use subprocess, then by all means, let's update
those.

Also: a point was made there that using os.system is essentially making an
assumption about teh shell in use, and thsu is not very portable accross
systems. MAybe that point could be explicitly added to the docs, rather
than just:

 "more powerful facilities for spawning new processes and retrieving their
results;"

When I read that, I think "system is working fine, I don't need  more
powerful facilities for spawning new processes or anything special in
retrieving their results;"

So more info, right there, about WHY you would want to use subprocess
instead would be helpful.


-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signalling NANs

[Chris Barker via Python-Dev]

works for me, too:

In [9]: x = cast_int2float(0x7ff80001)
In [10]: hex(cast_float2int(x))
Out[10]: '0x7ff80001'

In [11]: x = cast_int2float(0x7ff1)
In [12]: hex(cast_float2int(x))
Out[12]: '0x7ff1'

OS-X, conda build:

Python 3.7.0 | packaged by conda-forge | (default, Aug 27 2018, 17:24:52)
[Clang 6.1.0 (clang-602.0.53)] on darwin

I suspect it depends on the compiler's math library

But neither is raising an exception:

In [3]: x = cast_int2float(0x7ff1)

In [4]: y = cast_int2float(0x7ff80001)

In [5]: 2.0 / x
Out[5]: nan

In [6]: 2.0 / y
Out[6]: nan

When should it?

-CHB



On Fri, Nov 9, 2018 at 3:46 AM, Serhiy Storchaka 
wrote:

> 09.11.18 13:05, Steven D'Aprano пише:
>
>> py> x = cast_int2float(0x7ff1)
>> py> x
>> nan
>> py> hex(cast_float2int(x))
>> '0x7ff80001'
>>
>
> I got '0x7ff1'.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.
> barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Need discussion for a PR about memory and objects

[Chris Barker via Python-Dev]

On Mon, Nov 19, 2018 at 1:41 AM Antoine Pitrou  wrote:

> I'd rather keep the reference to memory addressing than start doing car
> analogies in the reference documentation.
>

I agree -- and any of the car analogies will probably be only valid in some
jurisdictions, anyway.

I think being a bit more explicit about what properties an ID has, and how
the id() function works, and we may not need an anlogy at all, it's not
that difficult a concept. And methions that in c_python the id is
(currently) the memory address is a good idea for those that will wonder
about it, and if there is enough explanation, folks that don't know about
memory addresses will not get confused.

This is what's in the docs now (3.8.0a0):

"""
Every object has an identity, a type and a value. An object’s identity
never changes once it has been created; you may think of it as the object’s
address in memory. The ‘is’ operator compares the identity of two objects;
the id() function returns an integer representing its identity.

**CPython implementation detail:** For CPython, id(x) is the memory address
where x is stored.
"""

I suggest something like the following:

"""
Every object has an identity, a type and a value. An object’s identity
uniquely identifies the object. It will remain the same as long as that
object exists. No two different objects will have the same id at the same
time, but the same id may be re-used for future objects once one has been
deleted. The ‘is’ operator compares the identity of two objects; the id()
function returns an integer representing its identity. ``id(object_a) ==
id(object_b)`` if and only if they are the same object.

**CPython implementation detail:** For CPython, id(x) is the memory address
where x is stored.
"""

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

[Chris Barker via Python-Dev]

On Wed, Feb 13, 2019 at 12:29 PM Barry Warsaw  wrote:

> I think we should aspire for this to be the case too, eventually.  When
> this has come up in the past, we’ve said that it’s not appropriate to
> change PEP 394 until Python 2 is officially deprecated.  OTOH, I appreciate
> that distros and others have to make decisions on this now.  I think it’s
> worth discussing where we eventually want to be as a community, even if we
> continue to recommend no official change until 2020.
>

I don't think end-users will see any discontinuity in 2020. For quite a
while now, more and more people have shifted from using python2 as the
default to using python3 as the default. On the other hand, some folks are
still using python2, and will continue to do so after 2020 (old, still
supported versions RedHat anyone?) Hopefully, after 2020 no one will start
anything new with py2, but it's going to be around for a long, long time.

So as there will be no time for a "clear break", we might as well make
changes when the time "seems" right. And has been pointed out in this
thread, there are a lot of folks not following the PEP anyway (virtual
environments, conda, ??). I myself have been telling my newbie students to
make a link from "python" to python3 for a couple years (before I saw that
PEP!).

I personally would like for `python` to be the latest Python 3 version (or
> perhaps Brett’s launcher), `python2` to be Python 2.7 where installed (and
> not mandatory).  `python3` would be an alias for the latest Python 3.
>

+1

Starting now.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Chris Barker via Python-Dev]

On Fri, Feb 15, 2019 at 11:58 AM Rob Cliffe via Python-Dev <
python-dev@python.org> wrote:

> A function with "microseconds" in the name IMO misleadingly suggests that
> it has something closer to microsecond accuracy than a 1-second granularity.
>

it sure does, but `delta.total_seconds()` is a float, so ms accuracy is
preserved.

However, if you DO want a "timedelta_to_microseconds" function, it really
should use the microseconds field in the timedelta object. I haven't
thought it through, but it makes me nervous to convert to floating point,
and then back again -- for some large values of timedelta some precision
may be lost.

Also:

_MICROSECONDS_PER_SECOND = 100


really? why in the world would you define a constant for something that
simple that can never change? (and probably isn't used in more than one
place anyway

As Alexander pointed out the canonical way to spell this would be:

delta / timedelta(microseconds=1)

but I think that is less than obvious to the usual user, so I think a:

delta.total_microseconds()

would be a reasonable addition.

I know I use .totalseconds() quite a bit, and would not want to have to
spell it:

delta / timedelta(seconds=1)

(and can't do that in py2 anyway)

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

[Chris Barker via Python-Dev]

On Fri, Feb 15, 2019 at 2:39 PM Brett Cannon  wrote:

> In my experience after using 'py' on Windows I consistently miss it on
> UNIX now, so to me there is enough of a benefit that I will continue to
> chip away at the project until it's done regardless of whether anyone else
> uses it. :)
>

And I would REALLY like it if as much was the same as possible on all
platforms...

-CHB




> -Brett
>
>
>>
>>
>>> Victor
>>> --
>>> Night gathers, and now my watch begins. It shall not end until my death.
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>>>
>>
>>
>> --
>> Gustavo J. A. M. Carneiro
>> Gambit Research
>> "The universe is always one step beyond logic." -- Frank Herbert
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

[Chris Barker via Python-Dev]

On Tue, Feb 26, 2019 at 9:58 AM Barry Warsaw  wrote:

> I see this question as having several parts, and the conflation of them is
> part of the reason why the unversioned `python` command is so problematic.
> Python is used for:
>
> * OS functionality
> * to run applications that aren’t critical to the OS but are delivered on
> the OS
> * as the entry point to the interactive interpreter
> * to run applications written and deployed on the OS but completely
> outside of it
>
> For OS functionality, there should probably be a separate command not
> conflated with /usr/bin/python.  The OS can make any adjustments it needs,
> calling it `spython` (as I think Nick once suggested), or whatever.  Nobody
> but OS maintainers cares what this is called or what version of Python it
> exposes.
>

I'm not sure that's necessary at all -- the OS should simply use an
unambiguous path!

I was a RedHat user way back when when in the midst of the python1.5 => 2.0
transition.

RedHat had a bunch of system scripts that had (I think):

#!/usr/bin/env python

In them (or maybe /usr/bin/python, but I'm pretty sure it was the env
version). In any case, when you installed python2 (to local) your system
scripts would ll break (even if they were python2 compatible, there wasn't
a lot of difference, but RedHat also depended on extra packages...)

So what we had to do was install python 2, remove the "python" command that
came with it, and use "python2" in all our scripts.

This was simply broken, and it was RedHat's fault. If they had used:

/usr/bin/python1.5

in their shebang lines, there would have been no problem. And users could
still use the system python1.5 if they wanted, or install an update, or
whatever.

My pint is:

Any OS that ships OS scripts that expect "python" to be a specific version
(or worse, a specific install) is broken.

Some distros are going to ignore the PEP anyway, so there is no harm (and
some good) in specifying in teh PEP the way we think it SHOULD be done, and
then see what happens -- we don't need to make the PEP match current
practice.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Chris Barker via Python-Dev]

This thread petered out, seemingly with a consensus that we should update
the docs -- is anyone doing that?

But anyway, I'd like to offer a counterpoint:

>From the OP, it is clear that:

* Folks have a need for timedeltas to be converted to numbers values, with
units other than seconds (milliseconds, at least).
* If left to their own devices, they may well do it wrong (Or at least not
as right as they should.

So: it would be good to provide a correct, simple,  intuitive, and
discoverable way to do that.

timedelta.total_seconds()

Provides that for seconds, but there is no equivalent for other units.

a_time_delta / timedelta(microseconds=1)

Is now possible in py3, and has been proposed as the canonical way to
convert to specific time units.

However, while it does provide a correct[1] way to do it, it is:

- not very simple
- not very intuitive.
- not the least bit discoverable

Each of these in turn:

simple:
=

compare

duration = a_timedelta.total_seconds()

to

duration = a_timedelta / datetime.timedelta(seconds=1)

Keep in mind that the timedelta object may have been generated by another
module somehow, so the coder that gets it and wants to turn it into a
number of seconds (or milliseconds, or ...) needs to import datetime and
reference the timedelta object. And if they are converting to a plain
number, it's probably because they don't want to be working with timedeltas
at all anyway. So no, not so simple.

intuitive:
==

A casual user reading the first will very likely know what it means -- a
casual user reading the second line will need to think about it carefully,
and probably have to go read the datetime docs, or at least do some
experiments to make sure it does what they think it does. Granted, a
comment:

duration = a_timedelta / datetime.timedelta(seconds=1)  # convert to seconds

would help a lot, but if you need a comment to explain a line of code this
simple, then it's not intuitive.

A couple more data points:

-- I am a physical scientist, I work with unitted quantities all the time
(both in code and in other contexts). It never dawned on me to use this
approach to convert to seconds or milliseconds, or ... Granted, I still
rely on python2 for a fair bit of my work, but still, I had to scratch my
head when it was proposed on this thread.

-- There are a number of physical unit libraries in Python, and as far as I
know, none of them let you do this to create a unitless value in a
particular unit. "pint" for example:

https://pint.readthedocs.io/en/latest/

In pint, you can create objects with units, including time:

In [50]: timespan = 2 * ureg.day

In [51]: print(timespan)

2 day

But if you divide a value of days by a value in seconds, you don't get a
unitless seconds per day:

In [54]: unitless = timespan / (1 * ureg.second)

In [55]: print(unitless)

2.0 day / second

Though pint does know it is dimensionless:

In [56]: unitless.dimensionless

Out[56]: True

And you can reduce it to a dimensionless object:

In [57]: unitless.to_reduced_units()

Out[57]: 172800.0 

And there is your seconds value.

But the "right" way to get a given pint object of time into particular
units is to convert, and then, if you want a plain number, get the
magnitude:

In [53]: print(timespan.to('second').magnitude)

172800.0

So no -- dividing a datetime by another datetime with the value you want is
not intuitive: not to a physical scientist, not to a user of other physical
quantities libraries -- is it intuitive to anyone other than someone that
was involved in python datetime development??

Discoverable:
==

It is clearly not discoverable -- the OP didn't find it, and no one other
than Alexander found it on this thread (I'm pretty sure). That could be
made much better with docs, but we all know no one reads docs -- I'm
willing to bet that even if we better document it, folks will still be
writing utility functions like the OP posted.

And  (this is also a doc issue) -- I wanted to know what the options were
for units we could specify to the datetime constructor, so I used the nifty
iPython ?, and got:

In [59]: timedelta?

Init signature: timedelta(self, /, *args, **kwargs)
Docstring:  Difference between two datetime values.


Gosh, that's helpful! (we really need to fix that regardless of this
thread). And someone earlier mentioned "weeks" without realizing that is
was already supported:

In [60]: timedelta(weeks=1)

Out[60]: datetime.timedelta(days=7)

So we have a real discoverability problem -- we really need to fix that.

On the other hand, if we add a few convenience methods, we will have a way
for folks to do this that is:

correct
simple
intuitive
discoverable

And we really don't need to add many. Looking at the docs ('cause the
docstring is useless), I see that the timedelta takes:

datetime.timedelta(days=0, seconds=0, microseconds=0, milliseconds=0,
minutes=0, hours=0, weeks=0)

So at most, we could have:

.total_microseconds()
.total_seconds()
.total_minutes()
.total_h

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

[Chris Barker via Python-Dev]

On Tue, Feb 26, 2019 at 3:25 PM Barry Warsaw  wrote:

> > Who gets to decide on PEP 394 changes?
>
> Honestly, I think it’s the active distro maintainers who need to make this
> decision.  They have the pulse of their own communities and users, and can
> make the best decisions and compromises for their constituents.  I
> personally am not part of that any more, so I have no problem having no say
> (despite still having opinions :).
>

The PEP is what the Python community recommends. The distro maintainers can
(and will) do whatever they want. IF we are going to let the distros drive
it, then there is no point to a the PEP.

Well, why look for more dissent?  If you can align Homebrew, Fedora-et-al,
> Debian-et-al, and we already know what Arch has done, and the PEP authors
> are in agreement, isn't that enough to JFDI?


more than enough :-)

-CHB

 -
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compact ordered set

[Chris Barker via Python-Dev]

On Tue, Feb 26, 2019 at 3:43 PM Barry Warsaw  wrote:

>  The behavior differences between dicts and sets is already surprising to
> many users, so we should be careful not to make the situation worse.
>

It's a nice to have, but other than the fact that we all used to use a dict
when we really wanted a set before set existed, I'm not sure the connection
is there to a layperson.

A mapping and a set type really don't have much to do with each other other
than implementation -- anyone that isn't familiar with python C code, or
hash tables in general, wouldn't likely have any expectation of them having
anything to do with each other.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Chris Barker via Python-Dev]

On Tue, Feb 26, 2019 at 7:22 PM Terry Reedy  wrote:

> > timedelta.total_seconds()
>
> To me, total_x implies that there is a summation of multiple timedeltas,
> and there is not.  So not intuitive to me.


THAT was a discussion for when it was added -- I can't say it's my favorite
name either. But at least a quick glance at the docstring will address
that. Hmm -- that one could use a little enhancement, too:

In [3]: td.total_seconds?
Docstring: Total seconds in the duration.

But at least it COULD be made clear in a docstring :-)

> (Neither us current obscure
> option).  It is also not obvious is answer is rounded to nearest second
> or not.
>

also could be made clear in a docstring -- the full docs say:

"""
timedelta.total_seconds()
Return the total number of seconds contained in the duration. Equivalent to
td / timedelta(seconds=1).

Note that for very large time intervals (greater than 270 years on most
platforms) this method will lose microsecond accuracy.
"""

That last point indicates that it is not rounded -- and a quick check will
note that it returns a float -- but those docs could be improved to make it
clear that a float is returned. (i.e. that is WHY it loses microsecond
accuracy). But anyway, you are nevert going to know all the intricacies of
a method by its name -- you need to know about the rounding and all if you
are trying to decide if you are going to us the method, but if you read it,
you probably have a pretty good idea what it does.

Anyway -- "I don't really like the name much" is really the answer to a
different question

(I don't like it either, but it's what's there)

> So at most, we could have:
> >
> > .total_microseconds()
> > .total_seconds()
> > .total_minutes()
> > .total_hours()
> > .total_days()
> > .total_weeks()
>
> I am also not enthusiastic about multiple methods doing essentially the
> same thing.  I might prefer one method, .convert? with an argument
> specifying the conversion unit, 'microseconds', 'seconds', ... .


yup -- that's been proposed, and is still on the table as far as I'm
concerned.

Though I'm not sure I like "convert", but would rather, say "in_unit" or
"to_unit". But any of these would address the intuitive and discoverability
issue -- very few folks would have any trouble guessing what any of:

a_datetime.convert("hours")
a_datetime.to_unit("hours")
a_datetime.total_hours()

mean -- at least generally, even if they con't know if they are getting a
float or an integer result.

And as for discoverable, any of these as methods on a dt object would
probably be checked out if you were looking for such a thing.


> I think this is in python-ideas territory.
>

I'd rather not :-)   --  have you SEEN the length of the thread on how long
the line limiet should be in PEP8.

Frankly, I think the arguments have been made, and the options on the table
-- if this were a PEP, it would be time for a pronouncement -- or at least
one of:

- not going to happen
or
- maybe happen but we need to iron out the details

I've lost track of where we are with the governance model, so is there a
way to get a clear idea for whether it's time to drop this?

BTW, I don't think this is worth a PEP, but if I get a go-ahead, and it
needs to be written up, I'd be willing to do so.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Chris Barker via Python-Dev]

On Wed, Feb 27, 2019 at 7:15 AM Paul Ganssle  wrote:

> As another data point, I also have a background in the physical sciences,
> and I actually do find it quite intuitive. The first time I saw this idiom
> I took it to heart immediately and only stopped using it because many of
> the libraries I maintain still support Python 2.
>
we know have two contrasting data points -- it's settled!

But I'd like to know -- I too will probably remember and use this idiom now
that I know it (if not take it to heart ...) -- but would you hvae figured
it out on your own?


>  Seeing the code `timedelta(days=7) / timedelta(days=1)`, I think most
> people could confidently say that that should return 7, there's really no
> ambiguity about it.
>
in that example absolutely -- but this is the one I'm worried about:

an_arbitrary_dateitme_returned_from_a_lib_you_didn't_write /
datetime.timedelta(days=7)

not so obvious.

> I think the analogy with pint's unit-handling behavior is not completely
> appropriate, because `timedelta` values are not in *specific* units at
> all, they are just in abstract duration units.
>
agreed -- but it is a use-case that is worth looking into -- folks will
either have no experience with units, or with a lib like that one (there
are more, and I think they all handle it a similar way)

And if you have a pint object representing time that you do not know the
units of, the way to get it into the units you want is to call teh to()
method:

a_time_in_arbitrary_units.to('second')

That is an obvious and familiar API.


> I agree that it is not discoverable (which is unfortunate), but you could
> say the same thing of *all* operators.
>

sure, but most operators are really widely used.


> There's no tab-completion that will tell you that `3.4 / 1` is a valid
> operation
>

well, THAT one had better be obvious :-)


> or that (3,) + (4,) will work, but we don't generally recommend adding
> methods for those things.
>

and that IS a source of confusion, but at least for the built-in types (and
ones that conform to the primary ABCs), people are going to learn it pretty
early.

But conversion of a datetime object into particular units is a pretty
specialized operation.

Also, in most (all?) cases with builtin (and most of the standard library)
the math operators return the same type (or a conceptional same type: int /
int => float -- all numbers)


> I do think the discoverability is hindered by the existence of the
> total_seconds method, because the fact that total_seconds exists makes you
> think that it is the correct way to get the number of seconds that a
> timedelta represents, and that you should be looking for other analogous
> methods as the "correct" way to do this,
>

well, yes. I for one, was very happy to have found it when I did :-) -- of
course that was py2, when it WAS the only easy way to do it.


> when in fact we have a simpler, less ambiguous (for example, it's not
> obvious whether the methods would truncate or not, whereas __truediv__ and
> __floordiv__ gives the division operation pretty clear semantics)
>

I don't think so, I was actually a touch surprised that:

a_timedelta / timedelta(microseconds=1)

yielded a float that may have lost precision, when the timedetla type
stores ms precision, and python ints can handle any size int.

it seems __floordiv__ does preserve precision. Which is kinda ironic --
after all, it's called "ture division" and "floor division", not "float
division" and "integer division"

My point being -- if you care about the details, you're going to have to
dig deeper no matter what method is used.

I think it's too late to *remove* `total_seconds()`, but I don't think we
> should be compounding the problem by bloating the API with a bunch of other
> methods, per my earlier arguments.
>

I'm not a big fan of bloated APIs -- but this is not a lot of methods, with
a clear naming pattern, on an API without a huge number of methods (seven
non-dunder attributes by my count) already.

>From a practicality beats purity standpoint -- a few new methods (or a
single conversion method) is less pure and "correct", but also a lot more
usable.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Chris Barker via Python-Dev]

Did we ever hear back from the OP as to whether they were using py2 or 3?

If they were unable to find timedelta division in py3 -- that's a pretty
good case that we need something else.


The OP:
"""
On Wed, Feb 13, 2019 at 9:10 PM Richard Belleville via Python-Dev <
python-dev@python.org> wrote:

> In a recent code review, the following snippet was called out as
> reinventing the
> wheel:
>
> _MICROSECONDS_PER_SECOND = 100
>
>
> def _timedelta_to_microseconds(delta):
>   return int(delta.total_seconds() * _MICROSECONDS_PER_SECOND)
>
>
> The reviewer thought that there must already exist a standard library
> function
> that fulfills this functionality. After we had both satisfied ourselves
> that we
> hadn't simply missed something in the documentation, we decided that we had
> better raise the issue with a wider audience.
>
"""

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Chris Barker via Python-Dev]

On Wed, Feb 27, 2019 at 3:04 PM Richard Belleville 
wrote:

> Timedelta division is quite a nice solution to the problem. However, since
> we're maintaining a python version agnostic library at least until 2020, we
> need a solution that works in python 2 as well.
>

So you were limited to a py2 solution.

But di you poke around in py3 before posting?

(you should have, python-dev is really about active development, i.e.
python 3 -- but that's not the point here)


> For the moment, we've left the code as in the original snippet.
>

If you care about microsecond precision for large timeseltas, you may want
to improve that.

I *think* this is the "correct" way to do it:

def timedelta_to_microseconds(td):
return td.microseconds + td.seconds * 1000 + td.days * 8640

(hardly tested)

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP-582 and multiple Python installations

[Chris Barker via Python-Dev]

Sorry somehow missed Steve Dower's post:

 that
discussion is mostly happening at
https://discuss.python.org/t/pep-582-python-local-packages-directory/963/

I"ll go there to comment.

-CHB


On Thu, Apr 4, 2019 at 9:02 AM Chris Barker - NOAA Federal <
chris.bar...@noaa.gov> wrote:

>
> > I'd like to raise a potential edge case that might be a problem, and
> likely an increasingly common one: users with multiple installations of the
> *same* version of Python.
>
> I would suggest that that use case is best addressed by a system that
> isolates the entire python environment, such as conda.
>
> > This is actually a common setup for Windows users who use WSL,
> Microsoft's Linux-on-Windows solution, as you could have both the Windows
> and Linux builds of a given Python version installed on the same machine.
>
> Sure, but Isn’t the WSL subsystem pretty isolated already? Would native
> Windows and WSL users be running in the same dir?
>
> That being said, I’m pretty skeptical of the PEP — I understand the
> motivation — I make a point of avoiding virtual environments in my intro
> classes, but at some point folks will need to learn them.
>
> I’ve had students think that virtualenv was a part of (or required by)
> e.g. flask, because the tutorials include it in the setup.
>
> But I think environments really need to be more distinct, not less, I’m
> quite concerned about mingling them in one place.
>
> Maybe I’m reading it wrong, but it seems that this could create serious
> clashes with other “environment” systems, such as conda.
>
> I suppose one could say: “don’t do that” — I.e. don’t create a
> __pypackages__ dir if you are going to use conda — but many folks want the
> same source to be runnable in multiple “styles” of Python.
>
> Also, I see a major benefit for teaching,  but it does go a bit against my
> philosophy of not hiding important details from newbies — that is, don’t
> teach using an approach that is not suitable for production.
>
> And newbies could be really confused by the fact that pip installs stuff
> differently depending on what dir they are in and what is in that dir.
>
> The PEP is listed as a draft — anyone know what’s going on with it?
>
> -CHB
>
>

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: dict.with_values(iterable)

[Chris Barker via Python-Dev]

On Fri, Apr 12, 2019 at 10:20 AM Brett Cannon  wrote:

>
>> This doesn't strike me as needing an optimization through a dedicated
> method.
>

maybe a new dict mapping type -- "shared_dict" -- it would be used in
places like the csv reader where it makes sense, but wouldn't impact the
regular dict at all.

you could get really clever an have it auto-convert to a regular dict when
any changes were made that are incompatible with the shared keys...

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: strip behavior provides inconsistent results with certain strings

[Chris Barker via Python-Dev]

This was quite extensively discussed on python-ideas recently:

https://mail.python.org/archives/list/python-id...@python.org/thread/RJARZSUKCXRJIP42Z2YBBAEN5XA7KEC3/#WIRID57ESUFUAQQQ6ZUY2RK5PKQQYSJ3

(I'm finding it hard to find a good thread view in the new interface -- but
that will get you started)

My memory of that thread is that there was a lot of bike shedding, and
quite a lot of resistance to adding a couple new methods, which I
personally never understood (Not why we don't want to add methods
willy-nilly, but why there was this much resistance to what seems like an
low-disruption, low maintenance, and helpful addition)

I think it just kind of petered out, rather than being rejected, so if
someone wants to take up the mantle, that would be great -- and some
support from a core dev or two would probably help.

-CHB




On Fri, Jun 28, 2019 at 10:44 AM Brett Cannon  wrote:

> Glenn Linderman wrote:
> > On 6/27/2019 3:09 PM, Brett Cannon wrote:
> > > My guess is that without Guido to just ask this will
> > > have to go to a PEP as it changes a built-in.
> > > How does adding two new methods change a built-in?
> > > Now if an extra parameter were added to modify lstrip, rstrip, and
> strip
> > to make them do something different, yes.
> > But adding new methods doesn't change anything, unless someone is
> > checking for their existence.
>
> Sure, but the built-ins are so widely used that we don't want to blindly
> add every method idea that someone comes up with either. We all very much
> share ownership of the built-ins, so we should all agree to changes to
> them, and getting agreement means either clear consensus and/or a PEP.
>
> -Brett
>
> > My preferred color is  pstrip and sstrip (prefix and suffix strip) since
> > lstrip and rstrip mean left and right.
> > And maybe there should be a psstrip, that takes two parameters, the
> > prefix and the suffix to strip.
> > Such functions would certainly reduce code in a lot of places where I do
> > if  string.startswith('foo'):
> >  string = string[ 3: ];
> > as well as making it more robust, because the string and its length have
> > to stay synchronized when changes are made.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SB6JECL6BUPF4Q37Q5SAGMDOQ2DI2GP5/
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MCV2ZKH6BUZ25XQSR4TJHUC5TLOOUCP4/

[Python-Dev] Re: Building Standalone Python Applications with PyOxidizer

[Chris Barker via Python-Dev]

Congrats on this new tool -- I'll be checking it out. With a quick glance
at the docs, I see this:

"""
 PyOxidizer loads everything from memory and there is no explicit I/O being
performed. When you import a Python module, the bytecode for that module is
being loaded from a memory address in the executable using zero-copy. This
makes PyOxidizer executables faster to start and import - faster than a
python executable itself!
"""

This made me think of a recent thread either here or on python-ideas about
optimizing the cPython start up time. In particular the idea of bundling
the core parts of the stdlib pre-loaded somehow. Multiple options were
discussed, but as far as I know no one actually prototyped any of them.

But it looks like you've done that.

My question is this: Can you use PyOxidizer partway -- that is, bundle up
some modules, while keeping the regular module loading system for
everything else. If so, then one could build an executable with all the
modules that always get loaded. It's quite a list judging from:

$ python -c "import sys; print(len(sys.modules))"

55

How much faster is the start up time when you make an exe that does, well,
nothing.

Also -- I have some vague memory of there being a mailing list for the
various "bundlers": py2exe, py2app, PyInstaller, etc... it would be good to
engage that community, it really is a bit different than the usual
"packaging" community, which is primarily focused on python packages
themselves rather than applications.

-CHB


On Mon, Jun 24, 2019 at 11:06 AM Gregory Szorc 
wrote:

> Hey Python developers,
>
> I just published the initial release of PyOxidizer - a utility for
> producing self-contained, potentially single file executable Python
> applications. You can read more about it at
> https://gregoryszorc.com/blog/2019/06/24/building-standalone-python-applications-with-pyoxidizer/
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QU6D62Q2XMYENKBLMBB6SNSSBIZWSUZV/

[Python-Dev] Unification of the Mac builds?

[Chris Barker via Python-Dev]

Sorry if I'm out of the loop here, but with Apple's new chip coming out, we
need new a build configuration (which I think has already been started, if
not done).

Perhaps we could take this opportunity to better modularize / unify the
build setup?

As it was last I checked, you really had only two options:

1) Ignore anything mac specific, and get a "unix" build.

2) Get a full "Framework" build, optionally with Universal support.

It would be nice to keep the Framework structure independent of the
Mac-specific features, if possible.

In particular, I'd love to get be able to get the "pythonw" executable
wrapper in an otherwise standard unix build [*].

It would also be nice if it were possible to get universal binaries in a
"unix style" build.

(option 3 would be to simply abandon the Framework Build altogether -- it's
still not clear to me what this really buys mac users)

Any chance of this happening? I'm afraid I know nothing of autoconf, so
can't be much help, but I'd be willing to help out with testing, or
documenting, or anything else that I have the skills to do.

Thanks,

-Chris

[*] The pythonw issue has been a thorn in the side of conda for years.
conda uses a standard unix build on the Mac, for consistency with other
unix systems. But there is no configuration set up to build the pythonw
wrapper outside of a "Framework" build. So instead, conda has creates its
own "pythonw" wrapper -- but that is a bash script that re-directs to a
different executable. This works fine on the command line (or #! line), but
it does not work with setuptools' entry_points. And the setuptools
community hasn't shown any interest in hacking around it. Anyway, few
enough people develop Desktop apps (particularly with conda) so this has
lingered, but it would be nice to fix.









-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S27ZRILE4QNVG7OZ3ROQW3TB7GRQHFVL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: unittest of sequence equality

[Chris Barker via Python-Dev]

On Wed, Dec 23, 2020 at 1:06 AM Steven D'Aprano  wrote:

> We're not obligated to take heroic
> measures to integrate numpy arrays with unittest methods. If we can do
> so easily, sure, let's fix it.
>
> I think Ivan's suggestion that the assertSequenceEqual method fall back
> on element-by-element comparisons has some merit.
>

If there are other common types this helps with, sure. But for numpy, as
pointed out elsewhere in this thread, it would still fail for numpy arrays
of > 1 dimension.

Personally I think this is really an issue with the structure of unitest --
having a custom assertion for every possibility is intractable.

If you want to test numpy arrays, use the utilities provided by numpy.

- CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NSZ7GLILVVITHOXIARWPVCMYHAEI7PE2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Story behind vars() vs .__dict__

[Chris Barker via Python-Dev]

This was discussed a bit over on python-ideas recently, so a note from me,
and one from that thread:

Or for that matter, not the reason to provide
> object's internal storage via object's attribute: obj.__dict__.
>

Well, it IS an implementation detail that it's a dictionary, but having a
dunder that gives you the object's namespace seems like a specification to
me -- would you be happier to call it obj.__names__ ?

And as for having vars() be the way to get that instead -- it's pretty much
universal in Python that dunders provide the implementation, while
functions (such as len()) provide a universal and clean interface to the
functionality -- so vars() seems quite consistent in that sense.

However, as pointed out on python-ideas, some objects have a .__dict__
and/or .__slots__ -- and THAT distinction seems to be an
implementation issue leaking out.

Maybe it would be too backward incompatible, but it makes sense to me that
vars() should return the union of .__dict__ and .__slots__

(though I notice that if you create __slots__ in pure Python, its names
show up in dict anyway -- so clearly I'm confused...)

But the question was about (thru-the-time) relationship of vars()
> vs .__dict__ ...
>

I believe Guido's recollection is that vars() was intended to provide easy
access, particularly in a REPL.

And that accessing .__dict__ directly in code is perfectly reasonable,
since you only need to do that when you are metaprogramming anyway -- which
inherently requires some knowledge of internal structure of objects anyway.

I'd be interested to hear if any other folks that have been around a long
time have other thoughts.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MG3BSQR7ZTVQV6OGJWUDHE4NBQVFTCSK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Unification of the Mac builds?

[Chris Barker via Python-Dev]

Ned,

Thanks -- I'll take further discussion to the python-mac list.

Ronald:

That’s a feature of the framework build. The unix build is exactly the same
> as a unix build on other platform.  Adding the same feature to the unix
> build should be possible, but would complicate the build.  I have no
> interest to work on this, but would be willing to review PRs,
>

fair enough.


> as long as those aim for feature parity with the framework build. That is,
> both pythonw(1) and python(1) should redirect through an embedded app
> bundle.
>

yes, that's what I'm thinking.

Done, just configure “—enable-universalsdk
> —with-universal-archs=universal2” without specifying a framework build.
>

Excellent, thanks!

Something I would like to work on, but don’t have enough free time for, is
> an alternative installation with an application bundle on the top level
> instead of a framework.  Installation would then entail dropping “Python
> X.Y.app” into your application folder, and uninstallation would be to drop
> the same bundle into the bin.
>

That would be nice -- and would, I think, actually buy us something useful
from this bundling :-)

Just don’t use conda ;-).  To be blunt, doing this properly is trivial.
>

which "this" are you referring to -- and if it's trivial, then why hasn't
anyone figured out how in multiple conversations over years?

But if it is -- tell us how and maybe we will do it (if it's really
trivial, maybe even I, with my limited build skills could do it)

Thanks,

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7YZ4JB5PTL2LKGRCNZF7X4TP7ZYTBW3H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Unification of the Mac builds?

[Chris Barker via Python-Dev]

Thanks Ronald,

This  is really helpful.

-CHB

On Fri, Jan 15, 2021 at 5:43 AM Ronald Oussoren 
wrote:

>
>
> On 14 Jan 2021, at 23:03, Chris Barker via Python-Dev <
> python-dev@python.org> wrote:
>
> Ned,
>
> Thanks -- I'll take further discussion to the python-mac list.
>
> Ronald:
>
> That’s a feature of the framework build. The unix build is exactly the
>> same as a unix build on other platform.  Adding the same feature to the
>> unix build should be possible, but would complicate the build.  I have no
>> interest to work on this, but would be willing to review PRs,
>>
>
> fair enough.
>
>
>> as long as those aim for feature parity with the framework build. That
>> is, both pythonw(1) and python(1) should redirect through an embedded app
>> bundle.
>>
>
> yes, that's what I'm thinking.
>
> Done, just configure “—enable-universalsdk
>> —with-universal-archs=universal2” without specifying a framework build.
>>
>
> Excellent, thanks!
>
> Something I would like to work on, but don’t have enough free time for, is
>> an alternative installation with an application bundle on the top level
>> instead of a framework.  Installation would then entail dropping “Python
>> X.Y.app” into your application folder, and uninstallation would be to drop
>> the same bundle into the bin.
>>
>
> That would be nice -- and would, I think, actually buy us something useful
> from this bundling :-)
>
>
> To be honest I’d expect that this will also lead to complaining about how
> this build is not a unix build ;-)
>
>
> Just don’t use conda ;-).  To be blunt, doing this properly is trivial.
>>
>
> which "this" are you referring to -- and if it's trivial, then why hasn't
> anyone figured out how in multiple conversations over years?
>
> “This” is having a pythonw that’s the same as in framework builds.
>
> I don’t know. I do know that nobody asked questions about this before.
>
>
> But if it is -- tell us how and maybe we will do it (if it's really
> trivial, maybe even I, with my limited build skills could do it)
>
>
>
> For anyone reading along that isn’t familiar with macOS: On macOS a number
> of system libraries, and in particular GUI libraries, require that they are
> used from an application bundle (‘.app”).   For Python we want to be able
> to just use the command-line for simple scripts (and testing of more
> complicated applications), instead having to build an app bundle every time.
>
> In framework builds this is solved by two changes relative to a unix build:
> 1. The regular python binary is in
> “{sys.prefix}/Resources/Python.app/Contents/MacOS/Python”
> 2. “{sys.executable}” (or “{sys.prefix}/bin/python3”) is a stub executable
> that does nothing but execv[1] the regular python binary. The source of
> this binary is in Mac/Tools/pythonw.c
>
>
> Note: All of this requires using a shared library build (—enable-shared)
> because of the way the stub finds sys.prefix.
>
> To add this to the unix build as well requires:
> 1. Make up a location for the Python.app
> 2. Change the Makefile(.pre.in) to build the stub executable install
> files in their new location, but only when installing on macOS with an
> —enable-shared build
> 3. Tweak the code in pythonw.c to calculate the correct path to Python.app
> 4. Possibly tweak the calculation of sys.prefix for unix builds, IIRC the
> special macOS sauce is enabled for framework builds only
> 5. Do some debugging because this list is likely incomplete, I typed it
> from memory
>
>
> Ronald
>
> [1] for the pedantically correct: it actually uses posix_spawn in exec
> mode to ensure that the correct CPU architecture is used. That’s why “arch
> -x86_64 python3.9” works when you use the universal2 installer for python
> 3.9, if we’d use execv the stub would launch the actual interpreter in
> native mode.
>
> Ronald
>
>
> Thanks,
>
> -CHB
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7YZ4JB5PTL2LKGRCNZF7X4TP7ZYTBW3H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>

[Python-Dev] Re: Advantages of pattern matching - a simple comparative analysis

[Chris Barker via Python-Dev]

On Mon, Nov 23, 2020 at 8:20 AM Brian Coleman 
wrote:

> Take as an example a function designed to process a tree of nodes similar
> to that which might be output by a JSON parser. There are 4 types of node:
>
> - A node representing JSON strings
> - A node representing JSON numbers
> - A node representing JSON arrays
> - A node representing JSON dictionaries
>
> The function transforms a tree of nodes, beginning at the root node, and
> proceeding recursively through each child node in turn. The result is a
> Python object, with the following transformation applied to each node type:
>
> - A JSON string `->` Python `str`
> - A JSON number `->` Python `float`
> - A JSON array `->` Python `list`
> - A JSON dictionary `->` Python `dict`
>
> I have implemented this function using 3 different approaches:
>
> - The visitor pattern
> - `isinstance` checks against the node type
> - Pattern matching
>

I've always thought that the alternative to a "switch case" construct in
Python (and I suppose most OO languages) is subclassing and method
overriding. I guess that's what the "visitor pattern" is here, but it seems
to be adding a bunch of unnecessary bolierplate.

So: given that you have a special "Node" object anyway, the thing to do is
to have those Node object know how to unpack themselves. Then the top
"traverse the tree" function becomes a single method or attribute access:

tree = node_tree.value

here's what the Nodes look like in this example:

class Node:
def __init__(self, val):
self._value = val

@property
def value(self):
return self._value


class StringNode(Node):
pass


class NumberNode(Node):
pass


class ListNode(Node):
@property
def value(self):
return [item.value for item in self._value]


class DictNode(Node):
@property
def value(self):
return {k: item.value for k, item in self._value.items()}

Of course, this requires that you have control of the Node objects, rather
than getting them from some other library -- but that seems to be what all
the examples here are anyway.

If you do need to parse out a tree of object that are not "special"
already, then you need to do some type of pattern matching / isinstance
checking. In this case, I wrote a function that builds up a tree of Nodes
from arbitrary Python objects:

def make_nodes_from_obj(obj):
if isinstance(obj, str):
return StringNode(obj)
if isinstance(obj, Real):
return NumberNode(obj)
if isinstance(obj, Sequence):
return ListNode([make_nodes_from_obj(item) for item in obj])
if isinstance(obj, Mapping):
return DictNode({k: make_nodes_from_obj(item)
 for k, item in obj.items()})

And that could benefit from pattern matching, I suppose, though it's not
very compelling to me.

And in "real world" code, I've done just this -- building a system for
saving / restoring dataclasses to/from JSON. In that case, each of the
dataclasses knows how to save itself and build itself from JSON-compatible
python objects (numbers, dicts, strings, lists) -- so again, no need for
pattern matching there either. And what I really like about the approach of
putting all the logic in the "nodes" is that I can make new types of nodes
without having to touch the code at the "top" that visits those nodes.

In short -- I'm still looking for a more compelling example :-)

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S6Y24VDUD5RL2SDU3LE56P3AONSAGMGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should the definition of an "(async) iterator" include __iter__?

[Chris Barker via Python-Dev]

Note: I am all for not enforcing anything here -- let's keep duck typing
alive!

If static type checkers want to be more pedantic, they can be -- that's
kinda what they are for :-)

But the OP wrote:

"""
 "[i]terators are required to have an __iter__()
 method"
which neither `for` nor `iter()` actually enforce.
"""

I'm confused -- as far as I can tell `for` does enforce this -- well, it
doesn't enforce it, but it does require it, which is the same thing, yes?
But does it need to?

On Wed, Sep 15, 2021 at 4:07 PM Guido van Rossum  wrote:

> Reminder about how for-loops work:
>
> This:
>
> for x in seq:
> 
>
> translates (roughly) to this:
>
> _it = iter(seq)
> while True:
> try:
> x = next(_it)
> except StopIteration:
> break
> 
>

exactly -- that call to iter is always made, yes?

The "trick" here is that we want it to be easy to use a for loop with
either an iterable or an iterator. Otherwise, we would require people to
write:

for i in iter(a_sequence):
...

which I doubt anyone would want, backward compatibility aside.

And since iter() is going to always get called, we need __iter__ methods
that return self.

However, I suppose one could do a for loop something like this instead.

_it = seq
while True:
try:
x = next(_it)
except TypeError:
_it = iter(_it)
x = next(_it)
except StopIteration:
break


That is, instead of making every iterator an iterable, keep the two
concepts more distinct:

An "Iterator" has a __next__ method that returns an item or raises
StopIteration.

An "Iterable" has an __iter__ method that returns an iterator.

That would mean that one couldn't write a single class that is both an
iterable and an iterator, and uses (abuses) __iter__ to reset itself. But
would that be a bad thing?

Anyway, this is just a mental exercise, I am not suggesting changing
anything.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5OBJCQPV2A6MB7BNYUEFASU2P4D23BWQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Python 2 death march

[Chris Barker via Python-Dev]

Regardless of the date of the final release, no one's Python2 install will
stop working, and people will still be able to download and install that
last release.

So I like the metaphor -- it's being "sunset" -- there will be a long dusk
.. a month or tow makes no difference to anyone's workflow.

-CHB





On Fri, Sep 13, 2019 at 6:39 PM Sumana Harihareswara 
wrote:

> Hi. I've joined python-dev to participate in this thread (I don't have
> email delivery turned on; I'll be checking back via the web).
>
> Benjamin, I am sorry that I didn't check in with you, and assumed that
> January 1, 2020 would be the the date of the final 2.7 point release.
> (My understanding was based on Guido's EOL announcement from March last
> year https://mail.python.org/pipermail/python-dev/2018-March/152348.html
>   -- I should have also gotten a review from you and not just the
> Steering Council in https://github.com/python/steering-council/issues/14
> .) I'm going to continue this discussion here so I can make sure I
> understand the policy decision properly, and then (if necessary) update
> the FAQ.
>
> Based on what I've read here and what I see in
> https://www.python.org/dev/peps/pep-0373/#maintenance-releases , it
> sounds like the timeline will go something like:
>
> * 2019-10-19: release of 2.7.17 October
> * October, November, and December 2019: developers continue to fix
> issues in 2.7
> * 2020-01-01: code freeze for 2.7.18 release candidate
> * January and February 2020: flexibility to fix any issues introduced
> since the 2.7.17 release, but no other bugs or security issues, and no
> 3.x backports
> * ~2020-04-02: release candidate for 2.7.18
> * 2020-04-17: final 2.7.18 release
>
> Is this right? (If so, I can submit an update to PEP 373.)
>
> This is a little more complicated than I had anticipated when
> communicating out about the sunsetting. But I can find a way either to
> concisely communicate this, or to point to a user-friendly explanation
> elsewhere.
>
> Thanks.
>
> --
> Sumana Harihareswara
> Changeset Consulting
> https://changeset.nyc
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/MXCGMTXDY7BX6JBBU36O5YFRWWBB3NQE/
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QM5B2SH5URWHF4WEWJ33KM4CXZA4QMFB/

[Python-Dev] Re: The Python 2 death march

[Chris Barker via Python-Dev]

Peter, I think that went just to me, which I suspect was not what you
intended, so I've brought it back on the list:

On Tue, Sep 17, 2019 at 12:06 PM Peter Wang  wrote:

> On Mon, Sep 16, 2019 at 5:55 PM Chris Barker via Python-Dev <
> python-dev@python.org> wrote:
>
>> Regardless of the date of the final release, no one's Python2 install
>> will stop working, and people will still be able to download and install
>> that last release.
>> So I like the metaphor -- it's being "sunset" -- there will be a long
>> dusk .. a month or tow makes no difference to anyone's workflow.
>>
>
> Metaphorically that is correct, but at the same time there are things like
> https://pythonclock.org which communicate a certain... precision and
> finality to the Dec 31, 2019 date.  I agree with Ned that as a community,
> we should have a unified messaging about this.  I already anticipate a
> final round of teeth-gnashing as the date draws near, so it would be good
> to minimize the room for misunderstandings.
>

Yes, it would, but if we are emphasizing that hard date, we need to
emphasise that it is a hard date on when the last patch would potentially
be applied, and other than that, there is no discontinuity ...

-CHB



>
> Cheers,
> Peter
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PLQ2SUAUJ5MPEAWY54Z5A6BVH5F7GRTB/

[Python-Dev] Re: The Python 2 death march

[Chris Barker via Python-Dev]

On Tue, Sep 17, 2019 at 10:39 PM Terry Reedy  wrote:

> I agree.  The thread title is a bit extreme.  There will be a long
> twilight.
>
> > Metaphorically that is correct, but at the same time there are
> > things like https://pythonclock.org  which
>
> is one person's very unofficial site.  It interprets and references
> https://www.python.org/dev/peps/pep-0373/#maintenance-releases


I can't see a way to contact the owner of that site (is it Guido??), but if
you are reading this list, maybe a sunset metaphor would be nice.

I like that: when you are watching a sunset, you can look out at the
horizon, hoping to see the green flash, and there is a clear moment when
the sun fully drops below the horizon -- but it doesn't suddenly get dark
:-)

> Yes, it would, but if we are emphasizing that hard date,
>
> But we core developers are not, other than what the PEP says:
> "Support officially stops January 1 2020, but the final release will
> occur after that date.  Actually, support has already been tapering off
> for over 3 or 4 years and 2.7 patches are now a slow dribble.  What is
> ending is free build and security from us.
>
> > we need to  emphasise that it is a hard date on when the last patch
> > would  potentially be applied,
>
> Not really your concern.


I'm just echoing (and agreeing with) Peter here -- there is a specific date
in the PEP (January 1 2020), and that has been adopted by pythonclock and
others, so it would be good to be clear what EXACTLY "support stops" means.

And I misspoke, it would be "date on when the last patch would potentially
be considered" And if that's not the case, then why have a date other than
the date of the last release ??

Personally, I have no angst over a few months here or there, but apparently
some people do.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EMDC6YMASGWTXJJZC6VLNCGKM6FONYHG/

[Python-Dev] Re: PEP 611: The one million limit.

[Chris Barker via Python-Dev]

I am not qualified to comment on much of this, but one simple one:

1 million is a nice round easy to remember number.

But you can fit 2 million into 21 bits, and still fit three into 64 bits,
so why not?

Ialso noticed this:

> Reference Implementation
> 
> None, as yet. This will be implemented in CPython, once the PEP has been
accepted.

As already discussed, we really can't know the benefits without some
benchmarking, so I don't expect the PEP will be accepted without a (at
least partial) reference implementation.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SD7WM5XHGH42HHVMXX53Z4TKGVZEOQRI/
Code of Conduct: http://python.org/psf/codeofconduct/