convenience

[Avi Gross via Python-list]

An earlier post talked about a method they used for "convenience" in a way 
they apparently did not understand and many of us educated them, hopefully.

That made me wonder of teh impact on our code when we use various forms
of convenience. Is it convenient for us as programmers, other potential readers,
or a compiler or interpreter?

The example used was something like:

varname = objectname.varname

The above clearly requires an existing object. But there is no reason it
has to be the same name. Consider the somewhat related idea used
almost always in code:

import numpy as np

Both cases make a sort of pointer variable and are a tad shorter to write.

But what impact does it have on interpreters and compilers?

I assume objectname.varname makes the interpreter look up "objectname" in
some stack of namespaces and find it. Then it looks within the namespace
inside the object and finds varname. Finally, it does whatever you asked for 
such
as getting or setting a value or something more complex. 

So what happens if you simply call "varname" after the above line of code has
set it to be a partial synonym for the longer name? Clearly it no longer does 
any
work on evaluating "objectname" and that may be a significant saving as the
name may be deep in the stack of namespaces. But are there costs or even errors
if you approach an inner part of an object directly? Can there be dunder 
methods not
invoked that would be from the standard approach? What kind of inadvertent
errors can creep in?

I have seen lots of errors in many languages and functions designed to make
some things superficially easier. Sometimes it involves adding a namespace
to the stack of namespaces that contains the names of all column names
of a DataFrame, for example to avoid extra typing, and this can make fairly
complex and long pieces of code way shorter. But this can also make other
existing names go deeper into the stack and temporarily shadowed or if
a similar technique is used a bit later in the code while this is in effect, can
shadow it as in opening another object of the same kind.

Sometimes there is a price for convenience. And even if you are fully
aware of the pitfalls, and expect to gain from the convenience to you
or even the interpreter, your work may be copied or expanded on by
others who do not take care. In the example above in a language like R,
you can add the namespace to the environment and use it in read-only mode
but writing to a resulting variable like "col1" makes a new local variable and 
does 
not make any change to the underlying filed in a data.frame.

I mean (again this is not Python code) that:

mydata$result <- mydata$col1 + mydata$col2

will create or update a column called "result" within the data.frame called 
mydata,

BUT

with(mydata, result <- col1 + col2)

That does not work well and leaves "result" alone or non-existent. Instead, a 
local variable 
called result is created and then rapidly abandoned as shown by code that 
displays it:

with(mydata, {result <- col1 + col2; print(result)})

The above returns 3 when the columns in mydata are 1 and 2.

Just FYI, there are other packages that handle such short names properly and 
well, but they 
are done carefully. Regular users may easily make mistakes for the sake of 
convenience.

So I wonder if some pythonic methods fit into convenience modes or can
veer into dangerous modes. Spelling things out in full detail may be annoying
but generally safer.
-- 
https://mail.python.org/mailman/listinfo/python-list

Difficulty in installing Python

[Reuel Lewis]

Hello,
I'm trying to install Python as I'd like to learn it. I'm a newbie in all
things related to software programming.
I have Windows 10 installed on my HP laptop. I tried to install Python but
I didn't Click on the 'Add python 3.10 to path'. So I tried to uninstall it
and install Python once more. But I only get options like Modify / Repair.
If I click on either of those options it does the Modify / Repair thing and
then I just see the 'Close' button. It does not go anywhere else from
there.
So what should I do to start this program so I can start learning? Thank
you and stay safe always.

Kind Regards,
Reuel R. Lewis
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for convenience

[Paul St George]

On 21/03/2022 18.04, dn wrote:

> On 22/03/2022 10.17, Chris Angelico wrote:
> > On Tue, 22 Mar 2022 at 08:13, Paul St George  > > wrote:
> >>
> >>
> >> When I am writing code, I often do things like this:
> >>
> >> context = bpy.context  # convenience
> >>
> >> then whenever I need bpy.context, I only need to write context
> >>
> >>
> >> Here’s my question:
> >>
> >> When I forget to use the convenient shorter form
> >>
> >> why is bpy.context not interpreted as bpy.bpy.context?
> >>
> > 
> > I don't understand the question. When you do that "for convenience"
> > assignment, what you're doing is creating a local variable named
> > "context" which refers to the same thing that bpy.context does (or did
> > at the time of the assignment, but presumably you only do this when
> > bpy.context won't get reassigned). It has no effect on any other name.
> > There's no magic happening here - it's just assignment to the name
> > context, like anything else.
> > 
> > What are you expecting to happen here?
> 
> 
> It's the way Python works.
> 
> try:
> 
> context = bpy.context  # convenience
> print( id(context), id(bpy.context) )
> 
> Remember that the 'relationship' between the two is established at
> run-time and at the data/address 'level' - and not at compile-time. Thus
> "context" points to a memory location, and does not 'stand for'
> "bpy.context" anywhere other than in your mind.
> 
> (which is why we often need to use a copy() when we want 'separate data'
> - see also the 'counters' Python uses to manage "garbage collection")
> 
> -- 
> Regards,
> =dn


Thanks dn,
I did exactly as you suggested.

import bpy

context = bpy.context  # convenience variable
print( id(context), id(bpy.context) )

4932508928 4932508928

The two are the same, and that makes it crystal clear. And, as a bonus you have 
explained a use of copy().

—
Thanks,
Paul




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for convenience

[Paul St George]

On 21/03/2022 18.02, Cameron Simpson wrote:

> On 21Mar2022 22:12, Paul St George  wrote:
> >When I am writing code, I often do things like this:
> >
> >context = bpy.context  # convenience
> >
> >then whenever I need bpy.context, I only need to write context
> >
> >
> >Here’s my question:
> >
> >When I forget to use the convenient shorter form
> >
> >why is bpy.context not interpreted as bpy.bpy.context?
> 
> Because it still has its original meaning. You haven't changed the 
> meaning of the word "context" in any position, you have simply made a 
> module level name "context" referring to the same object to which 
> "bpy.context" refers.
> 
> So your module global namespace contains, initially, "bpy". Then you 
> assign:
> 
> context = bpy.context
> 
> and now your module global namespace contains "bpy" and "context". But 
> "bpy.context" is still what it was, because "bpy" is an object and you 
> have done nothing to its ".context" attribute.
> 
> Consider this code:
> 
> class O:
> pass
> 
> o = O()
> o.x = 3
> 
> x = o.x
> 
> print(x)
> print(o.x)
> 
> I expect to see "3" twice. What do you expect?
> 
> "bpy" is no different to "o" - it's just a name.


Thanks Cameron,
What did I expect? Well, I expected to see  "3” twice, but I did not understand 
why. Now I do!

—
Thanks,
Paul








-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for convenience

[Paul St George]

On 21/03/2022 17.47, Avi Gross wrote:

> So, I ask Paul what other language than python he has used before, just out 
> of curiosity.

The other language I have used (and often) is Processing. Before that, and a 
long time ago, Lingo. 

—
Paul






-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for convenience

[dn]

On 23/03/2022 09.23, Paul St George wrote:
> On 21/03/2022 18.04, dn wrote:
> 
>> On 22/03/2022 10.17, Chris Angelico wrote:
>>> On Tue, 22 Mar 2022 at 08:13, Paul St George >> > wrote:


 When I am writing code, I often do things like this:

 context = bpy.context  # convenience

 then whenever I need bpy.context, I only need to write context


 Here’s my question:

 When I forget to use the convenient shorter form

 why is bpy.context not interpreted as bpy.bpy.context?

>>>
>>> I don't understand the question. When you do that "for convenience"
>>> assignment, what you're doing is creating a local variable named
>>> "context" which refers to the same thing that bpy.context does (or did
>>> at the time of the assignment, but presumably you only do this when
>>> bpy.context won't get reassigned). It has no effect on any other name.
>>> There's no magic happening here - it's just assignment to the name
>>> context, like anything else.
>>>
>>> What are you expecting to happen here?
>>
>>
>> It's the way Python works.
>>
>> try:
>>
>> context = bpy.context  # convenience
>> print( id(context), id(bpy.context) )
>>
>> Remember that the 'relationship' between the two is established at
>> run-time and at the data/address 'level' - and not at compile-time. Thus
>> "context" points to a memory location, and does not 'stand for'
>> "bpy.context" anywhere other than in your mind.
>>
>> (which is why we often need to use a copy() when we want 'separate data'
>> - see also the 'counters' Python uses to manage "garbage collection")
>>
>> -- 
>> Regards,
>> =dn
> 
> 
> Thanks dn,
> I did exactly as you suggested.
> 
> import bpy
> 
> context = bpy.context  # convenience variable
> print( id(context), id(bpy.context) )
> 
> 4932508928 4932508928
> 
> The two are the same, and that makes it crystal clear. And, as a bonus you 
> have explained a use of copy().
> 
> —
> Thanks,
> Paul


and thank you - it is refreshing, if not enervating, to receive feedback
on efforts-expended!

You will also notice, that now you understand the id() stuff, the
tag-team effect between @Chris and I (which we have often played, albeit
not by-design), now makes sense as an whole (if you didn't quite follow,
earlier).


My research-topic is Cognitive Psychology (how we learn - albeit not
usually in Python). I found this conversation useful, and may well apply
it as an example (with your permission, and suitably anonymised) - one
doesn't need to be a 'computer person' to follow the logic and thus
realise the dissonance!

While learning (this part of) Python and adding to 'previous
experience', you formed a "mental-model" of how things work (just as we
all do). However, when it came time to implement this knowledge:

- you created a 'situation'
- (all) things didn't 'work' (which also required realisation)
- you analysed and rationalised (but noted inconsistency)
- you asked a question (which many of us quickly understood)
- you've learned/corrected


The 'issue' is *not* a fault on your part, nor (necessarily) a lack of
learning or a lack of effort. So, no criticism from me!

The (under-lying) lesson, is that we (as trainers, but with application
to all helpers, pair-programmers, mentors, documentation-writers, et al
- working with less-experienced colleagues) shouldn't spout a whole load
of 'facts', 'rules', and formulae/s - which we expect to be committed to
memory. We need to help form a 'correct' mental-model ("correct" being
defined by the Python interpreter and 'the Python gods' who build it -
big "thank you" to them!).

Accordingly, my criticism of tests/exams which require recitation of
facts ("parroting"), compared with "mastery" (can you actually DO what
is being asked). More importantly, and finally getting to the point:
'tests' should be defined to reveal these (personal) 'quirks' of
learning/understanding, which led to a 'faulty' mental-model!

Your rationale made sense, was logical and understandable. How are you
to know that Python deems it 'wrong'? (until a 'test' shows you!)

The 'interest' should not be on the people who, and all the 'answers'
which, were 'correct'. What is more informative, is why someone (aside
from guessing, ie intelligent, reasonable, after learning the material,
exerting effort...) got it 'wrong' - but thought his/her path was true!
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Reducing "yield from" overhead in recursive generators

[Rathmann]

> On 19 Mar 2022, at 03:07, Greg Ewing  wrote:
>
> On 19/03/22 9:40 am, Rathmann wrote:
>> The other challenge/question would be to see if there is a way to implement
>> this basic approach with lower overhead.
>
> My original implementation of yield-from didn't have this problem,
> because it didn't enter any of the intermediate frames -- it just
> ran down a chain of C pointers to find the currently active one.
>
> At some point this was changed, I think so that all the frames
> would show up in the traceback in the event of an exception.
> This was evidently seen as more important than having efficient
> resumption of nested generators.

So that sounds like the original CPython implementation had an
O(depth) component, but with a lower constant factor than the current
version?  (Of course, since Python limits recursion depth, speaking of
O(n) is not quite right.)

Prioritizing stack traces over performance seems consistent with the
well known blog post on tail recursion elimination.  This is one of
the reasons I was motivated to try to attack this at the
library/application level -- when the programmer invokes a
transformation like I am proposing, they know that it is going to
affect the stack trace.

I am still hoping to find a way to implement the transformed
function/driver combination with lower overhead.  This is a little
tricky since the performance of the approaches is very dependent on
the Python implementation.  Below are some timings for 2.7 and 3.8
CPython and PyPy. For 2.7, for the untransformed version I used a loop
of yield statements, and timed with time.clock(), for 3.8, it is
"yield from" and time.perf_counter().  All times are for traversing a
balanced tree of 8 million nodes on X86_64 Linux.

3.8.10 (default, Nov 26 2021, 20:14:08) 
[GCC 9.3.0]
For depth= 22 nodes= 8388607 iters= 1 
  Elapsed time (yield from): 6.840768865999053 driver: 6.702329244999419

3.6.9 (7.3.1+dfsg-4, Apr 22 2020, 05:15:29)
[PyPy 7.3.1 with GCC 9.3.0]
For depth= 22 nodes= 8388607 iters= 1 
  Elapsed time (yield from): 4.037276787999872 driver: 2.3724582960003318

2.7.18 (default, Mar  8 2021, 13:02:45) 
[GCC 9.3.0]
For depth= 22 nodes= 8388607 iters= 1 
  Elapsed time (yield loop): 8.863244 driver: 13.788312

2.7.13 (7.3.1+dfsg-2, Apr 21 2020, 05:05:41)
[PyPy 7.3.1 with GCC 9.3.0]
For depth= 22 nodes= 8388607 iters= 1 
  Elapsed time (yield loop): 8.802712562 driver: 2.034388533

The most extreme variation was 2.7 pypy, where the driver was 4 times
faster.  (Of course, 2.7 is less used these days.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for convenience

[Avi Gross via Python-list]

I sent George a private reply as discussing other languages gets rapidly 
off-topic.

I want to add a small addendum here about the technique he used and a Dave Neal
and others are trying, a way to imagine things that is more compatible with how
a language like Python works so it meets expectation.

Your example was name= obj.name and you saw it as a convenience.

But consider some form of linked list, perhaps a multidimensional one
like a tree or generalized graph. If you are traversing the list there is a 
concept
of following a sort of pointer to the "next" and once there following that to 
the
"next" and so on, till you find what you want or reach a leaf node and back up
and so on.

So your routine may have multiple lines that look like:

here = here.next

As you traverse, "here" keeps moving and a sublist or subtree or whatever lies 
below you. It is not a convenience but a portable way to keep track of your 
current
location. It keeps changing. And, in some cases, you retreat back and follow
a nextleft or nextright pointer of sorts.

In examples like that, the construct is not for convenience but simply because
it makes no sense to approach an arbitrary data structure and have to
say first.next.next.next.next ...

Also noted is your use of a direct pointer had positive and also negative
ramifications to consider. It is not a synonym just for convenience.


-Original Message-
From: Paul St George 
To: python-list@python.org
Sent: Tue, Mar 22, 2022 4:52 pm
Subject: Re: for convenience


On 21/03/2022 17.47, Avi Gross wrote:



> So, I ask Paul what other language than python he has used before, just out 
> of curiosity.



The other language I have used (and often) is Processing. Before that, and a 
long time ago, Lingo. 



—

Paul













-- 

https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: convenience

[Greg Ewing]

On 23/03/22 7:00 am, Avi Gross wrote:

But are there costs or even errors
if you approach an inner part of an object directly? Can there be dunder 
methods not
invoked that would be from the standard approach? What kind of inadvertent
errors can creep in?


The attribute could be a property that returns different objects
at different times depending on some condition. In that case
you will need to think about whether you want the current value
of the attribute, or the value it had when you looked it up
before.

Concerning dunder methods, there are a bunch of them involved in
looking up an attribute of an object. Theoretically it's possible
for these to have side effects and for bad things to happen if you
skip them. But that would be very unusual and surprising, and we
generally assume such things don't happen.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Reducing "yield from" overhead in recursive generators

[Greg Ewing]

On 23/03/22 11:44 am, Rathmann wrote:

So that sounds like the original CPython implementation had an
O(depth) component, but with a lower constant factor than the current
version?


Yes, but so much lower that I would expect it to be unmeasurable.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

Re: convenience

[Avi Gross via Python-list]

Greg,

Yes, what I describe may not be common and some code goes to serious lengths 
precisely to make direct connections to internals on an object hard to access.

But Python indeed allows and perhaps encourages you to use what you consider
side effects but perhaps more. There are many dunder methods that allow
all sorts of side effects or direct effects.

You can set it up so an access of a variable increments a counter
but perhaps it can set it up so three tries at guessing a password disables it 
or refuses
to accept a number out of your bounds. Or it may be that some accesses to say 
write
a value might also update several other values. For example, if you store a 
distance and
also a time, any change of either one resets the value of a stored item holding 
a velocity.
Perhaps adding additional values (or removing any) results in an immediate 
recalculation
of the mean or standard deviation or whatever.

If you allow direct access then you can make changes that get around that and I 
see no reason
to assume programs do not use such ideas.

Perhaps worse is that items may be dynamic. If an object representing me and my 
library card
checks out a book, a new object may be created that holds the book. If you copy 
a pointer to
the reference to the book in the library card object and later the book is 
returned and a new book
checked out, then your reference is now pointing to the old book. If your goal 
is to get to the
CURRENT book, you failed!

So I stand by my suggestion that unless you know the code well and can be sure 
you won't
corrupt things, this kind of convenient access has possible dangers.



-Original Message-
From: Greg Ewing 
To: python-list@python.org
Sent: Tue, Mar 22, 2022 7:12 pm
Subject: Re: convenience


On 23/03/22 7:00 am, Avi Gross wrote:

> But are there costs or even errors

> if you approach an inner part of an object directly? Can there be dunder 
> methods not

> invoked that would be from the standard approach? What kind of inadvertent

> errors can creep in?



The attribute could be a property that returns different objects

at different times depending on some condition. In that case

you will need to think about whether you want the current value

of the attribute, or the value it had when you looked it up

before.



Concerning dunder methods, there are a bunch of them involved in

looking up an attribute of an object. Theoretically it's possible

for these to have side effects and for bad things to happen if you

skip them. But that would be very unusual and surprising, and we

generally assume such things don't happen.



-- 

Greg

-- 

https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list